The Gesturing Screen Art and Screen agency within postmedia assemblages
Charu Maithani A thesis submitted for the degree of Doctor of Phiolosophy
School of Art & Design February 2021
1
ABSTRACT This thesis investigates screens as key elements in postmedia assemblages where multiple technical devices and media platforms function relationally to activate new capacities. I characterize the agency of screens as a gesturality that re-arranges and sustains medial relations with and between other components. The gestures of screens reformulate mediality, but also shift experiences and open elements to novel formations and affects. In each re- organisation of the postmedial assemblage, the function and relations of screens are not pre-defined but emerge in process. I develop this dynamic conception of screens and assemblages to account for their diverse manifestations in postmedia. Here I employ an agential realist framework found in the work of Karen Barad and draw on the concept of gestures set out by Giorgio Agamben. This research contributes to a new understanding of postmediality in conjunction with a new conception of the agency of screens.
The thesis focuses on mainly digital screen-oriented artworks, repositioning these as heralding or firmly engaging with the postmedial condition. By challenging an understanding of screens that limits them to mere casings for images, this thesis expands the scope and role of screens in postmedia art practices stretching as far back as three decades. It argues that such art works and practices foreground the gesturality of screens and offers in-depth studies of works by Shilpa Gupta, Ulrike Gabriel, Natalie Bookchin, Blast Theory, Ragnar Kjartansson, Sandra Mujinga and Sondra Perry. Such works highlight how screens come to be relationally enacted in postmedia and how that enactment occurs through their performance of medial gestures.
I identify two kinds of gestures of postmedia screens that support and connect the technical, aesthetic and in some cases political components of an
2 assemblage. I turn to the multiplicity of frames both on-screen and distributed across screens observed by theorists of media such as Lev Manovich and Anne Friedberg. But the postmedial frame is consistently accompanied by what is out-of-frame – scrolling, swiping and ‘pinching’ continually calls on the out-of-frame to be moved on-screen. The out-of-frame is a postmedial screen gesture, then, that maintains an ongoing relation to the ‘inside’ of the frame, supporting and conditioning it. The multiple temporalities of postmedia assemblages – such as that of images, participants and software – allow an ‘out-of-frame’ to endure beyond the framed image. The second gesture is observed in the pervasiveness of chroma screens — blue and green screens used for compositing other images in postproduction. I suggest that this now ubiquitous technique suspends images from screens. Through a relational analysis of colour, and focusing on Perry’s work, I draw upon ways in which the blankness of chroma screens can be made to gesture a different enactment of race – ‘blackness’ as productive difference. Such gesture in postmedia entails the circulation and transference of social and cultural setting.
These two gestures of screens highlight multiple dimensions of the relations of postmedial screens beyond that of framed images, offering us ways to be attentive to enactments of screens as they continue to gather relevance in our expanded visual setting. As screens multiply, this research suggests alternate ways of conceiving the aesthetics and experience of screens’ persistent medial configurations.
3 LIST OF PUBLICATIONS Small sections of this thesis are included in the following papers:
Maithani, Charu. “Blan/ck Screens : Chroma screens performing race.” Media-N [forthcoming].
Maithani, Charu. "Chroma Screens–Intra-Actions, Connections and Gesturalities." In RE: SOUND 2019–8th International Conference on Media Art, Science, and Technology 8, pp. 253-257. 2019.
Maithani, Charu. "Intermediality of Screens in Post-Media Assemblages." Acta Universitatis Sapientiae, Film and Media Studies 17, no. 1 (2019): 63-80.
Maithani, Charu. "Screens as Gestures in Interactive Art Assemblage." AM Časopis za studije umetnosti i medija 17 (2018): 147-155.
1
TABLE OF CONTENTS
Acknowledgements ……………………………………………………………………i List of Figures ………………………………………………………………………… iv
INTRODUCTION ……………………………………………………………………1
1. Defining Screenness: Capacities and Intra-Actions of Screens…………………………………………………………………………………………..41 Screens in post media art assemblages 45 Screenness of Screens 51 Screenness in an agential realist framework 61 Screenness in and through material-discursive practices 66 Intra-Actions of contemporary screenness 76
2. Gesturality Of Screens………………………………………………92 Gesturing potentialities 97 Affective gestures 117 Who/What is gesturing 126 Distributed self and circulating gestures 135 The Third gesture 139
INTERLUDE ……………………………………………………………………….156
3. >>(Out-of) Frame Gestures<< Enactments of Multiplicities and Temporalities of and by Screens………………………………161 From Multiple Screens To Multiplicity Of Screens 164 Framing Frames of Space-Time 184
1
Out-Of-Frame 198 Multitudinal Temporalities In And Of Out-Of-Frame 209 Frames in Crystalline States 225
4. >>Blan/ck Screens, Absent Images<< Screen without Images Performing Race……………………………………………..233 Image-screen entanglement 238 Blan/ckness: Chroma screen practices and blue-green matterings 261 Blan/ck Opacity 278
POSTSCRIPT ………………………………………………………………………284
Bibliography ………………………………………………………………………295
2 ACKNOWLEDGEMENTS Throughout this research, I have received guidance and support from people around me. I am indebted to many people for their love, patience and kindness that has sustained me on this journey.
Thank you to my supervisor, professor Anna Munster who showed faith in me and gave me the opportunity to pursue research at UNSW. Over the years, she has encouraged me to be entangled with concepts and practices but, also guided me and reigned me in when I was getting lost in them. Her invaluable discussions and feedback have encouraged me to express myself better. Prof. Munster’s intellectual generosity, academic rigour and research ethics are inspiring, and I hope to carry what I have learnt about these values throughout my academic life. Over the last one year with my flailing motivation in a pandemic ridden world, her words infused me with much needed energy and confidence. With her guidance I have never regretted even once to move to Australia. A teacher, a mentor, an inspiration, her support has been integral to this project. My gratitude to professor Edward Scheer for his encouragement and valuable support. Your comments have been enlightening.
Thank you to my friends and colleagues on the fourth floor D block in the Paddington campus, particularly Anastasia Murney, Aneshka Mora, Mitchell Ryan, Alia Parker, June Kim, Scarlet Steven, Skye Wagner, Kirsten Drews for creating a comfortable space to share apprehensions, and acknowledge shortcomings. And for the laughter and silliness that have helped me get through a lot over the years. Most of all thank you for helping me to settle into a new country and a PhD routine. A special thanks to Anastasia Murney for being a wonderful neighbour and collaborator, Skye Wagner for sharing
i sharp, critical feedback on various versions of chapters and cheering me on in this journey, and June Kim for ideas and discussions that enriched the otherwise sloppy middle months of this journey. Thank you to the short-lived but intense Reading Group, particularly Scott East, Sam Spurr and Scarlet Steven to help in activating some of the thinking employed in this thesis. Thank you to artists Shilpa Gupta, Christa Sommerer, Laurent Mignonneau, Natalie Bookchin and Matthieu Cherubini for enlightening discussions about their work.
A big thank you to Ellen and Sally for putting up with the craziness of the fourth floor. I also wish to express my appreciation for the professional and administrative staff at Art and Design – Steven Shears, Denis Cooper, Ant Banister and Tobias Gilbert for helping to solve myriad issues while going about teaching and other duties on campus. The academic staff Astrid Lorange, Uros Çvoro, Tim Gregory, Katherine Moline and Verónica Tello who have chaired various review committees over the years. Your feedback and optimism have helped tremendously. A special thank you to Michele Barker for her valuable support while teaching. I would also like to acknowledge the helpful suggestions of Shivaun Weybury at the (former) Learning Centre for her assistance at the early stages. Thank you to Peter Blamey for his keen eye in proofreading this thesis.
Thank to my family who have supported me through their unconditional love. My parents whose love and support gave me confidence to venture out. Thank you to Ammaji and Babuji for their numerous expressions of deep care and love. Thank you to my brother Parth and his partner Neelam for their loving wishes, and the lip-smacking delicacies that have brightened dull days. My gratitude to extended family members, Ashish Nauriyal, Neelam Nauriyal and Avi for welcoming me in their home and helping to familiarise
ii
with Sydney. A big thank you to Judith Martinez and Craig Billingham for their open-hearted reception and generosity. The comfort and support provided by you both has certainly enriched these years. I am also thankful to Rewa Wright for motivational advice and always ready to lend a hand.
My warmest and biggest support has come from Ishan. Dragging him to Australia and asking him to plunge into this endeavour has been the most invaluable. He has nurtured and encouraged me, and I am thankful for his deep, unbounded love that keeps me going. I am indebted to him for taking on so much for which an expression of thanks does not suffice. Having to make some hard sacrifices over the years, I hope that we make a life that has been worth it.
Lastly, my grandparents whose sudden deaths has made me value goodbyes. The continuing restricted circumstances have robbed many of us of not being able to properly mourn the loss of loved ones. The circumstances have made me understand the significance of rituals of death and grief. Death may not seem a positive note to start a thesis, but the Hindi word of death, देहांत (dehānta), means the termination of body, not of life; life continues in form of ideas, relations and memories. I dedicate this thesis to my Daadi and Nana ji.
iii LIST OF FIGURES
Figure 1: John Gerrard, Solar Reserve (Tonopah, Nevada), 2014, realtime computer-generated simulation. Installation view, Lincoln Centre, New York, 2014.
Figure 2: John Gerrard, Solar Reserve (Tonopah, Nevada), 2014, realtime computer-generated simulation. Digital video still.
Figure 3: Unknown, A Louis XV style carved giltwood three-panel folding screen, last quarter 19th century, 171.5 X 59 cm, Sotheby’s. “19th Century Furniture, Sculpture, Ceramics, Silver & Works of Art, 26 October 2010, 10 AM EDT, New York.” https://www.sothebys.com/en/auctions/ecatalogue/2010/19th-century- furniture-sculpture-ceramics-silver-works-of-art-n08672/lot.119.html.
Figure 4: Nintendo Switch console in handheld mode. Image reproduced from Nintendo website. https://www.nintendo.com.au/nintendo-switch/.
Figure 5: Nintendo Switch console in TV mode. Image reproduced from MakeUseOf. https://www.makeuseof.com/tag/best-local-multiplayer- games-nintendo-switch/.
Figure 6: Shilpa Gupta, Shadow 3, 2007, interactive video projection incorporating the viewer's simulated shadow. Digital video still from documentation. Photographer: Anil Rane. Courtesy: The Artist & Galleria Continua / Le Moulin, San Gimignano / Beijing / Les Moulins / Habana.
Figure 7: Mapping the basic components of a mobile phone assemblage while navigating on foot in an urban location.
Figure 8: Detailing the components of a mobile phone assemblage while navigating on foot in an urban location.
Figure 9: Simon Weckert, Google Maps Hacks, 2020, performance and installation. Two stills documenting the performance. http://simonweckert.com/googlemapshacks.html
Figure 10: Aby Warburg, Mnemosyne Atlas, Panel 47, 1924-1929, photographs and paper on black cloth. Scans of original panels from Cornell
iv University Library and The Warburg Institute. https://warburg.library.cornell.edu/panel/47.
Figure 11: Aby Warburg, Mnemosyne Atlas, Panel C, 1924-1929, photographs and paper on black cloth. Scans of original panels from Cornell University Library and The Warburg Institute. https://warburg.library.cornell.edu/panel/c.
Figure 12: Eadweard J. Muybridge, Man Throwing an Iron Disk, plate 307 from Animal Locomotion (1887) 1884-86, 26 x 27.1 cm, Museum of Modern Art, New York, https://www.moma.org/collection/works/87258?artist_id=4192&page=1& sov_referrer=artist.
Figure 13: Étienne-Jules Marey, Successive phases of the pole vault, c. 1890, gelatin silver print, 22.2 x 48.9 cm, The Museum of Fine Arts, Houston, https://emuseum.mfah.org/objects/53535/movements-in-pole- vaulting?ctx=87e0947939fa1c0170e5aa361e98605cbc90ae5c&idx=1.
Figure 14: Pablo Gnecco, Gesture-Gesture, 2014, interactive installation. Installation view at Gallery-72, Atlanta, https://yopablo.studio/Gesture- Gesture.
Figure 15: Yang Fudong, The First Intellectual (right), 2000, from photo series The First Intellectual, Colour chromogenic print, 193 × 127 cm, ShanghART gallery, https://www.shanghartgallery.com/galleryarchive/simpleWork.htm?work Id=3246.
Figure 16: Yang Fudong, The Revival of the Snake, 2005, multi-channel video installation, 8 minutes looped. Digital video still.
Figure 17: Marin Arnold, Deanimated, 2002, digital video, 60 mins. Two digital video stills.
Figure 18: Videodrome. Directed by David Cronenberg, 1983, 87 mins, Canadian Film Development Corporation. Digital video still.
Figure 19: Videodrome. Directed by David Cronenberg, 1983, 87 mins, Canadian Film Development Corporation. Digital video still.
Figure 20: Videodrome. Directed by David Cronenberg, 1983, 87 mins, Canadian Film Development Corporation. Digital video still.
v Figure 21: Ulrike Gabriel, Breath, 1991/92, interactive installation. Digital video still from documentation, http://www.llllllll.de/breath_e.html.
Figure 22: Shilpa Gupta, Shadow 3, 2007, interactive video projection incorporating the viewer's simulated shadow. Digital video still from documentation. Photographer: Anil Rane. Courtesy: The Artist & Galleria Continua / Le Moulin, San Gimignano / Beijing / Les Moulins / Habana.
Figure 23: Myron Krueger, VideoPlace, 1985, interactive shadow installation. Video still from live interaction, https://aboutmyronkrueger.weebly.com/videoplace.html.
Figure 24: Rafael Lozano-Hemmer, Body Movies, Relational Architecture 6, 2001, interactive shadow installation. Digital video still from interaction at Atlantico Pavillion, Lisbon, Portugal, 2002. Photo by: Antimodular Research.
Figures 25-30: Hito Steyerl, Strike, 2010, HDV, 28 sec. Digital video stills.
Figure 31: George Landow, There Appear Edge Lettering, Sprocket Holes, Dirt Particles, Etc, 1966, 16 mm film, 5 mins. Digital video still, https://lux.org.uk/work/film-in-which-there-appear-sprocket-holes- edgelettering-dirt-particles-etc.
Figure 32: Olia Lialina, My Boyfriend came back from the War, 1996, net art. Screenshot from project website, http://www.teleportacia.org/war/.
Figure 33: Olia Lialina, My Boyfriend came back from the War, 1996, net art. Screenshot from project website, http://www.teleportacia.org/war/.
Figure 34: Ragnar Kjartansson, The Visitors, 2012, nine-channel video installation, 60 mins looped. Installation view, Institute of Contemporary Art, Boston, 2014. Photo by Charles Mayer Photography.
Figure 35: Natalie Bookchin, Now he’s out in public and everyone can see, 2012, 18-channel video installation, 16 mins looped. Screenshot from video documentation.
Figure 36: Natalie Bookchin, Now he’s out in public and everyone can see, 2017, digital film, 24 mins. Selection of video stills.
vi Figure 37: Natalie Bookchin, Now he’s out in public and everyone can see, 2012, 18-channel video installation, 16 mins looped. Selection of screenshots from video documentation.
Figure 38: Natalie Bookchin, Long Story Short, 2016, digital film, 45 mins. Two video stills.
Figure 39: Blast Theory, Can You See Me Now?, 2003, location based mixed media game. Digital video still from documentation of the game played in Tokyo in 2005.
Figure 40: Blast Theory, Can You See Me Now?, 2003, location based mixed media game. Digital video still from documentation of the game played in Tokyo in 2005.
Figure 41: Blast Theory, Can You See Me Now?, 2003, location based mixed media game. Selection of stills from the game played in Tokyo in 2005.
Figure 42: Ragnar Kjartansson, The Visitors, 2012, nine-channel video installation, 60 mins looped. Installation view at San Francisco Museum of Modern Art in 2017.
Figure 43: Bergson’s cone of memory, reproduced from Matter and Memory. Here S is the present moment where the past contracts. All memories are at AB and A’B’ and A”B” are different regions of the past (virtual circuits) that preserve the past.
Figure 44: Harun Farocki, Parallel I, 2012, two-channel digital video, 16 mins looped. Digital video stills.
Figure 45: Hsu Chia-Wei, Marshal Tie Jia - Turtle Island, 2012, one-channel HD video, one photograph 120cm×80cm, two documents, 6:35 mins. Digital video still.
Figure 46: Hsu Chia-Wei, Marshal Tie Jia - Turtle Island, 2012, one-channel HD video, one photograph 120cm×80cm, two documents, 6:35 mins. Digital video still.
Figure 47: Hito Steyerl, How Not to be Seen: A Fucking Didactic Educational .MOV File, 2013, one-channel HD video, 14 mins. Screenshot from digital video.
Figure 48: Hito Steyerl, How Not to be Seen: A Fucking Didactic Educational .MOV File, 2013, one-channel HD video, 14 mins. Screenshot from digital video.
vii Figure 49: Sondra Perry, installation view of exhibition flesh out at Squeaky Wheel Film and Media Art Centre, Buffalo, New York, on view January 20 - May 6, 2018.
Figure 50: Sondra Perry, Graft and Ash for a Three Monitor Workstation, 2016, video bicycle workstation, 9:05 mins. Installation view, The Kitchen New York, 2016.
Figure 51: Sondra Perry, Graft and Ash for a Three Monitor Workstation, 2016, video bicycle workstation, 9:05 mins. Screenshot from digital video.
Figure 52: Sondra Perry, Graft and Ash for a Three Monitor Workstation, 2016, video bicycle workstation, 9:05 mins. Screenshot from digital video.
Figure 53: Oscar G. Rejlander, The Two Ways of Life, 1857, printed in 1920s, carbon print, 40.6 x 76.2 cm, The MET, https://www.metmuseum.org/art/collection/search/294822.
Figure 54: Chroma keying on blue screen through the process of colour difference. Digital image, https://fxmakingof.wordpress.com/2014/01/21/the-1940s-an-overview-of- special-effects/
Figure 55: Sandra Mujinga, Throwing Voices, 2016, three lexan plates, perlon wires, HD video installation, 44:08 mins. Installation view at Oslo Kunstforening, Norway, 2016.
Figure 56: Sandra Mujinga, Throwing Voices, 2016, three lexan plates, perlon wires, HD video installation, 44:08 mins. Screenshot from digital video.
Figure 57: Matthieu Cherubini, Afghan War Diary, 2010, online website. Screenshot from documentation of the website, http://mchrbn.net/afghan- war-diary/.
Figure 58: Matthieu Cherubini, Afghan War Diary, 2010, online website. Focussed screenshot from documentation of the website, http://mchrbn.net/afghan-war-diary/.
viii Introduction This thesis was orchestrated through screens of different kinds in various locations. One kind of orchestration occurred through the screens of digital devices such as my laptop, mobile phone and desktops, all connected to the vast interlinked networks of the world wide web. Another mode of orchestration was via experiencing different kinds of media installations, sometimes as projections on various surfaces and on other occasions as displays on monitors and television screens. Travelling to these numerous venues where the artworks were located required me to navigate a sea of screens — maps on phones for location guidance, screens on train and bus stations, and even screens on an airplane. Meeting via screens, which became the norm during 2020, allowed conversations and analyses to take place with artists and academics. On these occasions, people from different locations converged in one frame on the screen. On many occasions, the space internal to the screen lingered on outside and in physical space. This was especially the case with screen installations, which created an immersive space beyond their displays, resonating with what was interior to the frames on-screen. Screens during this research, then, were part of different aggregates connected to multiple media, occurring in varied contexts. The very thinking and formulation of this thesis has been facilitated through, with and on screens; the research itself has involved highlighting various relations assisted by screens.
In an attempt to think about the operations of screens, this thesis proposes that screens come to acquire agency through enactments, which enable them to sustain the relations of mediality in postmedia assemblages. Comprising multiple media and coming together as contingent and unstable, the postmedia assemblage is where technical devices and platforms function
1 relationally to activate new medial potentialities. From the perspective of the postmedia assemblage, I look beyond the functionality of screens as mere display devices, to their roles in interconnecting, rearticulating and reconfiguring relations. This accord screens a certain kind of agency, which becomes the focus of this dissertation. But I also argue that screens themselves are enacted in and through the postmedia assemblage. I attend to the ways in which screens cannot be considered pre-given entities in postmedia – rather, they are configured through the work and operations of its assemblages. In this way, I intend to examine the agency of screens through what I will argue are their gesturalities. Such gesturalities maintain mediality even within an age that is designated as postmedial.
My research considers postmedia artworks that foreground two specific enactments of screens or types of gestures as I call them: the out-of-frame and the suspension of images. By gestures I mean the work carried by screens in arranging, reconfiguring and sustaining relations in and with the various components of the artwork. The out-of-frame gesture is highlighted in Blast Theory’s Can You See Me Now? (2003) which, performed over phones and computers, entangles physical and digital online spaces. Here, I recognise that in the networked devices of the participants, the out-of-frame is always moving into the frame; it is (be)coming the frame. I analyse the second gesture of suspending images in the artistic use of chroma screens — blue or green screens used for compositing images in post-production — where the blankness of images on-screen can be understood as an active gesture. In paying detailed attention to these modes of gesturing, I suggest that screens in postmedia assemblages emphatically perform rather than just contain or receive in postmedial screen artworks. My study of the gesturing screen offers insight into how mediality itself is maintained and experienced in the postmedia condition.
2 The ‘condition’ that this thesis attends to is identified as postmedia. The ‘post’ in postmedia gives a misleading impression that the medium is not so relevant anymore. This idea gathers more clout with the phenomenon of digital convergence. However, postmedia is not so much the irrelevance of the medial but rather the importance of medial collaborations and relationality. Explaining the cooperation of media in postmedia, Peter Weibel says, “no single medium is dominant any longer; instead, all of the different media influence and determine each other.”1 Postmedia/medium has been the subject of much debate in fields relevant to this research. In art history, conversations on the postmedium have been led by Rosalind Krauss, and her critique of
Clement Greenberg’s specificity of the physical medium in the work of art.2 In media arts, postmedia has been of concern to Peter Weibel and Lev Manovich, who account for the change in artmaking practices with the arrival of digital tools.3 In cinema studies, Francesco Casetti and Ágnes Pethő have led postmedial interrogations that acknowledge the transformation instead of the dissolution of medium in postmedia.4
I am less interested in the idea of the fusion of media to the level of their indistinguishability than the extent to which the blending and dissolution of media shift attention to the importance of their exchange and transfer. Here, the dynamicity of media is experienced as a collectivity of events and encounters that can be sustained over a period of time. But these collective medialities are not always the same. This study goes beyond the vision of
1 Peter Weibel, “The Post-Media Condition,” [2005] Mute, March 2012, https://www.metamute.org/editorial/lab/post-media-condition. 2 Rosalind Krauss, “Reinventing the Medium,” Critical Inquiry 25, no. 2 (1999): 289–305; Rosalind Krauss, Under Blue Cup (Cambridge MA: MIT Press, 2011); Clement Greenberg, “Towards a Newer Laocoön,” Partisan Review 7, no. 4 (1940): 296–310. 3 Weibel, “Post-Media”; Lev Manovich, “The Postmedia Aesthetics,” in (dis)LOCATIONS, ed. Astrid Sommer (Karlsruhe: ZKM, Centre for Art and Media/ Centre for Interactive Cinema Research, University of New South Wales, 2001), http://manovich.net/index.php/projects/post-media-aesthetics. 4 Francesco Casetti, “Back to the Motherland: The Film Theatre in the Postmedia Age,” Screen 52, no. 1 (2011): 1–12; Ágnes Pethő, ed., Film in the Post-Media Age (Newcastle upon Tyne: Cambridge Scholars Publishing, 2012).
3 postmediatic totality, as the notion of digital convergence may lead us to assume, to examine arrangements where different media elements and technical devices assemble to generate new potentialities and relationalities. I aim to develop an understanding of the multiplicity and openness of relations between medial elements and the ways they come to organise mediality. Hence, ideas about between and within media, that is, intermedia, also need to be thought through.
A productive discussion on postmedia is possible only by approaching the question of medium. While a full discussion of the term and its usage in art theory and history as well as media studies is beyond the scope of this thesis, nonetheless, a basic articulation is required in order to see how the concept of media travels within postmedia and intermediality. To state briefly, media and medium continue to be discussed regularly within the fields of communications, and art theory and film studies respectively.5 The main point of difference between the understanding of medium in these fields perhaps results from the fact that in art and film, the medium is also the method of production, in addition to the means of distribution and reception, whereas the latter two are more the focus in communications and media studies. Claus Clüver had already stated in 2007 that various definitions of medium were
5 Clement Greenberg “Avant-Garde and Kitsch,” Partisan Review 6 (Fall 1939): 34-49; Greenberg, “Newer Laocoön,”; Michael Fried, Art and Objecthood: Essays and Reviews,[1966] (Chicago: University of Chicago Press, 1998); Krauss, “Reinventing the Medium”; Krauss, Under Blue Cup. In cinema: Noël Carroll, Theorising the Moving Image (New York: Cambridge University Press, 1996); David Bordwell, On the History of Film Style (Cambridge, Mass.: Harvard University Press, 1997); Mary Ann Doane “The Indexical and the Concept of Medium Specificity,” Differences 18, no. 1 (2007): 128–52, https://doi.org/10.1215/10407391-2006-025; David N. Rodowick, The Virtual Life of Film (Cambridge, Mass.; London, England: Harvard University Press, 2007). In media and communications: Marshall McLuhan, Understanding Media: The Extensions of Man (London: Routledge & Kegan Paul, 1964); Raymond Williams, Television: Technology and Cultural Form (London: Collins, 1974). Félix Guattari’s post-(mass) media is located in a different context. His idea of post-media is the subjectivation by capitalist forces using media such as television and radio. Félix Guattari, Soft Subversions: Texts and Interviews 1977–1985, ed. Sylvere Lotringer, trans. Chet Wiener and Emily Wittman (Los Angeles: Semiotext(e), 2009), 291–306. Based on Guattari’s concept of post-media see Clemens Apprich, Josephine Berry Slater, Anthony Iles and Oliver Lerone Schultz, eds., Provocative Alloys: A Post-Media Anthology (London; Lüneburg: Mute and Post-Media Lab, 2013).
4 complicated by the use of digital technologies, leading to the development of newer media.6 Indeed, the increasing use of digital methods of production, distribution and reception in arts, film and communication has given us a wider theoretical framework for understanding the idea of medium, rather than restricting the term to a question of specificity. Jürgen E. Müller’s proposal to define medium via historical and sociocultural processes affords a greater flexibility for situating the term across varying fields, as well as accounting for its changing materiality and conditions of being.7 While different fields’ understanding of the term has allowed for the consolidation of critical practices, the medium-media discussion has also impacted on the articulation of postmedia.8 As we will see below, many arguments have crystallised around the purity and specificity of media at the expense of thinking through what dynamic possibilities are brought about by the convergence of media.
Within the field of art theory, Rosalind Krauss’ ideas on the postmedium condition were concerned with the significance and meaning of art-making tools at a time when installation art and conceptual art were gaining momentum.9 Identifying the postmedium condition to be satisfying a growing consumerist appetite for art throughout the twentieth century, she criticises the hold that conceptual art came to have on all aspects of art-making, including installation art from Duchamp onwards, accusing the latter of introducing art as idea, which ultimately lead to the demise of the importance
6 Claus Clüver, “Intermedia and Interart Studies,” in Changing Borders: Contemporary Positions in Intermediality, ed. Jens Arvidson, Mikael Askander, Jørgen Bruhn, Heidrun Führer (Lund: Intermedia Studies Press, 2007), 31. 7 Jürgen E Müller, “Intermediality Revisited: Some Reflections about Basic Principles of this Axe de pertinence,” in Media Borders, Multimodality and Intermediality, ed. Lars Elleström (New York: Palgrave Macmillan, 2010), 238. 8 Sven Lütticken, “Media to Mythology: Art in the Age of Convergence,” New Left Review 6 (2000): 139. 9 Krauss, “Reinventing the Medium”; Rosalind Krauss ‘A Voyage on the North Sea’: Art in the Age of the Post- Medium Condition (London: Thames & Hudson, 1999).
5 of the medium.10 Her analysis of postmedia art practices is also a reaction to Greenberg’s modernist definition of the medium, in which she saw the artwork reduced to nothing more than its physical properties.11 However, Krauss’s analysis did not account for the effect of technological changes on various media and art practices throughout the 1990s. This period saw a major shift towards digital modes of making and delivering art and can be distinguished from the modernist work of Greenberg’s analysis that centred on the period from the 1940s through the 1960s.
While acknowledging the difference between the term media used in communication and storage and medium as used in the arts, Krauss unpacks the term media as technology for communication and storage, identifying them as the contributing factors to the postmedium condition.12 However, Krauss’ idea of “technical support” is unclear; it takes into account the means to make art ranging from video to slides and video tape, but by using these unconventional materials for their time, Krauss suggests that artists “reinvented” the medium.13 This raises questions about whether technical support also included artists’ processes and practices. Moreover, Krauss ignores the changes occurring in the reception of art alongside those in artmaking. Instead, she lamented the loss of medium specificity and hailed artists such as William Kentridge, Bruce Nauman and James Coleman as “knights” of the medium who were able to reinvent by working within medial conventions.14 While sharing similar ideas on the end to medium specificity, Nicolas Bourriaud’s thinking diametrically opposed Krauss’. He deemed
10 Krauss, Under Blue Cup, 20. 11 Ibid., 7. While Greenberg regarded the purity of the medium as helping the artist focus on its materiality and properties, Krauss accused Greenberg’s medium specificity of being “tied to a physical substance.” She instead proposed a much more complex relationship to the work under the term ‘post-medium’. However, this was tied up in its own medium specificity. Krauss, Under Blue Cup, 7. 12 Ibid., 35–38. 13 Ibid., 18, 19. 14 Ibid., 102.
6 “radicant art” to be a signal of the end of medium specificity and bearer of a new world.15 For him, medium specificity was characterised by a lack of movement, unlike the ivy plant of the twenty first century — postmedia — which he saw growing in all directions, embracing movement and non- fixedness.16 The heterogeneity of material and practices were, for Bourriaud, part of the demise of medium specificity that had to be embraced.
The widespread uptake of digital technologies in the 1990s provided different possibilities for the creation and display of images. In the light of such changes, Peter Weibel assessed the postmedia condition as a drawing of equivalence between all media, directing attention to the intermixing of the older media of photography and film with video and computers, and also the non-technological traditional media of painting and sculpture.17 Here no particular media dominates and each refers to the other — an argument that first surfaces in Marshall McLuhan’s famous proclamation that “the content of any medium is always another medium.”18 As a simulator of all media, Weibel proclaims, computation is a major contributor to the postmedia condition.19 For Lev Manovich, digital computation also modifies the composition and deployment of information and leads to a new kind of aesthetics. In describing postmedia aesthetics, Manovich notes the inadequacy of older ways of organising and conceiving media in a post-digital and post-internet culture.20 He suggests that artistic media need to be rethought within the context of networked and computational culture using
15 Nicolas Bourriaud, The Radicant, trans. James Gussen and Lili Porten (Berlin: Sternberg, 2009), 53, 54. 16 Ibid, 51. 17 Weibel, “Post-Media Condition.” 18 McLuhan, Understanding Media, 3. 19 Weibel, “Post-Media Condition.” 20 Manovich, “Postmedia Aesthetics.”
7
concepts such as random access, information behaviour and information design.21
It is important to note here that Weibel and Manovich do not argue for an end to a medium per se, but signal the postmedia condition to be a movement, simulation and equivalence of media due to the use of digital computational techniques. As Weibel clarifies, “it does not render the idiosyncratic worlds in the world of devices or the intrinsic properties of the media world superfluous. On the contrary, the specificity and idiosyncratic worlds of the media are becoming increasingly differentiated.”22 Computational processes have made similar processes and operations, for instance editing techniques, available to traditional media forms such as film and television. But we also need to understand computation as not simply a set of technologies but as a mode of organising technologies and media. I take the understanding put forth by M. Beatrice Fazi and Matthew Fuller of computation as an ordering of objects, phenomena and media through discontinuous and discrete points via number and calculation.23 Computation, understood as a broader category by Fazi and Fuller, influences cultural and social modes of knowledge and meaning organisation and at the same time is “affected” by them.24 Digital arts are then established by the computational in “its technical, social, and cultural manifestations.”25 In this way, digitalisation has to be understood as an overall change in practices of media-making where the postmedial accounts for new and modified practices that occur in a particular context.
21 Ibid. 22 Weibel, “Post-Media Condition.” 23 M. Beatrice Fazi and Matthew Fuller, “Computational Aesthetics,” Companion to Digital Art, ed. Christiane Paul (Hoboken, NJ: John Wiley & Sons, 2016), 283. 24 Ibid., 282. 25 Ibid., 281.
8
However, mere identification of a postmedia condition is insufficient since this simply describes a state of affairs. This thesis takes Domenico Quaranta’s suggestion that postmedia is a point of departure instead of a place of arrival.26 He argues that postmedia collapses the boundaries not only between various media in art, but also between art-film and architecture-design “to arrive at a new, open vision of the visual realm.”27 Further, Quaranta argues that even though art, media and popular culture use the same means of distribution and production, art in postmedia is socially aware of its capacities to question, complicate and sensitise ongoing sociotechnical conditions and transformations.28 The postmedial, then, can be used to access and examine contemporary art to enable us to think about how particular art practices that deploy contemporary media transform existing connections and develop new ones. This supports my research undertaking to inquire into how postmedia also aesthetically conditions our experiences and relations.
This raises an important question about media in postmediality: while multiple media assemble together, how are they able to be ‘in media’ together; that is, how does postmedial circulation and exchange take place? I propose screens to be significant elements in the production and exchange of such medial circulation. Paying close attention to this common component between and across various postmedia aggregations, I posit screens as being crucial to the postmedia condition. It is hard not to acknowledge the ubiquity of screens, increasing especially over the last three decades. The accessibility of different kinds of media objects and sociocultural activities that screens facilitate might seem somewhat banal. However, this very characteristic demands analysis of the operations of screens that lifts them out of just being understood as display objects or content containers. While this has been
26 Domenico Quaranta, Beyond New Media Art (Brescia: Link Editions, 2013), 202. 27 Ibid., 205. 28 Ibid.
9
discussed in various fields, as I will elaborate further below, I take a different approach, locating them as major players and assemblers of postmedia. This thesis analyses specific instances in which screens undertake these performative enactments of the postmedial. At the same time, I also understand postmediality as something more than an aggregate of technical components; it has become a condition which itself influences screens and other media elements. This does not mean that an upsurge in screens is due to the postmedia condition, but rather that changes in capacities and operations of screens co-emerge with and in the postmedia condition. In summary, then, the postmedia condition in this thesis is examined along three axes: the ongoing medial transformations taking place within this condition; the agencies assumed and enacted by screens as technical actors within the condition; and the new kinds of artistic experiments that emerge out of these entanglements and relations. As such, this thesis contributes to media, digital and postdigital studies of the postmedia condition by arguing that the postmedial is more than a descriptor. It is an active set of forces that cultivates an intermediality of and by screens.
As the postmedial condition ascends in tandem with digital, technical and cultural transformations, I have restricted the focus of my analysis to artworks since the 1990s. This period also marks the increasing role of screens in facilitating engagement with media objects such as smartphones and cameras as well as the enabling of medial interconnections, an instance of which can be seen in using phone screens to take photographs and watch films. From the 1990s onwards, screens have been extensively used in image making, displaying, circulation and accessibility both within mainstream image-making production and within media arts. I engage with the works of Shilpa Gupta, Ulrike Gabriel, Natalie Bookchin, Blast Theory, Sandra Mujinga and Sondra Perry, among others in this thesis, as examples of the
10 ubiquitous deployment of screens in the arts from this time on. These works use techniques that are not restricted to one medium or artform, providing an opportunity to analyse the workings of screens as postmedial. For example, the green chroma screens used by Sondra Perry are also abundantly used in mainstream cinema and video production. Chroma screens are a compositing technique where visual elements from different sources are placed in a designated area indicated by the blue or green screen. But among artists who use chroma screens in their works, Perry is relevant to this study because the screens here do more than simply rearrange media, technologies and devices. They also interrogate a wider racial politics of colour, which connects how bodies come to be seen in and by media.
Understanding postmedia primarily as the convergence of different media addresses the transferability and exchange of content across different media platforms, but it does not take into account the reconfiguring of the relationships across and between different technical devices and platforms.29 These devices and platforms come together at a particular point yet are not just connected to each other. They can remain disconnected or in nonrelation with each other too, making them simultaneously available for other alliances. This way of conceiving the postmedial sets up the possibility of thinking it via the concept of the assemblage. Assemblage or agencement was first discussed by Gilles Deleuze and Félix Guattari at various places throughout A Thousand Plateaus.30 Deleuze additionally defines it as:
29 Henry Jenkins, for example, acknowledges the complexity in the interaction of old and new media, but his focus is on how this impacts the participation and behaviour of consumers. While I too argue that convergence is more than a technological change, I do not place the same importance as Jenkins does on consumer behaviour. Instead, I think of the co-constitution of material and discursive practices. I will elaborate more on this later. See Henry Jenkins, Convergence Culture: Where Old and New Media Collide (New York; London: New York University Press, 2006). 30 Gilles Deleuze and Félix Guattari, A Thousand Plateaus: Capitalism and Schizophrenia, trans. Brian Massumi (London: Bloomsbury, 2017), 7, 24, 356–57. Agencement was the French word originally used by Deleuze and Guattari. It has been poorly translated into English as assemblage. Agencement means to construct or arrange as opposed to the English meaning of assemblage which is to join or create a union of two or more things.
11
a multiplicity which comports many heterogenous terms, and which establishes liaisons, relationships between them, across ages, sexes, reigns — different natures. Also, the sole unity of agencement is that of co-functioning: it is a symbiosis, a ‘sympathy.’ What is important is never filiations but alliances and alignments; these are not heredities, descendants, but contagions, epidemics, the wind…31
Assemblage, then, has to be understood as relations between elements that are in the process of (already) assembling. In relations with each other, these elements already draw upon particular capacities. To be clear, such capacities are not their only attributes, but are brought into play within the particular relation and process. This thesis takes up Brian Massumi’s assertion that an assemblage does not require an analysis of what it signifies, but rather attends to what it functions with, what its multiplicities might be and how it transmits intensities.32 In postmedia art assemblages, media objects, technologies and human bodies are brought together. Each of these perform actions, practices, and construe mediality that may be habitual or novel but nonetheless are collectively the result of how their capacities affect each other.
I consider a postmedia assemblage as an arrangement that takes into account the various operations and gestures of screens in image-making, display and dissemination practices. The unstable, ever-changing relations in postmedia assemblages help us to understand the numerous enactments in which screens become. I uncover various expansions and relations of image, viewer/participant and interplay that screens perform differently in the postmedial than in media assemblages such as cinema. I posit that screens can
Moreover, assemblage suggests spatial organisation while agencement emphasises a dynamic set-up, which may be spatial. This has been clarified by John Phillips, “Agencement/Assemblage,” Theory, Culture & Society 23, no. 2–3 (2006): 108–109; Manuel DeLanda, Assemblage Theory (Edinburgh: Edinburgh University Press, 2016); Thomas Nail, “What is an Assemblage?” SubStance 46, no. 1 (2017): 21–37. 31 Gilles Deleuze and Claire Parnet, Dialogues, 66. 32 Brian Massumi, “Translator’s Foreword: Pleasures of Philosophy,” in Deleuze and Guattari, A Thousand Plateaus, 2.
12
guide us in thinking about various emergent operations and relations of image-making and display. I argue that screens in postmedia assemblages are a key element in actively reconfiguring and redistributing the relations of the assemblage.
Screens have been widely conceived as a device for the display of images or a prosthesis for the human gaze. They present themselves in various forms and modalities but while on display everywhere, are often under-investigated or taken for granted. As a common constituent of art, cinema and media, screens have been studied in all these fields, although most studies focus on displayed content rather than interrogate what screens do. A range of anthologies, such as Screen Space Reconfigured, Screen Genealogies: From Optical Device to Environmental Medium, and The Screen Media Reader: Culture, Theory, Practice, provide evidence that a diverse and wide-ranging debate on their capacities and modalities is present in scholarship.33 But it also indicates the challenges of thinking about screens from a single point of view. Instead, along with Vivian Sobchack we might go so far as to see screens and our engagement with them as a kind of diffuse system, in which the idea of a “screen-sphere” operates to organise our contemporary modes of knowing and being.34
Screen studies has a long scholarly tradition of analysing the gaze, spectacle, spectator and affect in an exploration of what occurs both on- and off- screen.35 Media arts theory also has an established tradition of analysing
33 Susanne Ø. Sæther and Synne T. Bull, eds., Screen Space Reconfigured (Amsterdam: Amsterdam University Press, 2020), https://doi.org//10.5117/9789089649928; Craig Buckley, Rüdiger Campe, and Francesco Casetti, eds. Screen Genealogies: From Optical Device to Environmental Medium, (Amsterdam: Amsterdam University Press, 2019), https://doi.org//10.5117/9789463729000; Stephen Monteiro, ed., Screen Media Reader: Culture, Theory, Practice (London: Bloomsbury, 2017); Scott McQuire, Meredith Martin, and Sabine Niederer, eds., Urban Screens Reader (Amsterdam: Institute of Network Cultures, 2009). 34 Vivian Sobchak, “Comprehending Screens: A Mediation in Media Res,” Rivista di estetica 55 (2014): 88. 35 Jean-Louis Baudry, “Ideological Effects of the Basic Cinematographic Apparatus,” Film Quarterly 28, no. 2 (1974): 39–47; Teresa de Lauretis and Stephen Heath, eds., The Cinematic Apparatus (New York: St Martin’s Press, 1980); Christian Metz, Psychoanalysis and Cinema: The Imaginary Signifier (London: Macmillan, 1982); Steven Shaviro, Post-Cinematic Affect (Zero Books, 2010).
13
screens as part of installations, expanded cinema and projection-based work in texts by Kate Mondloch, Susan Lord and Janine Marchessault, Jeffrey Shaw and Peter Weibel, Martin Reiser and Andrea Zapp, and Gene Youngblood.36 These enquiries have been oriented towards screen projections and film and video works in gallery spaces. Although helpful in understanding screen practices, they do not directly confront pressing questions about the mediality of screens or what operations and agency screens might enact in these works. On the other hand, media studies theorists Errki Huhtamo, Jay David Bolter and Richard Grusin, and Lev Manovich have treated screens as information surfaces, mediators and dynamic interfaces.37 These views have helped me in understanding the historical conditions in which screens have appeared in diverse medial formations. Additionally, I have approached screens as part of cinematic architecture through the work of Giuliana Bruno, Laura Mulvey, Christian Metz, Tom Gunning and Vachel Lindsay.38
Some of these thinkers, particularly, Gunning and Metz, have inquired into the relationship of on-screen images to the whole cinematic apparatus in order to think about the cinematic experience sociotechnically. Cross-
36 Gene Youngblood, Expanded Cinema (New York: New York, Dutton, 1970); Martin Reiser and Andrea Zapp, eds., New Screen Media: Cinema/art/narrative (London: British Film Institute, 2002); Jeffrey Shaw and Peter Weibel, eds., Future Cinema: The Cinematic Imaginary after Film (Cambridge, London: MIT Press, 2003); Susan Lord and Janine Marchessault, eds., Fluid Screens, Expanded Cinema (Toronto: University of Toronto Press, 2007); Kate Mondloch Screens: Viewing Media Installation Art (Minneapolis: University of Minnesota Press, 2010); Tamara Trodd, ed., Screen/Space. The Projected Image in Contemporary Art (Manchester: Manchester University Press, 2011). 37 Erkki Huhtamo, “Screen Tests: Why Do We Need an Archaeology of the Screen?” Cinema Journal 51, no. 2 (2012): 144–48, https://doi.org//10.1353/cj.2012.0011; Jay David Bolter and Richard Grusin, Remediation: Understanding New Media (Cambridge, Mass.: MIT Press, 2000); Lev Manovich “Towards an Archaeology of the Computer Screen,” in Cain, Abel or Cable? The Screen Arts in the Digital Age, ed. Thomas Elsaesser and Kay Hoffmann (Amsterdam: Amsterdam University Press, 1998), 27–43. 38 Laura Mulvey, “Visual Pleasure and Narrative Cinema,” Screen 16, no. 3 (1975): 6–18; Metz, Psychoanalysis and Cinema; Tom Gunning ,“The Cinema of Attraction: Early Film, Its Spectator and the Avant-Garde,” Wide Angle 8, no. 3-4 (1985): 63-70; Vachel Lindsay, The Art of the Moving Picture (New York: Macmillan, 1915); Giuliana Bruno, Surface: Matters of Aesthetics, Materiality, and Media (Chicago: University of Chicago Press, 2014).
14
disciplinary analysis has led to recognising ways in which screens operate variously as windows (Anne Friedburg), light (Christiane Paul and Sean Cubitt), interface (Brenda Laurel), and mobile (Wanda Strauven, Nanna Verhoeff).39 This rich literature has unveiled new perspectives on the history and contemporary operations of screens. I will take up some of this literature in chapter one to illustrate how specific qualities of screens, which have developed in and through certain practices, allow us to better understand their contribution to and enactment as postmedial.
In thinking through these various situations, disciplines and systems in which screens find themselves, this thesis steps back to ask a more fundamental question: what are postmedial screens? Such a broad question may seem ambitious given the apparent need to account for the overwhelming presence of screens in our urban surrounds. And, as we will see, there is no simple answer to this question, but rather the requirement that we attend to the relations enacted by screens in specific set-ups. Contending that screens do more than simply display, I suggest that their work of configuring and assembling relations of image, viewer/participant, and producer/maker are especially foregrounded in postmedia art works. This rearrangement of relations catalysed by contemporary screens is most evident in the process of making and displaying screen-based and directed visual arts that are cross- or intermedial. I acknowledge that it is not possible for this research to cover all domains of postmedial art practice, but what I seek to do is to lay the
39 For windows: Anne Friedberg, The Virtual Window. From Alberti to Microsoft (Cambridge: MIT Press, 2006). For light: Christian Paul “Mediations of Light: Screens as Information Surfaces,” in Digital Light, ed. Sean Cubitt, Daniel Palmer, and Nathaniel Tkacz (London: Open Humanities Press, 2015), 179-192; Sean Cubitt The Practice of Light: A Genealogy of Visual Technologies from Prints to Pixels (Cambridge, MA: MIT Press, 2014). For interface: Brenda Laurel, Computer as Theatre (Reading, Mass.: Addison-Wesley Publications, 1991). Location-based movements: Wanda Strauven, “The Archaeology of the Touch Screen,” Maske und Kothurn 58, no. 4 (2012): 69-79, Nanna Verhoeff, Mobile Screens. The Visual Regime of Navigation (Amsterdam: Amsterdam University Press, 2012), https://library.oapen.org/handle/20.500.12657/34549.
15
conditions for new discussions around the agency of screens in postmedial art.40
Screens have always functioned dually to both hide and display objects and images. To screen means to shield, protect from the dangers of, for example, projectiles, fire (fireplace screens, pole screens) or the sun (sunscreen). Here screens are positioned in the middle to hide or protect something or someone. In cinema, television and computation, screening refers to showing images, graphics and/or text. In medical terms, screening helps in detecting changes in the body that may indicate diseases even before symptoms start to manifest. The screen-printing technique uses a porous screen — a fine framed mesh with a negative stencil — to print on to a variety of materials, such as paper, textile and wood. From the above instances, we can conclude that as an object or function, screens can protect, hide or reveal. However, in many of these cases these dual functions occur simultaneously. John Durham Peters counts several cases where this duality occurs, both naturally and artificially, as in the absorption of sunlight and reflection in the earth’s atmosphere, or creatures such as squids that release/reveal ink as a screen to hide and escape from predators, and also phenomena such as online comments, which provide a space to voice opinions without having to reveal one’s identity.41 Such instances may help us think further about media practices of projection-protection and hiding-revealing. In cinema, screens show and hide. With the projection of light, screens reveal moving images and at the same time provide a barrier to and between the spectator from the
40 The use of screens in virtual and mixed media reality is out of the scope of this research. These practices have a different relationship to screens, one that requires articulation on a different conceptual base. 41 John Durham Peters, “The Charge of a Light Barricade: Optics and Ballistics in the Ambiguous Being of Screens,” in Screen Genealogies: From Optical Device to Environmental Medium, eds. Craig Buckley, Rüdiger Campe, and Francesco Casetti (Amsterdam: Amsterdam University Press, 2019), https://doi.org//10.5117/9789463729000; John Durham Peters, “Projection and Protection: On the Deep Optical and Ballistical Intersections of Screens,” part of “Genealogies of the Excessive Screen,” 16 February 2017, Yale University Art Gallery lecture series.
16
on-screen world.42 Perhaps screens can be thought to behave like membranes that restrict some objects and let others pass, such as flyscreens on windows, both protecting against insects and dust while allowing wind and light to permeate.43
Another kind of duality is provided by W.J.T. Mitchell, who argues that we see on and through screens — they can project or display, as seen in cinema and television, respectively.44 Sometimes screens show images on and through simultaneously, as seen in phantasmagoria where images are seen on the screen via the projection of light and through smoke and vapour.45 This on and through duality of screens can be extended to contemporary screens in both image-making and display. Images may be recorded on cameras through an LCD screen, created on a computer or digitised from a physical medium such as a sketch or a painting. These images may be edited, adjusted, viewed and distributed through the screen of a computer, mobile phone or a projector. Mitchell had already argued that ‘on’ and through’ could be expanded to ‘in’, ‘behind’, ‘between’ and ‘from’, highlighting the multidirectional capacities of screens.46 Stephen Monteiro, when assessing contemporary screens, encourages a looking ‘at’ and ‘around’, rather than ‘through’, in order to access a genealogy of representation, technical development and social spaces that have led to the screens of today.47 This task has in many ways already been undertaken by Huhtamo whose concept of screenology traces the historical manifestations of screens through media
42 Stanley Cavell, The World Viewed: The Reflections on the Ontology of Cinema (New York: Viking Press, 1971), 24. 43 The property of screens as membranes in the sense of textural materiality has been discussed by Giluiana Bruno in Surface, 76-105. 44 W.J.T. Mitchell, “Screening Nature (and the Nature of the Screen),” New Review of Film and Television Studies 13, no. 3 (2015): 234, https://doi.org//10.1080/17400309.2015.1058141. 45 Ibid., 233. 46 Ibid., 234. 47 Stephen Monteiro, “Introduction,” in The Screen Media Reader: Culture, Theory, Practice, ed. Stephen Monteiro (London: Bloomsbury, 2017), 2.
17
archaeology.48 Indeed, in the postmedial condition in which the image-making and display capacities of screens have multiplied, ‘on’ and ‘through’ have absolutely expanded to numerous other orientations. Via such multiplications and changes in functionality and modality, screens do not simply display but also transform the relationship of users with media.
New media employs other functions of screens where they are not, or not only optical devices, such as touch screens. Craig Buckley, Rüdiger Campe, and Francesco Casetti assert that the current profusion of screens cannot be understood via their optical histories but through other “spatial and environmental interventions that the screens have always performed.”49 Here, they do not just refer to early screens, which were façades used to temporarily divide space, hide or partially reveal something or someone, and protect from fire or wind.50 In their location of contemporary screens as a “range of surfaces that become screens”, Buckley, Campe and Casetti imply that screens are not pre-existent but become.51 This thesis aligns with, and in some aspects, expands on this idea. Buckley, Campe and Casetti trace and gather instances of a historical genealogy in which surfaces act like screens, quite unlike the media archaeology undertaken by Huhtamo. Focused less on a genealogical undertaking, I am instead interested in the dynamic eventfulness of postmedia where the becomings of screens are enactments that (re)arrange and facilitate assemblage relations.
48 Erkki Huhtamo, “Screen Tests”; Erkki Huhtamo, “Elements of Screenology: Toward an Archaeology of the Screen,” ICONICS 7 (2004): 31–82; Erkki Huhtamo, “Screenology; or, Media Archaeology of the Screen,” in The Screen Media Reader: Culture, Theory, Practice, ed. Stephen Monteiro (New York: Bloomsbury, 2016), 77 – 111. 49 Craig Buckley, Rüdiger Campe and Francesco Casetti, “Introduction,” in Screen Genealogies: From Optical Device to Environmental Medium, eds. Craig Buckley, Rüdiger Campe, and Francesco Casetti (Amsterdam: Amsterdam University Press, 2019), 7, 8, https://doi.org//10.5117/9789463729000. 50 Huhtamo discusses this history of screens, which I will cover in my discussion of the material-discursive practices of screens in chapter one. Huhtamo, “Elements of Screenology,” 82–84. 51 Buckley, Campe and Casetti, “Introduction,” 13.
18
Does such a prevalence and diversity of screens mean that they should be defined as platforms, formats, and/or media? Charles Acland in fact suggests that screens are an “in-between manifestation” of medium, format and platform. They are both how we see and are the connection between all the other media.52 With their persistent presence across different media formations, they have come to emphasize, interconnect and rearticulate the relations of the various elements at work in image-making and display. Following from Massumi’s provocation that “thinking must begin... immediately in the middle,” I focus on this middle domain or inbetweenness to better understand the postmedial screen.53 The inbetweenness of screens offers a particular understanding of postmediality that attends to and cultivates the dynamic potentiality of media, relations and practices.
Up till now, I have noted the multiple capacities of screens, their potentialities, and their activities and connections across different media. Circling back to the question I raised earlier about what constitutes the postmediality of screens, I want to propose that we look for the answer in their intermedial operations. The mixing of media in postmedia brings together different practices and components that activates new connections. The ease of movement of screens across and along different media makes them intermedially active. To be sure, such intermediality is not only a postmedial operation. But, in the postmedia condition this has accelerated and encompassed increasingly different media. Moreover, the co- determination of screens and postmediality, which is a particular proposition of this thesis, recognises the inbetween as their sphere of operation. The
52 Charles Acland, “The Crack in the Electric Window,” Cinema Journal 51, no. 2 (2012): 168. 53 Brian Massumi, Semblance and Event: Activist Philosophy and the Occurrent Arts (Cambridge, MA: MIT Press, 2011), 1.
19
inbetweenness of the intermediality of screens enables specific enactments of and for the postmedia assemblage.
Similar to postmedia, intermediality was also subjected to the exigencies of the debate on medium specificity in its earlier manifestations. The term intermedia was used by Dick Higgins in 1965 to address art that falls “between media,” exemplified by John Heartfield, Allan Kaprow and Robert Rauschenberg among others.54 Later, he expanded on intermedia artworks as ones in which various media “fused conceptually,” and could already be seen in both Futurism and Dadaism.55 This positioned intermedia as an approach towards artmaking, a practice in which the traditional medium became difficult to locate. In Gene Youngblood’s idealistic vision, ‘man’ became part of an “intermedia network,” consisting of nature and media such as cinema and television in which they would be “suffused in metamorphosis.”56 Referring to artists such as Carolee Schneemann and Aldo Tambellini as “intermedia artists,” Youngblood indicated how multiple media created new synesthetic experiences for the audience.57 The intermedia arts as an ‘expanded’ vision must disregard medium specificity and look instead towards interrelation and the mixing of mediatic elements. In the mixing of media, screens became prominent as spaces that gave rise to new kinds of images, as well as providing access to works and new means of distribution, for example, through video monitors. Importantly, it is in their capacities as “sense- extensions,” taking seeing and understanding beyond the biological capabilities of humans, that television and computer screens were seen by artists and theorists as important technologies during this period, according to Youngblood.58
54 Dick Higgins, with Hannah Higgins, “Intermedia,” [1965, 1981] Leonardo 34, no. 1(2001): 49. 55 Ibid., 52-3. 56 Gene Youngblood, Expanded Cinema (New York: Dutton, 1970), 54, 347. 57 Ibid., 311. 58 Ibid., 260.
20
Intermediality provided more than simply an expanded role for screens. Intermedial live performances and interactive art were from the outset facilitated by screens, allowing the viewing, distribution and reception of media in innovative ways.59 This, however, shifted from the 1990s onwards. The widespread consumer level and professional media uptake of digital technologies from the 1990s provided ease of access and better image quality in the creation and display of images than was available in the more limited context of art-making of the 1960s. Intermedia extended outside the field of art, to cinema, media and performance and also became the focus of scholarship. Scholars, artists and media practitioners became attentive to “configurations which have to do with a crossing of borders between media,” and to how materialities between media transformed media forms.60 Intermediality of the 1990s is distinguished from the intermediality of the 1960s in terms of its scope and context, as argued by Irina Rajewsky.61 But at the core of this distinction is an argument about the operations of the digital that suggests the question of medium has truly surpassed debate about physical materialities or specificities. Lars Elleström, for example, defines
59 Large and multiple screens became the feature of several other intermedia works of that time, such as the audio-visual performance HPSCHD (1969) by John Cage and Lejaren Hiller; the Space Theatre (1960s) of Milton Cohen; Black Gate (1969) by Aldo Tambellini; Monument (1967) by Ture Sjolander and Lars Weck; the diapolyceran screen in the Czech Pavilion at Expo ‘67; and the Vortex Concerts (1957–1960) by Henry Jacobs and Jordon Belson, to name a few. These works have been extensively studied as part of the new media or video media experience. See Jeffrey Shaw, “Movies After Film – The Digitally Expanded Cinema,” in The New Screen Media: Cinema/Art/Narrative, ed. Martin Rieser and Andrea Zapp (London: British Film Institute, 2002), 269; Peter Weibel, “Narrated Theory: Multiple Projection and Multiple Narration (Past and Future),” in The New Screen Media: Cinema/Art/Narrative, ed. Martin Rieser and Andrea Zapp (London: British Film Institute, 2002), 42–53; Lord and Marchessault, Fluid Screens; Mondloch, Screens; Tanya Leighton, ed. Art and the Moving Image: A Critical Reader (London, Tate Publishing, 2008). 60 Irina Rajewsky, “Intermediality, Intertextuality, and Remediation,” Intermédialités 6 (2005), 46. Ágnes Pethő uses the phrase ‘transgression of boundaries’ to define the coming together of different media in intermediality. See Ágnes Pethő, “The Garden of Intermedial Delights,” Screen 55, no. 4. (2014): 474-489. Balme uses ‘transposition of content between media’ as a way to understand intermedia. See Christopher Balme, “Intermediality: Rethinking the Relationship between Theatre and Media,” 2004, 7. Also Joachim Paech, “Intermediate Images,” Film and Media Studies 9 (2014): 31–49; Yvonne Spielmann, “Intermedia in Electronic Images,” Leonardo 34, no. 1 (2001): 55–61. 61 Rajewsky, “Intermediality, Intertextuality, and Remediation,” 51. Also see Eric Vos, “The Eternal Network: Mail Art, Intermedia Semiotics, Interarts Studies,” in Interart Poetics: Essays on the Interrelations of the Arts and Media, ed. Ulla-Britta Lagerroth, Hans Lund and Erik Hedling (Amsterdam: Rodopi, 1997), 325.
21
media as a complex set of modalities that are material, sensorial, spatio- temporal and semiotic. These modalities provide numerous intersections that lead to intermediality as a ‘state’.62
While the postmedia condition may fuel the study of intermediality, at the same time it complicates it if the focus is primarily on the qualities of media moving intermedially. A medium is in a constant state of flux in the postmedia condition, where it may be losing its original qualities and simultaneously gaining new ones. A photograph on a computer might not have the same materiality as that of a traditional photograph, but it can be manipulated through quick, easy and smooth operations.63 It can also be given a different form and agency by creating a different kind of image such as a GIF (Graphics Interchange Format). The question here is not whether GIFs have photographic properties, nor how these relate to the medium of photography even while they are in a state of intermediality. Instead, understanding intermediality now needs to take account of the concept of the medium itself changing in the postmedia condition. We should ask, then, not just how photographs exist in different forms or how the medium of photography lends itself to moving images. Nor should we simply explore techniques of photography such as zoom-in or depth of field employed by a format that is no longer lens-based. We should concern ourselves, instead, with what new capacities are gathered by the image in GIF form and the relations that screens sustain in editing and circulating an image as a GIF.
62 Lars Elleström, “The Modalities of Media: A Model for Understanding Intermedial Relations,” in Media Borders, Multimodality and Intermediality, ed. Lars Elleström (London: Palgrave Macmillan, 2010), 11, 12. 63 I am of course aware that photographs can be physically manipulated in similar ways to its digital counterpart. But manipulation on computer is faster, requires less time and most importantly, one can easily make copies of a photograph on a computer for faster circulation.
22
Intermediality has also been approached as the search for traces of media materialities. This is proposed by both Jürgen E. Müller and Joachim Paech, who discuss ways in which the interactions of media materialities can be located.64 Elleström, however, also acknowledges that intermedia fuse elements in such a way that it may no longer be relevant to determine which original media have been combined. He lists various forms of intermediality, such as finding traces of one medium in another, references of one medium in another, and the transformation of one medium into another where the original medium may or may not be identified.65 However, trying to trace the media at work in the intermedial process may not suffice for thinking through screens in the postmedia condition. Instead of approaching intermediality via the concept of media, Yvonne Spielmann suggests understanding intermedia as a concept that can be used to address the metamorphosis between media, transformation of content due to the metamorphosis, and as a tool for aesthetic change.66 Similarly, Rajewsky has argued that intermediality can be understood as a “fundamental condition or category,” or “a critical category for concrete analysis of specific conditions.”67 Both Spielmann and Rajewsky seem to suggest that intermediality lies in the lack of fixity of media. This echoes Higgins, who cautiously suggested that intermedia was not a movement or a special quality of the artwork.68 Hence, intermediality is situated in a practice and process of working with mediatic elements and cannot be abstracted to a separate domain of study. It can be understood as a posture engaged while making, rather than a category of media.69 As Jill Bennett argues, intermediality is “not just an issue of medium,” but becomes a
64 Jürgen E. Müller, “Intermediality and Media Historiography in the Digital Era,” Film and Media Studies 2 (2010): 15–38. On page 20 he specifically states that the social and historic aspects of a medium should not be neglected. 65 Elleström, “The Modalities of Media,” 34, 35. 66 Spielmann, “Intermedia in Electronic Images,” 60, 61. 67 Rajewsky, “Intermediality, Intertextuality, and Remediation,” 47–48. 68 Higgins, “Intermedia,” 53. 69 Elleström, “The Modalities of Media,” 39.
23
“transdisciplinary sphere of operation,” which sits at the juncture of different practices and technologies.70 This juncture can be just that “inbetweenness” described by Ágnes Pethő, where intermedia becomes, in the Deleuze and Guattari sense, as processes of ongoing unfolding that destabilise categories.71 Intermediality for this thesis, then, is a reservoir of and for medial potential.
Understanding intermediality through the ideas of inbetweenness, becoming and potentiality is helpful for this research, since it allows me to focus on aspects such as the dual functions of projection-protection and on-through for screens, which I discussed earlier. Postmediality catalyses the intermedial travels of screens, fuelling their inbetweenness and potentiality by exploiting these kinds of dualities. Moreover, screens are not only intermedial but also intramedial, that is, they are not only between media but also within a medium as well. Since the inter and intramediality often persist together, this simultaneity and its generativity are considered in the many works discussed in this thesis. Precisely how the inter and intramediality of screens are endemic operations of postmedia can be teased out by looking briefly at the work of prominent postmedia artist, John Gerrard. At different stages in the making of his work, we will see the different ways in which screens are used across multiple media. Focussing on the inter and intramedial operations of screens, the usual practices associated with a particular kind of screenic relation to a medium are not followed; for example, Gerrard’s use of computer gaming engines confounds the usual participatory role ascribed to the audience that plays out via accessing a screen. In following the ways in which artists such as Gerrard harnesses the inbetweeneness of postmedial screens, we can also draw attention to the openness of screens in the
70 Jill Bennett, “Aesthetics of Intermediality,” Art History 30, no. 3 (2007): 434. 71 Petho, “The Garden of Intermedial Delights,” 474-475; Deleuze and Guattari, A Thousand Plateaus, 341, 342.
24
postmedia assemblage as their relations are made and remade towards different performative outcomes.
Figure has been removed due to copyright restrictions.
Figure 1: John Gerrard, Solar Reserve (Tonopah, Nevada), 2014, realtime computer- generated simulation. Installation view, Lincoln Centre, New York, 2014. © Photographer: Iñaki Vinaixa
Gerrard creates works that are real-time computer simulations of physical places and events. He uses various media technologies at each step of the production of his works, starting with digital photographs that provide the basis for three-dimensional computer graphic renderings of a particular place. These are first created using modelling and imaging software such as Maya and Photoshop, and then taken into the gaming engine Unigine. Gerrard’s artmaking encounters screens at various steps as he traverses these media and platforms. His Solar Reserve (Tonopah, Nevada) (2014) simulates a solar power facility in Tonopah in Nevada, USA. Here, more than ten thousand sun- tracking mirrors, called heliostats, focus the heat from the sun onto a tower containing molten salt, generating electricity (fig.1). Gerrard’s slow-moving simulation shows the solar reserve from various angles over its 60-minute duration. The work tracks the real-time movement of the sun, mirroring the real-time sky of the original site in Nevada. Gerrard created digital models
25
from thousands of photographs of the facility. With the change in medium from photography to virtual modelling comes a change in the screen and its capacity. From the LCD screen of the camera, the photographs are transferred onto the screen of a computer for the virtual scene to be produced, where the heliostats are created as digital objects using some photographic detail for texture-mapping. Gerrard refers to such objects as “image objects,” due to their sharp resolution and detailing in their realistic simulation (fig.2).72 This lends them a sculptural quality — they are created by hand gestures but are mediated by a screen. These “sculptural photographs,” made on-screen, acquire a temporal character through the slow movement of the camera and the movement of the sun simulated in the work.73 This cross-media image-making process is facilitated by synthesising the multidirectional (on, through, behind, in) capacities of screens.
Figure has been removed due to copyright restrictions.
Figure 2: John Gerrard, Solar Reserve (Tonopah, Nevada), 2014, realtime computer- generated simulation. Digital video still.
72 Nicholas Forrest ,“Interview: John Gerrard on his ‘Slippery’ Sims at Thomas Dane Gallery,” Blouin Artinfo, February 16 2015. http://uk.blouinartinfo.com/news/story/1103413/interview-john-gerrard-on-his- slippery-sims-at-thomas-dane (site discontinued). 73 John Gerrard and Simon Groom, in conversation at the Scottish National Gallery of Modern Art, in John Gerrard (Madrid: Ivorypress, 2011), 127. The term can also be seen as referring to cinema, following director Andrey Tarkovsky and his book, Sculpting in Time, trans. Kitty Hunter-Blair (Austin, TX: University of Texas Press, 1986).
26
Unigine is used to create two- and three-dimensional worlds and simulations, mostly for video games. There are major differences between Gerrard’s scenography and that found in video game: the most glaring in this scenario is that games do not use only real places as a point of reference. Most games allow direct participation and engagement with a virtual world: the screen is an instrument to enter virtual space and explore it. Solar Reserve (Tonopah, Nevada) also creates a virtual world, but one that can only be observed from the outside. The large screen used during the display of the work evades the immersive potential and haptic interaction of a gaming interface, which is readily provided for by the Unigine software. Instead, the large screen creates a scale which works against the usual immersive intimacy of virtual on-screen space. Gerrard also creates slippages between the real and its representation.74 This is evident in the immense detailing of the scene and the real-time movement of the sun. Recognising that the heliostats continue to track the sun even when the observer has left makes the encounter with his work on- screen at a particular time of the day a memorable experiential event. Additionally, during the making of the work, the images of the actual facility in Tonopah are turned into data that is shared, edited and amplified via computer screens. Both the audience and artist’s relation to Solar Reserve (Tonopah, Nevada) creates an experience of screens as “information surfaces,” a term used by Huhtamo, which affords us a signal that something particular has happened to screens in the postmedial condition.75
The final largescale screen in this work that is viewed by the audience becomes just such an information surface — it has garnered specific capacities through various inter and intramedia screenic practices. In this way, the intermediality of Gerrard’s screens can be seen as just that operation
74 Gerrard and Groom, in conversation, 131; Forrest “Interview: John Gerrard.” 75 Huhtamo, “Screenology,” 32.
27
of inbetweenness that conjoins the diverse medial components and practices he deploys. In the intermedial travels of the making of the work, the screens maintain relations with different components of this assemblage. For instance, the scenes on-screen follow the real-time movement of the sun, but this is only simulated. This simulation on-screen is a kind of convergence of medialities – the photographic, the computational and gameplay – that are not collapsed. Rather, Gerrard’s screens preserve, or as I will propose, ‘gesture’ this relationship. This thesis explores this gesturing of screens where especially in this installation, they can be seen to embody several processes taking. In chapter two and three specifically, I examine the intensities, enunciations and movements of screens expressly comprising the postmedial art installation as they come to express the postmedial’s emergent qualities.
The idea of medial performances by screens has rarely been explored in the postmedial context. Giorgio Avezzù explores the pre-cinematic instances of screens by investigating the etymological roots of the word screen in order to explicate their historical mediations.76 While this may be useful for thinking about other alternate histories of screens, it does not help in understanding relations in postmedia. Instead, I propose that the mediality of screens is performative; that is, they gesture medial enactments. The enactments of screens and the inbetweenness which they expand and inhabit is supported conceptually by Giorgio Agamben’s concept of gestures. Conceived as supporting and maintaining mediality, Agamben’s notion of the gesture is one which sustains the ongoingness of mediality as an open-ended process. Drawing comparisons from dance, an activity replete with gestures, he insists we look beyond the communicative aspect of gestures: “If dance is gesture, it is so, rather, because it is nothing more than the endurance and the exhibition
76 Giorgio Avezzù, “The Deep Time of the Screen, and its Forgotten Etymology,” Journal of Aesthetics & Culture 11, no. 1 (2019), https://doi.org/10.1080/20004214.2019.1610296.
28
of the media character of corporal movements. The gesture is the exhibition of a mediality: it is the process of making the means visible as such.”77
This medial understanding of gestures helps me to account for the varying agential capacities of screens in different kinds of postmedia assemblages. Screens actively mediate, formulate and reconfigure mediality by productively maintaining the ongoingness of certain relations in a media assemblage. I combine these two ideas of mediality — being in the middle and ongoingness — to apprehend how and what screens do (perform) in a postmedia assemblage. By attending to the medial persistence of screens, this thesis demonstrates how different relations at work in an assemblage actually contribute to the condition, atmosphere or ecology that is postmedia. Gesturality, then, involves a complex set of relations where the agencies of other components in a media assemblage are being sustained. We will see how the intermedial travels of screens aid such gesturality. At the same time, the maintenance of mediality by screens highlights their potentiality not as simply passive receptacles but as technical elements with agency in a complex and shifting medial assemblage. This then becomes the dual operation of screens in postmedia: to gesture and to maintain those gestures.
A postmedia assemblage involves thinking about medial, technical and human components engaging with each other in different ways. These components can be, and certainly in the case of this research are, nonhuman entities such as screens. Hence this research also participates in a nonhumanist mode of enquiry encouraged by certain directions in new materialism. Agency, efficacy, materiality and performativity are thrown into contention for both the human and nonhuman within such strands of thinking. One of the most
77 Giorgio Agamben, “Notes on Gesture,” in Means Without End: Notes on Politics, trans. Vincenzo Binetti and Cesare Casarino (Minneapolis: University of Minnesota Press, 2000), 58. Emphasis in original.
29
prominent influences here has been Karen Barad. My research is informed by their agential realist approach.78 The concept of agential realism understands that entities are not preformed but emerge in practices. A postmedia assemblage, as an arrangement of parts, can, using such an approach, be argued to be of “neither individual entities nor mental impressions, but entangled material agencies.”79
I propose that in order to understand the agency of screens we need to ask What are the entangled intra-actions that constitute screens; what are the intra-acting agencies of screens and what are they producing? Intra-action is Barad’s key concept for thinking how relations allow matter and entities to emerge. Agency in the agential realist framework is an “enactment” and not an inherent property; it is “‘doing’ or ‘being’ in its intra-activity.”80 Barad uses intra-action to signal the “mutual constitution of entangled agencies.”81 Intra- action, as opposed to interaction, is developed by Barad to indicate that entities are not predetermined but become determinate (to an extent) in relations. They clarify that intra-activity constitutes entities differentially, since matter is constantly forming and unforming.82 Hence their concept of agential realism happens through intra-action.83
In relation to the postmedia assemblage, intra-action indicates that screens emerge in and through practices and via relations with other entities. In the entangled relationships that screens are constantly negotiating, they not only connect, relate, reorganise and participate but also produce sensations,
78 Karen Barad, Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning (Durham, NC: Duke University Press, 2007), 32-4, 132-9, 207, 211, 225-26; Karen Barad, “Agential Realism. Feminist Interventions in Understanding Scientific Practices,” in The Science Reader, ed. Mario Biagioli (New York, London: Routledge, 1999), 1 – 11. 79 Barad, Meeting the Universe Halfway, 56. 80 Ibid., 178. 81 Ibid., 33. 82 Ibid., 178, 32. 83 Ibid., 175.
30
affects, transitions and experiences for and with different parts of the assemblage. Entities and agencies are constantly being enacted through discursive, material, social, cultural, scientific and natural practices. Hence, different kinds of screens, their agencies, functions and materialisation depend on the relations, intra-actions and material-discursive practices in which they are imbricated. Chapter one delves into screens and their agency in postmedia assemblages drawing on Barad’s approach.
For the purpose of this thesis, it may be useful to examine the agency of screens along two lines, analysing particular intra-actions of screens in the entangled agencies that they are embedded in, and exploring the differential constitution of screens due to their enactments in and through specific material-discursive practices. Practices, however, are not separate from each other, nor are they separate from discourse. In order to think of screens and their materialisations in and through practices, it is important to consider that the material and the discursive have to be considered together, entangled. Being entangled does not only mean being connected but co-constituted, “to lack an independent, self-contained existence.”84 The material-discursive also implies socio-economic and techno-cultural dynamics do not merely influence each other but are likewise co-constituted.85 Indeed, in agential realism these categories are unclear. Barad tells us:
The world isn’t naturally broken up into social and scientific realms that get made separately. There isn’t one set of material practices that makes science, and another distinct set that makes social relations; one kind of
84 Ibid., ix. 85 Several works that have guided my material-discursive application in this research: Rebecca Coleman, “Pricing Bodies: A Feminist New Materialist Approach to the Relations Between the Economic and Socio- cultural,” Distinktion 19, no.2 (2018): 230–248, https://doi.org//10.1080/1600910X.2018.1495658; Katie Warfield, “Making the Cut: An Agential Realist Examination of Selfies and Touch,” Social Media + Society 2, no. 2 (2016): 1–10, https://doi.org//10.1177/2056305116641706; Wanda J. Orlikowski and Susan V. Scott, “Exploring Material-Discursive Practices,” Journal of Management Studies 52, no. 5 (2015): 697–705, https://doi.org/10.1111/joms.12114.
31
matter on the inside, and another on the outside. The social and the scientific are co-constituted. They are made together — but neither is just made up. Rather, they are ongoing, open-ended, entangled material practices.86
Technological changes are not distinct from nor do they effect cultural habits of screen viewing but are entangled, enacting specific screens. This is why postmedia and digital practices have to be considered together rather than one being an effect — technical or cultural — of the other. Screens in postmedia are not just materially constituted differently but discursive practices play a role in their particular materialisations. The material enactments of screens in and through specific relations are foregrounded in chapter one. Screens are actively produced in and through practices that constitute both meaning and materiality, and practices are key to maintaining postmediality of and by screens. I explore some key postmedial practices to extract the specific gestures of screens. Through image-making and display practices, such as chroma key and multiple screens, I focus upon particular enactments of screens in chapters three and four.
86 Barad, Meeting the Universe Halfway, 168.
32
Figure 3: Unknown, A Louis XV style carved giltwood three-panel folding screen, last quarter 19th century, 171.5 X 59 cm, Sotheby’s. “19th Century Furniture, Sculpture, Ceramics, Silver & Works of Art, 26 October 2010, 10 AM EDT, New York.” https://www.sothebys.com/en/auctions/ecatalogue/2010/19th-century-furniture-sculpture- ceramics-silver-works-of-art-n08672/lot.119.html.
This thesis structurally unfolds following the design of an old-fashioned panelled folding screen used for privacy (shielding/protecting) and dividing a room to reveal a new space. The three or four folds in such screens are usually joined by hinges. Figure 3 shows one such painted panel from the nineteenth century. Viewing this thesis in a similar vein, two sections – each of which further divide in two – are hinged together. In the first two chapters, I argue that screens sustain and reconfigure the dynamic and variable relations in postmedia assemblage; the second section hinges on the argument of the first but now unfolds into a demonstration of how screens sustain and
33
configure relations according to particular gestures. But unlike the hinge of a door that moves only one way, the hinge of the folding screen moves both ways — inwards and outwards — providing orientation, openings and revelations of different spaces. The postmedia assemblage, too, opens towards intra-connection in the assemblage and to other material-discursive practices external to it. Moreover, postmedial screens can acquire more hinged panels. This means that there are more gestures of the postmedial screens than the ones analysed in this thesis; several other ways through which screens maintain mediality in postmedia. This research, then, also opens up further opportunities for exploration of what future gestures and enactments might be undertaken by postmedial screens.
In chapter one, I focus on identifying screenness as the agency of screens in postmedia assemblages. This chapter frames screenness in an agential realist framework, arguing that it is both performative and dependant on the relations of the postmedial assemblage in which it is situated. I analyse the work Shadow 3 (2002) by Shilpa Gupta to argue that screens actively reconfigure and redistribute relations in the assemblage. The agency of screens seen in Shadow 3 is more than that of display. Instead, they become a threefold space for display, intra-action and performance, opening up scope to investigate their postmedial agency. Such an analysis reveals not just the relations in which screens are embedded but the conditions of postmediality.
Via discussion of material-discursive practices, I lay out specific medial performances of screens, such as their roles in magic lanterns and cathode ray tube monitors, to illustrate how screens have emerged with different materialities and capacities. The chapter concludes with a close analysis of an instance of a postmedial assemblage: the complex of software and hardware that supports and enables Google Maps. This assemblage provides insights
34
into intra-actions of practices and materials such as crowdsourcing, scalability of maps through screen pinching, as well as the technical elements of smartphones (gyroscope, microphones) and infrastructural support (cell phone towers) in which the material, formal and expressive role of screens assembles postmedially. In navigating through Google Maps, I consider intra- actions of and with screens that not only generate locatability and mobility, but also reconfigure experiences of time and space. Screenness, it will be shown, entails the ability of screens to reconfigure, as well as sustain and express the relations of the assemblage. This leads us to the question of how this performativity of screens, or screenness persists.
Chapter two develops an argument for a gesturality of screens as the specific performativity undertaken by screens in postmedia. I argue that screens facilitate a kind of gesturality in postmedia that both sustains connections and performs their redistribution. Here I draw on Agamben’s notion of gestures. His understanding of gestures as the continuation of medial activity allows me to focus on how screens are able to sustain actions or movements while being part of a postmedia assemblage where media are nevertheless unstable and undergoing incessant processes of transformation. By attending to the maintenance of mediality, we shift from a concern with how media themselves perform or enact phenomena. Instead, we might see how they contribute to conditions, atmospheres or ecologies. Here screens provide a collective experience of being-in-the-medium — a kind of mediality. I show that this gesturing entails an affectivity that is expressed between images and among entities involved in screenic assemblages rather than something encountered or generated within a single image or medial element. This highlights the ambiguity of who or what is gesturing in postmedia. By further discussing Shadow 3 and then Breath (1991–1992) by Ulrike Gabriel, I elaborate upon the entangled relations of the screen, image and human body,
35
making a case for a new understanding of the gesturality of screens that takes into account the relations and conditions of postmediality. Finally, I propose that the redistribution of bodies and relations that take place in postmedia indicates a new kind of gesturality, which can be ascribed to nonhuman entities such as screens.
Ultimately, Agamben’s concept of gestures as “pure, endless mediality” leads us to recognise gestures to be other than meaning- or intention-laden. Integrating this idea of gestures into a postmedial scenario allows me to consider a gesturality of screens in three ways. First, screens support the endurance of relations in the assemblage. Second, screens perform the redistribution and reconfigurations of these connections. Third, screens enact relations particular to the assemblage of which they are a part. In the dynamic postmedial situation, screens are able to sustain actions, movements and relations and in doing so, the postmedial condition itself. How this gesturality is performed under specific conditions and practices is the focus of the second half of the thesis. This second section provides an in-depth analysis of two key gestures within the postmedia condition through works created and performed after 2000. This challenges the perceived function of screens as simply framing images by highlighting the relations of screens beyond and outside of frames and images. Each of these explorations brings out other gestures of screens.
Chapter three explores the multiplication of both screens and frames on- screen as a dominant quality of postmedial artwork. Some early attention was directed to this aspect of screen-based new media work by Lev Manovich.87 However, I aim to show that frames are positioned on-screen not merely as
87 I refer to Manovich’s discussion on split screens as spatial montage. Lev Manovich, The Language of New Media (Cambridge, MA: MIT Press, 2001), 322, 323.
36
spatial constructs but that their multiplicity is temporal. While a spatial numerosity, as seen in split screens, renders images as a quantitative multiplicity, this may not be adequate in accounting for postmediality where variation and changeability of relations is at play. Instead, a multiplicity of kinds of frames and their relationality is what is at stake. With the help of Henri Bergson’s qualitative and quantitative multiplicities, and Gilles Deleuze’s concept of movement-image and time-image from his seminal books on cinema, I seek to establish the multiplicity of frames and screens as one that involves differing temporalities.88 I want to demonstrate how the screenic gesture of multiplying frames, windows and screens has been productive of qualitative durational experiences.
In order to ascertain this, the chapter has three interrelated moves. First, I illustrate the nuances of the spatio-temporal performances of the frame beyond preoccupations with the merely quantitative. I propose that these performances lead us to think of the gesturality of screens as being primarily temporal in postmedia. By analysing Natalie Bookchin’s video editing and framing techniques, with a special focus on Now he’s out in public and everyone can see (2012), I show how her temporal layerings produce qualitative experiences. In tracing the various kinds of movements of images in and outside the frame, I highlight a different kind of relationship between screens and frames not restricted to spatial overlap. This allows me to arrive at the kind of gesturing ‘framing’ performs in the postmedial assemblage.
Second, I describe multiple temporalities of postmedia assemblages where, beyond the framed image, the out-of-frame prevails. I argue that in postmedia,
88 Henri Bergson, Time and Free Will: An Essay on the Immediate Data of Consciousness, trans. F. L. Pogson (London: Allen & Unwin, 1950); Gilles Deleuze, Cinema 1: The Movement-Image, trans. Hugh Tomlinson and Barbara Habberjam (Minneapolis: University of Minnesota Press, 1986); Gilles Deleuze, Cinema 2: The Time- Image, trans. Hugh Tomlinson and Barbara Habberjam (Minneapolis: University of Minnesota Press, 1986).
37
the out-of-frame performs an ongoingness (Agamben’s continuation of medial conditions) of the multiple temporalities generated by an artwork. This can itself be considered a gesture of screens. In a postmedia assemblage, screens sustain and re-constitute several relations in and through the out-of-frame. I will demonstrate that the out-of-frame — or the various outsides of screen space — maintains an ongoing relation to the inside of the frame while also supporting and conditioning it. Moving onto a different configuration of time in postmedia, I will examine the out-of-frame as a relation in a postmedia assemblage that always simultaneously provides a possibility of moving into the frame. ‘Out-of-frame’ can therefore be understood as a ‘becoming’ frame. Becoming-frame as the temporal rather than spatial movement of the out-of- frame is closely examined in Blast Theory’s Can You See Me Now? (2003). Here multiple frames and screens are arranged in a way that produces both discrete and continuous durations.
Third, I describe a particular temporal experience that is activated in the relations that audience members moving around the installation space form with screens. In analysis of Ragnar Kjartansson’s work The Visitors (2012), a nine-screen installation, I contend that it is through audience movement and out-of-frame gesturing that affect is generated and experienced through temporal shifts and relations. Importantly, this interpolation of audience movement into the postmedial assemblage together with the out-of-framing gesture of screens means that the The Visitors must be thought of not simply as a multi-channel installation but as a complex space-time relationality of the nine screens with the performers, moving and gesturing together in relation to audience sight- and ear-lines. It is these relations of the postmedial (art) assemblage that produce its qualitative multiplicity.
38
After the focus on multiple images and screens in chapter three, I turn finally to a mode of encountering screens where they are ‘without’ images. Chapter four offers an analysis of screens under conditions where elements, components and functionalities long associated with screens — images in this case — are neither immediately present nor directly influencing their performance. Such a discussion evokes the question of whether there can be deliberations about medial screens that are without images. The phrase ‘without images’ in this context does not indicate a complete absence of images, but highlights instances where they do not dominate screen space. The chapter focuses on chroma screens — blue or green screens used in filmmaking, television and graphics during image capture — which facilitate compositing during post-production. Chroma screens illustrate a unique relationship between images and screens where images wait to be called onto screens. By considering chroma screens in and through material-discursive intra-action, I demonstrate that the green or blue colour of chroma screens can be considered as a cultural enactment of racialised medial practices. I historically trace how chroma screens have been medially optimised for performance with white bodies. The question of what and how a gesture is performed by screens in the production of contemporary images can be comprehended not only medially and aesthetically but also politically. I analyse two works where chroma screens are deployed without images. First, in Hsu Chia-Wei’s Marshal Tie Jia - Turtle Island (2012), I analyse the intra- action of political and marginal histories in relation with green screens; second, in How Not to be Seen: A Fucking Didactic Educational .MOV File (2013) by Hito Steyerl, I explore the politics of the visibility of images and propose that chroma screens can be a way to hide from over-visibility by becoming a part of the image.
39
From there, I take the image blankness of chroma screens as the condition for a performance of race. Sondra Perry’s installation work Graft and Ash for a Three Monitor Workstation (2016) opens up how racial practices and concepts around blackness have functioned in the very blankness of chroma screens. The blan/ckness relationship in Perry’s work is then leveraged to become generative of a differencing in which the black body differences and blends with the blue chroma screen. Through this chapter, I show that neither blackness or whiteness (nor any colour) are predetermined but are caught up with a powerful imaging and medial techniques, co-constituted in, with, and through chroma screens.
Both the out-of-frame and chroma screens perform gesturalities of postmedial screens providing two major instances of gestures in a range of postmedia artworks. Accordingly, this study comes to comprehend postmediality through such gesturalities of screens. Offering an expanded conception of screen agency, I aim to track and investigate screen gesturalities so as to offer a way of thinking of the relations that endure and are reconfigured in and through a postmedia assemblage. This study thus provides both an understanding of the gestures of screens, and of screens as actively performing agentic entities of and for postmedia. Reconsidering screenic artworks as active contributors to this framework also provides insights in to the dynamic and intermedial spaces – such as the inbetweenness – that have become characteristic of postmedia.
40
1. DEFINING SCREENNESS: capacities and intra-actions of screens
No machine or technical tool exists by itself, for these artifacts only function in an assembled [agencé] milieu of individuation, which constitutes its conditions of possibility: there is no hammer without a nail, and thus the interaction between a multitude of technical objects makes the fabrication of hammers and nails possible, while also forming the conditions of their utilisation and the practices and habits associated with them.
— Anne Sauvagnargues, Artmachines: Deleuze, Guattari, Simondon1
If a standard object is understood to have specific set of qualities and affordances in one context, one set of dimensions of relationality, how can we use this constrained understanding of its capacities in another? Media are experimented on, not simply in terms of their affordances as standards, but also in terms of what may be mobilized or released when they come into odd conjunction with another scale, dimension of relativity, or drive.
— Matthew Fuller, Media Ecologies: Materialist Energies in Art and Technoculture2
The two quotations above offer a way to understand how a thing becomes a certain thing in relation with other things. To elaborate, the handheld video gaming console Nintendo Switch follows the lineage of gaming devices in which audio, video and control units converge on the one device. The screen
1 Anne Sauvagnargues, Artmachines: Deleuze, Guattari, Simondon, trans. Suzanne Verderber and Eugene W. Holland (Edinburgh: Edinburgh University Press, 2016), 513. 2 Matthew Fuller, Media Ecologies: Materialist Energies in Art and Technoculture (Cambridge, MA: MIT Press, 2007), 172.
41
size of the Switch, slightly over six inches (15.7 cm) diagonally, makes it one of the largest screen sizes in this category. Like other devices in this group, its popularity lies in portability, high-end visual graphics and access to video games. The on-screen action in games takes place through control pads on both sides of the screen, known as Joy-Con controllers (fig. 4). Sauvegnargues, in the above quote, comments on practices and applications that standardise and strengthen the association of a thing to particular conditions. Joy-Con are joysticks that control action and movement. Joysticks are widely employed in manoeuvring mechanical devices such as aircrafts, cranes and other mobility equipment. In the Switch set-up, the screen is used in relation to the Joy-Con where pressing of buttons or slight movements of the Joy-Con changes the action of objects on the screen. At the same time, the actions unfolding on the screen alter the use of the controller. Some on-screen actions also provide tactile feedback in the form of vibrations of the Joy-Con. Moreover, the mobility of the Switch means that the gameplay could take place anywhere (location) or in any position of the player’s body (lying down, sitting up). This adds another dimension to the already complex experience of gameplay. The screen here is a technical element embedded in the practices and provisions offered by playing games on handheld devices.
42
Figure has been removed due to copyright restrictions.
Figure 4: Nintendo Switch console in handheld mode. Image reproduced from Nintendo website. https://www.nintendo.com.au/nintendo-switch/.
Figure has been removed due to copyright restrictions.
Figure 5: Nintendo Switch console in TV mode. Image reproduced from MakeUseOf. https://www.makeuseof.com/tag/best-local-multiplayer-games-nintendo-switch/.
By way of comparison, the Switch also provides another way of operating. The screen can be separated from the Joy-Con, docked and connected to an external screen or monitor (fig. 5). This ‘TV mode’ takes away the portability of the device since it is now connected to an external screen. Since the Joy- Con are detached in this situation (Joy-Con Left and Joy-Con Right), they can be used by two players, unlike handheld mode, which only allows for a single player. The relations of the screen with other technical elements in
43
these two configurations permit different arrangements of display, the bodies of players, gameplay, control panels (hardware) and software.3 Even though the engagement of screen and game-playing is present in both handheld and TV modes, different kinds of screens emerge and diverse connections with other media are forged. At the same time, associations of the screen hooked to the Joy-Con are distinct to the relations when watching a film on that same screen. Albeit a different context, the latter alliance of the screen follows Fuller’s provocation of “another scale”. Screens in both arrangements — playing games and watching films — not only demonstrate that the relations of elements are different but also show how these relations are contingent upon the context in which they are enacted.
This is not merely a rhetorical exercise for listing the diverse ways and means in which screens appear and are used. Rather, I want to think about how different conditions for technical objects and elements, and their entanglements at semiotic, material and pragmatic levels, draw out and condition the different capacities of things. Sometimes such conditions and entanglements disclose latent potentials. The latent abilities of technical things can open up new doings as well as their current uses. But more importantly, understanding these conditions and entanglements makes us attentive to how and where their capacities are modified, organised, and activated differently.
This chapter articulates screenness as a way to comprehend what screens do in the postmedia condition. My particular focus is on media and postmedia
3 For example, there are changes in the resolution of the game and the availability of games in the two modes of operation. Although the resolution and frame rate also depend on the individual game, when connected to an external screen the Switch can present resolutions up to 1080p and a maximum frame rate of 60 frames per second. See Allegra Frank, “Everything We Know about Nintendo Switch (Update),” Polygon, March 2, 2017, https://www.polygon.com/2017/1/13/14241898/nintendo-switch-details. Some games are offered only in one mode, e.g. Super Mario Party and Just Dance are designed for TV mode.
44
artworks. I argue that these works take screens in directions that can reveal or push at the conditions under which assemblages coalesce in this postmedia condition. In other words, they give us an intensive instantiation of the screenness of screens. Screenness, understood in an agential realist framework, is performative and contingent upon the relations of the postmedia assemblage, considered here as an arrangement that takes into account the various functionings of screens in image-making, display and dissemination practices. The unstable, ever-changing relations in postmedia assemblages, then, help in understanding the numerous enactments of screens. This performativity of screens in postmedia assemblages is what I articulate as screenness. Developing Barad’s notion of entanglement in the postmedia context, I will explicate the material-discursive practices in which variations of screens have emerged. I will demonstrate that the entangled intra-active agencies are separated out into determinate techno-medial entities such as screens by material-discursive practices. It is through practices that certain boundaries — agential cuts — are executed, producing a variety of screens. In an attempt to demonstrate the performativity of screenness, I will finally discuss screen intra-actions — intensity and mode of mattering — that can be gleaned via the particular activity of navigation using a screen-based device and the Google Maps mobile phone app.
SCREENS IN POSTMEDIA ART ASSEMBLAGES In a dark room, participants entered to face a large screen. In the interactive media art installation, Shadow 3 (2007) by Shilpa Gupta, the participants stood in front of the screen, their shadows projecting on it (fig. 6). These were not quite shadows, but silhouettes captured via the Kinect motion sensing input device mounted just below the screen in front of the participants. As they took their position in front of the screen, a string seemed to attach to the participants’ shadow on the screen. Further shadows of objects such as
45
umbrellas, trees, balls and other shapes passed through the string towards the shadows of the audience members and ‘attached’ to them as well. With this the two sets of shadows combined into one. As the participants moved, the attached objects moved with them. Moreover, other participants could interact with these newly formed associations of shadows by transferring the silhouetted objects from and to other participants. When the participant’s silhouette assembled with shadows of other objects, they both became different objects.
Figure has been removed due to copyright restrictions.
Figure 6: Shilpa Gupta, Shadow 3, 2007, interactive video projection incorporating the viewer's simulated shadow. Digital video still from documentation. Photographer: Anil Rane. Courtesy: The Artist & Galleria Continua / Le Moulin, San Gimignano / Beijing / Les Moulins / Habana.
The participants’ bodies conjured a shape-shifting image, where changes occurred by attaching their projections to other objects. The shadow-parts — the participants’ shadow and objects’ shadow — did not exist in isolation on the screen. Instead, they generated a togetherness exceeding their appearance as separate units. This fusion of the shadows of participants and objects enacts a visual creation of an assemblage, which, as I have outlined in the
46
introduction, comprises a temporary arrangement made in a particular time and space. These shadow assemblages could be modified and reshaped by adding or borrowing shadows; that is, participants could extract objects from other people-object aggregates to add to their composition, return them or passing on other shadow objects in due course.
Assemblages are formed by and are forming the connections of the participants’ bodies and objects by drawing on their particular capacities. To be clear, the assemblages of the shadows on the screen are not the only assemblages in Shadow 3. The installation itself can also be considered as a media assemblage with human (participant) and nonhuman/technical elements (Kinect, software, screen, shadows). However, these elements should not be mistaken for well-defined purposeful parts. Rather, they are emerging in the process. The capacities of the elements lie in relation to the capacities of other components. These interactions between the components give rise to emergent properties of the assemblage and its components. This means that the assemblage is “structured and structuring.”4 The emergent properties, also referred to as synergistic properties, are “contingent” instead of being “necessary or transcendent.”5 The attributes of the assemblage are dependent on particular intra-actions — entanglements and mutual constitution — of the components and are not intrinsic to the whole.
Furthermore, the installation Shadow 3 is a particular kind of assemblage that this thesis is concerned with— a postmedia assemblage. Such an assemblage is not simply an assemblage in, of, or about postmedia, but one in which its capacities and arrangements draw upon and generate postmediality. As outlined in the introduction, a postmedia assemblage does not have a specific
4 Ian Buchanan, “Assemblage Theory, or, the Future of an Illusion,” Deleuze Studies 11, no. 3 (2017): 463. 5 DeLanda, Assemblage Theory, 12. Emphasis in original.
47
media materiality and is one in which exchange across media materialities intensifies. In Shadow 3, the Kinect is able to process the images of the participants almost instantaneously, producing silhouettes that are projected not onto but from a screen. Here different devices exist yet the movement and circulation of images between and across them proliferates. The postmedia assemblage is based on just this relationality of different technical devices and platforms with each other. These devices and platforms come together at a particular point yet are not just connected to each other but can remain outside each other as well. This is why we need a notion of the assemblage rather than of convergence, as elaborated in the introduction, when understanding the postmedial. In Shadow 3, we see the ways that the shadows of objects and participants remain open to ongoing connections with other participants and objects. Relations form between and through the elements but are not limited to them.
The postmedia assemblage is conditioned by movement and relations that are dynamic, as the conjunctions of devices and platforms — a feature of postmedia — constantly activates new potentialities. With different media, relations between various elements such as images and participants are organised, activated and sustained. My focus in this chapter is to establish how screens interrelate and perform certain functions in a postmedia assemblage. The silhouette of the participant’s body in Shadow 3 connects with the shadows of objects that are not physically present to the audience members. The screen, then, offers a space where the participant’s body could be transformed and transfixed in a relationship with elements that had no substantial physical manifestation beyond their shadowy forms. Moreover, the participant’s body evokes the outline of the body that we have become familiar with from user-end motion capture devices such as Kinect and Wii that facilitate bodies being displayed as an on-screen avatar interacting with
48
virtual objects. Through such interactions, the screen becomes a threefold space for display, interaction and performance. I want to argue, then, that screens in postmedia assemblages actively reconfigure and redistribute the relations that constitute them; as such they are a key element of the postmedia assemblage. Engaging in numerous interactions, screens undertake roles, participate in processes, and produce new events and configurations. They co-produce and facilitate various interactions of the assemblage, whether maintaining its stability or propelling the assemblage towards instability and change. In the process, screens constantly rearrange and recompose the relations of and with other components. In this way, screens are also constantly producing and being produced.
Screens exist physically even if they are just constituted by a projection on the wall; they have a material presence. It may seem as though screens are the ‘carriers’ of images, text, source code and so on, between a user, participant, or viewer and an apparatus. But, as I discussed in the introduction, their mediality is not only explained by their location between the machine and user. Rather I will show, screens in fact arrange and compose relations between various elements in the assemblage. So, to understand the performative role of screens in postmedia, we require a revision of the idea of mediality in this context. Moreover, screens have an agency that is not restricted to that of display. I argue that the investigation of the different agential capacities of screens in postmedia reveals more about screens, and the postmedia condition and its processes. The agency of screens is performative and generative, bringing interactivity and participation out of their relegation to a zone of action-reaction, and instead shifts screens to a role of sustaining and continuing relations in the assemblage.6
6 Massumi notes that action-reaction in interactive art has to deal with the “flow of actions” and the “qualitative relational-aspect.” Interactive art has to make sure that “thinking-feeling,” that is, the qualitative
49
Here assemblage theory as understood through the work of Gilles Deleuze and Felix Guattari, Manuel DeLanda and others can lead to a genuine enquiry into how screens can be used to examine the techno-cultural space that they have come to occupy. Their capacities and emergent properties, which intensify in postmediality, point to their instability as strictly medial objects and also indicates postmediality to be a space of becoming. In the analysis below, I will map out the material, formal and expressive roles of screens in their various interactions or more appropriately in the context of this thesis, their intra-actions. Alongside this, I will focus on the mechanisms that they present for maintaining or disrupting the identity of the assemblage that they co-constitute.
In the postmedia assemblage, there are no clearly bounded media, devices or bodies; all are contingent on their relations. Thus, what we call screens are equally dependent upon the relations which enmesh them. This raises two significant sets of questions: first, how are screens differentiated from other entities in the assemblage? Can screens be identified through their screenness? In other words, what makes screens screens in postmedia? The second set of questions take up the agency of screens: can a specific agency for screens be identified? If the relations of the assemblage are unstable, constantly forming and unforming, how are screens identified as agents with certain kinds of discrete activities? In the following sections, I intend to reformulate screenness by resituating the agency of screens via Barad’s agential realist framework. Apprehending screenness as agency that is enacted through material discursive practices, I argue, will help to illustrate what screens do and how they do it.
dimensions of potentialities and qualities of the event is foregrounded, instead of causal (re)action. See Brian Massumi, “The Thinking-Feeling of What Happens,” Inflexions 1 (2008): 1–40.
50
SCREENNESS OF SCREENS In an assemblage, thinking agency through capacities in relation, in which both capacities and relations have effects, would mean that the assemblage’s entities are also capable of entering into different relations or modify existing ones. This can be ascertained in the roles and functions of screens in the various media objects in which they participate. For example, the common display function of screens as seen in cinema and television; or the combination of interactive surface and display in computers and smartphones, where screens are accessed through external hardware such as a mouse, stylus or human fingers. In the assemblage of a television, it can be argued that the agency of screens is to display the signals via cathode ray tube (CRT) scanning. In a smartphone, the agency of screens, arguably, is the display of images and being responsive to human touch. However, the problem in understanding screens this way is twofold: firstly, screens are presumed as preformed entities that participate in an assemblage, which is antithetical to the idea of assemblage. Secondly, screens are assumed to be objects upon which actions and ideas are impressed. This notion is founded on the affordances of screens, which does not assume screens to be inactive but definitely more closed and functionalist. Consequently, this approach is not helpful as it does not take into account the becoming of screens. Moreover, it does not consider the activity, formation, and ‘response-ability’ of screens.
When screens are so easily configurable — since, for example, a blank wall can become a screen to display images — then the fundamental yet complicated question is no longer what are screens, rather how do they become? In other words, is screenness a quality that can be defined? If so, what are the practices through which such a quality can be identified and supported? I am proposing that in the postmedia context, screens emerge in
51
and from the relations of various medial and nonmedial entities; that is, screenness lies in the shifting alliances of the assemblage. I aim to highlight a performative agency of screens here that allows them to execute various roles and functions within a media set-up that is not restricted to displaying images bounded by frames.
Screenness as a topic has been examined in order to explicate different aspects of screens. Lucas D. Introna and Fernando M. Ilharco describe screenness in phenomenological terms.7 Screens are encountered, they argue, in a pre-existing and known world, for example, in a cinema hall the screen is known to be showing the film.8 The screenness of screens, for Introna and Ilharco, lies in the “promises to bring into being, to make visible that which is relevant in that world.”9 This assumption of a ‘given’ world in which screens ‘appear’ is problematic as screens don’t simply hold our attention or frame something (images, text) relevant to us, as they suggest.10 For example, the engagement with screens in cinema is not the same as computer screens. We don’t “look through them to encounter our way of being-in-the-world”;11 rather we look, encounter and produce the world with them. I want to challenge their notion of screens as being fundamentally a background by arguing that screens come into being through the relations they form with other entities. So, specific conditions for the formation of relations between architecture, participation and subjectification of the human (user vs viewer, for example) are important to note because the execution of the functions and capacities of screens lies beyond grounding the display of images. To understand the
7 Lucas D. Introna and Fernando M. Ilharco, “On the Meaning of Screens: Towards a Phenomenological Account of Screenness,” Human Studies 29, no. 1 (2006): 57–76; Lucas D. Introna and Fernando M. Ilharco, “The Ontological Screening of Contemporary Life: A Phenomenological Analysis of Screens,” European Journal of Information Systems 13, no. 3 (2004): 221–234, https://doi.org//10.1057/palgrave.ejis.3000503. 8 Introna and Ilharco, “On the Meaning of Screens,” 62, 64. 9 Ibid., 64. 10 Ibid., 65. 11 Ibid., 66. Emphasis in original.
52
nuances of the different ways of engaging and producing screens, we have to take into account the specific conditions and material-discursive practices in which screens emerge. If screens are not the same within a particular set of practices, historical conditions and so on, then screenness, too, must be a quality that changes.
Several theorists have situated screenness in socio-material practices that have illuminated its other aspects. Sonya Hofer explicates screenness through the performances of musician-artists, describing it as an “experiential and expressive power” of screens, specifically referring to their use of laptops.12 Screenness can come alive in different, sometimes contrasting ways for the viewers by fragmenting sensorial experiences or blending different sensations.13 These different aesthetic experiences, Hofer discusses, are mediated through screenness.14 While screenness features here as a capacity of screens, the question of how it is experienced is compelling. Alexander Styhre elaborates on screens amid organisational practices. He is careful not to conflate screenness with interface and argues that screenness precedes the role of screens as interfaces, calling it a “nexus between the human user and the technological system.”15 Naming aesthetics and code as the two aspects of screenness, Styhre describes the former as being visible to the user while the latter is concealed.16 Styhre demonstrates the aesthetics of screenness of digital media (which he situates in the ‘new visuality’ of graphic design) through a history of visuality that he traces through an oral tradition.17 Code as machinic performativity is based on a ‘doing’ that administers humans,
12 Hofer illustrates screenness through examples of live performances. See Sonya Hofer, “Screenness in Experimental Electronica Performances,” Music and the Moving Image 10, no. 2 (2017): 16–32. 13 Ibid., 19–23. 14 Ibid., 23, 26. 15 Alexander Styhre, “Screenness and Organizing: Sociomaterial Practices in Mediated Worlds,” VINE 43, no. 1 (2013): 8, https://doi.org//10.1108/03055721311302115. 16 Ibid., 6, 8. 17 Ibid., 9–10.
53
technological systems and organisations.18 Although Styhre locates code and aesthetics in material and discursive practices, he treats these practices separately and not entangled as I will discuss later. Moreover, he overlooks other significant practices. His discussion on the aesthetics of screens fails to take into account the pre-cinematic media objects and film practices that have had a profound effect on the history of visuality in which Styhre is situating screenness. Similarly, his analysis of code does not recognise how coding practices themselves have changed with the development of programming languages, increases in data compression, and the microprocessing abilities of computers. More importantly, to write code also requires a screen, although it is also the case that code can run invisibly in the background. The entanglement of screens in the making of images and writing code has not been accounted for in the discussion. Styhre’s emphasis is on how screens display, organise and processes information for humans.19 I, however, extend this to argue that the capacities of screens also create, perform and generate relations and interactions with humans.
Although Nigel Thrift also discusses code in relation to mediation, he keeps it separate from screenness. Screenness for Thrift is a condition that is produced by screens. He identifies screens as mediators between the human body that interprets what is seen on the screen and the calculation or the code that displays it.20 He then defines screenness as one of the registers of mediated materialities that lies between interpretation and calculation.21 Helen Verran argues for screenness as the ways screens “organize interaction and knowledge practices.”22 Here, screens can function as indices of learning,
18 Ibid., 12. 19 Ibid., 13. 20 Nigel Thrift, “Beyond Mediation: Three New Material Registers and Their Consequences,” in Materiality, ed. Daniel Miller (Durham, NC: Duke University Press, 2005), 244. 21 Ibid., 237. 22 A course conducted by Helene Verran and Lucy Suchman is detailed in the special issue of the journal
54
especially when seen in the educational context. However, for my purposes it may be more interesting to think of screenness as a way of arranging bodies and practices drawing upon the work of someone such as Lucy Suchman, who takes up screenness in an ethnographic context.23 While such an argument may be useful in accounting for the interaction between humans and media objects, it does not go far enough in establishing how screens become, which is a quality of postmediality. Suchman’s analysis can tell us the position of screens in a particular assemblage, but not how screens continue to become something else and indeed how the assemblage itself is transformed through them.
I am proposing another way to look at screenness; that is, as an enactment of specific relations. While the functions of screens emerge through and in connections with other components of an assemblage, screens are able to form relations outside a specific instantiation of an assemblage, which then provide them a certain autonomy. To signal such agency for screens, and thereby a new understanding of screenness, we will need to forge an understanding of the postmedial assemblage through an agential realist framework. We begin by challenging ideas of agency as a given ability to act and instead rethink it in the context of entangled matter and practices as already been proposed by Barad’s work on other kinds of physical and scientific nonhuman phenomena.24
The conception of agency as operating purely in the domain of human actions, beliefs and intentionalities — an idea proposed by Aristotle — has
STS Encounters. Brit Ross Winthereik, Peter A. Lutz, Lucy Suchman and Helen Verran, “Attending to Screens and Screenness: Guest editorial for special issue of Encounters,” STS Encounters 4, no. 2 (2011): 2. 23 Ibid. 24 For example, Barad’s discussion on the Stern-Gerlach experiment that illustrated particle spin, in Meeting the Universe Halfway, 166–168.
55
been challenged by proponents of nonhuman agency.25 Instead of accepting agency framed in terms of intentionality and causal events, nonhuman-centric arguments for dissociating agency from mental representation have been made. For example, Xabier Barandiaran, Ezequiel Di Paolo and Marieke Rohde described the idea of minimal agency, where an entity is doing something on its own, such as bacteria having minimal agency to survive.26 Agency, then, is understood as a capacity or power in both humans and nonhumans.
Preoccupations with the relationship between human and nonhuman entities permeates media studies, network and communication studies. Already with Marshall McLuhan’s theories, the work of nonhuman agents such as television and radio to reorganise social relations was apparent. His proclamation of media as an extension of human senses is an attempt to understand the role of media in reorganising experience as a complex technoperceptual phenomenon.27 Arguably, this is equally an enquiry into the agency of media and networks. Yet while the agency of nonhuman media objects is acknowledged, it is examined from the perspective of augmenting human communication. This human-centred approach differs from mine here, which engages instead with co-creation and co-production of events and bodies.
My concern with the agency of screens follows recent attention to nonhuman objects, materials other than human, and practices that highlight more than the human. Theoretical developments such as actor-network theory, affect
25 For a quick background on the sequential movement of ideas around agency, see Markus Schlosser, “Agency,” Stanford Encyclopedia of Philosophy, 10 Aug 2015, https://plato.stanford.edu/entries/agency. 26 Xabier Barandiaran, Ezequiel Di Paolo, and Marieke Rohde, “Defining Agency: Individuality, Normativity, Asymmetry, and Spatio-Temporality in Action,” Adaptive Behavior 17, no. 5 (2009): 367–386. 27 McLuhan, Understanding Media, 11.
56
theory, systems theory, new materialism among others have similar interests.28 These may share different concerns, contexts and methods, but they all in different ways are attempting an understanding of the nonhuman such as plants, animals, machines, micro-organisms and/with the human. The ‘nonhuman turn,’ proposed by Richard Grusin, summarises these concepts, theories and approaches focussing on the agency, vitality, affectivity and relationality of both nonhumans and humans.29 He identifies that these ideas are embedded in nonrepresentational, non-linguistic frameworks that are capable of examining the togetherness of all entities.30 Underpinning my enquiry into screens, this rich discourse at the intersection of human, nonhuman and more-than-human helps by being attentive to the synergies of a postmedia assemblage co-constituted by material and discursive practices.
Enquiring about ways of being together with nonhuman entities such as plants, animals and machines may not be new, but the method of such investigation has changed. The resurgence of interest in a materialist understanding of the world has two aspects that are important to this thesis.31 Firstly, there is an attention to the performance and choreography of the
28 The list of works is too long and widespread, but important texts for this research include New Materialisms: Ontology, Agency, and Politics, ed. Diana Coole and Samantha Frost (Durham, NC: Duke University Press, 2010); The Nonhuman Turn, ed. Richard Grusin (Minneapolis: University of Minnesota Press, 2015); and The Affect Theory Reader, ed. Melissa Gregg and Gregory J. Seigworth (Durham, NC: Duke University Press, 2010). 29 Richard Grusin, “Introduction,” The Nonhuman Turn, viii. 30 Ibid., x. The field of feminist new materialism has perhaps been the strongest voice in considering material practices and the agency of nonhumans. Among several significant studies, particularly see Felicity J Colman, “Agency,” New Materialism Almanac, 17 May 2018, http://newmaterialism.eu/almanac/a/agency.html; New Materialisms: Ontology, Agency, and Politics, eds. Diana Coole and Samantha Frost (Durham, NC: Duke University Press, 2010); Rosi Braidotti, The Posthuman (Cambridge: Polity Press, 2013); Stacy Alaimo, Bodily Natures: Science, Environment, and the Material Self (Bloomington: Indiana University Press, 2010). 31 For a small sample explicating the link between social and discursive practices see W.J. Orlikowski, “Sociomaterial Practices: Exploring Technology at Work,” Organization Studies 28 (2007): 1435–1448; P.M. Leonardi, “Materiality, Sociomateriality, and Sociotechnical Systems: What Do these Terms Mean? How are they Different? Do We Need Them?” in Materiality and Organizing: Social Interaction in a Technological World, ed. P.M. Leonardi, B.A. Nardi, and J. Kallinikos (Oxford: Oxford University Press, 2012), 25–48; Myra J Hird, “Review: Feminist Engagements with Matter,” Feminist Studies 35, no. 2 (2009): 329–346.
57
togetherness of the human and nonhuman.32 Here the co-production, collectivity, intersections, and contributions of different kinds of matter are closely studied and not considered as a given. Moreover, an entity is understood to be created not just through discursive, but material-discursive practices where matter is also acknowledged as agential. Barad argues that “the world kicks back” pointing to how matter(s) have active agency in the making of the world and how we understand it.33 Secondly, the rise of new materialist enquiries is propelled by an interest in the use of technologies in the context of growing ecological concern. The urgency of environmental degradation has shifted focus not only towards the kind of materials used, but the practices foregrounding them, be they scientific, technological, social or economic. In this regard, Rebecca Schneider argues that matter, human and nonhuman, are both agential and discursive. This means that not only does matter engage with other matter but is also a way to know the world that is different from a semiotic way of knowing.34 Making a case for the agency of biology, Samantha Frost explains that new materialism pushes feminists to consider that culture and biology produce each other instead of bodies being produced as a result of structures of power.35 These ideas around agency not only emphasise thinking about co-production and a decentring of typical causal relations, but also focus on practices in which things become operational.
In the context of thinking agency within the assemblage, Jane Bennett’s work is significant. Bennett defines agency as a kind of power of things that can
32 Dualism is rejected in new materialism. Diana Coole, “Agentic Capacities and Capacious Historical Materialism: Thinking with New Materialisms in the Political Sciences,” Millennium 41, no. 3 (June 2013): 454, https://doi.org/10.1177/0305829813481006. 33 Barad, Meeting the Universe Halfway, 215. 34 Rebecca Schneider, “New Materialisms and Performance Studies,” The Drama Review 59, no. 4 (2015): 7. 35 Samantha Frost, “The Implications of the New Materialisms for Feminist Epistemology,” in Feminist Epistemology and Philosophy of Science: Power in Knowledge, ed. Heidi E. Grasswick (Dordrecht; New York: Springer, 2011), 69–83. For a discussion on similar propositions by Elizabeth Grosz and Susan Oyama see page 77.
58
“produce (helpful, harmful) effects in human and other bodies.”36 She further explains that the “distributed agency” of an assemblage is made up of micro and macro actants.37 Each component of the assemblage has its own “vital force,” a capability, different to the agency of the assemblage. The power of these actants intensify in the assemblage.38 However, the assemblage is more than the sum total of its elements, so while the energies of its elements are changing, the capacities of the assemblage change differently to its components.39 The issue with Bennett’s conceptualisation is that even though she is talking about the instability of the assemblage, there is not enough emphasis on the relations.40 More importantly, she has a tendency to think each actant’s capacities as already coming to the assemblage. As Rebekah Sheldon points out, in detailing the entities of an assemblage, Bennett sometimes misses the opportunity to elaborate on the relations between them and the emergent capacities of the assemblage.41
While keeping in mind the new materialist attention to nonhuman entities in examining the relationality of materials, I turn towards Barad’s agential realist framework to think about agency as an enactment.42 Such theoretical grounding supports thinking at a more fundamental level of how entities emerge from intra-actions, from entanglements of matter(s). It is particularly useful to apprehend screens constituted of matter and meaning, materialising in and through various practices. Barad argues that agency is not something that is possessed; it is not something an entity, human or nonhuman has, since
36 Jane Bennett, Vibrant Matter: A Political Ecology of Things (Durham, NC: Duke University Press, 2010), xii. Emphasis in original. 37 Ibid., 21. 38 Ibid., 24, 21, 34. 39 Bennett considers the member-actants of an assemblage to be “out of phase” with the assemblage. Ibid., 35. 40 Thomas Lemke elaborates on similar points. He adds that Bennett does not consider the destructive tendencies of things and only considers things to have a positive effect. See Thomas Lemke, “An Alternative Model of Politics? Prospects and Problems of Jane Bennett’s Vital Materialism,” Theory, Culture & Society 35 no. 6 (2015): 31–54. 41 Rebekah Sheldon, “Form/Matter/Chora: Object-Oriented Ontology and Feminist New Materialism” in The Nonhuman Turn, ed. Richard Grusin (Minneapolis: University of Minnesota Press, 2015), 209. 42 Barad, Meeting the Universe Halfway.
59
that would take us back into a humanist orbit.43 Rather, agency is an enactment of the possibilities for (re)configuring entanglements.
Turning back to screens here, screens don’t simply ‘have’ agency in terms of the capacity to act. Rather, screenness can now be understood as the agentic performance of certain relations and practices.44 Barad deploys a posthumanist idea of performativity, which builds upon Judith Butler’s ideas among others such as Donna Haraway, Bruno Latour, Joseph Rouse and Andrew Pickering.45 Such a notion of enactment can not only tell us something about the formation of screens but also about the conditions of their production. As Barad argues, since performativity produces “the matter of bodies,” it is imperative to know about the attributes of this production.46 The idea of the conditions of/for the enactment of screens is important because it tells us when screens might be functioning as screens; that is, it highlights their screenness. Moreover, this is not just about how screens come into being (display or touch screen) in a particular assemblage but that the conditions include relations, systems, and durations through which they come
43 Karen Barad, “‘Matter feels, converses, suffers, desires, yearns and remembers’: Interview with Karen Barad,” Rick Dolphijn and Iris van der Tuin, in New Materialisms: Interviews and Cartographies, ed. Rick Dolphijn and Iris van der Tuin (London: Open Humanities Press, 2012), 54; Barad, Meeting the Universe Halfway, 235. 44 I strongly support the use of the term enactment as opposed to performance, following Mol’s assessment that enactment highlights the activity without conjuring the ones that are performing. Enactment is not as loaded a term as performance. See Annemarie Mol, The Body Multiple: Ontology in Medical Practice (Durham, NC: Duke University Press, 2002), 32, 33. 45 See Barad, Meeting the Universe Halfway, 191-2 for a discussion on Butler by Barad, who surveys the shortcomings of Butler’s arguments. From the range of texts by these authors that Barad draws upon, I am mainly referring to: Judith Butler, Bodies That Matter: On the Discursive Limits of ‘Sex’ (New York: Routledge, 1993); Donna Haraway, “Manifesto for Cyborgs: Science, Technology, and Socialist Feminism in the 1980s," Socialist Review 80 (1985): 65-108; Donna Haraway, "Situated Knowledges: The Science Question in Feminism as a Site of Discourse on the Privilege of Partial Perspective," Feminist Studies 14, no. 3 (1988): 575-99; Bruno Latour, Politics of Nature: How to Bring the Sciences into Democracy, trans. Catherine Porter (Cambridge: Harvard University Press, 2004); Andrew Pickering, "After Representation: Science Studies in the Performative Idiom," talk given at the Philosophy of Science Association Annual Meeting, New Orleans, 1994; Andrew Pickering, The Mangle of Practice: Time, Agency, and Science (Chicago: University of Chicago Press, 1995); Joseph Rouse, How Scientific Practices Matter: Reclaiming Philosophical Naturalism (Chicago: University of Chicago Press, 2002). 46 Karen Barad, “Posthumanist Performativity: Toward an Understanding of How Matter Comes to Matter,” Signs 28, no. 3 (2003): 808.
60
to be identified as having such properties. It does not mean that we should simply identify different kinds of screens (such as a television or mobile phone). Rather, we need to give careful consideration to the relations that persist in the assemblage; for example, an image changes its angle and size when the orientation of a smartphone is changed. Upon flicking the smartphone from a vertical position to a horizontal one, the image on the screen also adjusts its position with respect to the screen and the viewer. In the action of changing the orientation of the phone it is the relation of the image and screen that prevails. My next section elaborates upon these ideas by further unfolding an agential realist understanding of screens.
To summarise momentarily, I am proposing that screenness is the capacity enacted by anything or anyone to become (a) screen, and that this is an unbound, shifting capacity in postmedia. Barad’s agential realist framework offers the possibility to move beyond a mere typology of screens and observations about their ubiquity in the postmedial condition. In the following sections, I will consider two operations of screenness. Firstly, the particular intra-actions of screens in the entangled agencies of the postmedia assemblage; and secondly, the differential constitution of screens due to the enactment of agential cuts by specific material-discursive practices.
Screenness in an agential realist framework Agency, in agential realist philosophy, is revealed and performed in intra- activity; that is, via synergies between matter. Intra-action, as discussed in the introduction, is developed by Barad in opposition to interaction, to indicate that entities are not already determined.47 Interaction in the medial context, and with specific reference to my area of concern, assumes the existence of
47 Barad, Meeting the Universe Halfway, 139.
61
screens with specific boundaries, while intra-action gets at the ontogenesis of screens, specifically the materialities and features of screens emerging via intra-active performance.48 Although several possibilities of intra-action exist in an assemblage, the ones that are enacted depend on the material-discursive practices that create boundaries and exclusions. Hence, specific practices disclose certain agential possibilities. For instance, a mobile phone screen is a viewfinder while taking photographs but the common social practice of taking selfies with face filters, popularised by applications such as Snapchat, adds objects and alters faces onto the screen view. Understanding screenness via an agential realist framework takes into account the numerous possibilities for an assemblage to perform and undergo changing relations. This does not mean that the screenness is a mere result of the assemblage, rather screenness is a simultaneous intra-activity with and of the assemblage.
Moreover, if the meaning and form of entities depend on their relations, then this calls for a consideration of the specific relations in which screens are emergent. But, if relations are not fixed in a postmedia assemblage and there is an inseparability of the agentially intra-acting components, then how do screens sustain themselves? That is, since human and nonhuman agencies are constantly producing-reproducing and configuring-reconfiguring each other, how then can screens maintain some specificity? The separation of the intra- acting agencies into entities must be constituted through specific practices; that is, specific kinds of agential intra-actions — which Barad calls “agential cuts” — are marks through which the boundaries and properties of things are determined.49 These cuts may be repeated as boundary drawing practices that
48 Barad advises not to consider agential separability as individuation, because entities are always in the making, they are never fully formed. I am not claiming that screens are entities that are complete but attempting to illustrate how they come to be rather than trying to reach one definition for all kinds of screens. 49 Karen Barad, “Karen Barad: Intra-actions,” interview with Adam Kleinman, in Mousse 34 (2012): 80; Barad, Meeting the Universe Halfway, 148.
62
come to constitute objects within an assemblage in a particular way. The objects are, then, “with in” and “part of” these practices.50
The enactment of entities in a range of practices has been scrutinised by Annemarie Mol. She has studied how medical practices, for example, enact ‘a’ disease. In practices such as a patient’s understanding of disease, a doctor’s diagnosis based on symptoms, x-ray reports and patients’ blood tests, the entity or object of observation (the disease) is not one stable thing but comes into being differentially through these practices. That is, objects are “enacted” via these practices.51 The cuts or boundaries between entities create agential separability. This does not simply mean that entities have different agencies;52 instead, as Barad suggests, the entities can also enact agencies outside of their boundaries.53 They face out to other kinds of practices and matter but are still entangled (exteriority-within-phenomena).54
In a postmedia assemblage, entities are not formed separately before relating to each other; they are already connected. However, certain practices and set- ups make them discrete. Screenness is not just an enactment of agency as affordance or function but an enactment of intra-actions through specific material-discursive practices. In the context of screens, practices such as filmmaking and the design of digital computer interfaces draw out screens and their screenness in specific ways through different intra-actions with components in an assemblage. For example, in filmmaking the decision to shoot on a 16 mm film roll instead of the more common contemporary practice of recording digitally onto hard drives may position the associated
50 Barad, Meeting the Universe Halfway, 208. 51 Mol, The Body Multiple, 5, 33. 52 This seems to be the suggestion in Katie Warfield “Making the Cut: An Agential Realist Examination of Selfies and Touch,” Social Media + Society 2, no. 2 (2016): 1–10. 53 Agency is never foreclosed, even while intra-acting. Barad, Meeting the Universe Halfway, 214. 54 Ibid., 184.
63
playback screen to be situated within a more cinematic context with an appropriate film projector, instead of being distributed via digital means and destined for digital cinema, computer or television screens.55 Practices have to be closely analysed to trace how screens as technical objects materialise through and in diverse intra-actions across the technical apparatus, social context, medial history, viewing habits and architectures.
Although elements of this approach permeate many kinds of screen studies analyses, the emphasis to date has mainly been to approach the screen by way of media-specific contexts. However, the fluidity of entities enacted and foregrounded via postmedia assemblages requires us to shift toward a more enactive approach. Such an enquiry also opens up an examination of the conditions in which such enactment of screens occurs. Screenness it can be said, then, is contingent; it is not a stable quality but one that changes as per situations and through the variability of relations. This comprehension of how screens become will allow us, in the last two chapters, to explore conditions where the assumption of the relationship of screens with images and frames is challenged.
It is important to unpack both the material and discursive aspects of practices. Continuing with an agential realist position, discursive practices are also material as they unfold. As Barad emphasises, “In agential realist account, discursive practices are material (re)configurings of the world through which the determination of boundaries, properties, and meanings is differentially enacted.”56 Screens are articulated and materialised differentially in the intra-action of their practices, ideations and materialisations. Thus, rather than taking
55 The DVD release of such a film would be digital, but the distribution and exhibition of such a film would take place largely in cinemas with 16 mm projectors. Among popular films in the last few years, Christopher Nolan’s Dunkirk was shot extensively in IMAX 65 mm, and screened in IMAX, 70 mm and 65 mm theatres. 56 Barad, Meeting the Universe Halfway, 148.
64
knowing (a discursive activity) to be different to making (creating a material), Barad focuses on practices as being material-discursive.57 Material and discursive are not separate but in a reciprocal relationship. She writes:
Neither [material and discursive] is articulated/articulable in the absence of the other; matter and meaning are mutually articulated. Neither discursive practices nor material phenomena are ontologically or epistemologically prior. Neither can be explained in terms of the other. Neither has privileged status in determining the other.58
Moreover, Barad notes that knowing is not an activity limited to humans; instead, it is in and through the “differential responsiveness” of intra-action that the world is known.59 Particular things are known and understood only in specific agential intra-actions.60 So screens have certain meanings, functions and screenness, but these attach to specific practices. In the following section, I will explicate those practices that configure and reconfigure screens and demonstrate how screens come to be differentially bounded by them. Screenness is executed differentially as the intra-acting agencies are separated in and through various ways of doing and acting. Sometimes prior conditions of these possibilities are taken into account and boundary-making practices reiterate particular kind of screenness. For instance, screens displaying images is a common use of screens. This sets up my ongoing enquiry into other practices in subsequent chapters that are also boundary-making practices. But, as I will show, these nevertheless preserve a dynamism and generation of screens that has otherwise gone unnoticed.
57 Orlikowski and Scott, “Exploring Material-Discursive Practices,” 699. 58 Barad, “Posthumanist Performativity,” 822. 59 Barad, Meeting the Universe Halfway, 149. 60 Ibid., 150.
65
Screens in and through material-discursive practices In this section, I attempt to survey only some of the myriad social, scientific, cultural and economic instances or agencies of observation that have extracted a certain performance for or quality of screens. This gives us a few examples of the ways screens have variably emerged via the intra-actions in and of diverse medial assemblages. It is not possible to consider all of the explicit and implicit intra-acting agencies that may have fuelled the specificities of screens. Rather, this section should be viewed as an attempt to illustrate the relations, viewing habits, technological innovations and so forth in which particular screens have developed. This section, then, is to be viewed as a prelude to the material-discursive way to think about screenness in postmedia. Screens have not always been stable, in other words, but have emerged and continue to do so in and through these practices that condition their agential cuts.
To be clear, the various effects of the social, economic and technological impacts on the conditioning of screens and their screenness that I will briefly touch upon centre around the history and materiality of screens understood to be related to image-making devices, viewing habits and social spaces, human physiological deficiency, and surveillance practices. I, on the other hand, am recommending the material-discursive entanglement of practices through the agential realist framework which is a slightly different approach. The latter articulates the enactments of screens and their discursive materialisations differently. The considerations of screens below by Erkki Huhtamo, Lev Manovich, and Wanda Strauven among others account for the interventions in generating specific effects and show how and what we do produces differences. This is not to say that these articulations are not important, they are constructive and valuable in gathering the multiple possibilities of understanding screens. However, the ‘material with the
66
discursive’ approach of material-discursive practice accounts not just for how practices matter in recognising our connections, our situatedness, but also demands us to be mindful of doings by humans and nonhumans. As Barad says, “the point is not merely that knowledge practices have material consequences but that practices of knowing are specific material engagements that participate in (re)configuring the world.”61 This discussion will be followed with the material-discursive exploration of a postmedia assemblage.
Theorists who have undertaken the task of drawing a history of screens in order to account for their changing and developing nature have taken diverse approaches to articulating the relations between social and cultural contexts, and technological innovations. Huhtamo traces the history of screens from older technologies, such as magic lanterns and the kinetoscope. To outline the relationship between screens and its spectators, he proposes the term screenology, describing it as a study of screens as “information surfaces.”62 Huhtamo’s media archaeological approach is helpful in revealing connections that may not otherwise be obvious.
By inspecting pre-cinema devices, Huhtamo strings together a history of screens in media objects, their cultural application and relationship with viewers. He analyses the engagement of the body of the viewer with various kinds of screens, pointing out that objects such as the mutoscope (peep boxes) and zograscope are for individual viewing while panorama paintings and magic lantern shows are for collective viewing.63 Huhtamo acknowledges Charles Musser’s study of screen practices.64 Musser’s concern with the
61 Ibid., 91. 62 Huhtamo, “Screen Tests”; Huhtamo, “Elements of Screenology”; Huhtamo, “Screenology; or, Media Archaeology of the Screen.” 63 Huhtamo, “Screen Tests,” 146. 64 Charles Musser, “Toward a History of Screen Practice,” Quarterly Review of Film Studies 9, no. 1 (1984): 59- 69.
67
archaeology of cinema screen has restricted his analysis to practices and objects that trace a linear history of cinema, such as magic lantern. This way Musser has ignored practices of ‘peeping’ that have contributed to the development of cinema screen.65 Huhtamo identifies three main screen media practices: peep practices, which engage the individual viewer’s body in a frontal pose; touch practices, which have a tactile relationship with the viewers as seen in room camera obscura; and mobile practices, where the viewers are in motion while the observed image or object could be still or moving along with the observer. Two instances of mobile practices can be seen in circular panoramas and dioramas.66
In this way media archaeology provides us with an account of the social and cultural framework where screens can be situated. More importantly, it illuminates various actions and practices that we associate with contemporary screens, such as mobility and touching, are not new. Wanda Strauven finds that pre-cinematic devices such as the mutascope and phénakistiscope were not only about images but also the direct use of the viewer’s hands in moving the image. This “hands-on practice,” according to Strauven, indicates that touching the media was always a part of the viewing experience.67 As Huhtamo notes the tactility of camera obscura, Strauven also illustrates the use of display screens in camera obscura for the purpose of tracing.68
In addition to examining viewing habits of people pertaining to media objects, the focus on pre-cinematic objects also throws light on how viewing spaces
65 Erkki Huhtamo, “The Four Practices? Challenges for an Archaeology of the Screen,” in Screens: From Materiality to Spectatorship — a Historical and Theoretical Reassessment, ed. Dominique Chateau and José Moure (Amsterdam: Amsterdam University Press, 2016), 116–124. 66 Ibid., 117–118. 67 Wanda Strauven, “The Archaeology of the Touch Screen,” Maske und Kothurn 58, no. 4 (2012): 73–74, https://doi.org/10.7788/muk-2012-0405. 68 Wanda Strauven, “The Screenic Image: Between Verticality and Horizontality, Viewing and Touching, Displaying and Playing,” in Screens: From Materiality to Spectatorship — a Historical and Theoretical Reassessment, ed. Dominique Chateau and José Moure (Amsterdam: Amsterdam University Press, 2016), 148.
68
were inhabited. For example, in the nineteenth century, Huhtamo elaborates, shops in parts of Western Europe were fitted out with media objects in order to attract passers-by and prolong their stay, something that is not very different to the neon lights and other screen media attractions in the twentieth century.69 These shop displays can be argued to be screens, where they fulfill the dual functions (as discussed in the introduction) of displaying or showing objects and hiding the unappealing interiors of the shop. Furthermore, it was common to cover the passageways around the shops and convert them into sheltered spaces, blurring the boundaries between an exterior space of a market and the intimate space of a home.70 However, it cannot be asserted that shop windows were simply a marketing opportunity, as the strolling habits of people, a centralised market area and other social practices contributed to making markets a site for perceptual experience.
Such practices in a social and technological setting change media objects and ways of being in the city. A current instance of how social practices and media objects intra-act can be observed in the introduction of new laws due to the habit of talking or texting on the mobile phones while driving.71 Viewing habits and innovation, some argue, have established channels of control through contemporary screens. Sean Cubitt offers a classification of public screens based on the urban environment, the audience and media innovation. First are the television screens in cafes and bars that have the same function as the ones found in homes; second are the huge advertising screens in locations with large numbers of people, such as a traffic
69 Ibid., 118. 70 Walter Benjamin, The Arcades Project, trans. Howard Eiland and Kevin McLaughlin (Cambridge, London: Harvard University Press, 1999), 41-2. 71 Bans on phone use while driving have been difficult to implement due to tinted windows, night time and other factors. But the latest software, which uses artificial intelligence (implemented in the state of New South Wales in Australia in 2019) has been more successful. Verity Truelove et al., “Sanctions or Crashes? A Mixed-Method Study of Factors Influencing General and Concealed Mobile Phone Use while Driving,” Safety Science 135 (2021), https://doi.org/10.1016/j.ssci.2020.105119.
69
intersection or city squares;72 third are handheld screens of phones and iPads, which are highly interactive and intimate.73 Cubitt compares the large advertising screens to posters and small handheld devices to a book.74 He argues the use of big screens colludes with advertising to maintain order in public spaces, whereas in personal and mobile screens advertising can take the form of either public announcements and modify behaviour, or personalised and targeted ads.75
In an attempt to tease out an agential material-discursive method of analysis, I propose to read screens in media objects as not only offering visibility of images and objects to the spectators, but also the visibility of them. Here it may be useful to invoke Michel Foucault’s reading of modes of subjectivity in disciplinary society — a power mechanism that uses “instruments, techniques, procedures, levels of application, [and] targets,” which is managed by specialised institutions such as correctional facilities, schools, prisons, and familial relations and set-ups.76 As “branches of knowledge” and “apparatus of punishment,” they became places that set rules and defined “normal categories.”77 Jonathan Crary argues that these disciplines gave the subject visibility by transforming the observer into a subject, that is, the
72 A renewed interest in urban screens has engaged with ways in which they are employed to remediate public spaces. This interest led to a number of projects and conferences on urban screen, including the Urban Screens movement that started in Europe and Australia in the early 2000s, and held the ‘Urban Screen project’ conference in 2005 in Amsterdam. See Nanna Verhoeff, Jean Dubois, Dave Colangelo, and Claude Fortin, “Disrupting the City: Using Urban Screens to Remediate Public Space,” presented at ISEA, August 2015, Vancouver, Canada; Mirjam Struppek, “Urban Screens — The Urbane Potential of Public Screens for Interaction,” Intelligent Agent 6, no. 2 (2006), http://www.intelligentagent.com/archive/Vol6_No2_interactive_city_struppek.htm. 73 Sean Cubitt, “Large Screens, Third Screens, Virtuality, and Innovation,” in The Oxford Handbook of Sound and Image in Digital Media, ed. Carol Vernallis, Amy Herzog and John Richardson (Oxford: Oxford University Press, 2013), 77-8, https://doi.org//10.1093/oxfordhb/9780199757640.013.023. 74 Ibid., 78. 75 Ibid., 84, 85. 76 Michel Foucault, Discipline and Punish. The Birth of the Prison, trans. Alan Sheridan (New York: Vintage Books, 1995), 215, 216. 77 Ibid., 216, 304.
70
“observer became an object of investigation.”78 Finding faults to regard something as abnormal helped in the construction of homogenous subjects and new technologies to correct their ‘faults’. This also contributed to the production of specific visual practices in the form of new instruments. For example, stereoscopes were a way to evaluate and establish the physiological operation of binocular vision.79 In this way the pre-cinema screen objects examined by Huhtamo are politically situated as modes of subjectivation.
In a different examination of the genealogy of screens, Manovich argues that the window interface of a computer screen has more affinity with graphic design than cinema.80 Situating the history of computer screens in military surveillance, Manovich identifies three kinds of screens: classical, dynamic and radar.81 Several media objects overlap in this classification as it is based on the relationship between the viewers, screens, and the images on-screen: classical screens have a frontal display of static images, such as paintings and computer screens; dynamic screens show images that change over time, such as television, cinema and video; and radar screens, on which the real-time changes in the displayed image can be seen as the media device (radar, or television) scans.82 Manovich’s archaeology of screens is not as extensive as Huhtamo’s and mainly highlights the combination of increased military surveillance since World War II and the development of computer graphics.83 His point of view is more technocentric, which does not account for changes in viewing practices. Rather, the classification focuses on the kind of images that are displayed.
78 Jonathan Crary, Techniques of the Observer: On Visions and Modernity in the Nineteenth Century (Cambridge, MA: MIT Press, 1990), 16. 79 Ibid. 80 Lev Manovich, “An Archaeology of a Computer Screen,” in New Media Logia/New Media Topia, ed. Edited by Irina Alpatova (Moscow, Soros Center for the Contemporary Art, 1996), 6. 81 Ibid., 3–9. 82 Ibid. 83 Ibid., 8–12.
71
I now turn to how physical or material practices have considered screens in relation to notions of presence, hypermediacy and immediacy.84 Screens may have been used as a display and projection object, but in a few instances they were meant to be hidden, such as in phantasmagoria, where the edges of the screens were concealed. As Huhtamo describes, the eighteenth century phantasmagoria was one of the earliest instances of screens emerging into the public realm.85 Several techniques were used to make the screens invisible, such as making them wet, and painting their frames black to collapse the boundary between the viewing space and the outside world. A less discernible edge was thought to make it easier to ignore the presence of screens, leading to increased immersion in the images. While watching the ghosts and monsters, people were either not aware of the screen or chose to ignore it, willingly surrendering to immersion in the spectacle. In addition to a framed screen, a thick cloud of smoke was often used in phantasmagoria as a projection material.86 This, accompanied by ghostly music, added to the frightening spectacles common to phantasmagoria shows.
Another instance of immersion via dissolving the frame is seen in the practice of producing very large frames that engulf viewers. In this instance of immediacy, the popular panorama paintings and moving panoramas of the late eighteenth and nineteenth century produced an immersive view engulfing the spectator’s field of vision. This provided a “heightened sense of embodiment,” and made them one of the earliest examples of large screens in public.87 The arrangement of different yet framed objects, images, and sounds
84 Hypermediacy indicates an awareness of the medium, whereas immediacy renders the medium transparent. See Jay David Bolter and Richard Grusin, “Immediacy, Hypermediacy, and Remediation,” in Remediation: Understanding New Media, ed. Jay David Bolter and Richard Grusin (Cambridge, MA: MIT Press, 2000), 20–51. 85 Huhtamo, “Elements of Screenology,” 35. 86 Castle, Terry. "Phantasmagoria: Spectral Technology and the Metaphorics of Modern Reverie." Critical Inquiry 15, no. 1 (1988): 33, http://www.jstor.org/stable/1343603. 87 Alison Griffiths, Shivers Down Your Spine. Cinema, Museum and the Immersive View (New York: Columbia University Press: 2008), 37.
72
on-screen, can create an immersive experience for the viewer. Admittedly, viewing practices construct screens alongside immersed viewers. Framing practices will be discussed in detail in chapter three, where I consider a temporality of and for frames. There, I will take a closer look at practices that define frames, the out-of-frame and screens in particular ways.
The physical composition of screens was also highlighted in dioramas. Louis- Jacques-Mandé Daguerre gave instructions as to the type of cloth and lighting to be used in dioramas, which could be seen as proto-cinematic spaces in the mid-nineteenth century.88 Dioramas consisted of two paintings lit from the front and back. Daguerre instructed the use of either percale or calico for this purpose. In terms of lighting, he suggested lighting the first painting from the front or top and the second from the back, at least two meters away from the fabric.89 The dioramas, along with panoramas, were black-boxed and moved mechanically.90 However, even here we see that it is not just the type of cloth but the intra-action between surfaces and other material elements such as lights and music that formulate screens and their capacities for their viewers. Continuing the textile materiality of screens in the context of contemporary installation, Giuliana Bruno imagines the covering of screens as “surface tension.” She considers screen surfaces not through their material composition, but likens them to a fabric where “the architecture of light can be reflectively sensed in surface tension.”91 Since screens as flat surfaces are able to add depth and dimension to images, Bruno argues that the light and projection create architectural structures on
88 Helmut Gernsheim, L.J.M. Daguerre: The History of the Diorama and the Daguerrotype (New York: Dover, 1968), 43. 89 Louis-Jacques-Mandé Daguerre, “Description of the Process of Painting and Lighting in…Pictures of the Diorama (1839)” in The Screen Media Reader: Culture, Theory, Practice, ed. Stephen Monteiro (New York: Bloomsbury, 2016), 163–166. 90 Wolfgang Schivelbusch, Disenchanted Night: The Industrialization of Light in the Nineteenth Century (Berkeley: University of California Press, 1995), 218-19. 91 Giuliana Bruno, Surface: Matters of Aesthetics, Materiality and Media (Chicago, University of Chicago, 2014), 60.
73
screens.92 Such ideas highlight the luminous and structural emergence of screens and the functions of screens other than that of display.
A material-chemical compositional analysis of screens has been done by Cubitt. Cathode ray tubes (CRT) were commonly used in televisions, computers and gaming consoles during the twentieth century.93 Cubitt’s analysis of the material composition of screens highlights their chemical composition and its damaging effects on the environment. He comments that phosphor was used in CRTs because of its capacity to illuminate and fade quickly. Although it is a toxic material, it was used extensively in televisions and computers.94 Liquid crystal displays (LCD) have since replaced CRTs but are also made of a number of toxic materials such as liquid crystal and borosilicate or silicon dioxide glass. Moreover, LCD and LED (light emitting diode) screens add to the growing problem of e-waste disposal and management.95 Here we can see ways in which environmental intra-actions urge a rethinking of the materials of screens.
These analyses bring together various fields and practices, opening up numerous intra-actions and entanglements in and through which screens are produced. I have tried to reconstitute an iterative understanding of screens out of the historically and culturally specific intra-actions of material- discursive practices, for instance, taking into account the materials used in constructing the screen, dispersing with its material presence and its use in practices of subjectivation. However, we have to keep in mind that screens are not simply to be regarded as an object, with inherent properties. Indeed, the materialisation of screens takes place, such as the cotton cloth in
92 Ibid., 63. 93 Sean Cubitt, “Current Screens,” in Imagery in the 21st Century, ed. Oliver Grau with Thomas Veigl (Cambridge, MA: MIT Press, 2011), 40. 94 Ibid., 40, 41. 95 Ibid., 42.
74
dioramas, LCD and CRT screens in television and computers; but they are enactments of particular material cuts along with and are part of the discourse, such as making the screens invisible for a terrifying experience in phantasmagoria materialised in screens that were hardly noticeable.
Keeping these articulations of screens in mind, it is important to consider the enactment of screens in a postmedial setting that reconfigures relations and sees new potentialities for screens emerging. One way to examine intra-active agencies in postmedia, I propose, is by focussing on how the openness of screens (their becoming, which postmediality intensifies) highlights their propensities to form and/or mediate conjunctions and relations. Screenness, I want to argue, is an enactment whereby screens are not only configured in a particular way but is also a making visible of relations and temporalities that only appear along with the assemblage. There is, then, no a priori screenness.
The next section interrogates the entangled intra-actions that constitute screens in a postmedia assemblage. The enactment of screenness will be illuminated by analysing particular intra-actions of screens as entangled agencies. The confluence of practices, materials, stagings, and networking in postmedia requires a closer examination of the ‘doings’ of humans and nonhumans, and relations that are sustained in a postmedia assemblage. These boundary-making practices provide insights into executions of screenness in different conditions and how screens become. Moreover, emerging entities and practices co-constitute particular kind of screens, or in other words, specific cuts enact screens. The analysis also pays attention to the differential constitution of screens in postmedia due to agential cuts produced by specific material-discursive practices. Here everything is accountable to the practice of meaning making.
75
INTRA-ACTIONS OF CONTEMPORARY SCREENNESS In the following section, I will discuss the intra-action of screens and other human and nonhuman entities. In doing so I will unpack an example that demonstrates the emergence of screenness out of entangled agential relations. Here entities emerge as ‘screen’ (or any other medial entity) through the agential cuts enacted by specific material-discursive practices. In my analysis of the complex intra-actions of postmedial assemblages, I will consider intra- actions of and with screens that not only generate locatability and movability, but also reconfigure time and space. Screens of mobile phones, I will demonstrate, when navigating using Google Maps, are separated from the intra-actions of hardware and software of the smartphone in this mobility. Hence, any analysis will have to consider screens, smartphones and apps, along with digital computation and algorithms that comprise the intra-active agency of such an assemblage.
For this discussion, I am considering a smartphone, which is described as a phone with computing speed comparable to most laptops and desktops, with access to the internet. They have touchscreens, which means that human bodies can input to this assemblage directly by touching the screen. The diagrams below (fig. 7 and its detailed version fig. 8) illustrate the particular instance of entangled relations that constitute the act of human navigation using the mobile phone app, Google Maps. It shows some of the important relations that screens are not just caught up in but also enact. The discussion will elaborate screenic entanglements that produce new kinds of spatio- temporal experiences, while also highlighting new forms of digitality.
76
Figure 7: Mapping the basic components of a mobile phone assemblage while navigating on foot in an urban location.
77
Figure 8: Details of the components and some of their relations in a mobile phone assemblage while navigating on foot in an urban location.
This particular postmedia assemblage of mobile phone navigation is commonly deployed when moving through and exploring areas in cities and semi-urban locations. The Google Maps application is used widely by people as well as transport and food delivery services such as Uber and Deliveroo. This highlights the intra-actions of screens in viewing, navigating, finding and locating. In this set-up, screens function between the human body and a smartphone’s communicational capacities. As shown in in figure 8, the phone can be divided into its hardware, which mainly consists of sensors, motherboard, silicon chips and battery; and its software, comprising of its operating system (OS), which is commonly either Android (Google) or iOS (Apple), and its applications. Applications or apps provide a range of
78
functions and activities for the mobile phone user. Describing the smartphone’s “layered interface,” Nanna Verhoeff identifies three general layers of screen navigation that are connected to each other and not easy to separate: “internal interfacing,” where the back end of the app/software is included, for example, the application programming interface (API) through which two pieces of software are able to merge; “spatial positioning and connectivity,” where all the internal hardware linked to the outside world is included, for example, wi-fi connectivity; and “user interaction,” which allows for exchanges between the first and second level, and accounts for the user’s inputs and actions in combination with the other two levels.96 While these layers have been identified as functioning together and their assimilation important for the process of navigation by Verhoeff, they however are “filtered by the user interaction interface,” at the level of the screen.97 Although this highlights screens merely as a communicative device, the layers described by Verhoeff are produced in and through material-discursive practices. Therefore, some of these ideas are important to this discussion. They will be elaborated below by focusing on the entangled agencies through which entities, including screens, are unfolding in the particular instance of navigation.
Explication of these various phenomena will aid the analysis of the agential cuts that separate several entities in this assemblage. The agencies of sensors,98 screens and the user are phenomena in the agential realist sense as they are entangled and do not easily allow separation between components.99 Specific agential intra-actions define each of these entities and their material
96 Nanna Verhoeff, Mobile Screens: The Visual Regime of Navigation, 150, 151. 97 Ibid., 152. 98 Some of the several sensors in a smartphone include accelerometers, gyroscopes, microphones, cameras, magnetometers, barometers, pedometers, light sensors, and humidity sensors. 99 Barad, Meeting the Universe Halfway, 139.
79
articulation.100 For instance, the intra-acting agencies ‘mark’ the user’s body — that is, they separate the user out as agent — by generating clues to their location. The location button has to be enabled in order for the phone to be detected. There are numerous ways to find out the location of a phone, including through the mobile service provider, Wi-fi and the Global Positioning System (GPS).101 With the growing focus on data privacy, users do not always enable location tracking. However, this does not mean that a phone cannot be located; rather, covert ways can be used to ascertain the location, such as the use of numerous phone sensors.102 Additionally, consider the phenomenon of the human hand touching the screen — typing on the on- screen keyboard can produce minute tilting of the gyroscope and sounds that can be captured by the stereoscopic microphones inside the phone.103 These two sensors have been employed to determine texts typed on the phone as the gyroscope tilts away from the side of the keyboard that is touched and the microphone captures the loudness and the time taken for the sound to reach it.104 Here the cut that separates the screen out as an entity that is differentiated from the touch of the human hand is enacted by the gyroscope and the microphone as they recognise the tilts and sounds of different kinds of touching. While it can be argued that the screen is an extension of the hand when the hand is touching it, here the very act of touching — due to the
100 Ibid. 101 In smartphones, location is not entirely ascertained using GPS, but uses Assisted GPS (A-GPS), which incorporates a number of additional methods, such as service tower information. 102 One way to infer the location of the phone is to collect gyroscope and accelerometer data from the phone of the person who is travelling. With the compass of the phone determining the direction of travel, the gyroscope measuring the tilt at each turn and the accelerometer measuring if the phone is moving or stopping. All this information can be assembled to possibly identify the route taken. 103 An experimental app was developed by the researchers at Northeastern University (USA) using smartphone sensors, specifically the gyroscope and the stereoscopic microphones, instead of the screen, to find out what the user was typing. See Sashank Narain, Amirali Sanatinia and Guevara Noubir, “Single- stroke Language-Agnostic Keylogging using Stereo-Microphones and Domain Specific Machine Learning,” in WiSec ’14, Proceedings of the 2014 ACM Conference on Security and Privacy in Wireless & Mobile Networks (Oxford, UK, 2014): 201–212. 104 The experiment utilised the amplitude of the audio signals as well as the time delay of the audio reaching the microphone. See ibid., 204. For the details on the process of filtering, extracting and sampling of the audio signals captured by the microphones, see ibid., 205.
80
entanglement of other entities such as sensors — separates the hand and the screen. Furthermore, the practice of filtering sound and tilt data to be recognised as alphabetic characters is another complex process of making agential cuts enacted by other practices of data collection.
However, the sensors are apparatuses in the Baradian sense in that they have been produced through other specific (historical or genealogical) material- discursive practices and materialised via intra-action with other apparatuses. As Barad explains, “Apparatuses are not pre-existing or fixed entities; they are themselves constituted through particular practices that are perpetually open to rearrangements, rearticulations, and other reworkings.”105 For example, the idea for a gyroscope, similar to several other mechanical instruments, emanates from toys and gaming devices.106 Gyroscope originated in the spinning top that could maintain an upright position while moving at high speed. But it was not until the late eighteenth and early nineteenth century that gyroscopes were used as instruments for navigation.107 Numerous variations of the gyroscope are used in a range of fields such as aeroplane navigation, astronomical telescopes, mining and smartphones. These iterations have developed over time from several practices such as design and engineering compulsions, manufacturing material that is subject to market forces and economic capital.
As discussed through the work of Huhtamo and Manovich in the previous section, it is important to consider insights from older practices, which indicate how contemporary objects have come to be used in particular ways. Combining this approach with Barad’s attention to material-discursive
105 Barad, Meeting the Universe Halfway, 203. 106 Lewis Mumford, Technics and Civilization (London: Routledge and Kegan Paul, 1955), 101. 107 Giuseppe Iurato, “On the Historical Evolution of Gyroscopic Instrumentation: A Very Brief Account,” HAL archives, 2015, https://hal.archives-ouvertes.fr/hal-01136829; Ljiljana Veljović, “History and Present of Gyroscope Models and Vector Rotators,” Scientific Technical Review 60, no. 3–4 (2010): 101.
81
practices helps in thinking about the enactment of agential cuts through which objects, along with their boundaries and properties, are produced. Surely material-discursive practices are ongoing processes that materialise phenomena differently.108 This is evident in the multiple ways in which gyroscopes are used, each depositing practices and prior intra-actions. Here space and time do not pre-exist the assemblage. They are also produced in and through intra-activities and rearranged in the construction of material- discursive boundaries.109 Below I analyse the ways in which multi-layered spaces are produced on and through mobile phone screens and how navigation activity produces screenness or a performance of screens.
An application such as Google Maps offers information about a particular location — place, street, city, country — ranging from traffic conditions, route planning, geographic details, distance, and available transport, which can be visualised via satellite images, Street View or simple graphic street maps.110 Additionally, the app provides information about the locations of restaurants, cafes, hotels, shopping centres, galleries and museums (often accompanied by reviews of these locations) and services such as ATMs, service stations and post offices, among other facilities. It is not just a geographical map that can be used to get from one location to another, but also offers a culturally specific navigation of a place. For instance, in India Google Maps can be accessed in ten regional languages and provides the option of landmark-based navigation in a number of cities, highlighting their historical monuments while users move through the city.
108 Barad, Meeting the Universe Halfway, 234. 109 Ibid., 180, 234. 110 Google Maps is also a website. It provides the same level of engagement as the mobile phone app.
82
The specific information seen on the screen is collected in a number of ways. In the intra-active map of navigation within a postmedia assemblage above (fig. 8), I have divided the information into two broad methods of syndicating data in the application: crowdsourced information and satellite-based sources. However, there is no clear-cut distinction between these sources since some crowdsourced information utilises satellite images. Google Maps also uses a range of other methods for gathering information: traffic conditions input directly into the app; the estimation of traffic at a given location by collecting information from the number of smartphones that have location services enabled; the provision of information to the app from organisations such as government transport institutions and airports (which have their own means for collecting data); petrol and diesel prices direct from distributors; and private businesses, such as shops and restaurants, who furnish details about their services and even supply interior maps of their spaces — these in turn could be rated and written about by people who have used their services. In many of these provisions of data, screens are used to input as well as display the information.
The practice of crowdsourcing data in the way described above signals changing relations in a social ecology. In its intra-actions, it can be argued that in these instances screens are constantly reconfiguring the possibilities of relations that people can have with places and other people. Describing postsocial relations, Karin Knorr Cetina and Urs Bruegger argue that screens are able to aggregate relations, bringing humans in contact with each other as well as generating feelings and experiences.111 In their analysis of the Swiss foreign exchange market, Knorr Cetina and Bruegger remark on the “self- historicizing” properties of screens when they show price histories,
111 Karin Knorr Cetina and Urs Bruegger, “Traders’ Engagement with Markets: A Postsocial Relationship,” Theory, Culture & Society 19, no. 5–6 (2002): 163–4.
83
transaction histories and electronic programs underneath.112 In the postmedia assemblage, the intra-actions of people operating and engaging via screens not only takes place in the present but shows up as historical events (previous intra-actions). In this way, the current entangled agencies of/for screens are not just constituted in but are instead configuring space and time.113 Moreover, in Google Maps, the on-screen map is constantly altered as data from various users gets updated. There is a mutual reconfiguring of space-time and image on the screen. This coincidence of on-screen and off-screen spaces is termed screenspace by Verhoeff.114 With data being inputted from numerous sources, the screenspace constitutes multi-layered and multi-temporal intra-activity.
Screens are part of the wider network that includes various social and material practices that assemble information and relations for this one app. Highlighting information and alliances that were hidden in earlier technologies such as telephones has already been attributed to screens, as Knorr Cetina and Bruegger have noted.115 The accessibility, transferability (such as sharing location) and scalability (zooming in and out of an area, and increasing or decreasing the amount of information that can be seen) of the map is enacted by screens. Screens, then, do not merely present a reality that is out there but are co-constitutive of it.116 This is a co-constitution of the agency of screens with other agencies executed in various practices, such as sharing experiences of a place and customising maps.
The performativity of screens is very much a part of the space that is produced by the user as they move along a location while new information is
112 Ibid., 167. 113 In agential realism, space-time does not exist prior to the intra-actions but is produced and iteratively reconfigured in them. See Barad, Meeting the Universe Halfway, p 179. 114 Verhoeff, Mobile Screens, 150. 115 Knorr Cetina and Bruegger, “Traders’ Engagement with Markets,” 163. 116 Ibid., 166.
84
being aggregated on the screens in the intra-action of software, hardware and human bodies. While attention to this kind of networked mobility is not new, the role of screens is a recent focus.117 Verhoeff’s concept of performative cartography is of use here. Performative cartography is a particular kind of navigation where maps are made collectively. As Verhoeff explains, “fixed maps do not dictate the itinerary but rather maps and views evolve and emerge along the way. In performative cartography there is a timespace collapse between making images and viewing them.”118 This kind of mobility that co-creates the map of movement, Verhoeff argues, has been made possible by screen-based interfaces.119 In the making and displaying of space, screens also emerge in this performance. To illustrate this, let us go back to the Google Maps mode of navigation where the phone’s location service is turned on.
When the location signal has been turned on by the user in order to commute from one location to another, the physical location of the user on the smartphone screen offers various perspectives.120 By default, it shows a bird’s eye view;121 and as discussed above, allows a lot of information about the area
117 Karlis Kalnins coined the term locative media, see Drew Hemment, “Locative Arts,” Leonardo 39, no. 4 (2006); Marc Tuters and Kazys Varnelis, “Beyond Locative Media. Giving Shape to the Internet of Things,” Leonardo 39, no. 4 (2006): 357–363. Tristan Thielmann’s geomedia is a combination of locative media and media localities, see Tristan Thielmann, “‘You have Reached your Destination!’ Position, Positioning and Superpositioning of Space through Car Navigation Systems,” Social Geography 2, no. 1 (2007): 63–75; Tristan Thielmann, “Located Media and Mediated Localities: An Introduction to Media Geography,” Aether 5 (2010): 1–17. Eikens and Morrison proposed the term navimation, which brings animation and navigation together to define “the intertwining of visual movement with activities of navigation”; see Jon Olav H. Eikenes and Andrew Morrison, “Navimation: Exploring Time, Space & Motion in the Design of Screen- based Interfaces,” International Journal of Design 4, no 1 (2010): 1–16. A significant voice in the study of locative media is Jason Farman, who interrogates the socio-cultural shifts that delivered by the mobility of media. See Jason Farman, Mobile Interface Theory: Embodied Space and Locative Media (London; New York: Routledge, 2012). 118 Verhoeff, Mobile Screens, 145. 119 Ibid., 13. 120 Other ways to ascertain the location of smartphones include the ‘network-based’ technique, which determines location as phones connect to their service provider through the closest station/tower. Wi-fi positioning systems can also be used to locate devices via Wi-Fi hotspots. However, this works in conjunction with the GPS. 121 The turns can be viewed in street view. Augmented reality views are available on phones such as the Google Pixel 3 handset.
85
of navigation. As the map affords scalability, the screen provides an overview of a larger space, far more than the immediate surroundings. The user can see their body in relation to places five kilometres or ten kilometres away, or even another city, while they intra-act with the blue dot to orient their body. The user’s position as blue dot on the screen not only embodies them but is also part of the signaletic processes that are taking place together alongside the movement of the human body. This is a phenomenon in the agential realist sense, where multiple agencies —satellite signals, microchips and receivers in the phone, software code and the body of the user — intra-act. In this intra- action the impossibility of discerning which entities are (separately) working together is evident from the brief description below of the technical process of finding the location of a mobile phone using the Global Positioning System (GPS) and Geographic Information Systems (GIS).
Global Positioning System (GPS) receivers are present in smartphones.122 They generally collect information from four GPS satellites to determine their location on Earth, with an accuracy of a few meters.123 GPS receivers do not need to be connected to the internet to receive location, but in the Google Maps app it is important to be networked to update the map as the user
122 GPS is the commonly known version of the system, NAVSTAR GPS, owned by the US government. This means that the US government can potentially deny this service to any party. Due to the American monopoly of the GPS, other countries have developed their own navigation and positioning systems, such as GLONASS (Russia), BeiDou Navigation (China), NVIC (India) and the Galileo global navigation satellite system (EU). Moreover, in smartphones, there is Assisted GPS (A-GPS), which draws assistance from other sources, such as the signal form the network provider, to get the position of mobile phones. 123 Out of the four satellites, three zone in on the position of the receiver on Earth, while the fourth helps to correct time deviations of the satellites with respect to the GPS receiver. The time deviation is very important, as each satellite broadcasts its time and position. Each satellite has an atomic clock that is synchronised with Earth time. Depending upon when the signal from the satellite was received, the distance of the satellite from the receiver can be gathered (the delay in receiving the time signal multiplied by the speed of light will give the distance between the receiver and the satellite). The distance between the receiver and the satellite and the position of the satellite can help us in working out the receiver’s position along a possible course. As more of these possible lines of position are gathered from other satellites, the position is determined with an accuracy of up to 10 to 15 meters. In this massive system, there is a lot of opportunity for errors such as the delay between the clocks on the satellite and on earth, and disturbances in the ionosphere, an area in the earth’s atmosphere that has free electrons, that can cause deviation and week number rollover.
86
moves along.124 The connection to the internet is provided by the mobile phone network provider as wireless internet access (commonly known as data services).125 In other words, GPS gives the location of the smartphone and the wireless internet provided by the mobile network is able to then transmit the location and receive an update on the movement of the user. This physical movement of the person is seen as the moving blue dot. This process becomes more complex when other methods of location are combined, such as service tower information. Moreover, the GPS intra-acts with Geographic Information Systems. Since their first use in the 1960s to conduct land resource inventory, these systems have developed into frameworks, techniques and processes that collect and evaluate geographical and related data (for example, meteorological data). One of their most significant and widespread uses is in the analysis of data of a location over a period of time; for example, a GIS is used in navigation services such as Google Maps on a mobile phone, where it visualises the map of a particular location. Simply put, a GIS interprets the data provided by GPS and other systems.
In order to understand the various agential cuts made by material-discursive practices in this set-up, it is important to acknowledge the complex intra- actions of humans, nonhumans, signals, matter, networking, and coding. The satellite signals are agentially cut when they are interpreted as information by GPS receivers. This information is translated and stored in a standardised way by the GIS programs, making several more agential cuts. Moreover, through specific practices of coding and design in the OS of the phone and Google Maps, humans enact agential cuts on the intra-acting agencies, that is,
124 GPS is not just used in navigation but there are other uses as well such as farming, the financial sector, and games (e.g. Pokemon Go). 125 Higher speeds of internet, commonly known as 2G, 3G and 4G, use different transmission technologies. For example, 2.5G uses GPRS (General Packet Radio Service), 4G uses HSPA + (Evolved High Speed Packet Access), seen as “H+” on Android smartphones.
87
on the already existing connections, to produce a specific materialisation such as the navigational set-up of Google Maps.
In this assemblage, screens are demarcated both by the human body and the smartphone. The marking of the screen by the human body is not just through touching it, as discussed earlier, but also through the user moving as/with the blue dot on the map. There is a combination of mobility and tracking in the relation between the two bodies, the physical body of the human and the on-screen blue dot. While these two may appear to converge — an actual physical person and their on-screen representation — each has its own set of networked relations while still being connected to each other. Similarly, the screen is separated by the software processing of the app as well as the hardware on the phone, such as the current that lights up the pixels with any change of image on the screen.
Such navigation that is dependent on screens and software-hardware entanglement is also continuously reconfiguring on-screen and off-screen relationships. While the on- and off-screen are co-produced in their entanglement, their enactments can generate patterns or prompt certain quantifiable behaviours that can then in turn be used for prediction.126 On Google Maps, a road with a red line is understood to be congested with vehicles, which can prompt a change in the route of others planning to move in that direction. However, this relationship can be problematised when these two seemingly separate spaces, on-screen and off-screen, are made indiscernible in the screenspace. In the performance installation Google Maps Hack (2020) Simon Weckert took ninety-nine smartphones with their location
126 Nanna Verhoeff and Heidi Rae Cooley, “Performativity/Expressivity: The Mobile Microscreen and its Subjects,” in Technē/Technology: Researching Cinema and Media Technologies – Their Development, Use, and Impact, ed. Annie van den Oever (Amsterdam: Amsterdam University Press, 2014), 212.
88
turned on. He carried them in a handcart on several streets of Berlin (fig. 9) which turned them red on Google Maps, indicating heavy traffic on those roads.127 However, that traffic congestion could only be seen on Google Maps; there was a ‘traffic jam’ without vehicular traffic. Surely, this was not a “virtual traffic jam” as some described it.128 There was congestion as per the GPS signals from the phones, but not of vehicles. The assumption of the red colour denoting increased number of vehicles or people was misleading as the traffic was that of phones, not cars. So, the phones were present, albeit in a handcart and not close to a driver driving a vehicle as one would assume. While commuters navigating with their phones were forced to rethink their course and look for alternative routes, this in turn may have increased traffic on other routes. The on- and off-screen relation here is problematised as the presumed correlation fades or is disrupted. The screenspace is not simply a performance of on- and off-screen relationships but a deliberate blurring of those boundaries. The space is not a physical bounded area but one that is ‘framed’ by the phone screens. Questions around how screens emerge in such postmedial conditions of on- and off-screen entanglements is of interest of this research. This will be undertaken by interrogating ways in which artists have expanded the gestures of screenic performances in postmediality.
127 Weckert points out that that movement was necessary for this work to function. Since the app constantly updates, they had to go up and down the street a few times before the road turned red on the Map. Moreover, if a car went past them the map showed the street as green and not jammed. See Sebastian Eder, “This Reality Does Not Exist,” Frankfurter Allgemeine, 2 March, 2020, https://www.faz.net/aktuell/stil/trends- nischen/berliner-kuenstler-simon-weckert-hat-google-maps-ausgetrickst-16615421.html. 128 Design Boom, “Simon Weckert Causes a ‘Virtual Traffic Jam’ with 99 Smartphones and a Handcart,” 3 March, 2020, https://www.designboom.com/art/simon-weckert-virtual-traffic-jam-99-smartphones-handcart- 02-03-2020; Jessica Stewart, “Artist Creates Virtual Traffic Jam on Google Maps With a Wagon Full of 99 Smartphones” My Modern Met, February 17, 2020, https://mymodernmet.com/google-map-hacks-simon- weckert; Ben Schoon, “Google Maps ‘hack’ uses 99 Smartphones to Create Virtual Traffic Jams,” 9To5Google, 4 February, 2020, https://9to5google.com/2020/02/04/google-maps-hack-virtual-traffic-jam.
89
Figure has been removed due to copyright restrictions.
Figure 9: Simon Weckert, Google Maps Hacks, 2020, performance and installation. Two stills documenting the performance. http://simonweckert.com/googlemapshacks.html.
In this chapter, I have considered the emergence of screens in the postmedia assemblage through and in material-discursive practices. Investigating the intra-actions in the assemblage, I have articulated how screens are configured in and through material-discursive practices. It is important to be attentive to where different kinds of screens emerge and to how screenness is executed. Here, I have paid attention to screenness as the capacity to enact specific relations that are particular to practices; that is, the entangled agencies in an assemblage are differentiated in and through specific practices. Screens emerge in the process of making and differentiating in intra-actions. I illustrated how moving image production practices, viewing practices, and the materiality of screens have produced screens with different capacities. Taking this into account, the postmedial condition offers a dynamic engagement of and with screens as they configure that cannot be determined prior to the assemblage and its relations.
The next chapter continues to build on the proposition for the enactment(s) of screens by attempting to understand the ascription of this enactment. The performance of screens entails its ability to continue and reconfigure the
90
relations of the assemblage, then the question becomes how or what does that entail? In postmedia, while one medium can exist within another, the specificity of the medial elements is not completely lost. As I have shown in this chapter, screens are able to maintain their properties, which then make them part of cinematic, mobile, and networked media but may also reconfigure and/or maintain the relations of those assemblages. I propose the concept of gesturality in the next chapter to articulate just this ongoingness of mediality that screens help to provide.
91
2. GESTURALITY OF SCREENS
Gestures and emoji don’t break down into smaller parts. Nor do they easily combine into larger words or sentences (unless we’re using a clunky version of the grammar of our language).
While there are preferences, there is nothing “grammatical” about using � instead of �. Rather, what is most important is context. � could be a reference to your own dog, a good dog you saw while out for a walk, or a sign of your fondness for puppers over kitties.
— Lauren Gawne, Emoji Aren’t Ruining Language: They’re a Natural Substitute for Gesture ���1
By drawing a comparison between gestures and emojis, Lauren Gawne illustrates how emojis, not unlike gestures, bear meaning and that only develops out of specific extralinguistic contexts. Hence, she argues that emojis are similar to gestures. Emojis, then, are substitute words that compact sentences, making efficient use of letters and reducing the effort that goes into typing them. Since emojis are purposefully deployed in place of words, their use replaces the linguistic with a different mode of exchange. However, emojis do not merely substitute for language but, as Gawne demonstrates, can be used alongside words to emphasise a point in the same way gestures are used in conjunction with speech. Usually, gestures — movement of hands, body and facial expressions — have been understood as subordinate to speech, although their use, either instead of words (paralinguistic) or to emphasise them, has been of interest in studies of human communication.2
1 Luren Gawne, “Emoji Aren’t Ruining Language: They’re A Natural Substitute For Gesture ���,” The Conversation, July 5 2019, https://theconversation.com/emoji-arent-ruining-language-theyre-a-natural- substitute-for-gesture-118689. 2 See David McNeill, Hand and Mind: What Gestures Reveal about Thought (Chicago: University of Chicago Press, 1992); P.E. Bitti and I. Poggi, “Symbolic Nonverbal Behavior: Talking Through Gestures,” in Fundamentals of Nonverbal Behavior, ed. R.S. Feldman and B. Rime (New York: Cambridge University Press, 1991), 433–456; Adam Kendon, Gesture: Visible Action as Utterance (Cambridge, MA: Cambridge University
92
However, this meaning-finding conceptualisation of gestures impedes their medial understanding. What I will contend in this chapter is that a medial understanding of the gestural places it in the middle of speech and language, of bodies and movement, of image and meaning. It is this expanded understanding of the medial, as not simply a channel but a middling that I want to extend to gestures, and which, I will argue, the gestural brings to an understanding of media. In postmedia assemblages, the idea of mediality sustained via gesturality helps in comprehending how relations are configured, rearranged and sustained. These operations are central to postmedia assemblages where different kinds of media are transmutating. In this chapter as I analyse specific artworks, my enquiry will focus on how the key issue for screen gesturality is the movements generated to sustain mediality in postmedia. This gesturality of and by screens is specific to the condition of postmedia due to the (intermedial) position of screens in its assemblage as well as their use in various media and stages of production, presentation and dissemination (intramedial). The gestures of screens take on specific enactments in postmedia.
This consideration of the gesturing screen builds on Agamben’s concept of gesture. Gesture, he says, “is not the sphere of an end in itself but rather the sphere of a pure and endless mediality,” that is, a “means without an end” where mediation is not goal-oriented but sustains action.3 This chapter develops a framework for a gesturality of screens by proposing that Agamben’s shift to ongoingness can also help us understand the postmedia condition and the way in which screens are enacted in different technical set- ups. The term gesturality refers to the capacity of an entity to perform diverse
Press, 2004); Simon Harrison, “The Production Line as a Context for Low Metaphoricity: Exploring Links Between Gestures, Iconicity, and Artefacts on a Factory Shop Floor,” in Metaphor in Specialist Discourse, ed. J. Berenike Herrmann and Tony Berber Sardinha (Amsterdam: John Benjamins Publishing, 2015), 131–61; David McNeill, Gesture and Thought (Chicago: University of Chicago Press, 2008). 3 Agamben, “Notes on Gesture,” 59.
93
kinds of gestures, albeit not always implemented on all occasions. For example, watching a video on a smartphone asks for a different engagement with screens than photographing a person with the smartphone’s camera. Here I am not simply indicating different actions that are made by users, but instead want to direct our thinking towards the different relations screens maintain in these actions.
Agamben’s idea of gestures as the continuation of activities allows me to focus on how screens are able to sustain actions or movements while being part of a postmedia assemblage, where media is itself unstable or undergoes incessant processes of transformation. By attending to the maintenance of mediality, we are no longer concerned with how media themselves perform or enact phenomena. Instead, we might see how they contribute to a condition, atmosphere or ecology. Gesturality, then, can be argued to be a complex situation where intra-acting agencies are being sustained alongside the demarcations that I argued were a result of material-discursive practices in the last chapter. Furthermore, I will discuss how the intermedial travels of screens aid their gesturality. When examining in detail the gesturality of screens in both this and subsequent chapters, I will attempt to unravel how screens enact movements that make them intermedial; that is, they are able to (re)configure relations between media. At the same time, the maintenance of mediality by screens highlights the potentiality of screens as not simply receptacles but as technical elements with agency (screenness) in a complex and shifting medial assemblage.
Taking into account the understanding of gesture as provided by Agamben, this chapter will continue its investigation of Shilpa Gupta’s Shadow 3 (2007) and will also look at Ulrike Gabriel’s Breath (1991–92). I will examine how in these works the gestural agency of screens lies in preserving continuing
94
actions, which are open-ended. To arrive at this, I will be tracing two related discussions in screen studies and media arts scholarship on gestures and screens: the meaning ascribed to gestures, especially filmic gestures; and the location of these gestures. In order to explore the meaning of gesture in a postmedia screen context, I will be moving away from a representationalist paradigm, which focuses on descriptions and meaning making, and instead will be examining the doings, practices and conditions of production of what is happening on and with screens.4
I will look towards the ambivalence and multivalency of screen gesture. This is also accompanied, especially in the postmedial context, by an inability to definitely identify which medial entity is performing a given gesture or gestures. Here the postmedia context of media fluidity and instability catalyses movement across and within media and other technical assemblages. Hence, the question of where gesture is to be located — at the site of the screen, image or interactant, for example — also needs to be discussed. This chapter does not attempt a typology of screens or other gestures (for example, those performed by agents normally ascribed intentionality, such as humans). Instead it flags a range of gestures so as to demonstrate how their performativity can be grasped across a range of (technical and human) entities.
Screens, I will argue, facilitate a kind of gesturality in postmedia, which both sustains connections and performs their redistribution. I begin this
4 Representationalism raises questions about who is representing whom, and how and why. Ideas around representation have been challenged in fields such as poststructuralism and feminism, including Barad’s concept of agential realism, where representationalism “marks a failure to take account of the practices through which representations are produced. Images or representations are not snapshots or depictions of what awaits us but rather condensations or traces of multiple practices of engagement.” Barad, Meeting the Universe Halfway, 53. Instead Barad urges to focus on practices and performances that question the basis of representations. Barad says, “performative account makes an abrupt break from representationalism that requires a rethinking of the nature of a host of fundamental notions such as being, identity, matter, discourse, causality, dynamics, and agency, to name a few.” Ibid., 49.
95
exploration with the gestures of human bodies, such as the waving of the hand, seen as a mode of on-screen meaning-making by Michelle Langford.5 Moving into the context of gestural performativity, I will discuss the implications of using images of gesturing human bodies in various projects that precede yet signal towards the cinematic, such as Aby Warburg’s Mnemosyne Atlas (1924–29), and pre-cinematographic studies undertaken by Eadweard Muybridge and Étienne-Jules Marey. Moving away from the meaning-making function of gestures, I will then examine the loss and retrieval of gesture in cinema, as suggested by Agamben.6 The compositional gestures of the filmmaker or artist must also be accounted for when analysing films and artworks. Indeed, this may lead us to an affective understanding of gestures that is, as Vilém Flusser suggests, attentive to how affect arises between two or more gestures.7 Here gestures, thought through Sergei Eisenstien’s theory of montage, could be understood as more relational than symbolic.
Finally, I will suggest that the gesturing entity in relation to media, and hence where gesture arises, may no longer necessarily be a human body. In the postmedial assemblage, what or which ‘body’ gestures is ambiguous and multifarious: the artist/filmmaker, image, audience, or gestures made by technical entities such as screens, interfaces and cameras. Furthermore, it is not even a question of a single entity but the juxtaposition of different kinds of entities, and their interrelations and effects. The examination of the affective-ness of gestures denotes their relationality, which is further complicated as gestures migrate with reassembling media in postmediality. With various examples of artworks, I will show that the postmedia condition
5 Michelle Langford, Allegorical Images: Tableau, Time and Gesture in the Cinema of Werner Schroeter (Bristol: Intellect, 2006), 137; Maurice Merleau-Ponty, Phenomenology of Perception, trans. Colin Smith (London: Routledge, 2004), xx–xxi. 6 Agamben, “Notes on Gesture,” 54. 7 Vilém Flusser, Gestures, trans. Nancy Ann Roth (Minneapolis: University of Minnesota, 2014), 6.
96
endows images and screens with greater mobility, where both can traverse various media forms with ease, taking on new capacities, building new connections and leaving traces.
In this entanglement of images, screens and human bodies, gestures cannot be said to be produced by one entity. I will argue that the multifacial modalities of screens help in arranging distributed bodies (of all kinds) across media. Here I note that the arrangement of bodies pertains to particularities of material-discursive practice where often screens are situated in the middle of users or participants and the technical apparatus, such as in the mobile phone location assemblage explored in chapter one. The performance of bodies depends on the intra-actions in and of the assemblages that are continuously reconfigured, such as the body of the Google Maps user in chapter one changing its direction depending on the information received on the map. This redistribution becomes central to a new kind of gesturality that we might ascribe to nonhuman entities such as screens. This will be discussed in relation to the framework of gestures provided by Agamben. Here the focus is not on a specific movement being performed but how conditions are created that sustain the performativity of various entities. In this respect, the intermediality of screens as a process or condition that fuels inbetweenness also becomes important to the discussion.
GESTURING POTENTIALITIES In anthropology, the development of gestures in humans, along with language and the ability to use tools pivots on the transition of humans towards bipedalism.8 Anthropologist and archaeologist André Leroi-Gourhan’s work
8 Bipedalism (walking upright on two feet) has been argued to have developed as a result of humans moving into open grasslands or savannahs. For instance, see John D. Kingston, Bruno D. Marino and Andrew Hill, “Isotopic Evidence for Neogene Hominid Paleoenvironments in the Kenya Rift Valley,” Science 264 (May 13, 1994):955-9, https://doi.org//10.1126/science.264.5161.955. PMID: 17830084.
97
is seminal in studying the evolution of gestures. The term chaîne opératoire coined by Leroi-Gourhan, refers to the distinctly human behaviour of making goal-oriented gestures using materials (be that with tools or other things).9 Directly translated as ‘operational sequences,’ chaîne opératoire is extended to address activities based on a combination of gestures and tools used by humans as they evolved.10 Leroi-Gourhan presented a table relating tools to the actions performed while using them. Bipedalism, he argued, gave rise to the hand-tool and face-language development in human beings.11 Leroi- Gourhan is significant for my argument as he situated gestures in the human body as an aspect of technogenesis, even though he narrowly cast them as operational gestures; that is, gestures that are bound to the use of tools.
Elsewhere in the humanities, gestures have, classically, been understood as a form of communication that plays out via the movements of the human body.12 Here gestures signify meaning through their expressiveness. This understanding recognises gestures as a means to transmit information about feelings and ideas, however ambiguous or opaque they may be. More recently, specifically in relation to on-screen gestures, Langford has suggested two main types of gestures to be found in cinema: histrionic and naturalistic.13 Histrionic gestures comprise codes that have developed over time to contain a specific meaning. Naturalistic gestures, according to Langford, are more impulsive and motivated towards an action.14 In fact both kinds are encoded with specific meanings. Langford counters this classification of gestures with
9 Randall White, introduction to Gesture and Speech, [1964] by Andre Leroi-Gourhan, trans. Anna Bustock Berger, (Cambridge MA, London: MIT Press, 1993), xviii. 10 Andre Leroi-Gourhan, Gesture and Speech, [1964] trans. Anna Bustock Berger (Cambridge MA, London: MIT Press, 1993), 114. 11 Leroi-Gourhan, Gesture and Speech, 240. Leroi-Gourhan gave importance to language in accelerating the human intellect. Tools, he argued, change much faster than language. See Françoise Audouze, “Leroi- Gourhan, a Philosopher of Technique and Evolution,” Journal of Archaeological Research 10, no. 4 (2002): 292. 12 For instance, Irenäus Eibl-Eibesfeldt, “Similarities and Differences Between Cultures in Expressive Movements,” in Nonverbal Communication, ed. R.A. Hinde (London: Cambridge University Press, London, 1972), 297–314. 13 Langford, Allegorical Images, 137. 14 Ibid., 137.
98
examples where gestures do not have a fixed meaning — an idea that I will also explore in the forthcoming sections.15 Nevertheless, her understanding of gestures is a semiotic one, where they are seen as a mode of communication along with or in addition to speech. Such a comprehension of gestures, as I will demonstrate below, is inadequate as it does not take into account the performativity of gestures, both in their sociotechnical context and in terms of what kind of body performs them.
A narrow understanding of gestures remains in some fields of contemporary research in which questions of gesturality and mediality play a role. For example, interpretations of bodily movement are widespread in the field of social robotics. In this domain, much work is underway teaching human gestural communication to robots in order to facilitate successful human- robot collaboration.16 However, it would be a mistake to assume gestures can so easily substitute for semiotic communication alone, when, for example, a nod is understood to signify only agreement. In several instances, the commingling of gestural signification is such that consideration of direct semiotic decoding renders analysis of the gesture incomplete, as gestures also carry a history of corporeality embedded in the sociality of a specific time and place. In some cultures, such as in Greece and Bulgaria, a nod is considered to be a refusal.17 However, my point is not about whether specific actions or
15 For example, Langford analyses the works of Werner Schroeter as “gestural cinema” and analyses the images as “allegorical figures.” Langford, 133–67. 16 Paul Bremner et al., “The Effects of Robot-Performed Co-Verbal Gesture on Listener Behaviour,” Proc. 11th IEEE-RAS International Conference on Humanoid Robots 11 (2011): 458–465. Chien-Ming Huang, Bilge Mutlu, “Modeling and Evaluating Narrative Gestures for Humanlike Robots,” Robotics: Science and Systems (2013), https://doi.org//10.15607/RSS.2013.IX.026; S. Costa et al., “Using a Humanoid Robot to Elicit Body Awareness and Appropriate Physical Interaction in Children with Autism,” International Journal of Social Robotics 7, no. 2 (2015): 265–278. 17 Marianne LaFrance and Clara Mayo, “Cultural Aspects of Nonverbal Communication,” International Journal of Intercultural Relations 2, no. 1 (1978): 71–89. The culturally dissimilarity in the meaning of gestures has become a point of consideration in human-computer interaction (HCI) studies. See: Paul Chippendale, “Towards Automatic Body Language Annotation,” Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition (2006): 487–92. Elsewhere, Walter Benjamin noted that in Kafka’s work, gestures do not have any symbolic meaning. Instead meaning is determined from the context and the combination in
99
movements can be definitively decoded, but rather that gestures are already multivalent. In the field of social robotics, where the prevailing approach to the study of specific kinds of gestures as part of human behaviour is primarily representationalist, ironically the multivalency of these human gestures is ignored.
The multivalence of gestures is further complicated by the context of their performance, where they might serve as an operation, movement or response. Hence gestures indicate not only a body that performs but also the conditions under which performance takes place. A close analysis of Aby Warburg’s ambitious project, Mnemosyne Atlas (1924–1929) illustrates this. I also turn to this visual project as a site for thinking through gesture beyond the semiotic as it is of interest to Agamben. The Atlas aimed to animate the cultural and scientific progress of Europe using photographs and sketches of art and non- art images from the Renaissance, through to newspaper cuttings, advertisements, personal photographs and maps.18 At least seventy-nine panels were originally planned, but by Warburg’s death in 1929 only sixty- three panels had been loosely completed.19 These panels were covered in black cloth with photographs stuck on top (fig. 10, 11). A few have texts and description, but most of them remain incomplete. Here various gestural forms of human bodies, sculptures and events are captured imagistically, with the gesturing human body photographically captured. Agamben argues that the Mnemosyne Atlas is primarily an attempt to represent gestures through the medium of images.20
which the gestures lie. Walter Benjamin, Illuminations, ed. Hannah Arendt, trans. Harry Zohn (New York: Schocken, 1968), 120. 18 There are also several references to Arabic astronomy. See the panel themes at Cornell University Library at: https://warburg.library.cornell.edu/about/mnemosyne-themes. 19 Christopher D Johnson, “About the Mnemosyne Atlas,” accessed 31 January 2021, https://warburg.library.cornell.edu/about. 20 Agamben, “Notes on Gesture,” 54. Christopher D. Johnson calls the Atlas “a wordless attempt to chart the Nachleben of the classical Gebärdensprache (language of gestures) in Renaissance art and beyond.” See Memory, Metaphor, and Aby Warburg’s Atlas of Images (Ithaca, New York: Cornell University Press, 2012), 9.
100
Warburg’s Atlas has been the subject of great interest in the fields of art history and iconography. By structuring different types of imagery from multiple sources around a central idea, the Atlas presents fragments of an idea, and offers numerous entry points into the cultural and social history of Europe. It has been the subject of a range of inquiries, such as the study of pathos in Western imagery, and the preservation and provocation of memory.21 The gestures of its images have also been studied as metaphors presenting fragmented thoughts unconstrained by disciplinary boundaries.22
Figure has been removed due to copyright restrictions.
Figure 10: Aby Warburg, Mnemosyne Atlas, Panel 47, 1924-1929, photographs and paper on black cloth. Scans of original panels from Cornell University Library and The Warburg Institute. https://warburg.library.cornell.edu/panel/47.
21 Johnson, Memory, Metaphor, and Aby Warburg’s Atlas of Images. Also see Georges Didi-Huberman’s exhibition ATLAS: How to Carry the World on One’s Back? Museo Reina Sofía, November 2010–March 2011. This exhibition contemplated the atlas as a model for viewing, understanding, and gaining knowledge of the world. 22 Johnson, Memory, Metaphor, and Aby Warburg’s Atlas of Images, xi.
101
Figure has been removed due to copyright restrictions.
Figure 11: Aby Warburg, Mnemosyne Atlas, Panel C, 1924-1929, photographs and paper on black cloth. Scans of original panels from Cornell University Library and The Warburg Institute. https://warburg.library.cornell.edu/panel/c.
However, in the Atlas there are not merely gesturing images, but also an overall aesthetic gesture that occurs, which works conceptually and expressively. The grouping of images together functions as an aesthetic gesture towards collating expressive movements, which denote a similar idea. Agamben contends that the Atlas is a “representation in virtual movement of Western humanity’s gestures from classical Greece to Fascism.”23 These gestures are arranged in a way of representing a particular idea, such as panel 47 (fig.10) centred around the imagery of nymphs; or panel C (fig. 11) concerned with Mars as both an astrological and astronomical object. Philippe-Alain Michaud argues that the images are essentially a study of all of the gestures that have survived from Antiquity to the present.24 He further states that the eloquence of the gestures in the Atlas instigates a comparison with how
23 Ibid., xi. 24 Philippe-Alain Michaud, Aby Warburg and the Image in Motion, trans. Sophie Hawkes (New York: Zone Books, 2004), 265–270.
102
contemporary images enact expressions.25 The action of putting together a set of images fragments one set of meanings (for gestures as individual expressive movements) into a plurality of potential significations and contexts. Setting up relations between the gestures seen in each image can create multiple meanings. The spectator not only observes, analyses and questions, but in these processes relates and connects their understanding to the actions that they see in and across the images. Here the incompleteness of the Atlas’ panels works against a prescribed understanding and overarching meaning-making, and instead stimulates actual assembling gestures and movements of viewing. The spectator, in a way, extracts and continues the conceptual gestures laid out compositionally by Warburg. So, it is not only in the gestural significations of the bodies that the Atlas provides semiosis; it is also generated through the arrangement of images. I will return to this use of photographs in a montage-like manner in the Atlas a little later.
In the exploration of gestures of human bodies in images, it is also important to clarify the relationship between gestures and movements and how their relationship is understood in the context of this thesis. Some might argue that gestures are meaningful movements that can be interpreted, based on the understanding that gestures are goal-oriented actions accompanying speech or are used instead of speaking.26 However, I am investigating if gestures can be located aside from their association with the human body and direct meaning-making. If this is possible, then we may need to comprehend gestures and movements not as things that occur in bodies (both human and nonhuman) but rather that open up their potentialities. To explicate the
25 Ibid., 271. 26 As discussed previously, fields of cognitive and HCI that rely on such understanding. See, David Efron, Gesture and Environment (The Hague: Mouton, 1972); David McNeill, Hand and Mind: What Gestures Reveal About Thought (Chicago, London: University of Chicago Press, 1995). In HCI studies: Maria Karam, and M.C. Schraefel, “A Taxonomy of Gestures in Human Computer Interactions,” Technical Report ECSTR-IAM05-009, Electronics and Computer Science, University of Southampton (2005), http://eprints.soton.ac.uk/id/eprint/261149.
103
different workings of movements and gestures, I will discuss the experiments of Eadweard Muybridge and Étienne-Jules Marey.
It can be argued that the experiments of Muybridge and Marey are studies in movement and not gesture. This, I will argue below, does not apply when movements and gestures are considered aside from the moving body and its form. The studies of human and animal motion by Muybridge and Marey were in their time of interest to the medical sciences, but have also been claimed as part of moving image history, with their works extensively studied as predecessors to cinema in terms of technological and medial innovations.27 However, my interest in Muybridge and Marey is twofold in their appeal to the workings of both gestures and movements. Firstly, I believe we can see in their work a sense of gestures that is other than communicative and is indeed non-representational. Gestures may be a kind of movement, but they cannot be wholly subsumed under movement as human motion. In a way, then, this chapter is concerned with separating gestures from movement by interrogating the capacities of gestures to be more than human, to operate throughout postmedia and to function as more than semiotic. I will argue later that gestures are less connected to human motion and are instead more indicative of conditions of operations; conditions that allow movements and relations across, for instance, postmedial assemblages. Secondly, by way of a direct contrast between gestures and movements, the clinical gaze that supported and apprehended the works of Muybridge and Marey arguably led to the abandonment of gesturality.28 Later in this chapter, I will discuss the role of cinema in reclaiming gestures for the human body, as proposed by
27 See Akira Mizuta Lippit, “Phenomenologies of the Surface: Radiation-Body-Image,” Qui Parle 9, no. 2 (1996): 31–50; Mary Ann Doane, “Temporality, Storage, Legibility: Freud, Marey, and the Cinema,” Critical Inquiry 22, no. 2 (1996): 313–343; Maria Tortajada, “Marey and the Synthesis of Movement: The Reconstruction of a Concept,” in Cine-Dispositives: Essays in Epistemology Across Media, ed. Maria Tortajada and François Albera (Amsterdam: Amsterdam University Press, 2015), 93–114. 28 Agamben, “Notes on Gesture,” 53.
104
Agamben, whereby this circling back to the imagistically captured human body comes to re-gesture, albeit via the cinematic image.
Both gestures and movements have been understood in the realm of experience and feeling, but movement, when considered as something not tied to human motion, can be argued to be that which holds potentialities. Susanne Langer describes gestures as “vital movements” that can be both experienced and seen.29 While studying the spiralling ornamental border of a tablecloth, Langer refers to its design as providing a “sensation of movement,” yet the spiral as such is not moving, indeed, nothing really is.30 Based on Langer’s observations, Brian Massumi proposes that this movement is not an illusion but can be thought of as an abstraction.31 This abstraction, Massumi argues, is derived from the form that is seen. The abstraction is seen “with and through the actual form.”32 This abstraction lies in the form (in this case, the spiral) merging with the experience of the form (spiralling).33 In other words, we feel the movement in and through the design. The form itself, then, holds potential for movement, which we see and feel imperceptibly:
Form is full of all sorts of things that it actually isn’t—and that actually aren’t visible. Basically, it’s full of potential. When we see an object’s shape we are not seeing around to the other side, but what we are seeing, in a real way, is our capacity to see the other side. We’re seeing, in the form of the object, the potential our body holds to walk around, take another look, extend a hand and touch.34
29 Susanne K. Langer, Feeling and Form: A Theory of Art (New York: Charles Scribner’s Sons, 1953), 174. This is not to be confused with Jane Bennett’s notion of vitality. Here vital is used by Langer to mean that the everyday gestures are fundamental movements functioning as “signals or symptoms of our desires, intentions, expectations, demands, and feelings.” Ibid., 174. 30 Ibid., 64. 31 Massumi, Semblance and Event, 41. 32 Ibid., 41. 33 Ibid., 41. 34 Ibid., 42.
105
Experiencing the virtual — one way in which semblance is explained by Langer and Massumi — takes place in and through a set of (affective) movements.35 In this sense, we can argue that movement carries the abstraction of potentiality. Situating gestures as a kind of movement amid this understanding speaks to virtual potential and to how this potential actualises. Gestures then are potential movements actualised. Langer differentiates between everyday gestures as actual movements and dance movements as gestures that are imagined before being performed.36 But she considers gestures as a kind of movement situated in the realm of symbols, performed consciously or unconsciously. Although Massumi does not explicitly address gestures, the relationship between gestures and movements can be derived from his text above. In Massumi’s conceptualisation of semblance (thinking- feeling), gestures can be argued to be movements that may be actualised yet may not carry meaning. Gestures can then be considered as aliveness and as experiences of forces and flows.37
Another separation between gesture and movement is attempted by Vilém Flusser. He differentiates between gestures and movements by deliberating on the approach of the performer who is gesturing or moving. Flusser proposes that not all bodily movements, for example, dilation of the pupils, are gestures; only “intentional” movements can be considered gestures.38 However, since intentions are very subjective, Flusser does not consider them to indicate the dissimilarity between gestures and movements.39 Attributing
35 Ibid., 15–18; Langer, Feeling and Form, 50. 36 Langer, Feeling and Form, 175. 37 These are “vitality effects,” theorised by Daniel Stern in 1985 and later revised as “dynamic forms of vitality” or “vitality forms,” a coming together of forces, motions, aliveness, space and content in temporal intensity. Daniel Stern, Forms of Vitality (Oxford: Oxford University Press, 2010), 40–43. Massumi expands on Stern’s initial concepts to articulate semblance: “The design calls forth a certain vitality affect—the sense we would have, for example, of moving our eyes down a branch of rustling leaves, and following that movement with our hands. But that life dynamic comes without the potential for it to be actually lived. It’s the same lived relation as when we ‘“actually’” see leaves, it’s the same potential. But it’s purely potential. We can’t live it out. We can only live it in—in this form—implicitly.” Massumi, Semblance and Event, 43. 38 Flusser, Gestures, 1. 39 Ibid., 1.
106
intention to gesture brings the ownership of the gesture back to the human body when, for Flusser, gestures are already in relation to things exterior to the human body. Moreover, intentionality here also denotes a cycle of causality, where a psychological force is realised by the human body as it affects its surroundings. Instead, Flusser proposes that gestures can be understood as “a movement of the body or of a tool connected to the body for which there is no satisfactory causal explanation.”40 Although, he clarifies, a causal explanation for gestures can always be provided.41 Nonetheless, such causes do not provide any clarification on what gestures are as they only throw light on to why a particular gesture is performed.
In this short discussion on movements and gestures, I have argued that gestures can be considered to be a kind of movement which is indeterminate, providing a range of possible gestures to be executed. Whatever gestures actualise is not directly connected to a causal chain or sequence. This understanding sits well with a postmedial assemblage as I have described it in the previous chapter, where linear relations and effects are abandoned in favour of intra-active dynamic relations. In the discussion further on, I will argue that the causes of gestures become redundant in postmedia since one specific gesturing body cannot be ascertained. But before I get to this, a discussion on Muybridge and Marey will shed some more light on how gestures differ from human motion. Following this an examination of the multivalency of gestures in cinema and media art practices will help us to trace the shift towards multiple, non-definite meanings of gestures.
40 Ibid., 2. 41 Ibid.
107
I now turn to the ways in which gestures come to be separated from human movement in the image by looking at the work of Marey and Muybridge.42 After his successful experiment of photographing a running horse’s motion in 1877, Muybridge studied the movement of the human body between 1884 and 1887 at the University of Pennsylvania. He photographed it performing various movements (which, taking into account my expanded understanding above, can also be called gestures), such as walking, hopping, and fencing. These movement-gestures (fig. 12) were used to reinforce the ideal masculine body as athletic and strong while the female body was seen as domesticated, delicate and sexualised.43 Simultaneously in Europe, Étienne-Jules Marey also studied human motion mainly through the method of chronophotography.44 While Muybridge captured movement using sequential images, Marey captured the whole movement performed on one photographic plate using multiple exposure, resulting in a flow of action. His most well- known work between 1882 and 1901 included photographs of men walking, pole vaulting and jumping, as well as the movements of birds and animals (fig. 13).
42 I am aware that when considering the human body as a gesturing one, questions of power and ideology cannot be ignored. Power and social relations can be conveyed in even the slightest of human gestures. In examining ordinary human movements such as walking and jumping, it may be useful to consider the works of various theorists who have elaborated on how human bodies are socially configured, historically transmitted and culturally constructed. Several concepts have influenced a generation of thinkers in different disciplines. The important ones that I am thinking of here are Pierre Bourdieu’s ‘habitus’, Michel Foucault’s ‘docile body’ and study of discipline, and Judith Butler’s performativity of gendered bodies. See Pierre Bourdieu, Outline of a Theory of Practice (Cambridge, UK: Cambridge University Press, 1977); Michel Foucault, Discipline and Punish: The Birth of the Prison (New York: Random House Vintage Books, 1977); Judith Butler, Gender Trouble and “Gender as Performance: An Interview with Judith Butler,” Radical Philosophy 67 (1994): 32–9. This rich literature provides the impetus to consider the movements of human bodies not as simple functional movements but as performative. This perspective, while not a main focus of my research, will be taken up later in this chapter when I consider questions of agency, especially in a posthuman context. 43 Tim Cresswell, On the Move. Mobility in the Modern Western World (New York: Routledge, 2006), 65. Also see Rebecca Solnit’s analysis of the impact of Muybridge’s study on human perception. Rebecca Solnit, Motion Studies: Time, Space and Eadweard Muybridge (London: Bloomsbury, 2003). For a detailed view of his photographic plates, see Eadweard Muybridge, The Human Figure in Motion: Eadweard Muybridge 1830–1904 (New York: Dover Publications, 1955). 44 Didi-Huberman considers the chronophotographic works of Marey as not only an examination of movement, and thus part of the history of cinematography, but also as an “inscription of time.” Georges Didi-Huberman, The Surviving Image: Phantoms of Time and Time of Phantoms: Aby Warburg’s History of Art, trans. Harvey L. Mendelson (University Park, PA: Pennsylvania State University Press, 2017), 69.
108
Figure has been removed due to copyright restrictions.
Figure 12: Eadweard J. Muybridge, Man Throwing an Iron Disk, plate 307 from Animal Locomotion (1887) 1884-86, 26 x 27.1 cm, Museum of Modern Art, New York, https://www.moma.org/collection/works/87258?artist_id=4192&page=1&sov_referrer=arti st.
Figure has been removed due to copyright restrictions.
Figure 13: Étienne-Jules Marey, Successive phases of the pole vault, c. 1890, gelatin silver print, 22.2 x 48.9 cm, The Museum of Fine Arts, Houston, https://emuseum.mfah.org/objects/53535/movements-in-pole- vaulting?ctx=87e0947939fa1c0170e5aa361e98605cbc90ae5c&idx=1.
109
Marey’s invention was significant for many fields including photography and cinema as well as his intended research into physiology. The particular series where reflective markers on the human body trace the gesticulations of the body in the form of lines are comparable to the contemporary practice of light painting. Light painting mostly makes use of long exposure time to capture the motion of light as it moves or is moved due to gestures of a human body. In both cases, human gestures are captured.45 However, the human form itself is entirely eliminated in light painting photographs, leaving only the traces of light made by the moving body. Contrary to this, a different kind of gestural design, one that records the gesturing body, is seen in the descendent of Marey’s chronophotography — cinema.46
A different aspect of the images of Muybridge and Marey is provided by Erin Manning, who I draw on to help formulate an understanding of gestures as being other than symbolic of human meaning or expression. Manning considers that Muybridge’s work centres on “movement poses,” whereas Marey’s work is concerned with the “durational force” of movements.47 Bringing in previous discussions by Massumi and Langer, Muybridge’s series position movement in the body itself, being specifically trained upon what is moving. Marey’s work, on the other hand, observes movement itself. As Manning explains,
While Muybridge’s images look like cinematic stills, it is the movement’s interval and duration that is palpable in Marey’s images, rather than the
45 There is a long list of names associated with light art performance photography, including Man Ray’s Space Writing (1935) and Pablo Picasso’s Light Drawings (1949). It continues to be popular in commercial and artistic photography and performances. 46 However, the invention of the Phonoscope by Marey’s associate Georges Demenÿ’ is considered more significant in the development of cinema. After Marey and Demenÿ’s falling out, Demenÿ improved the chronophotograph gun, leading to the invention of Phonoscope. He filed a patent for the Phonoscope in 1892. Demenÿ developed the beater mechanism to move the film, which was later adopted in projectors. Considering several experiments in moving images were taking place almost simultaneously, there is not one but several forerunners of cinema. See Deac Rossell, “Chronophotography in the Context of Moving Pictures,” Early Popular Visual Culture 11, no. 1 (2013):10–27, https://doi.org//10.1080/17460654.2012.756651. 47 Erin Manning, Relationscapes: Movement, Art, Philosophy (Cambridge, MA: MIT Press, 2009), 9.
110
actual stills themselves. In Marey’s work, duration is felt, whereas in Muybridge, duration is divided. Muybridge’s images do not incorporeally flow into one another, and are in fact often shuffled for narrative purposes.48
The movement in Marey’s works is felt; its seeing becomes feeling. The progression of the moving body is visible as it changes from one state to another. But it is in the intervals across states where, Manning argues, a virtuality of movement arises, full of “microperceptions, with microexpressions” that are not actualised in the images but are nonetheless registered by the viewer.49 These intervals can be felt in a similar way to Langer’s ‘spiralling’. The gesture that Marey’s movements execute, then, is just this affective sensing of the potential for movement. This expanded sense of gestures will be taken up further when discussing affective gestures in the next section. But before discussing affective gestures, it is imperative to tackle the affinity of cinema with gesture, starting with silent cinema. Although, as I will analyse, various cinema and media art practices employ the varied meaning-making of gestures in media art.
Perhaps we can track a growing interest in human gestures in the late nineteenth century at the level of the imaging assemblages such as cinema. According to Agamben, cinema was an attempt to retrieve gestures from an introspective and psychological scrutiny initiated in the second half of the nineteenth century.50 Cinema, he remarks, “leads images back to the homeland of gestures.”51 Agamben credits cinema for recovering gestures, allowing their preservation in an image as well as hiding gestures via the
48 Ibid., 108. 49 Ibid., 95. 50 Agamben, “Notes on Gesture,” 53. 51 Ibid., 56.
111
movement of images.52 Hence the two-way and somewhat paradoxical nature of gestures’ revealing and obscuring character became embedded in the very medium of cinema.
With a lack of synchronised sound in early cinema, the silent film era relied heavily on conveying meaning through the bodily movements of actors. The gesturing human body was widely used in early cinema as a means of expression. Béla Balázs strongly argued that gestures, especially in silent cinema, were cinema’s “primordial form of expression.”53 Words, he argued, made the body retreat from its humanity, becoming instead a merely biological organism.54 The semantic aspect of gestures is not negated by Balázs; in fact, it is augmented by emphasising gestures as an alternative form of articulation in which people were seen to be at ease with their bodies. However, as we have discussed, gestural meaning-making is far from direct or singular; it is a mode of communicative exchange that is not precisely accomplished, always leaving room for misunderstandings and incompleteness.55 As gestures took on the responsibility of communication in silent cinema, a semiotic understanding of gestures dominated. Here several filmmakers and actors utilised gestures to complicate and confuse, such as Werner Schroeter. Langford classifies Schroeter’s films as a “gestural
52 Ibid., 55. 53 Béla Balázs, Béla Balázs: Early Film Theory, ed. Erica Carter, trans. Rodney Livingstone (Oxford: Berghahn Books, 2010), 11. 54 Ibid. 55 Roland Barthes charges gestures with muddling meaning and generating myriad connotations due to their unfixed nature. He conceived of gestures as the “surplus of an action,” distinguishing them from message and sign, which communicate information and idea respectively. Even though gestures may start with the intention to communicate, they instead obscure communication by producing contrary meanings. The value of gestures in excess to the sign is also argued by Julia Kristeva. Although she does not regard gestures as meaningful, she nevertheless firmly situates them in corporeal proceedings. For Kristeva gestures are not situated in the signification of the verbal or bodily sign, instead they are “indicative, relational, empty.” By comparing gestures to the use of ‘anaphora’ in language (repeating a sequence of words generally for emphasis), gestures can be understood as pointing towards the actual meaning without actually conveying the meaning themselves. Roland Barthes, The Responsibility of Forms: Critical Essays on Music, Art and Representation (Oxford: Blackwell, 1986), 160; Julia Kristeva, “Gesture: Practice or Communication?” [1968], trans. Jonathan Benthall, in The Body Reader: Social Aspects of the Human Body, ed. Ted Polhemus (Harmondsworth: Penguin Books, 1978), 269.
112
cinema,” a style that utilised gestures to connect as well as disturb the action.56 Gestural cinema is characterised by the prominence of unfixed and obscure meanings of gestures, as opposed to the clarity of plot and characters.57
Such an impetus towards the multivalency of gestures has contemporary resonance. In order to appeal to a larger audience, video and media artists such as Yang Fudong and Pablo Gnecco make use of the multivalency of meaning in human gestures. Their works substitute words with actions and encourage multiple meanings implied by gestures. Minimal dialogue and text in Yang’s works make the viewer rely only on the gesturing body in the image to follow the action.
In his participatory work, Gesture-Gesture (2014), Gnecco invited viewers to record and share their hand gestures by putting their arms in a box. A three- second recording of each gesture inside the box was converted into a GIF and played in the exhibition space (fig. 14) as well as archived online. The gestures are not explained in any particular way and it is up to the audience to find meaning in them.
56 Langford, Allegorical Images, 134. 57 Ibid., 134–135.
113
Figure has been removed due to copyright restrictions.
Figure 14: Pablo Gnecco, Gesture-Gesture, 2014, interactive installation. Installation view at Gallery-72, Atlanta, https://yopablo.studio/Gesture-Gesture.
While analysing the art gallery setting as a place for gestural cinema, Chris Berry highlights that the obscurity and multiple meanings generated in Yang’s cinematic works can function in various cultural settings.58 Studying Yang’s photo series, The First Intellectual (2000), where we see a young man with an injured forehead and a brick in his hand (fig. 15), Berry explains the multiple meanings that this image presents: Is the man ready to throw the brick or has it been thrown at him, or perhaps he has hit himself with it — all these meanings are available.59 In Yang’s The Revival of the Snake (2005), a soldier tries to get out of a deserted place where he is stuck. He climbs a tree and looks up towards the sky (fig. 16). The same ambiguity of gesture although all within the possibility of the narrative, can be seen here as well. Is the soldier trying to get a better view of the place from atop the tree or did he spot someone? In the works of Gnecco and Yang we see that the same gesture is understood in different ways, bringing out the subtleties of the work. Like Langford, Berry defines gestural cinema as favouring gestures
58 Chris Berry’s analysis of Yang Fudong’s films, “The New Gestural Cinema. Yang Fudong and the Gallery Film,” Film Quarterly 67, no. 3 (2014): 22. 59 Ibid., 22, 23.
114
over dialogue, but goes a step further in extending the purview of gestural cinema beyond silent cinema.60
Figure has been removed due to copyright restrictions.
Figure 15: Yang Fudong, The First Intellectual (right), 2000, from photo series The First Intellectual, Colour chromogenic print, 193 × 127 cm, ShanghART gallery, https://www.shanghartgallery.com/galleryarchive/simpleWork.htm?workId=3246.
Figure has been removed due to copyright restrictions.
Figure 16: Yang Fudong, The Revival of the Snake, 2005, multi-channel video installation, 8 minutes looped. Digital video still.
60 Ibid., 18.
115
What we can draw from this discussion is that it is not just cinema that uses multiple meanings of gestures, but the multivalence of gestures lies at the very core of the operations of images and screens in contemporary media art. Oksana Bulgakowa also analyses which gestures have been preserved, modified and abandoned by comparing actors in films to the behaviour of people in real life.61 Her DVD and later installation, The Factory of Gestures: Body Language in Film (2008), made in collaboration with Dietmar Hochmuth, is an archive of gestures taken from Russian films and placed in the larger American and European context of the gesturing human body. She emphasises the variation of gestures in terms of race, culture, class, gender and social settings. In some respects, Bulgakowa’s study of the preservation, as well as the modification of human gestures, brings together Agamben’s suggestion of cinema preserving the gestures and the efforts of Warburg’s project as a temporal archive of gestures.
So far, I have analysed the gestures of human bodies in images and cinema. I have dealt mainly with discussions about gestures situated in the human body and that have been understood within a broad semiotic framework. Gestures here are seen to replace or add to communication. As seen in the images of Mnemosyne Atlas and the neurological gaze in early photography, gestures made by the human body have to be decoded in some way. However, I have also noted that gestures resist direct or indexical interpretation due to their multivalency, the arbitrariness of the movements that initiate them and due to gesture itself being produced medially. Indeed, the ambiguousness of gestures has been utilised by contemporary artists, such as Yang and Gnecco, to direct their works toward more diverse audiences.
61 Oksana Bulgakowa and Dietmar Hochmuth, The Factory of Gestures: Body Language in Film, Potemkin Press, 2008, DVD; Oksana Bulgakowa and Dietmar Hochmuth, The Factory of Gestures: Body Language in Film, 7- channel installation, 2008.
116
In the next section, I want to turn towards a more affective approach to gestures where I will suggest that they gain expressiveness between images and between entities involved in screen assemblages rather than simply within a single image or medial element. This is accompanied by an interrogation of gestures and gesturing that is accomplished by nonhuman entities; that is, in other aspects and relations of the postmedia assemblage. However, I will begin with the cinematic apparatus and examine some of the literature on gesture, affect and mediality that has arisen there. This will point us to the ways in which gestures in medial assemblage are already being thought beyond the human and the communicative.
AFFECTIVE GESTURES Flusser’s exploration of affect leads him to an interrogation of gestures. He locates the human body as the source of affect in demonstrating that human gestures express and articulate affect.62 Our being in the world can be comprehended as bodies that gesture.63 Hence, affect is generated via gesturing bodies in numerous ways, such as writing, searching, loving and making, among various activities.64 While this concept of locating affect in human gestures may work for Flusser in understanding affect, it is unfavourable for understanding gestures. As affect, gestures cannot be described or prescribed easily and are always dependent on the capacities of the affecting entities entering into relations. I argue that an understanding that locates affect in the gestures of human body does not work to determine the meaning of gestures, as affect does not generate clear or fixed meanings, nor does it reside in one body. Drawing on contemporary conceptions of affect we will see that these lead to an understanding of gestures as having manifold interpretations.
62 Flusser, Gestures, 4. 63 Ibid., 142. 64 Ibid.
117
Affect in and of gestures also suggests that gestures are not located in or contained by ‘a’ body but between bodies. The disestablishment of gestures in a ‘body’ continues as I turn towards cinema. Starting with Sergei Eisenstein, I want to show how montage had already worked as a gestural and affective technique. I then link the concept of mimetic gestures, provided by Lesley Stern, with digesture — gestures of nonhuman bodies — outlined by Akira Mizuta Lippit, to extrapolate that gestures do not originate in a body but are shifting and migrating in a cinematic apparatus. Building on this understanding, the next section will help to get a grasp of gestures in postmedia assemblage.
Affect is “born in in-between-ness” and dwells in “beside-ness”, as identified by Greg Seigworth and Melissa Gregg.65 Focussing on something other than the signification of the image, the spaces in between the images assist the generation of forces of feelings and intensities. But as Massumi warns, affect is not simply emotion, as it is generally used, but a capacity to affect or be affected.66 We need to understand images through their relations then, which is why I turn to montage. Kurt Forster had compared Warburg’s Mnemosyne Atlas to the photomontages of the 1920s.67 Michaud takes this comparison to another level and argues that the Atlas is closer to cinema.68 Further, the montage quality of Mnemosyne Atlas can be argued to be affective. The arrangement of images on the panels pertains to the “cinematic mode of thought,” where thinking is not geared towards questions of representation but rather to the production of affects.69 The construction of images spaced
65 Melissa Gregg, Gregory J. Seigworth, “An Inventory of Shimmers,” in The Affect Theory Reader, ed. Melissa Gregg and Gregory J. Seigworth (Durham and London: Duke University Press, 2010), 2. Emphasis in original. 66 Brian Massumi, Parables for the Virtual: Movement, Affect, Sensation (Durham and London: Duke University Press, 2002), 27, 15. 67 Kurt Forster, Aby Warburg: His Study of Ritual and Art on Two Continents,” October 77 (Summer 1996): 19. 68 Michaud, Aby Warburg and the Image in Motion, 278. 69 Ibid., 278.
118
out on a black cloth nurtures the affective qualities of the Atlas. The images of the Atlas act on each other as well as being affected by their general assembling. It is through these montage-like relations that shifts in the intensive qualities of the images occur. The capacities of images are revealed in these shifting qualities. In the intervals between images, here provided by the black cloth, affect is dispersed when relations to viewing shift. As Manning concludes,
Affect passes directly through the body, coupling with the nervous system, making the interval felt. This feltness is often experienced as a becoming-with. This becoming-with is transformative. It is a force out of which a microperceptual body begins to emerge. This microperceptual body is the body of relation.70
This way the distribution of affect is a transformative relation for images, and arguably for the viewers as well.
The construction of images on a black cloth with space between them in the Atlas begs comparison with the logic of montage propagated by Eisenstein, in that an image does not exist by itself but has to be seen with other images it is grouped with.71 But just as the inbetweenness of the black cloth can be gathered affectively, so too can early cinema montage, particularly as theorised and practised by Eisenstein, be considered a mode of affective gesturing. This also helps to position the technique of montage within a gestural context. As Schulzki has observed, Eisenstein’s thinking on gestures remains a rather underexplored area.72 Eisenstein described montage as a
70 Manning, Relationscapes, 95. 71 Ibid., 283, 289. Another overlap in the formulations of Warburg and Eisenstein is that both consider their works ‘pathosformel.’ See Eisenstein’s ‘nonindifferent nature’ in Antonio Somaini, “Cinema as ‘Dynamic Mummification,’ History as Montage: Eisenstein’s Media Archaeology,” in Sergei M. Eisenstein: Notes for a General History of Cinema, ed. Naum Kleiman and Antonio Somaini (Amsterdam: Amsterdam University Press, 2016), 67, 68. 72 Irina Schulzki, “The Underlying Gesture: Towards the Notion of Gesture in Jean d’Udine and Sergei
119
technique where two images produce a meaning of another order.73 Interestingly, Eisenstein studied the active presence of montage in various other arts such as painting, sculpture and literature. However, in cinema, he concludes, it finds its “highest expression.”74 He details different forms of montage as being placed in between frames of a shot (micro-montage) and between shots and scenes (macro-montage).75 Macro-montage can be seen as a collection of several montage units or micro-montages. By analysing two techniques and ideas of Eisenstein in micro- and macro-montage, I will illustrate the gestural and affective understanding of montage.
Eisenstein argued for the potential of montage to create conflict — a collision of meanings, movements and intensities, which could absorb spectators.76 Methods to create cinematic tension, such as the use of sound, are prescribed by him. In cinema, sound is not merely used to lend a voice to characters and objects, but to develop and perfect the montage. As Eisenstein asserts, “The first experiments in sound must aim at a sharp discord with the visual images.”77 Such deployment of sound and images would rupture meaning, an interruption that would confuse any complacent viewers. In this sense, montage generates affects not only through the gestures of actors, but also by the use of sound.
Eisenstein,” in From Sensation to Synaesthesia in Film and New Media, ed. Rossella Catanese, Francesca Scotto Lavina and Valentina Valente (Newcastle upon Tyne: Cambridge Scholars Publishing 2019), 103. 73 Sergei M. Eisenstein, Selected Works, Volume 1 Writings, 1922-34, trans., ed. Richard Taylor (London: BFI Publishing and Indiana University Press, 1988), 139. 74 Geoffrey Nowell-Smith, “Eisenstein on Montage,” in Selected Works, Vol II: Towards a Theory of Montage by Sergei M. Eisenstein, ed. Michael Glenny and Richard Taylor, trans. Michael Glenny (London: BFI Publishing, 1991), xv. Also see Sergei M. Eisenstein, Selected Works, Vol II, 116, 296–97. 75 Sergei Eisenstein, “Laocoön,” in Selected Works, Vol II, 109. Eisenstein changed his position on montage from the late 1920s to late 1930s where he worked on the criticism of montage and presented his reworked position on montage. See Jacques Aumont, Montage Eisenstein, trans. Lee Hildreth, Constance Penley and Andrew Ross (London: BFI Publishing and Indiana University Press, 1987), 145–199; and Sergei Eisenstein, “Montage 1938,” in Selected Works, Vol II, 296–299. 76 Eisenstein, Selected Works, Vol 1, 145. 77 Ibid., 114.
120
In montage theory, Eisenstein also gives importance to the gestures of the actors, albeit not just as a meaning-making activity. “Mise en geste” describes the importance of actors’ gestures to convey the motives of the character in a way that brings together the nuances of the story.78 The social, cultural and psychological relations of gestures were given primacy by Eisenstein. Consequently, his gesturology is based on conflicting forces, internal and external, converging in the body of the actors that then construct gestures from a mechanical, biological, social or political viewpoint.79 Eisenstein does not limit the cinematic image to that of the gesturing body of the actor, but suggests an interrogation of how the actors’ gestures are produced in particular ways. We can then perhaps combine the gestures of the actors with that of the other technicians of the film and even the arrangement of objects in a scene that allow only certain movements of the actors. This way the gesturing image on-screen conflates the gestures of the actors with out-of- frame objects and bodies. By looking to Eisenstein’s ideas and practices of montage, we can see a dynamism attributed to gestures on-screen and off. These are not simply located in the movement or expression of a single human body but between images and in the mediality of cinema itself. By aiming to generate intensities and using the ‘forces’ of images and sound, Eisenstein implicitly provides us with an affective understanding of montage.
Yet, gestures or their affectivity are not limited to images and sound in cinema. Moshe Barasch comments that gestures in art have not only functioned as encoded significations but also arouse the emotions of the viewers.80 However, this affective response is not only generated by images.
78 Sergei Eizenshtein, Neravnodushnaia priroda. Tom 1. Chuvstvo kino, ed. Naum Kleiman (Moskva, 2004), quoted in Ana Hedberg Olenina and Irina Schulzki, “Mediating Gesture in Theory and Practice,” Apparatus. Film, Media and Digital Cultures in Central and Eastern Europe 5 (2017): 4, http://dx.doi.org/10.17892/app.2017.0005.100. 79 Ibid., 5. 80 Moshe Barasch, “Gesture,” Grove Art Online, 2003, https://doi.org/10.1093/gao/9781884446054.article.T031819.
121
As discussed above, Eisenstein’s ideas of sound as providing conflictual cinematic elements allows us to regard it as a part of the cinematic repertoire of gestures. In a similar vein, Stern interrogates gesturalities that are set up in cinema in the relations between the actors, director, cameraman and editor, and the equipment and devices used by them.81 Bringing together the concept of mimesis and pathos, she argues that gestures are mimetic as each gesture calls for a gestural response.82 In cinema, the gestures of actors elicit a gestural response from the spectators as well which does not solicit only repetition, but transfers the intensity of the gesture to spectator’s bodies affectively. Hence, the mimesis lies in the intensity of the experienced gesture and not at a representational level.83 The gestures of the camera, or of editing or in cinematic sound, do not lie only in what their physical movement symbolically convey but also in moving viewers.
Different gesturing bodies in cinema also provide various kinds of gestures. Lippit has examined the gestures of nonhuman entities in order to question the possibility of performing or inscribing gestures without (human) bodies in the cinematic assemblage.84 He broadly identifies two gesturing bodies: the actors along with the various cinematic apparatuses such as the camera, which gestures by panning, tilting, point of view shots, and so on; and editing, which gestures a space-time continuity or discontinuity via continuous or disjunctive movements in consecutive shots.85 Lippit coined the term digesture to talk about the gestures of these two sets of ‘bodies’.86
81 Lesley Stern “Ghosting: The Performance and Migration of Cinematic Gesture, Focussing on Hou Hsiao- Hsien’s Good Men, Good Women,” in Migrations of Gesture, eds. Carrie Noland and Sally Ann Ness (Minneapolis: University of Minnesota Press, 2008), 206, 188. 82 Ibid., 202. 83 Ibid. 84 Akira Mizuta Lippit, “Digesture: Gesture and Inscription in Experimental Cinema,” in Migrations of Gesture, eds. Carrie Noland and Sally Ann Ness (Minneapolis: University of Minnesota Press, 2008), 117. 85 Ibid., 116, 117. 86 Ibid., 117.
122
In various works of experimental cinema, Lippit describes how these two sets of bodies could give rise to new gestures. For example, in Martin Arnold’s films Passage a l’acte (1993), pièce touchée (1988) and Alone. Life wastes Andy Hardy (1998), movements made by actors in Hollywood films, are worked upon. Arnold edits so as to repeat the movements of actors over and over again, glitching, slowing, reversing and inverting them. The gestures of the actor’s body together with the edits give rise to new gestures that are not in the original work.87 Lippit describes Arnold’s later film, Deanimated (2002), as an instance where he digitally erases bodies. Arnold edits out the actors or their mouths (fig. 17) leading to gestures without bodies.88 Similarly to Stern, Lippit also recognises the agency of other nonhuman actors in cinema such as the camera and editing. While Stern observes the gestural dialogue between various bodies and its transformation into affect, Lippit is concerned with the effect of gestures on the bodies and the erasure of those bodies. However, both are concerned with the migration of gestures across bodies, medial elements and contexts.89 Moreover, along with their movement, we also see an interest here in the question of how gestures are inscribed; their recording highlights how these are transmitted to and have effects upon both human and nonhuman entities. There is always someone or something witnessing, recording, effected or affected by gestures.
87 Ibid., 121. 88 Ibid., 127. 89 Carrie Noland, introduction to Migrations of Gesture, eds. Carrie Noland and Sally Ann Ness (Minneapolis: University of Minnesota Press, 2008, xvi.
123
Figure has been removed due to copyright restrictions.
Figure 17: Martin Arnold, Deanimated, 2002, digital video, 60 mins. Two digital video stills.
In this unravelling of gestures as fixed by Stern and Lippit through attending to their migratory movements, the location and meaning of gestures become complicated. In terms of their location, we can already see that they have moved away from the human body and that nonhuman bodies are also being interrogated. Stern moves toward the intensity of gestures as affective, and Lippit is concerned about the image in cinema as produced by a multiple gesturing body (or bodies), and not always human.
However, Stern and Lippit’s discussion is limited to the cinema and cinematic apparatus. The role of spectator is also limited to being one who is simply affected. In the next section, I will analyse the migration of gestures to contemporary media spaces. The postmedia condition encourages and facilitates various entities beyond human bodies and cinematic apparatuses as gesturing. Here gesturality extends beyond screens as an entity of cinema to the network of humans, screens, images and various technological apparatuses. There are not just two gesturing bodies, as Lippit argues, but several other bodies contributing to the gestural conversation.90 Following from this, I will suggest that in the postmedia condition it may be more helpful to think of gestures as persistently circulating since entities are
90 Lippit, “Digesture,” 122.
124
postmedially assembled, making it almost impossible to distinguish which body is gesturing. This not only complicates the location of gestures but also what they are doing or how they are functioning, as contextual changes produce new capacities and expressions for gestures.
We see this flux of gestures with postmedia assemblages in a number of ways. For instance, a change in the orientation of a mobile phone from vertical to horizontal also changes the orientation of the image. Here, the movement of the viewer/user’s hands almost simultaneously changes the orientation of the screen and image. It can be argued that the image is gesturing so as to reveal itself in the change of orientation. Even a modest action such as changing the direction of the mobile phone is cryptic as to which body is gesturing — hand/human or phone/screen. This leaves us with the question of how we start talking about the gestures of, by and with postmedial screens without referring to the work being also performed by the image or a human body.
To complicate things further, there are other technical entities gesturing when the orientation of a phone changes, such as the accelerometer that changes the orientation of the image, the actual change in orientation made by the images on the screen,91 the movement of the arm that flicks the phone, the support frame of the screens that becomes visible in the gesturing or an affective gesturing of the body of the viewer. In this next section, I will argue that in these shifts across and among gestures there is not one gesturing body but a number of entities that produce gesturality, which we might think of as a kind of relational gestural performance. Further, I will examine how some gestures become conflated in screen space as screens perform operations across various entities and reorganise relations between them. As observed in the example above, the orientation of the phone brings, in most cases, a larger
91 By images here I mean the expression and movement of graphics, text, pictures on the screen.
125
view of the image on-screen. But with that, other relations also readjust such as the on-screen keyboard, which will be of a different shape and arrangement to the one summoned in a vertical orientation of the phone. Thus, several relations, as I will discuss below, are formed by, through and across the screens of postmedial assemblages.
Up until now, we have gathered a varied understanding of gestures including how affective gestures are produced not only by the movement of a body or images but through the relations between them. Moreover, I have suggested that what is at stake is not the kind of gestures, but rather the ambiguity of who or what is gesturing. In the next section, I will further elaborate upon the entangled relations of the screen, image and human body we now find ourselves confronted with, making a case for a new understanding of the gesturality of screens that takes into account the relations and conditions of postmediality.
WHO/WHAT IS GESTURING? The agency of screens as operative, performative and gestural has been underdone in the discussion on images, medium and bodies by various contemporary scholars of the digital and postdigital such as Hans Belting, Pasi Väliaho and W.J.T. Mitchell.92 I contend, however, that the relations between screens and the movement of images within and across them is important for understanding what a postmedia assemblage is, so we need to pay attention to such relations and movements. Furthermore, both of these are also deeply entangled with human bodies. I turn now to explore the nexus
92 See, for example, Hans Belting, An Anthropology of Images: Picture, Medium, Body, trans. Thomas Dunlap (Princeton, NJ; Oxford: Princeton University Press, 2011); Hans Belting, “Image, Medium, Body: A New Approach to Iconology,” Critical Inquiry 31 (2005): 306; Pasi Väliaho, Biopolitical Screens: Image, Power, and the Neoliberal Brain (Cambridge, MA; London: MIT Press), 2014; W.J.T. Mitchell, What do Pictures Want? The Lives and Loves of Images (Chicago: University of Chicago Press, 2004).
126
between screens, images and bodies through a discussion on screenness, the agency of screens. As discussed in the last chapter, agency involves intra- actions in and through which screens and other bodies continuously emerge. Keeping this in mind, I will also illustrate how screens perform the distribution of gestures in the shifting relations of postmediality.
Screens play a significant role in forging relationships with images. In addressing the confusion between the image and medium, Belting proclaims that images have a “mental quality” that is perceived through the “material” medium.93 The image here is not quite present but is made so through the medium. For such an exchange to take place, Belting explains, the medium has to become “transparent” for the image to be perceived by the body.94 In this way the image and medium are both present and absent and the image comes to be ‘present’ through the cognitive capacities of viewers. For Belting, screens are a “pictorial medium” that “transmit” images.95 Väliaho differentiates between screens and images using similar terms. He outlines screens as the “technical materialisation” that provide support and accessibility to images as they move through different media.96 Yet Belting’s idea that screens are a medium of transmission and Väliaho ‘s argument about screens being carriers of images both overlook various operations performed by screens, such as that of the interface, where screens are not only used to view images but to interact with them. Moreover, some screens are activated via sound, movement and graphics, and also modify the behaviour of the interactant. I will expand on this in the following section when discussing Ulrike Gabriel’s work Breath to get at a complex nexus that can operate between bodies and screens.
93 Belting, An Anthropology of Images, 20. 94 Ibid., 20. 95 Ibid., 20. 96 Väliaho, Biopolitical Screens, 5
127
Additionally, Väliaho and Belting also ignore the agency of screens in activating and reconstructing relations between different parts of an assemblage. However, they do not discount the significance of the human body. Belting argues that human bodies are the “living media that makes us perceive, project, or remember images that also enables our imagination to censor or transform them.”97 Väliaho goes a step further, outlining that the human body, along with technological platforms, transport and multiply images, and affect the present and future behaviour of the viewers.98 If bodies already carry images, according to Väliaho , then what happens when they intra-act with screens that potentially introduce more images; what happens in the confluence of these two kinds of image modalities? Moreover, if images materialise and are carried in the body, then this already assumes that images have migrated from screens to the body, a significant movement that cannot be ignored when studying (inter)medial interconnections. I argue below that the migration of images takes place in and through screens, including instances where the screens become the image. In the contemporary scenario, where images are accessed via screens, overlooking the agency of screens in the image flow does not do justice to the entire picture.
To detail the role of screens in the movement of images, let us analyse a scene from director David Cronenberg’s cult classic, Videodrome (1983). Videodrome is the story of Max Wren, head of a television channel that delivers pornography and violent programming. In order to show unique material, he illegally starts broadcasting a program called Videodrome made in Malaysia, which shows an ongoing stream of violence and torture. After he initiates its broadcast, bizarre incidents start happening to him making it impossible —
97 Belting, “Image, Medium, Body,” 306. 98 Väliaho, Biopolitical Screens, 4–6.
128
on the level of both his own and other’s sexualised and violent behaviour — to differentiate between the television show and reality.
Prof. Brian O’Bilivion, an expert on television and media technology, speaks to Max only via videotapes prophesising the future of television. In one particular scene, Prof. O’Bilivion is murdered on-screen (fig. 18) while Max is watching, although by now Max is unclear if what he is seeing is a recording or a live broadcast. The person who seems to have killed Prof. O’Bilivion is Max’s girlfriend, Nicki Brand. As Max watches the death of Prof. O’Bilivion, Nicki turns to him after finishing the act and seductively calls out to him to be with her. As she provocatively commands Max, her different facial features such as eyes and mouth are focused upon and become absorbed into and converge with the televisual screen space (fig. 19). As Nicki is calling out, Max notices that the body of the television set is contracting and expanding as though something is moving inside. As Max touches the body of the television, Nicki can be heard calling him continuously. It seems as though Nicki is physically at one with the television set, and the set moves (fig. 20) as she also moves her body to entice Max. Max’s heavy breathing, moving hands on the television set, and the screen suggests that even though he is slightly confused at the turn of events, he is trying to reach Nicki. The image on-screen fixates on Nicki’s lips. In Max’s heavy breathing, mixed with Nicki’s appeals to him, the screen conflates all of these: body, screen, medium. Then the flat screen suddenly converts into a three-dimensional surface whose lips become the medium. This emergence of the screen-as-body seduces Max and he immediately buries his face in the screen.
129
Figure has been removed due to copyright restrictions.
Figure 18: Videodrome. Directed by David Cronenberg, 1983, 87 mins, Canadian Film Development Corporation. Digital video still.
Figure has been removed due to copyright restrictions.
Figure 19: Videodrome. Directed by David Cronenberg, 1983, 87 mins, Canadian Film Development Corporation. Digital video still.
Figure has been removed due to copyright restrictions.
Figure 20: Videodrome. Directed by David Cronenberg, 1983, 87 mins, Canadian Film Development Corporation. Digital video still.
130
This scene has been discussed by W.J.T. Mitchell to highlight the instability of the medium as we enter into a postbroadcast era.99 He argues that the components of sender, receiver, code and message, usually associated with a classic definition of a communications medium, are blurred in the volatile new medium we see emerging in this scene. The image on television that should be consumed by the viewer becomes the consumer and the producer of images — Max is the one that is devoured.100 I argue that, in addition to the medium of television, this scene demonstrates the instability of the screen, and not just in a literal sense.101 In such a scenario, screens become spaces where roles are blurred, embodying both the image and the medium. As the medium shifts and turns, the screen space performs these actual movements. The screen embodies the lips of Nicki, which emerge from their on-screen flatness to become a convex protrusion inviting Max. In the literal inflation of the screen, it also expands with desire; the desire of Max to be with Nicki. While this film is widely regarded as a forerunner of the reconfiguration of the medium of television, it also indicates a new kind of image that circulates and preys on human bodies willing to surrender to its spell.
In the postmedia condition, the movement of images is accompanied by screens and their movements. The creation, display, circulation, modifications and various other operations on images and other media elements take place in and through screens. Images, according to Mitchell, are like living organisms that have demands and desires and migrate from one medium to another generating contradictory feelings and values.102 As images travel, screens aid in navigating the media. Mitchell states that the materialities of media serve as “habitats or ecosystems,” where images live and evolve parallel
99 Mitchell, What do Pictures Want? 219. 100 Ibid., 219. 101 I have already discussed how we can separate screens from medium as well as the technological apparatus in chapter one. 102 Mitchell, What do Pictures Want?, 91–93.
131
to human bodies.103 This gives us an idea about how screens then get to be mistakenly viewed as mere habitat for images. While images move across various media, several processes take place between various kinds of screens as well. The human body does not just assist these movements but is interpellated into all the processes.
However, it is not simply that screens aid the circulation, multiplication and movement of images, but also the speed with which these postmedia operations take place. Even poor images, as noted by Hito Steyerl, are not defined by their resolution but via speed and proliferation.104 The digital networked image is dissociated from its source; the source is not important anymore because in the process of movement and circulation the image acquires new meanings, and contexts and relationships are (re)made.105 With their multi-facial modalities, how is it that screens negotiate or assist in the distribution of bodies? An analysis of Ulrike Gabriel’s Breath may be able to shed some light on the reconfiguring capacities of screens in postmedia.
103 Ibid., 91. 104 Hito Steyerl, “In Defence of the Poor Image,” in The Wretched of the Screen (Berlin: Sternberg Press, 2012), 40. 105 Some theorists propose that the metadata of the image is more important in the accessibility, visibility and circulation of the image. See Daniel Rubinstein and Katrina Sluis, “A Life More Photographic,” Photographies 1, no. 1 (2008): 20.
132
Figure has been removed due to copyright restrictions.
Figure 21: Ulrike Gabriel, Breath, 1991/92, interactive installation. Digital video still from documentation, http://www.llllllll.de/breath_e.html.
Breath (1991–92) is an installation that consists of a belt made up of sensors that the participant wears around their waist. A screen in the space (fig. 21) displays graphics of numerous polygons. The sensors in the belt capture the breathing of the participant, which then is used to transform polygons on the screen. An audio output accompanies the changing shapes on the screen. The geometric shapes change according to the rhythm of the participant’s breathing and the volume of their breath generates the shapes of the polygons. However, neither the actual breath nor the activity of breathing is directly visualised on the screen. The polygons on-screen also factor in previous participants’ breaths in their movement. The previous breaths along with the latest input are computed by the algorithms of the installation and given an abstract geometric form on-screen. Accordingly, the breath of the participant is computationally distributed in the network of the installation to be processed and visualised not once but several times. As a result, even before a participant wears the belt, the polygons on screen are moving as the algorithm processes data from previous breaths.
133
Importantly, the transformation of the shapes on-screen takes place slowly; that is, the breath of the participant will only gradually influence the shapes on-screen. At the start of each new interaction, the on-screen shapes come from the data of both the previous and new participant. But at no point is the breath of the participant directly translated into the movements on-screen. As the breath of the participant is processed by an algorithm taking into account previous results, the images are both more autonomous and dependent on the participant and the data. In Breath, numerous parts of the installation, such as the sensors on the belt and the program processing the data from the sensors, form new connections with the body of the participant, transforming the associations between these elements of the work as well as with the participant.
However, the work cannot simply be regarded as a generative artwork where the images on-screen are produced by the breath of participants. When the participant learns that the changes on-screen are a result of their breathing, they adjust their breathing; the altering of the participant’s breathing impacts the input of the data, which in turn changes the shapes of the polygons visible on screen. Yet, as Anna Munster cautions, this does not simply mean that the body is asserting its presence.106 She advises to instead think in terms of the relations between the capacities afforded by both the human interactant and the technological apparatus.107 Here the screen is not merely displaying the data, but actually becomes both a receptive and regulatory space for the participant and the image. The more relaxed and rhythmic the breathing, the more complex and random the structures of the polygon and the sound
106 Anna Munster, Materializing New Media: Embodiment in Information Aesthetics (Lebanon, NH: Dartmouth College Press, 2006), 4. 107 I am aware that Char Davies’ Osmose (1995) uses the breath and balance of participants to navigate virtual worlds. However, it uses a head mounted display and real-time motion tracking. This is a virtual reality work that falls under ‘post-screen’ practices, a substantially different perceptual and aesthetic experience, which I have already indicated is outside of the scope of this thesis.
134
become. As a result, the participant’s gestures are modulated via the movement of polygons on-screen. For the participant’s body, the screens combine image with the movements of the body. Additionally, the residual data on the screen effects the perception of the participant. By displaying shapes, the screen actively transforms the relationship of the participant via the sensors on the device, the images on-screen and with the computer that transform the data into images. It seems the participant is having a conversation with the screen where both the participant and the images are affected. In this way, the screen’s capacity to regulate the behaviour of the participant has to be highlighted. We have already seen in Väliaho’s argument that images exert pressure on the future behaviour of the bodies; in Breath the screen’s participation in regulating behaviour is even more apparent.108 Below I discuss a few more postmedia assemblages to illustrate the different ways in which the human body is arranged by screens.
Distributed self and circulating gestures The gestural distributed human body in relation to images in postmedial screens can be evaluated in two ways. Firstly, the human body is (re)distributed and incorporated in various algorithmic processes, for example in video games and as seen in Gabriel’s work above. The gestures performed by bodies in front of the screen while gaming produce different experiences involving both the materiality and phenomenality of the screen and images.109 However, within the constraints of the operational framework of the game, the user still has more input into the images on-screen than in other media, such as cinema. Users create specific spatio-temporal and affective experiences as they interact with images on-screen in gaming
108 Väliaho, Biopolitical Screens, 4–6. 109 James Ash, “Emerging Spatialities of the Screen: Video Games and the Reconfiguration of Spatial Awareness,” Environment and Planning A 41 (2009): 2119.
135
environments. Seeing themselves in a different body on-screen, the user/player’s lived temporality differs from the temporality of images unfolding on-screen.110 In video games, the interactant co-constitutes images not only on-screen but also off-screen. The different mode of seeing, perceiving and connecting in games, James Ash argues, affects the way things are touched by users and moved around in everyday life. In an attempt to show how the different modes of seeing and moving in videos games produce a “synaptic rewiring” of the body, Ash explicates that grasping in video games requires engagement with buttons by applying pressure on them.111 This is different to the way things are held in everyday life: “haptic and visual sense perceptions are reversed: the visual intensity of light and colour from the screen take on the haptic function that weight and texture play in holding an object or navigating in extended space.”112 This kind of engagement with images goes beyond the haptic quality of images as discussed by Gilles Deleuze and Laura Marks, and indicates how these images have rewired the movement of body when not playing games.113
The second way in which I consider the relationship between the gesturing, distributed body and images concerns the ways in which gestures performed in front of screens are not the only gestures to be scrutinised. We should also consider gestures that modulate a participating body. Various technologies are able to process human bodily gestures as data, measure them, compare them with other data, and display them on-screen for evaluation. As discussed earlier, this is seen in Breath but another example would be the Fitbit device and app, a wearable technology that tracks numerous activities of human
110 Here I am thinking of Juhani Pallasmaa’s discussion on Tarkovsky’s Nostalghia (1983) and the experience of on-screen time by the viewers. Juhani Pallasma, The Architecture of Image: The Existential Space in Cinema (Helsinki: Rakennustieto, 2007), 79-86. 111 Ash, “Emerging Spatialities of the Screen,” 2106, 2117. 112 Ibid., 2117. 113 Gilles Deleuze, Francis Bacon: The Logic of Sensation (London: Continuum, 2004); Laura Marks, Touch: Sensuous Theory and Multisensory Media (Minneapolis: University of Minnesota Press, 2002).
136
body, such as the number of steps walked or climbed, calories burnt, sleep hours, and even the quality of sleep. The data generated by the Fitbit device can be monitored on other devices as well. The monitoring of bodily functions via screens to increase performance signals a direct intervention of screens in the regulation of behaviour. This is another way in which the human body is entangled in the network of screen, image and politics.114 This instance demonstrates that screens can receive and transport data and images and hence cradle the constant exchange between external apparatuses and the human body.
Gestures, however, do not always have to be performed in front of screens. In 2018, Apple was granted a patent for “motion and gesture input from a wearable device.”115 The Apple Watch uses some of the gestures that this patent covers, but most of these gestures are yet to be included in the Apple Watch or any other device; nevertheless, the patent can be viewed as a list of possibilities for a device in the future. However, the functioning of the device is not just based on the gestures of a human body but are very much an illustration of the gestural capacity of the technical elements in the device. Apart from the fairly common optical, inertial and contact sensors that are already used in Apple Watches, there are plans to add myloelectric sensors that can sense finer degrees of body movement, such as those of muscles and
114 As more human activities are being quantified, these are contributing to the phenomena of ‘big data,’’ massive amount of information that are distributed across networks, which carry the potential to be used by government and private organisations. This leads to a range of issues, such as surveillance, civil rights, and data ownership. Critical discussion of these issues have been undertaken by: Rob Kitchin, The Data Revolution: Big Data, Open Data, Data Infrastructures and Their Consequences, London: Sage, 2014); José van Dijck, “Datafication, Dataism and Dataveillance: Big Data between Scientific Paradigm and Ideology,” Surveillance and Society 12, no. 2 (2014): 197–208; Chris Till, “Exercise as Labour: Quantified Self and the Transformation of Exercise into Labour,” Societies 4, no. 3 (2014): 446–462; Rita Raley, “Dataveillance and Counterveillance,” in Raw Data is an Oxymoron, ed. Lisa Gitelman (Cambridge, MA: MIT Press, 2013), 121– 146; Danah Boyd and Kate Crawford, “Critical Questions For Big Data,” Information, Communication & Society 15, no. 5 (2012): 662–679, https://doi.org//10.1080/1369118X.2012.678878. 115 Andrzej Baranski et.al. Motion and gesture input from a wearable device. US Patent: US20160091980A1, filed February 6, 2015, and issued, November 26, https://patents.google.com/patent/US20160091980A1/en.
137
tendons, in order to activate certain functionalities.116 The movements of the body can modulate the device to perform certain functions as the watch in turn would effectively determine the gestures and commands associated with it. Touching the screen to perform gestures would not be required for functions to be activated.117
Such patents provide the impetus for people to be gesturing profusely and perhaps involuntarily as they go about other basic tasks such as walking. This could lead to circumstances similar to the late nineteenth century where Agamben noted that gestures lost their meaning as everyone was gesticulating frantically. Moreover, in contemporary times, such gesturing has a new way of feeding into the screen and images without frantically being performed in front of or on the screen. Instead, subtle gestures are automatically detected then silently fed into the machine. The nexus that forms between software, the human body and screens changes from the usual keyboard-hand-eye- screen arrangement.118
I have so far discussed the conditions of postmedia assemblages where screens, bodies and images create a gesturality in which the location of the gesturing body is obfuscated. What or who is gesturing cannot be answered easily, more so because gestures are not initiated by one entity alone. Rather, gestures circulate through the multiple relations of entities in the assemblage.
116 Michael Peterson, “Apple Patents Method to Control Watch with Hand Gestures and Movements,” Apple Toolbox, May 1 2018, https://appletoolbox.com/2018/05/apple-patents-method-to-control-watch-with- hand-gestures-movements. 117 See, for example, the work of Atau Tanaka, whose practice explores the use of embodied sonic interaction gestures. 118 The scope of the thesis does not allow me to dwell on this topic. But some sources worth noting are the studies on inter-sensorality and sense perception undertaken by the Centre for Sensory Studies, Concordia University, Canada, http://centreforsensorystudies.org. On sensory capitalism and the role of senses in the political-socio-technical nexus, see Chris Salter, “Disturbance, Translation, Enculturation: Necessary Research in New Media, Technology, and the Senses,” Visual Anthropology Review 34, no. 1 (2018): 87–97. For a discussion of how wearable technology governs human habits, see Natasha Dow Schüll, “Data for Life: Wearable Technology and the Design of Self-care,” BioSocieties (2016) 1–17; Danielle Wilde, “A New Performativity: Wearables and Body-Devices,” in Re:live Media Art History, Melbourne, November 2009, conference proceedings, http://www.mat.ucsb.edu/Publications/burbano_MAH2009.pdf.
138
This circulation of gesture utilises the capacity of entities to form and deform relations. In this sense, gestures are already set up as part of the potentiality of all bodies; gesturality is always emergent in the relations of the entities of the assemblage. Although gesturality may already be present, it is screens that set up the potentiality for gestures to circulate. Being in between body and image, body and machine, and between and across image and software, screens interface between these multiple entities and, in the process, facilitate a particular kind of gesturality. The gestural capacity of screens to create, break or expand relations with various parts of an assemblage means that they surpass the status of being a mere container of images. Their gestures indicate that screens support and condition the possibilities through which several components associate and engage with each other. The generation of this gestural space, I argue, maintains contemporary mediality; that is, this gestural space is part of the conditions for the ongoing performance(s) of postmedial gesturality. In this sense, I argue that the gestures of screens also maintain (post)mediality, considered here not as medium specificity but as the continuance of the assemblage itself. To elaborate on this understanding of gestures as that which sustains activity, we will turn to Agamben once again.
THE THIRD GESTURE Until now my investigation into gestures has followed two paths: one that tracks the function of gestures and the other that examines the location of gestures. By discussing the uncertainty about and confusion over both meaning and location of gestures, I have demonstrated that there is neither one meaning for gestures nor one body performing gestures in a postmedia assemblage. In the last section, I also suggested that the gesturality of a postmedia assemblage is situated in the movement and engagement of elements of an assemblage. The interplay between these elements can be
139
considered as gesturing; that is, the performance of gestures lies inbetween the entities. In ascertaining the gesturality of screens, I argue that screens sustain mediality in postmedia. Such an understanding builds on the dynamic conditions of postmedia assemblage where non-semiotic gestures and indeterminate gesturing bodies are considered. In a postmedia assemblage, entities do not pre-exist but are in a state of emergence. The intra-acting capacities of the assemblage, as discussed above, attest to the unlocatability of gestures and their unfixedness. Here gesturality has to be reformulated to take into account the ongoing performance of screens. I propose to do this with Agamben’s definition of gestures as pure means. This analysis will also extrapolate the significance of the intermedial position of screens in gesturality in postmedia.
As discussed earlier, Agamben refrains from seeing gestures as providing psychological insights nor does he assign them symbolic meaning.119 Gestures can also not be expressed in language; not simply because gestures and language are two different modes of communication, but because language is incapable of communicating gestures.120 Instead, Agamben elaborates on a third role for gestures, placing them in the realm of non-representation.121 By stating that gestures offer “pure and endless mediality,” Agamben refers to their status as between causality and end, effect or result.122 Drawing upon Aristotle’s poiesis (making) which is a means to an end, and praxis (action) where the action itself is the end, Agamben situates gestures in the middle of poesis and praxis, where they become means without end.123
119 Agamben, “Notes on Gesture”, 50–51. 120 Ibid., 59. While considering Eisenstein’s use of sound discussed earlier, vocalisation, whether using words or not, should be excluded from the linguistic production of ‘language’ here. 121 Agamben, “Notes on Gesture”, 58. 122 Ibid. 123 Ibid., 57.
140
Nothing is more misleading for an understanding of gesture, therefore, than representing, on one hand, a sphere of means as addressing a goal (for example, marching seen as a means of moving the body from point A to point B) and, on the other hand, a separate and superior sphere of gesture as a movement that has its end in itself (for example, dance seen as an aesthetic dimension). Finality without means is just as alienating as mediality that has meaning only with respect to an end.124
This passage suggests a thinking of gestures as mediality that is ongoing and not a simple passage to reach an end. Agamben argues that both a pornographic film and mime deploy gestures that are “suspended in and by their own mediality,” as reaching the ‘goal’ in both these cases would complete the act and in a certain way end the purpose of the action itself.125 These gestures then have to be understood as maintaining the action; in other words, they are medial. If gestures are means that never arrive at a specified goal, then their function is to “endure and support” the action.126 This means that gestures can be understood as movements that continue action, propelling as well as cultivating it, without concerns over its completion. Agamben’s essay “Notes on Gesture,”, which forms the basis of my analysis here, has been largely inspired by Max Kommerall’s ideas on gesture.127 Here gestures are the “communication of a communicability,” that refers more to the potential for meaning and transmission than to the expression of fixed signification.
In this sense, some works that I have mentioned earlier in a different context, such as Mnemosyne Atlas, can also be studied as gestures that sustain an action.
124 Ibid., 58. 125 Ibid., 57. 126 Ibid., 56. 127 Giorgio Agamben, “Kommerell, or On Gesture,” [1991] in Potentialities. Collected Essays in Philosophy, trans. Daniel Heller-Roazen (Stanford, CA: Stanford University Press, 2000), 77–85. Parts of this text overlap with Agamben’s “Notes on Gesture.”
141
Agamben also gives examples of gestures producing “pure and endless mediality” in other artists’ work, for example Sergei Diaghilev’s ballets, Isadora Duncan’s dance and Rainer Maria Rilke’s poetry.128 In these cases, it may seem that Agamben is concerned with the gestures of human bodies only. However, I will later extrapolate Agamben’s idea of gestures in images and use this to further my arguments about gestures in postmediality.129 My concern in studying this ‘other’ meaning of gestures is to open a door to a discussion on the instability of screens, on how they become through material- discursive practices, and to elicit screenness — that is, the enactment of agential relations — in the contemporary postmedia assemblage. Through detailed discussion on (inter)medial gesturality of screens, I will pave the way for a more example-led discussion across the next two chapters of the thesis.
If gestures produce mediality, then this calls for examining understandings of mediality in the context of this research. Mediality of screens in postmedia, I argue, can refer to two states of being medial. One supports Agamben’s concept of mediality as “being-in-a-medium.”130 This is not to be confused with being the medium. It is easy to confuse the screens with the medium, but they are not media itself; instead they maintain mediality as some of the earlier discussion have indicated. Second is the position of being in the middle or ‘inbetween’ two entities. These two meanings of mediality are not the only ways to understand mediality nor are they exclusive, albeit that they are relevant to the gesturality of screens. In a brief discussion below, I ascertain how these two ideas of mediality are enacted by screens.
128 Ibid., 53. 129 Moreover, Agamben is concerned with the ethical dimension of gestures in order to make connections between gestures and philosophy, and philosophy and cinema. This gives us an idea about where gestures figure in the larger context of Agamben’s philosophy on ethics, power and politics in human life. See Giorgio Agamben, Homo Sacer: Sovereign Power and Bare Life (Stanford, CA: Stanford University Press, 1998); Giorgio Agamben, State of Exception, trans. Kevin Attell (Chicago: University of Chicago Press, 2005). 130 Agamben, “Notes on Gesture,” 58.
142
First let us discuss how Agamben deploys a concept of mediality. He remarks that gestures put mediality on display by making the means visible.131 Here he does not indicate that gestures highlight the medium itself — understood within contemporary media studies through concepts of presence, immediacy, hypermediacy and transparency — as these are transmitted via different forms of screen display.132 Such discussions have accounted for both awareness of the medium as well as its invisibility and both aspects are seen as facilitated via screens. Mediality in this context is also not what Jørgen Bruhn has described as being “related to the process of mediation in communicative situations.”133 Bruhn places mediality in the communication model between the sender and receiver.134 The communication model involves a message and its transfer emphasising transfer of signals directed by the sender to the receiver. Bruhn then is not concerned with the meaning of the message but uses the term mediality as a process of mediation, referring to its being between. While this can be useful for our discussion here, however, Bruhn regards medialities as a “crucial part of any communicative process” placing it in two specific “passages” of signal: those between the user and the message, and the receiver and the message.135 Since his framework is grounded in communicative messaging, Bruhn’s idea of mediality fundamentally conflicts with our understanding of gesturality that rejects the function of gestures as primarily communicative.
131 Giorgio Agamben, “The Author as Gesture” in Profanations, trans. Jeff Fort (New York: Zone Books, 2007), 58. 132 See, for example, Matthew Lombard and Theresa Ditton, “At the Heart of It All,” Journal of Computer Mediated Communication 3, no. 2 (1997), https://doi.org//10.1111/j.1083-6101.1997.tb00072.x; Thomas Elsaesser, “The ‘Return’ of 3-D: On Some of the Logics and Genealogies of the Image in the Twenty-First Century,” Critical Inquiry 39 (2013): 217–46; Jay David Bolter and Richard Grusin, Remediation: Understanding New Media (Cambridge, MA: MIT Press, 2000). 133 Jørgen Bruhn, The Intermediality of Narrative Literature. Medialities Matter (London: Palgrave MacMillan, 2016), 16. 134 Ibid., 17. 135 Ibid. Bruhn uses the term mediality to replace ‘medium’. See Bruhn, “The Intermediality of Narrative Literature,” 16.
143
Instead, Agamben talks about gestures sustaining mediality. While discussing how images can be gestures in cinema, he argues that images become “pure means” by displaying such a mediality that “[the image] shows itself as such,” for example when an actor looks directly into the camera in order to address the audience.136 In other words, images become gestures that highlight their mediality, that is being-in-a-medium.137 To be sure, this means that the images are the focus without becoming the medium.138 Mediality in this sense is a complicated state. In order to explore this more, let me deliberate on how Agamben has discussed the concept of imagelessness, a being-in-a-medium of images in certain conditions. But since Agamben has not used the word gestures, the conditions of such mediality have to be derived from a close reading of the text.
Agamben calls the state where images are not representing anything but are themselves “imageless.”139 The name may seem contradictory but instead implies that the meaning of the images do not have be derived from an external referent. He elaborates that this state of images where they have to signify their being an image (in a way that a sign signifies that it is signifying) can only be shown and not said; it is “unutterable” in that sense.140 This again alludes to the image as performing gesturality without Agamben directly saying so. The two seemingly contrary ways suggested by Agamben to show such imagelessness occur first in pornography and advertising where imagelessness is seen in the insufficiency of an individual image since there are always more images to be seen. The second is to show an “image as image” as seen in Guy Debord’s early work Hurlements en faveur de Sade (Howls for
136 Giorgio Agamben, “Difference and Repetition: On Guy Debord’s films” [1995] in Guy Debord and the Situationist International: Texts and Documents, ed. T. McDonough (Cambridge, MA: MIT Press, 2002), 318. 137 Even though Agamben does not use the word gestures, “pure means” alludes to them. 138 Agamben, “Difference and Repetition,” 318. 139 Ibid., 319. 140 Ibid.
144
Sade) (1952); that is, there are no more images to be seen.141 In fact, imagelessness is another gesture that maintains mediality, a ‘state’ where images do not mean anything other than imaging itself. In other words, imagelessness is a gesturality of images.
In considering this similarity between the gesturality of images and the gesturality of screens, I want to extend the argument to say that a gesturality of screens lies with screens supporting the medium without being the medium; that is, screens are able to create conditions that enable them to persist. Screens are, then, being-in-a-medium.142 If this is the case, then what happens to the gesturality of screens in the postmedia condition where the medium is unstable? How are they able to sustain actions and engagements? Screens are not determined through a particular medium — they are not only intermedial but also intramedial. This is one of the reasons why screens are an important part of the postmedia assemblage as they perform the gesture of converging various media.
The second idea of mediality is produced by a gesturality that derives from being in the middle. The inbetweenness of screens in an assemblage can be associated with this kind of gesturality. Screens support the actions of the machinic components (hardware, software) and humans in an interactive art installation by constituting an interface for engagement across participant and computation. But this interface is not simply what one sees or what is displayed. Screens perform interfaces as both revealed and hidden spaces of engagement. The algorithmic functions, calculations and signal transfers made by software are concealed by screens. On the other hand, screens
141 Ibid., emphasis added. 142 I am aware that this can seem similar to the concept of immediacy, the transparency of a medium, given by Bolter and Grusin in Remediation: Understanding New Media. But here, ‘being-in-medium’ is in the context of postmedia, where media are complicated and not easy to identify one medium from the other. Screens in postmedia can arguably gesture towards the medium itself, which contradicts the idea of immediacy.
145
express what is required and understood by the participant and leave out the processual information of the task. This way the gesturality of screens lies in the traversing capacity of screens to sustain both the hidden and revealed.
The inbetweenness of screens is more than their spatial position; it speaks to their abilities to form connections. Discussing inbetweenness in interactive art, Munster points out that “the inbetween is not an occupiable space but rather a dynamic and moving register of tendencies.”143 Munster looks beyond the spatial connotation with which ‘betweenness’ is generally understood and instead defines inbetweenness as a potential for deep and ever-changing relations between technical entities and humans. This idea of inbetweenness can be viewed as what comprises the gesturality of screens in their capacity to form relations with various parts of the assemblage in which they participate. Through relations, screens are able to execute certain functions, take on roles and express affects. Inbetweenness would not be ascribed a location here but is produced through the (ongoing) arrangement of entities.144 The configuration of entities in turn depends upon their capacities to form relations. Movement, then, takes place according to these relations.
This idea that a body’s movement is relational is echoed by Manning. She suggests that a movement “move[s] into gesture” when the relation changes and that the gesture is not its beginning.145 Gestures are always in the middle, sustaining relations with bodies based on the latter’s’ potentialities, but also shifting experiences and opening bodies to emergent potentials. As discussed previously, the circulating gesturality between bodies, images and screens develops through the capacities of these entities to form relations.
143 Anna Munster, “Into ‘inter’: The Between in Interacting,” Rivista di estetica 63 (2016): 59. 144 In postmedia, inbetweenness is not just a property of one thing but can also be a quality of an assemblage. I will discuss this further in chapter three. 145 Manning, Relationscapes, 76. Emphasis in original.
146
To comprehend the inbetweenness of gestures, we need to pay closer attention to the specific situations of the assemblage. Elaborating on Agamben’s concept of gestures, Jill Bennett remarks that gestures reveal the “conditions of mediality.”146 This provokes us to think in terms of the relations that sustain gestures in certain situations. These relations are not established but are continuously negotiated as and through agential capacities and cuts (as discussed in the previous chapter). The connections not only produce movements that we are able to see, but also “excavat[e] inward movements that endure at a deeper level.”147 The gestures of screens are generated as shifts in intensities and affects, leading, for example, to the sensorial experiences of a media art installation.
Further, the inbetweenness of screens is also supported through the prospect of further action. In their relational entanglements, screens present new instances of interaction that may spur further actions. Breath provided an example of how this occurred. Agamben argues for consideration of the author as a gesture when they mark themselves as reflexively present in a work; for example, they disturb the continuity of the story by constantly unfolding the plot.148 Here he makes gesture an undercurrent that directs or provides tendencies for other elements. Similarly, the mediality sustained by screens makes them an interface offering a similar absence-presence role that remains “unexpressed in each expressive act.”149 In this way, screens are not only gesturing, but can in some instances be considered gestures of media art installations in and of themselves.
146 Jill Bennett, “Aesthetics of Intermediality,” in Art History 30, 3 (2007): 436. 147 Patricia Pisters, “Image as Gesture: Notes on Aernout Mik’s Communitas and the Modern Political Film,” Journal for Cultural Research 19, NO. 1 (2015): 69–81, https://doi.org//10.1080/14797585.2014.920186. 148 Agamben, “The Author as Gesture,” 70. 149 Ibid., 66.
147
In an analysis of Shilpa Gupta’s Shadow 3 (2007), and also Myron Krueger’s VideoPlace (1985) and Rafalel Lozano-Hemmer’s Body Movies (2001), I want to use this conception of gesturality as sustaining mediality to help us think through what happens in a media art assemblage. This discussion roams over works that strictly speaking do not belong to the postmedial condition, but nevertheless signal it. However, these works put screens to work in specific ways that begin to allow us to track how screens come to function gesturally to support the ongoing production of the postmedial assemblage.
The installation Shadow 3 gives space to participants to use their silhouettes (which appear as shadows on a screen in front of them) to connect with the shadows of various objects that appear on-screen (fig. 22). The use of silhouettes to realise on-screen the actual bodies of participants off-screen is not new in media art. Several works have used silhouettes on-screen in similar ways, the most significant being Krueger’s VideoPlace, a pioneering work in interactive media art. Here, the silhouette of a participant is projected on to the screen in front of their moving body. They can move to change the dimensions, shape and colours of the silhouette itself and then interplay with other on-screen objects (fig. 23) such as mountains, seas, and miniature versions of themselves.150 This interaction can take place with others, alone or with another participant in a separate room.
150 See details here: Myron W. Krueger, Thomas Gionfriddo and Katrin Hinrichsen, “VIDEOPLACE—An Artificial Reality,” ACM SIGCHI Bulletin 16, no. 4 (1985): 35–40.
148
Figure has been removed due to copyright restrictions.
Figure 22: Shilpa Gupta, Shadow 3, 2007, interactive video projection incorporating the viewer's simulated shadow. Digital video still from documentation. Photographer: Anil Rane. Courtesy: The Artist & Galleria Continua / Le Moulin, San Gimignano / Beijing / Les Moulins / Habana.
Figure has been removed due to copyright restrictions.
Figure 23: Myron Krueger, VideoPlace, 1985, interactive shadow installation. Video still from live interaction, https://aboutmyronkrueger.weebly.com/videoplace.html.
Other works that use shadows on-screen, such as Multiple Shadow House (2010) by Olafur Eliason and Frost Frames (2000) by Shiro Takatani, are fundamentally different from VideoPlace and Shadow 3, relying on the shadows of the audience members but not their silhouettes. Multiple Shadow House and Frost Frames require the audience to stand or move in front of the lights in the
149
installation and cast shadows on the screen. On the other hand, both VideoPlace and Shadow 3 use cameras and sensors to record the image of the participant before (re)projecting them as silhouette-shadows. Body Movies, from the Relational Architecture series by Lozano-Hemmer, also uses the shadows of audience members. This work does cast shadows by obstructing light, but the shadows of the participants reveal other portraits that are projected on the screen (fig. 24). These other portraits are not seen against the brightly lit building used as the projection surface in the work, but can only be seen when the audience obstructs the light, casting their shadow and exposing the portraits.151 Hence I will discuss, Body Movies as lying more closely conceptually and at the level of gesturality and screens with VideoPlace and Shadow 3 .
Figure has been removed due to copyright restrictions.
Figure 24: Rafael Lozano-Hemmer, Body Movies, Relational Architecture 6, 2001, interactive shadow installation. Digital video still from interaction at Atlantico Pavillion, Lisbon, Portugal, 2002. Photo by: Antimodular Research.
151 For further details on the artwork and its numerous locations where it has been exhibited, see: http://www.lozano-hemmer.com/body_movies.php.
150
In VideoPlace, Body Movies and Shadow 3, there is a connection that is formed not just with the images or shadows that are projected on the screens, but with the screen itself, although there are different modalities at work in these installations. Screens are gesturing by sustaining mediality in the two ways described above. The inbetweenness of screens sustains mediality as they reconfigure the relationship of the participants’ bodies with other entities. The details of the bodies of the interactors are hidden, while their shadows reveal different connections with objects on-screen in case of Video Place and Shadow 3 and images in case of Body Movies. There is a constant reconfiguring of relations, assisted by screens, between audience members and the elements of the installation. For example, in Shadow 3, the audience can transfer the shadow objects to other silhouettes; while in Body Movies, different sized portraits on the building are revealed inside the different lengths of audiences’ shadows. This way the screens are also supporting and relating the actions of different components in the medium. Thus, they are being-in-a-medium.
It may seem that the gesturality of screens is functionally the same in all three works — that they hide and reveal, create human-machine assemblages, relate different parts of the assemblage, and participate in the intra-actions between machines and participants. However, the performance of that gesturality is different as each installation emphasises a particular reconfiguration of relations among entities. All three works use silhouettes and shadows, so the imagistic details of bodies are missing. Yet given such similar situations, the gestural performances of the screens are different. While discussing VideoPlace, Munster has pointed out that since the silhouette on-screen is not the exact copy of the body in front of the screen, it is a “body of information” rather than one of “representation.”152 The physical body tries
152 Munster, Materializing New Media, 146.
151
to adjust to the on-screen visualisation without much success. There is a perpetual gap, Munster identifies, between the physical and informatic bodies.153
I argue that the screen’s gestural capacity lies in maintaining this gap. The software provides instances that force the body to act in different ways. At the same time, the incapacity of the software at visually replicating the participant’s body means that the participant is constantly relating to the image on screen.154 In this way, screens facilitate the relation or the lack of it between the two kinds of bodies. In Body Movies, the screens reveal images that have already been projected but have not been seen. The new images that are seen in and through the shadows of the participants on-screen are a ‘revealing’ that takes place from ‘inside’ of the body. As the images are seen only in the darkness of the participants’ shadow, it is the gesturality of screens that reveals the image. Comparatively, in VideoPlace and Shadow 3 the silhouettes are opaque and only seem to connect and reveal their capacities by connecting to other objects externally.
Focusing on Shadow 3 reveals a transformational gesturality at work for screens. Here the screen is used by participants to connect and co-compose with their silhouettes. It is an object of orientation through which participants engage their own bodies. Watching their silhouettes on-screen combine with various objects, participants experience their body external to themselves. The gesturality of screens constantly reconfigures relations between the shadows of objects and the participants, and also between the ‘informatic bodies’ and the lived corporeality of the participants. The continuous reworking of relations and the becoming of entities brought in to mediality by
153 Ibid. 154 Ibid.
152
screens requires a further expansion of the inbetweenness of screens. For this, I suggest Richard Grusin’s radical mediation provides an understanding that is sympathetic to my discussion of the gesturality of screens in postmedia.
Grusin’s radical mediation takes into account the two underlying aspects of this research: firstly, exchange and circulation across media, and secondly, the entanglement of humans with technologies. Radical mediation does not reduce mediation to media technologies but instead regards mediation as a process that can be registered by both humans and nonhumans.155 Grusin’s expanded concept allows for different kinds of mediation:
Radical mediation does not take mediation as a unifying or totalizing epistemological concept that holds together disparate and heterogeneous practices, events, and entities. Nor does it maintain ontologically that there are only disparate and heterogeneous objects and things that do not relate to each other. Rather, radical mediation takes everything as a form of mediation. Because mediation is always transformative, one of the things that is radical about mediation is its ability dramatically to change scale, moving among smaller and larger, simpler and more complex, briefer and more extended assemblages or entities.156
From this passage, mediation can be thought of as transformative where entities are created in the process of mediation. The intra-activity in postmedia assemblage that I have discussed tells us that entities are co- emergent with each other. The screen in Shadow 3 is not a ‘screen’ without the shadow-silhouettes and the shadow-silhouettes only become such by ‘appearing’ on the screen. The shadows of the objects attach to the shadows of the participants and take a different form in the fusion. Screens are doing
155 Richard Grusin, “Radical Mediation,” Critical Inquiry 42 (2015): 126. 156 Ibid., 145–46.
153
much more as they transform the experience of the participants as they move their bodies, act, mime and combine with other shadows that fall on-screen.
Grusin claims that radical mediation is “affective and experiential rather than strictly visual.”157 The affective experience of a non-physical object attaching to a body in Gupta’s work may burden the participants, reminding them of the baggage that they constantly carry in various forms such as habits, histories and conventions as well as previous experiences.158 As the participants transfer the shadows of the objects to other participants, they not only relocate the shadows of the objects, but also transport and transpose an experience of attachment with the objects. The ability to gather this space of transference into a flat plane also facilitates the relations of one participant to another. During these transfers, the screen is gesturing, literally maintaining the action by facilitating two important connections. Firstly, the combination of the shadows of the objects and the bodies of participants is different with each fusion, as are the shapes and affect produced. Secondly, at any given time the silhouettes of a number of participants are on the screen. Hence the screen provides and produces a collective experience of being-in-a-medium. Both the reconfiguring of relations and the variable and affective experiences of the installation can be comprehended as and through the gesturality of screens.
Ultimately, Agamben’s concept of gestures as “pure, endless mediality” leads us to recognise gestures to be other than meaning-laden and intent on realising particular objectives. Integrating this idea of gestures into a
157 Ibid., 132. 158 The shadows in Shadow 3 have been analysed from the perspective of Jungian-Buddhist philosophy by Nancy Adajania, who regards them as the “lost shadow material of consciousness” which have to be accosted for them to disappear. See Nancy Adajania, “Darkness is What Light Will Never Be: Shilpa Gupta’s Experiments with Truth,” in Shilpa Gupta, ed. Nancy Adajania (Munich: Prestel Verlag, 2010).
154
postmedial scenario, I have considered a gesturality of screens in three ways. First, screens allow relations to endure in the assemblage. Second, screens perform the redistribution and reconfiguration of these relations. Third, screens enact relations particular to the assemblage of which they are a part. If the gesturality of screens can be accepted as a performance of mediality in a condition where relations are transformational, then screenness, a specific agency of screens in postmedia, can also be highlighted. In such a dynamic postmedial state, screens are able to sustain actions, movements and relations and, so doing, the postmedial condition itself. How this gesturality is performed under specific postmedial conditions and practices will be the focus of my interrogation in the next two chapters.
155
S T R I K E
An interlude
156
Figure has been removed due to copyright restrictions.
Figure has been removed due to copyright restrictions.
Figure has been removed due to copyright restrictions.
157
Figure has been removed due to copyright restrictions.
Figure has been removed due to copyright restrictions.
Figure has been removed due to copyright restrictions.
Figures 25-30: Hito Steyerl, Strike, 2010, video HDV, 28 sec. Digital video stills.
158
White letters in bold on a black background A woman looks at something on her left She walks towards a big, blank, black screen Bringing with her a chisel and hammer The screen reflects them — the off-screen creeps into the frame But not as on-screen content. With her chisel and hammer she strikes the screen The screen lights up, With an image.
The blackness filling the screen overtaken by an image. Its blankness couldn’t be tolerated, was too much. Black-blank becoming interchangeable. The blackness might swallow everything around it, so hurry up and supress it… put something there, replace it, hide it, insert an image over it. Take away the emptiness and replace it with representation. Images illustrating, always saying something, always taking ‘us’ somewhere, through someone, to another screen. An encompassing blackness no more. Is it so difficult to bear it? Aren’t we all born in blackness?
In the twenty-second sequence of Steyerl’s work Strike (2010), a simple gesture of striking coincides with the screen gesturing the image.
This act of striking is a gesture where both the thing that is hit and the thing that hits change. They connect, touch, impress, move, release, break. ‘Strike a pose’ — a command, a gesture, a position. All movements bring out affectivities. A striking screen launches, accelerates the processes of circulation, movement, transformation. But it can also strike — refuse to work. Like the workers who strike to resist or change practices, and to be heard and seen, perhaps a striking screen also takes a stand. What is this
159
stand that screens take when striking with images or striking to not show them? What does their blankness show?
The chisel and the hammer in Steyerl’s hand indicates a strike that is also a sculpting action. The artist strikes the stone to carve, modelling bodies into poses. In a similar fashion, the artist sculpts the images on-screen — moulds, composes, composites them. The screen then also creates. Not just the images, but we who emulate those images. Images may indeed go beyond the screen but instead of being representations, they move through affects, percepts, gestures. Even if the screen is empty of content, does this necessarily mean it will do nothing? Could such black blankness also be generative? The screen on strike is a blank-black screen of a gesture about to become.
Strike is, after all, videoing blackness: Steyerl is dressed in black; the set uses black drapes; the lighting highlights rather than fights blackness; there is a black television on a stand; the blank screen is filled with blackness. And in the end, when an image leaps on-screen, it doesn’t take over; blackness still escapes through cracks in the screen.
Having collected my conceptual tools across chapters one and two — my chisel and hammer — I am ready to strike and attend to the gestures of screens. Screens are not simply objects to be ‘struck’ but ‘strike’ as well. This will be elaborated going forward where we will see that the out-of-frame and black/blank screens become important gestures for understanding the particular strikings of screens. Indeed, screens are not objects, screens are an act; screens enact.
160
3. >>(OUT-OF-)FRAME GESTURES<< ENACTMENT OF MULTIPLICITIES AND TEMPORALITIES OF AND BY SCREENS
A simple idea of frames is prevalent in the recording and display of medial images. While taking a photograph, the photographer ‘frames’ the scene to include elements that they want to capture, which are subsequently organised according to arrangements of depth of field (focus, foreground and background), angle (top shot, low angle) and size of the image’s subject (long shot, mid shot, close-up). This spatial enclosing of the frame can be contrasted to its temporal viewing, which for a moving image is explicitly activated by the frame rate —the speed of a sequence of images over a specified time. In an attempt to understand the spatial and temporal aspects of the frame, it is also imperative to account for its countability in terms of physical dimensions, relations to focal length and depth, quantities per unit of time and so on. While all this spatialisation and enumeration renders images in multiple quantitative ways, this kind of multiplicity may not be adequate to the postmedial condition where variation and changeability of relations makes quantity alone insufficient. Here a multiplicity of types of frames and their relationality have to be interrogated.
This chapter aims to analyse how the multiplicity of frames, so common to postmedia, can be rethought via a temporal understanding of the gesturality of screens by way of the out-of-frame. In order to pursue this, the chapter has three interrelated moves. First, I will illustrate the nuances of the spatio- temporal performances of the frame beyond preoccupations with the many frames of postmedia screens, or the merely quantitative. I propose instead
161
that the qualitative screenic enactments lead us to think of the gesturality of screens as being primarily temporal in postmedia. With a focus on Natalie Bookchin’s Now he’s out in public and everyone can see (2012), I analyse how multiple frames and screens do not only produce a quantitative spatial distribution but also create temporal differences. The work unites multiple frames, screens, and speakers in the exhibition space, but both Bookchin’s editing techniques and the display of the work produce temporal fragmentation and multiple rhythmic flows. There seems to be many temporalities operating. Yet, this works in tandem with the multi-durational performance of the work where we experience time qualitatively; that is through stretched, asynchronous, contracted, and variable intensities. Bookchin’s work points us towards considering that its temporal qualities are not constrained by the frame in spite of its many appearances in her work, rather we are drawn beyond frames. Reading through Gilles Deleuze’s theories of cinema, the ‘whole’ that the framed image on-screen is connected to is out-of-field, and this motivates us to look beyond the frame. Moreover, in postmedia frames themselves lose fixity, bringing us to our next move.
Second, then, I describe multiple temporalities of postmedia assemblages that allow the out-of-frame to endure beyond the framed image. I argue that in postmedia, the out-of-frame as the ongoingness of multiple temporalities can itself be considered a gesture of screens. This leads to the production of different durational experience in a postmedia assemblage where screens endure and reconstitute several relations both in and through the out-of-frame. It is imperative to move beyond the idea of screens ‘containing’ frames and frames ‘enclosing’ images that marks an older understanding of screen media. Indeed, this has only metaphorically carried over into digital media and now in the postmedial condition it is vacated. We need to turn, rather, to the multiple temporalities of the out-of-frame in postmedia and their explication.
162
I will demonstrate that the postmedial out-of-frame — or the variable outsides of screen space — maintains an ongoing relation to the ‘inside’ of the frame, and in fact supports and conditions it. Moving onto a different configuration of time in postmedia, I will examine the out-of-frame as a relation in a postmedia assemblage that always provides the possibility to move into the frame. It can therefore be understood as a becoming-frame. This potentiality of the out-of-frame is arguably an enactment of the temporal. Becoming-frame as the temporal movement of the out-of-frame towards the frame will be closely examined in Blast Theory’s work Can You See Me Now? (2003), where multiple frames and screens are arranged in a way that produces both discrete and continuous durations. I will discuss how in this game movements in space are disorienting, as space itself is convoluted by the mixing of physical and online spaces. At the same time players can feel a continuous succession of moments fused together; that is, as a qualitative experience. Moreover, in a particular instance in the artwork, a direct image of time or crystal-image is also experienced. In discussing Can You See Me Now?. I will explicate affective experiences of these temporalities.
This discussion culminates in a third move, where I elucidate how the multiplicities and simultaneity of frames and out-of-frames are experienced, resulting in a new kind of gathering of temporalities on one screen. I describe a particular temporal experience that is activated in the relations that audience members, moving around an installation space, form with screens. Here the postmedial multiplicity of screens is enacted in and through the intra-actments of the frame, out-of-frame and the actual space of display and experience. In Ragnar Kjartansson’s work The Visitors (2012) nine screens arrange an actual space into numerous simultaneous space-time-frames in the installation space. The multiplicity of frames punctures the various space-time of the performers, which then combines in a single frame towards the end of
163
the work. Adopting Deleuze’s concept of crystals of time, I propose that each screen here carries time-images where the sheets of past and peaks of present gather. The end image produces an affective temporality where spatial and temporal foldings of the frame and the out-of-frame are repeated in the space- time-body of audience members. The gestures of screens and audience movements in the installation space provides a differential yet continuous experience of the relational space-time conjoining of on-screen and physical spaces.
FROM MULTIPLE SCREENS TO MULTIPLICITY OF SCREENS In this section I trace how multiple frames have been formulated in media arts and cinema, such as a collection of frames on one screen and as multiple frames on multiple screens. I reconsider this multiplicity, following Lev Manovich, as “spatial montage,” a simultaneous presentation of a number of images possibly of different sizes on the screen, or simply as “windowing” in the context of computers, following Alexander Galloway.1 The popular technique of splitting screens has been used to mark a static simultaneity of time in different spaces. Both Manovich and Galloway consider this kind of montaging of space as an alternative to the temporal montage of cinema, where the images come in a sequence and have to be waited upon, whereas in a spatial montage all images are available together.2 In visual arts, multiscreen installation art fractures the attention of viewers by introducing non-linearity and provided multi-medial experiences. By way of explicating this multiplying of frames in cinema and installation art, I argue that we should consider this multiplicity as something already surpassing a spatialised montage on-screen and signalling a sense of durational experience.
1 Manovich, The Language of New Media, 322; Alexander R. Galloway, The Interface Effect (London: Polity, 2012), 114, 115. 2 Manovich, The Language of New Media, 323. Galloway calls this a kind of cut that is opposite to suturing. Galloway, The Interface Effect, 115.
164
Here I propose that the out-of-frame helps provide an understanding of framing in the postmedial context. The out-of-frame is coupled with the frame and supports it from the outside. It can even become the frame and hence entangle temporalities. With the help of Bergson and Deleuze, I seek to establish this registration of an out-of-frame that is made possible on-screen via the gesturality of screens. By revisiting what has been seen as the spatial splitting of screens, I want to demonstrate how gesturing screens have nonetheless been productive of qualitative durational experiences. This section forms the basis of my chapter, which later focuses on specific examples. It will lead us to understand how the explosion of configurations of multiple frames playing out through and across the gestures of screens has become the hallmark of postmedial moving image experience.
Traditional screen media regard frames as boundaries that enclose a view while also indicating that this continues beyond the frame. We ‘see’ only a part, then, of the full setting. This viewing of and through frames is analogously compared by theorists, from John Berger through to Bolter and Grusin, to the way in which looking out of windows composes a specific view rather than simply looking at the external world. Anne Friedberg scrutinises the relationship between frames and windows, noting that Leon Battista Alberti used windows to advocate perspectival study.3 Instead of considering windows as representations of reality — the outside world seen from architectural portals — they have to be understood as a ‘frame’, where historical figures and events than those of the viewer are depicted.4 The windows in Alberti’s case therefore emphasise a ‘framing’ of the view rather
3 Alberti’s use of windows has also been discussed by Grayson and Panofsky. See Anne Friedberg, The Virtual Window, From Alberti to Microsoft (Cambridge, MA: MIT Press, 2006), 12. Richard DeCordova has also examined how the cinema frame is radically different to that of Renaissance perspective. See Richard DeCordova “From Lumière to Pathé: The Breakup of Perspectival Space,” in Early Cinema: Space, Frame, Narrative, ed. Thomas Elsaesser (London: BFI, 1990). 4 Friedberg, The Virtual Window, 32, 35.
165
than its realism. John Berger, too, dismisses the idea of paintings as a “framed view” in favour of paintings being used to highlight the economic prospects of their owners.5 Both Berger and Friedberg are referring to the frame in relation to the image, as opposed to the rigid physical casing of the paintings.6 Thinking about remediation, Jay David Bolter and Richard Grusin remark that unlike the Renaissance paintings, the computer screen frames are windows through which the viewers enter the (cyber)space.7 Similarly, while thinking about what is inside the frame, its relation to the outside is acknowledged by both Friedberg and Berger. The importance of this relation, as I will argue, is crucial in the consideration of postmedial screens.
Friedberg’s conception of the window metaphor as a spatio-temporal separation for the viewer is closer to the idea of the cinema viewer. In cinema, the enumerability of frames has to be recognised, since in the recording process the camera frames a scene, which is then registered as a recorded ‘frame’, and then its screen display is a different kind of framing. However, instead of differentiating the screen and the frame, the conflation of screens and frames has been more common. Stanley Cavell merges screens, frames and images to indicate that they all are the same when light is projected: “The screen is a frame; the frame is the whole field of the screen — as a frame of film is the whole field of a photograph, like the frame of a loom or a house.”8 On the other hand, Vivian Sobchack engages with the experience of viewers watching a film. Accepting its “geometrical rectangularity,” she positions frames as the “secret boundary” of film’s vision, which is unbounded, and is
5 John Berger, Ways of Seeing (London: Penguin Books, 1972), 99. 6 The formalist aspect of frames as images in paintings has been accentuated by painters such as Piet Mondrian, who arranged lines and colours in a way that the images itself are frames. 7 Bolter and Grusin, Remediation, 180, 235, 236. 8 Cavell, The World Viewed, 25.
166
unlike frames visible to the audience.9 For Sobchack, frames are an organising principle that are encountered differently by film and audience. The frame of the film is a performance of the narrative and technical disposition of the profilmic space— things such as the film set and actors, which are in front of the camera — while for the audience the frame is the boundary of the film along with a physical framed screen on which the film is viewed. This describes a relationship between two kinds of frames.
The multiple kinds of framing activities are disregarded by an inaccurate consideration of frames functioning solely as perspectival windows. In moving images, framing exists at multiple levels — that of the camera, screen and the media format — although all of them are connected and influence each other. The frame rate is the speed at which images are recorded and displayed so as to appear as moving. The standard frame rate of 24 fps (frames per second) for film and 25 or 30 fps for video denotes the number of frames used to produce a continuous ‘moving image’ for the human eye.10 In this way, several frames are viewed continuously and sequentially.
In addition to this, the frame rate can also decide the act of framing itself. Cinema framing aesthetics are starkly different to the framing approach of smaller media formats such as television and gaming.11 This can also lead to
9 Vivian Sobchack, The Address of the Eye: A Phenomenology of Film Experience (Princeton, NJ: Princeton University Press, 1992), 131. 10 The NTSC system uses 30 fps as standard while PAL uses 25 fps. However, there is a variety of frame rates deployed in films, video and computers. Early films had a lower frame rate, generally of about 16 fps. These frame rates varied throughout as the film reel in the camera was turned by hand. Now films are shot as high as 60 fps (for example James Cameron’s Avatar 2) as higher frame rates can provide sharper images. Frame rates during the recording also vary depending on the aesthetic choice. A slower frame rate when played back at standard frame rate shows action in high speed while a higher frame rate shows the action in slow-motion. 11 Yu-Ching Chang and Chi-Min Hsieh, “Filmic Framing in Video Games: a Comparative Analysis of Screen Space Design,” Multimedia Tools and Applications 77, no. 6 (2018): 6531–6554, https://doi.org/10.1007/s11042- 017-4564-6; Jane Stadler, and Kelly McWilliam, “Cinematography: Writing in Light and Movement,” in Screen Media: Analysing Film and Television (Sydney: Allen & Unwin, 2009), https://doi.org/10.4324/9781003117230. Here the discussion steers towards the first-person point of view
167
other framings when media is converted, specifically from film to video. In the 1950s and ‘60s, widescreen formats, such as CinemaScope and Cinerama, were developed as an immersive experience for the audience.12 Charles Barr claimed that widescreen provided an immense awareness of the space offering a “greater physical involvement” of the viewer.13 These formats required specific a frame rate, film size and screen size for projection.14 For television broadcast, the frames of widescreen film formats had to be adjusted to suit the smaller screen ratio of the video format. Visually this adjustment utilised techniques such as letterboxing, where the film maintained its original aspect ratio by adding black bands in the top and bottom to fit the video format without having to compromise the panoramic dimensions of the image. Here, the cinematographic image of the film is framed within black bands, which in turn is framed by the television.
The change in aspect ratio from the original frame size of the image can also be used to communicate medial modification for the audience. Harper Cossar comments that the techniques of letterboxing are used as a “cinematic trope” in contemporary advertisements, animation and games.15 In video games, the cutscenes — non-interactive scripted sequences that add to the story and its characters — are letterboxed, signalling a change of behaviour by the gamer
frame that is offered in games, which is uncommon in cinema: Geoff King, Tanya Krzywinska, “Film Studies and Digital Games,” in Understanding Digital Games, ed. Jo Bryce and Jason Rutter (London: Sage, 2006), 115–119. 12 Charles Barr, “Cinemascope: Before and After,” Film Quarterly 16, no. 4 (1963): 4–24; Guy Gadney, “Broadband Futures,” LAMP, Australian Film Television and Radio School, 7 September 2005, podcast; James Kendrick, “Aspect Ratios and Joe Six-Packs: Home Theater Enthusiasts Battle to Legitimize the DVD Experience,” Velvet Light Trap 56 (2005): 58–70. 13 Barr, “CinemaScope,” 4. 14 For example, CinemaScope uses anamorphic lens to shoot and generally refers to aspect ratios of 2.35:1, 2.39:1, 2.40:1 or 2.55:1. Antonio Somaini makes a direct link between format and form by evaluating that the size of the frame of the film where images are imprinted is also the format of film, such as 8 mm, 16 mm, and so on. For a detailed discussion on the relationship between format and medium, see Antonio Somaini, “The Screen as ‘Battleground’: Eisenstein’s ‘Dynamic Square’ and the Plasticity of the Projection Format,” in Format Matters: Standards, Practices, and Politics in Media Cultures, ed. Marek Jancovic, Axel Volmar and Alexandra Schneider (Lüneburg: Meson Press, 2020), 219–236. 15 Harper Cossar, “The Shape of New Media: Screen Space, Aspect Ratios, and Digitextuality,” Journal of Film & Video 61, no. 4 (2009): 3.
168
from participatory to spectatorial.16 The frame size change is used to trigger a perceptual habit on the part of the medial body. But we should also consider how letterboxing does not alter the shape or scope of the images inside the frame; it only scales them down. By adding black bands, the size of the frame is effectively reduced while another frame is added. Another common technique of reframing films for television was pan and scan. This reduced the original frame by concentrating on the area where the action was happening.17 This reframing of the picture-frame demolished the original framing and manipulated the compositional aesthetics of the film. In an attempt to control the process of transferring the wider aspect ratio to fit a narrower television frame, Cossar uses the example of A Bug’s Life (1998), as a film that ‘reframed’ by reworking the frames of the animation in order to maintain the original composition.18 The makers modified the frames to recompose the elements of the image from its original aspect ratio of 2.35:1 to fit the television frame’s aspect ratio of that time at 1.33:1.19 In this way, the composition of the frames was maintained for a smaller screen size, modifying its previously compromised condition.
16 Ibid., 9–10. 17 This recomposition of the frame commonly happened without the filmmaker’s intervention. Woody Allen’s Manhattan (1979) was one of the first where the director had control over the transfer process from film to video for VHS and television. See John Belton, Widescreen Cinema (Cambridge, MA: Harvard University Press, 1992). 18 Cossar, “The Shape of New Media,” 11. 19 Ibid., 11–14.
169
Figure has been removed due to copyright restrictions.
Figure 31: George Landow, Film in which there appear Edge Lettering, Sprocket Holes, Dirt Particles, Etc, 1966, 16 mm film, 5 mins. Digital video still, https://lux.org.uk/work/film-in- which-there-appear-sprocket-holes-edge-lettering-dirt-particles-etc.
The idea that frames simply enclose images was challenged by expanded cinema and video artists. In particular, the 1966 work by George Landow, Film in which there appear Edge Lettering, Sprocket Holes, Dirt Particles, Etc makes use of the edge of the frame by offsetting the image centre and focusing on the sprocket holes and letters at the boundary of the celluloid (fig. 31). Live performances using videos unchained the frame from one location to disperse multiple spatial and temporal readings. The frame became free to move around in the viewing space, such as in Stan Vanderbeek’s Movie-Drome (1963–65) and Imi Knoebel’s Projektion X (1972), both of which have been widely analysed in terms of media/medium discussions, role of the artist/filmmaker, attention to process of making art, questioning the filmic apparatus and creating new experiences for the audience.20 But the
20 To list a few from a long list of sources, see A.L. Rees, “Expanded Cinema and Narrative: A Troubled History,” in Expanded Cinema: Art Performance and Film, ed. Steven Ball (London: Tate Publishing, 2011), 148– 156; W. Raban, “Reflexivity and Expanded Cinema: A Cinema of Transgression?” in Expanded Cinema: Art Performance and Film, ed. Steven Ball (London: Tate Publishing, 2011), 98–107; Peter Gidal, “The Theory and Definition of Structural/Materialist Film,”, in Structural Film Anthology, ed. Peter Gidal (London: British Film Institute, 1976); Beatriz Colomina, “Enclosed by Images: The Eameses Multimedia Architecture,” Grey Room 2 (2001): 6–29.
170
multiplying of frames through multiple screens and projections needs to be also emphasised as it helped develop non- and mixed narrative strategies. This occurred, for example, in the five-screen structure of In the Labyrinth (1967) by Roman Kroitor, Colin Low and Hugh O’Connor at the International and Universal Exposition of 1967 and also in the diapolyceran screen in the Czech pavilion of the same Expo, where 160 frames could be viewed simultaneously.21 The multimedia/intermedia work HPSCHD by John Cage and Lejaren Hiller involved a live performance where 8000 slides and 100 films were projected onto forty-eight windows of the University of Illinois in 1969.22
Similar to these multiscreen and multiple projection scenarios of the 1960s, the video art and computer art of the 1990s utilised emerging technologies to produce new experiences, methods of dissemination and participatory experiences. This was coupled with the rise of installation art and moving image screenings in galleries and museums.23 Arguably the growth of installation art alongside the presentation of moving images in galleries and museums had a profound effect on screen projection practices and audience viewing patterns, although installation art provided the impetus for drawing attention to the transformational capacities of screens. However, I am more
21 The Czech pavilion also presented Kinoautomat by Radúz Činčera, one of the first interactive films viewed internationally. At several points in the film, the action stops, and the audience members have to choose between two scenes to decide the course of the story. 22 There are conflicting records of the number of screens. This particular figure is quoted from Weibel, “Narrated Theory,” 44. Also see “An Extravaganza by John Cage Due,” New York Times, May 3, 1975, https://www.nytimes.com/1975/05/03/archives/an-extravaganza-by-john-cage-due-4hour-multimedia- concert-set-in.html. 23 Before the 1990s it had been noted that adding the ‘experimental’ tag to practices using emerging technologies kept the aforementioned practices on the fringes. See Weibel, “Narrated Theory,” 46. Moreover, a distinction between experimental films and artists’ films was sought. Chrissie Iles curated Into the Light: The Projected Image in American Art 1964-1977, which brought the work of ‘experimental filmmakers’ such as Michael Snow and Paul Sharits into a gallery/museum space. It was one of the first prominent exhibitions to do so. See Brett Kashmere, “Underground Film, Into the Light: Two Sides of the Projected Image in American Art, 1945-1975” Synoptique 8, (2008), http://www.synoptique.ca/core/en/articles/kashmere_underground; Jonathan Walley, “Modes of Film Practice in the Avant-garde,” in Art and the Moving Image: A Critical Reader, ed. Tanya Leighton (London: Tate Publishing, 2008), 182–199.
171
concerned with how various framings were being performed and organised by screens in some of these works. Here the out-of-frame and off-screen frames are of particular importance due to their relations with what is inside the frame. Alison Butler notes a “deictic turn” in how spaces in galleries have changed in the wake of digital gallery films, and also the differences in the deixis of expanded cinema, theatre and gallery films in their reference to on- screen and off-screen space.24 Multiple screens in different contexts made the screening space an extension of the frame.
To be clear, the multiscreen works discussed here demonstrate two kinds of multiple-ness of frames: 1) multiple frames on a single screen, and 2) multiple frames with multiple screens. Although I have discussed nested frames in the form of frame rate in the display of moving images and the conversion of wide format films for television viewing, it is the technique of split screens — several separate frames within a larger frame — that demands attention with reference to the postmedial condition.25 Multiple frames organised on a single screen and the incremental use of multiple screens in visual arts since the 1990s are both noteworthy as signalling a ‘coming’ postmediality, in addition to those ‘frames’ such as windows within windows produced by internet- enabled computers, which that have rapidly reconfigured the visuality of contemporary assemblages. In particular, the creation and experience of screens have changed as new ways of engagement through multiple windows became possible. The experimentation of this by early net artists drew our attention to these new screen gestures.
24 Deixis, in linguistics, refers to words that are context specific, such as ‘there,’ ‘they,’ and ‘you’. Butler employs this term to locate medium-specific tendencies of then and now, here and there. Alison Butler, “A Deictic Turn: Space and Location in Contemporary Gallery Film and Video Installation,” Screen 51, no. 4 (2010): 311, 312. 25 Multi-frame visuals are common to other media such as graphic books and newspapers. See Scott McCloud, Understanding Comics (Northampton, MA: Kitchen Sink Press, 1993); Ng Suat Tong, “An Open Debate about Closure,” The Comics Journal 111 (1999): 77–79.
172
Split-screen techniques in films involve multiple frames on a single screen, usually employed to emphasise the separation of space and simultaneity of time.26 Jim Bizzocchi gives us three ways in which split screens have been used in cinema: narrative, structural and graphic.27 These centre around the use of split screen in moving the plot or establishing relationships between characters (narrative use); establishing formal relationships between frames in space and time (structural use); and design, shapes and numbers of frames on a screen (graphical use).28 Widely seen in phone conversations that are part of the filmic narrative, for example in Pillow Talk (1959), and multiple point of views of the same event, such as in Timecode (2000), split screens show coincidental events albeit with different results. Split screens advance the story and the relatedness of the characters in Pillow Talk, while Timecode uses split screens to fracture the narrative and establish the overlapping yet disjointed storylines of the characters. For the viewer though, Friedberg suggests, four frames on a single screen in Timecode do not ‘split’ attention into viewing multiple frames, rather the viewer watches multiple frames at the same time.29 The simultaneity of split screens is common to the interplay of multiple windows and tabs we use in digital devices.30 On a computer, multiple windows present fragments of different spaces but the user
26 Double exposure in early films can also be considered as split-frames, although they were not employed to emphasise the separation of space and time; rather it insists on the fusion of space. Later on, with split screen, the separation between spaces was not just obvious but overt. Compositing techniques of double exposure, where the same portion of the film is exposed twice, made different images appear in one frame. While Friedberg compares this to the split-screen effect, she is quick to ascertain that this technique of double exposure as split screen fused the two parts as one, rather than separate them. This makes the technique closer to the polyscenic painters of the 16th century. See Friedberg, The Virtual Window, 198. 27 Jim Bizzocchi, “The Fragmented Frame: The Poetics of the Split-Screen,” 2009, 2, http://web.mit.edu/comm-forum/legacy/mit6/papers/Bizzocchi.pdf. 28 Ibid. 29 Friedberg, The Virtual Window, 218. 30 Split screen techniques are not only an editing practice but also viewing practice, such as in CCTV control rooms. In television debates the split screen is used to show the interaction between the speakers. For an overview of split screens in “communicative, instructional, observational purposes” see Lorenza Mondada, “Video Recording Practices and the Reflexive Constitution of the Interactional Order: Some Systematic Uses of the Split-Screen Technique,” Human Studies 32 (2009): 67–99, https://doi.org//10.1007/s10746-009- 9110-8. Bolter and Grusin frame the multiple frames of the television screen as being closer to practices of graphic design and offer an ‘immediacy’ where the medium interface is transparent. Bolter and Grusin, Remediation, 40, 189–190.
173
concentrates on one frame at a given time. Friedberg evaluates the computer interface as postcinematic, as the user has a “window view” when several frames confront the viewer like a variety of windows to look in to, as well as gets a “desktop view” where the frames stack on top of each other and viewing is similar to seeing objects on a table from above.31 Manovich finds that the windowed view in computers unfixes the viewer’s attention. He remarks that “rather than showing a single image, a computer screen typically displays a number of coexisting windows… No single window dominates the viewer’s attention.”32 In a similar vein, Galloway believes that the multi- framed view is closer to the informatic distributed network.33 The fragmented viewing offers a multiplication of spaces into frames/windows as well as the viewer’s attention.
Figure has been removed due to copyright restrictions.
Figure 32: Olia Lialina, My Boyfriend came back from the War, 1996, net art. Screenshot from project website, http://www.teleportacia.org/war/.
31 Ibid., 231, 232. 32 Manovich, The Language of New Media, 97. 33 Galloway, The Interface Effect, 117.
174
Figure has been removed due to copyright restrictions.
Figure 33: Olia Lialina, My Boyfriend came back from the War, 1996, net art. Screenshot from project website, http://www.teleportacia.org/war/.
Touching briefly on net art projects, we find that multiple frames were used early on as a device for users to generate narrative in relation to the multiplying of windows, in part a new design affordance of the browser. Olia Lialina’s My Boyfriend Came Back from the War (1996) is a website arranged in graphic and text-based sequences. Initially the viewer is presented with a statement: “My boyfriend came back from the war. After dinner they left us alone.”34 This statement is a hyperlink on an otherwise black screen, and clicking it takes the viewer to two images: one is of a window; the other shows a seated couple (fig. 32). The latter is also clickable, which further divides the frames with more hyperlinked images and texts (fig. 33). Later I will discuss how the space across and beyond the frame of the screen can be accessed, navigated and sustained in works that develop possibilities beyond the browser. For now, I want to focus on the multiplying of space created when dividing the screen.
34 Olia Lialina, My Boyfriend Came Back from the War (1996), http://www.teleportacia.org/war.
175
Here screen space can be differentiated from frame-space. Lialina uses the frame of the computer screen as a mode of navigating the artwork. Combining graphic novel aesthetics with interactivity, she works to explore screen space. Manovich terms this a new kind of spatial montage where there is “addition and coexistence,” unlike cinematic montage in which images are supplanted by other images over time.35 While moving along the options provided in My Boyfriend Came Back from the War, all the previous frames continue to be displayed, as if the viewer’s movement in time and space can be retraced. The action of the viewer changes the frame by adding new ones, but the new frames do not replace the existing ones, instead coexisting with them. For Manovich, this represents only an inversion of traditional cinema, where “time is no longer privileged over space, sequence is no longer privileged over simultaneity, montage in time is no longer privileged over montage within a shot.”36 However, it is also possible to experience Lialina’s work as an exploration of online spatio-temporality: multiple frames can be rearranged both spatially and temporally by going back to a previous division and clicking on a different link. In this way, the screen’s gesture of multiplying carries the potential for a qualitative experience of time where the movement of forward and reverse sequences of frames can lead to new temporal possibilities.
Before proceeding further, it is important to explain what a temporal consideration of frames in a postmedia assemblage entails. Seemingly, and as we have seen through the scholarship considered thus far, the spatial links of multiscreen set-ups are prioritised.37 In an installation this literally enfolds via
35 Manovich, The Language of New Media, 325, 323. 36 Ibid., 326. 37 Colomina, “Enclosed by Images,” 22–25; Mondloch, Screens, 30–35; Janine Marchessault, “Multi-Screens and Future Cinema: The Labyrinth Project at Expo 67,” in Fluid Screens, Expanded Cinema, ed. Janine Marchessault and Susan Lord (Canada: University of Toronto Press, 2007), 29–51; Kate Mondloch, “Be Here (and There) Now: The Spatial Dynamics of Screen-Reliant Installation Art,” Art Journal 66, no. 3
176
a relation to space, which is highlighted as viewers move through the space of viewing and/or are connected to media objects. But multiple screens in postmedia assemblage are also participatory or experiential spaces where there is, as Mondloch puts it, “interpenetration between the space ‘on’ the screen, the space between the viewer and the screen, and the space of the screen object.”38 Considering frames are constructed in relation to the image, sound and media hardware, frames simply conceived as spatial devices that configure the inside and outside of the image or content neglects the ‘framing’ of the time, space and activity in and through screens. Moreover, screen spaces embedded in ubiquitous media objects such as phones intensify the entanglement of human and nonhuman parts of the assemblage and permeate the experiences of movement and temporality that such entanglement entails. There is not just a multiplicity of frames and screens, but a multiplicity that involves differing temporalities. Here we need to solicit some of the considerations of framing and temporality that Deleuze provides in his writings on cinema, where he provides a well-examined critique of the characterisation of cinema as a spatialised medium. 39 However, I am particularly interested in his work on the frame since it provides us with an unfixing of the frame from spatialised constraints and allows us to draw on this aspect of its relation with screens in the postmedial context.
Deleuze gives a deceptively straightforward definition of framing as “the determination of a closed system, a relatively closed system which includes everything which is present in the image — sets, characters and props.”40 This emphasises
(2007): 20–33; Catherine Fowler, “Into the Light: Re-Considering Off-Frame and Off-Screen Space in Gallery Films,” New Review of Film and Television Studies 6, no. 3 (2008): 253–267, https://doi.org//10.1080/17400300802418578; Tamara Trodd, ed., Screen/Space: the Projected Image in Contemporary Art (Manchester: Manchester University Press, 2011). 38 Mondloch, Screens, 78. 39 Deleuze, Cinema 1, 12; Deleuze, Cinema 2, 271. 40 Deleuze, Cinema 1, 12. Emphasis in original.
177
the deliberate action of including something in the frame. Moreover, as a ‘closed system’, frames should supposedly contain within their bounds all the information required to examine the image. Deleuze, however, warns that this definition may have to be revised as the “visual image has a legible function beyond the visible function.”41 Taking this as a cue to look beyond the frame (as does he), I will explicate the relationship of the frame to the image it encloses, but also to the things outside of it and to screens themselves.
Linking screens with respect to the frame and image, Deleuze comments that, “the screen, as the frame of frames, gives a common standard of measurement to things which do not have one — long shots of countryside and close-ups of the face, an astronomical system and a single drop of water — parts which do not have the same denominator of distance, relief or light.”42 Screens standardise image making and enables it to be compared, substituted or exchanged. Where screens may be fixed, images or parts of the frame change. Factors such as shot size, angle, object size, depth do the work of internal transformation. But we should consider that Deleuze is speaking of cinema screens here, and therefore this issue of standardisation must be re-evaluated for postmedial screens if not already for the medialities that precede it, which Deleuze only considered at the close of Cinema II.43 In postmediality, the agency of screens is not just a unifying and homogenising factor but also a
41 Ibid., 15. 42 Ibid., 14, 15. 43 When talking about linking images and the movement of those images, Deleuze refers to the brain as being the screen. He considers the brain is what unifies the connections that the viewer sees on the screen, and meaning is derived not just from images but also one’s bodily reactions to them. He elaborates: “Cinema, precisely because it puts the image in motion, or rather endows the image with self-motion [auto-mouvement], never stops tracing the circuits of the brain.” Gilles Deleuze, “The Brain Is the Screen,” trans. Marie Therese Guirgis, in The Brain Is the Screen: Deleuze and the Philosophy of Cinema, ed. Gregory Flaxman (Minneapolis: University of Minnesota Press, 2000), 366. The screen-brain relationship is of interest to Patricia Pisters who interrogates a new kind of image in networked digital media with insights from neuroscience. Patricia Pisters, The Neuro-Image: A Deleuzian Film- Philosophy of Digital Screen Culture (Stanford: Stanford University Press, 2012).
178
dispersing one in which, via the heterogeneity of screens, multiple affects, perceptions and relations are generated. Of course, as I have also discussed above, multiple frames on a single screen do not just figure in postmedia. But postmedial assemblages encourage both heterogeneity across the elements of the assemblage yet also maintain the mediality (ongoingness and the being-in- the-middle of) the assemblage. The diverse elements of the assemblage are arranged and reconfigured by screens spatially and temporally. The multiplicity of screens and frames make these processes dynamic and contingent. That sense of being ‘in-the-middle’ of all this when in a postmedial installation is substantively produced by the ways in which screens here gesture a different kind of framing.
I argue that we need to consider both qualitative or heterogeneous, continuous experiences and quantitative or homogeneous, simultaneous spatio-temporal experiences as the gestures performed by screens in postmedia. As a trailer for how this plays out, I want to begin to tease out these possibilities through exploring a work I will turn to towards the end this chapter — Ragnar Kjartansson’s installation, The Visitors (2012). This work presents nine screens (fig. 34) where artists and musicians are shown individually playing the same song, with each using a different musical instrument. For the audience they are playing like an orchestra, but in separate spaces. Their individuality in different and multiple screen spaces comes together for the audience in the gallery space. Towards the end, the performers leave their on-screen space to assemble outdoors. As they come together, they cross each other’s frame-space, revealing that the multiple screens variously frame a shared although initially separated physical space. That the performers were playing in the same location at the same time makes the artist’s deployment of multiple screens offering multi spatial-temporal engagement an aesthetic strategy for postmedial assemblage. Here we
179
experience screens and frames both synchronised with and loosened from each other, allowing both an experience of the passing of time that takes place simultaneously yet differentially. Drawing on Henri Bergson’s concepts of qualitative and quantitative multiplicities, The Visitors shapes experience as intensities in which the passing of time as qualitative and as spatially distinct (quantitative differences) can be perceived together.44
Figure has been removed due to copyright restrictions.
Figure 34: Ragnar Kjartansson’s installation, The Visitors, 2012, nine-channel video installation. Installation view, Institute of Contemporary Art, Boston, 2014. Photo by Charles Mayer Photography.
In order to understand this distinction between qualitative and quantitative multiplicities with respect to screens and media we will need to pass through Deleuze’s ideas in his two seminal books on cinema to understand time in two ways. However, these do not need to be restricted to cinema and I will use them to think about duration in the gesturality of postmedial screens. First, his concept of the movement-image gives us an idea of time through spatialised movement with temporality cinematically achieved through causal and spatial links.45 Second, with the time-image the temporal dimension and
44 Bergson, Time and Free Will, 101–102. 45 Deleuze, Cinema 1, 22-3.
180
experience are no longer dependent on movement ( ‘false movement’ depends on time) but rather a direct image of time is presented cinematically.46 It is important to note that the understanding of duration that Deleuze articulates here is not specific to cinema but is generated by a “cinematic consciousness,” which produces concepts to understand the world.47
Movement in cinema can be considered in various ways, such as the movements of actors inside the frame, camera movement and editing sequences, including montage. However, Deleuze examines how a shot enacts a pure movement-image for cinema, where the movement is provided by the camera or by editing.48 The movement of the bodies (when the camera or bodies in frame move) or change in the arrangement of bodies (as seen in a montage sequence), both seem to be taking place in space. However, both of these highlight the mobility of bodies rather than changes in a predetermined or pre-constituted space external to them; that is, such movement asks us to be attentive to it and not its spatialised trajectory: “The essence of the cinematographic movement-image lies in extracting from vehicles or moving bodies the movement which is common substance, or extracting from movements the mobility which is their essence.”49 The idea that “movement is distinct from the space covered” is imperative in the qualitative understanding of duration. This movement, according to Deleuze, can then produce both continuous (qualitative) and discontinuous (quantitative) durations.50
Movements of various elements in the frame (actors, props) and the movement that relates these parts (through camera and/or editing) change the
46 Deleuze, Cinema 2, 271. 47 Felicity Colman, Deleuze and Cinema: The Film Concepts (London: Bloomsbury, 2011), 1 48 Deleuze, Cinema 1, 25. 49 Ibid., 23. Deleuze follows Bergson, who identifies that motion is not homogeneous and divisible, but space is. Bergson, Time and Free Will, 110. 50 Deleuze, Cinema 1, 22–27.
181
duration of the whole continuously.51 As the position of elements change, the duration of the whole changes qualitatively:
Movement relates the objects of a closed system to open duration, and duration to the objects of the system which it forces to open up. Movement relates the objects between which it is established to the changing whole which it expresses, and vice versa.52
Notably the movement of elements also changes their specific durations, which “reunite” in the duration of the whole.53 We can understand the movements of elements to be a quantitative change — the divisible, successive movements that take place in space. These movements reconstituted as a new ‘whole’, then produce a qualitative change, yielding a different continuity.54 The concepts of qualitative and quantitative changes as multiplicities, which differ in kind, provide different temporal experiences. This also offers a way forward in scrutinising the different kinds of changes of relations that characterise a postmedia assemblage.
Multiplicities are especially helpful for thinking about frames in postmedia. Deleuze concludes that “all framing determines an out-of-field.”55 The out-of- field is here identified in relation to the frame of the movement-image, thereby presenting a concept of time tied to sequential movement unfolding via frame rate. In this chapter, however, I deliberate upon the out-of-frame, reconfigured through the relations in postmedia assemblages. I am arguing that the entanglement of frames and screens in postmedia can be understood through the out-of-frame. Later in this chapter, I will discuss how the postmedial out-
51 For Deleuze, temporality in the movement-image is guided by the “sensory-motor schema,” which are linear and causal situations. Deleuze considers the frames as immobile sections as well as objects or parts of a set. See Cinema 1, 2, 11. 52 Ibid., 11. 53 Ibid., 20; Deleuze, Cinema 2, 34. 54 Bergson, Time and Free Will, 126–128. 55 Deleuze, Cinema 1, 16.
182
of-frame is constantly brought into the frame and that this is one of the characteristic gestures performed by screens in postmedia assemblages. In doing so, a qualitative temporal experience is produced, in which we become users, players and participants in non-linear and multiple temporal relations. Contending that the out-of-frame is to be understood as a gesture of screens, I will analyse instances where the out-of-frame endures and is maintained in and through screens. In describing relations, sensations, affects, generators and transformations of the out-of-frame as it is pulled in and out of the frame, I will have recourse to a range of media and postmedia art works that exemplify these moves.
I will start by discussing the various spatio-temporalities disclosed by multiple screens in the analysis of Natalie Bookchin’s works, with a specific focus on Now he’s out in public and everyone can see (2012). Although my main concern will focus on this work in its initial form as an installation, I will constantly contrast this with the film version created by Bookchin in 2017, as well as refer to other Bookchin works such as Long Story Short (2016) in order to bring out the differential methods of temporal movements. By analysing Bookchin’s editing techniques, I will illustrate the multiple temporalities that are experienced in and through the specific gesturing that screens perform in postmedia. These temporalities depend on various kinds of movements, particularly that of editing, which is most clearly seen in Bookchin’s work. In the process of editing, Bookchin generates a movement of images as a new relational ‘whole’ through the modification of the original position of YouTube-sourced video clips. This is effected, for example, by making them reappear on different screens in relation with each other, or only displaying a small section of the original clip on-screen. These movements produce a quantitative multiplicity of space and time where duration becomes measurable and is dependent on space. However, I will also suggest that
183
certain kinds of temporal multiplicities are produced through the use of montage in Bookchin’s work. Her work stands, then, in a kind of transitional space between Manovich’s characterisation of digital cinema and my characterisation of postmediality.
FRAMING FRAMES OF SPACE-TIME In the previous section I demonstrated the ways in which a quantitative multiplicity of frames can produce spatial fragmentation for users/viewers. Multiple frames and multiscreen installations can fractalise the space-time continuum of narrativity, delivering differing audience experiences of installations. The following section proposes that such multiple frames and multiple screens do not only produce a quantitative spatial distribution — as has been the focus of digital art analysis via the idea of ‘spatial montage’— but also temporal differences. Such qualitative multiplicity is not only a postmedial experience but the sheer number of screens, frames and voices to be traversed make it ripe for investigation. How do we understand the variations and divergences of screens that make a difference? By analysing Natalie Bookchin’s video techniques, with a special focus on Now he’s out in public and everyone can see (2012), I will discuss the temporal layerings that produce qualitative experiences in this work. I trace this via the various kinds of movements made in and outside the frame. This highlights a different kind of relationship between screens and frames that is not restricted to spatial overlap, allowing us to get at what kind of gesturing ‘framing’ performs in the postmedial assemblage.
184
Figure has been removed due to copyright restrictions.
Figure 35: Natalie Bookchin, Now he’s out in public and everyone can see, 2012, 18-channel video installation, 16 mins looped. Screenshot from video documentation.
Now he’s out in public and everyone can see was first presented as an 18-channel installation in 2012 and then later as a single-screen short film in 2017. In the installation, 18 screens of different sizes are suspended at different heights and angles in a way that the audience cannot view all of them at once. Walking into the installation, the viewer can position themselves anywhere in a sea of screens (fig. 35). Bookchin has compiled excerpts from video logs (vlogs) and video diaries of people voicing their opinion on news stories about African American men, but without revealing the names of the people discussed. As various screens light up with video, sometimes many at once and other times just one at a time, the viewer, immersed in the installation, hears voices on screens from in front, behind and to either side. The film, figure 36, on the other hand, is a single screen work largely with one person speaking at any given time. In the installation, the videos often play simultaneously (fig. 35); at other times it is just the speakers’ voices that overlap. In the film, the emphasis is more on the convergence of voices than presenting multiple frames on-screen. Bookchin compiles disparate voices into one (screen) space while maintaining their distinctness through multiple
185
frames. This is different to the multiple frames on a single screen that Bookchin uses in other works such as Mass Ornament (2009) and Long Story Short (2016). In the film of Now he’s out in public mostly the voices of people are juxtaposed, unified and scattered, while Mass Ornaments and Long Story Short are a visual and sonic arrangement of multiple frames on one screen. Bookchin has diverse techniques for separating the uniqueness of each speaker in the unified space of the screen.
Figure has been removed due to copyright restrictions.
Figure 36: Natalie Bookchin, Now he’s out in public and everyone can see, 2017, digital film, 24 mins. Selection of video stills.
Several of Bookchin’s works, including Mass Ornament (2009), Laid Off (2009), and My Meds (2009), are a ‘collection’ of videos. Using publicly available videos from websites such as YouTube and Vimeo has been Bookchin’s method for acquiring material since the late 2000s. The networked relatability offered by vlogging is recomposed into new ideas of sociality and interaction in her artworks. Bookchin is interested in the social experiences that the online space offers, as it is another form of public space accessed and delivered through screens that can connect and exchange voices and experiences. She states, “What I am trying to do through my editing and compilation is reimagine these separate speakers as collectives taking form as a public body in physical space.”56 The work Now he’s out in public recognises sociality by segregating, arranging and displaying the individual voices in a
56 Natalie Bookchin and Blake Stimson, “Out in Public: Natalie Bookchin in Conversation with Blake Stimson,” in Video Vortex Reader II: Moving Images Beyond YouTube, ed. Geert Lovink and Rachel Somers Miles (Amsterdam: Institute of Network Cultures, 2011)308. Also available at https://rhizome.org/editorial/2011/mar/09/out-public-natalie-bookchin-conversation-blake-sti.
186
way that makes them “more of a social experience than it currently is.”57 Although on several occasions these multitudinous voices across numerous screens create a cacophony where everyone is speaking yet no one is listening.
Bookchin brings out heterogeneous experiences in the numerous frames that are seemingly united. Screens become the field where spatial and temporal differences among frames come to overlap. I have mentioned the distinction between speakers that Bookchin has tried to explicate through editing. In the process, the spatial stasis of the framing of the speakers in Now he’s out in public is also arranged on multiple screens, producing dynamic movement. I propose that this multiplication of frames and screens can be understood as a qualitative experience rather than simply a quantitative increase of voices or people. Although Bookchin’s editing generates movements through which we experience time as simultaneous yet discontinuous, this nonetheless provides an opportunity to explicate a durational experience as well. We can use Deleuze’s ideas around frame, shot and montage as well as his commentary on Bergson for an analysis of multiple temporalities in Bookchin’s works, especially the installation Now he’s out in public. I will first illustrate the relationship between frames and shots as they come together on multiple screens of the installation and their relationship to the whole. This ‘whole’ is not the entire installation, but the ‘whole of durée’. After this I delve into the specific techniques that Bookchin uses to create multiple temporal rhythms and movements from which the whole can be determined.
As I have already indicated, a frame can be regarded as a closed system according to Deleuze, albeit one that has been artificially composed.58 He
57 Ibid., 314. 58 Deleuze, Cinema 1, 12.
187
discusses it in regard to cinema, but it can be explicated in the postmedia context, for the act of framing here too asks us to consider the setting up of the frame. Arguably, the vlogs and video diaries in Now he’s out in public (installation) reproduce the default aesthetics of frontality associated with cameras attached to a computer. But the setting up of the frame to signal a certain appearance is evident in some of these vlogs. For example, a few speakers are set against the backdrop of an American flag; someone else has an Afro-American flag; a few others have Barack Obama’s image in the form of posters and even a t-shirt with Obama’s face in the background. Moreover, angles and lighting in these frames vary. Figure 37 illustrates a sample of the various angles and lighting that are included in the work. The compositionality of the frames is made possible due to shared cultural and social practices.59 The speakers are mostly framed in a home environment, which discloses their personal space, adding details about their identity in addition to their recorded opinions. In the viewing of the installation, these frames are also ‘framed’ by the display screens.
59 Colman, Deleuze and Cinema, 44.
188
Figure has been removed due to copyright restrictions.
Figure has been removed due to copyright restrictions.
Figure has been removed due to copyright restrictions.
Figure 37: Natalie Bookchin, Now he’s out in public and everyone can see, 2012, 18-channel video installation, 16 mins looped. Selection of screenshots from video documentation.
189
Bookchin takes only sections of the vlogs, snippets that are akin to shots or shot sequences in cinematic terms. The shot, according to Deleuze determine the movement of the parts and thus provide “an articulation of duration.”60 There are various types of shot, defined by distance from the camera (wide shot or close-up) or movement of camera (tracking shot). As noted by Bogue and Colman, Deleuze uses all these understandings of the shot to highlight its relationship to movement.61 In the installation, the fading in and out of the videos creates a movement of images, bodies and objects at different scales and angles through the organisation of elements inside each frame, as well as through the arrangement of screens in the gallery space. The careful edits place the speakers in a particular relation to each other (as discussed above), illustrating the variety of opinions and at the same time consolidating them at two levels: the semantic-aural level, registering as the specific points where voices overlap; and the general social-contextual level, which is focus of the work — African American men in news media.
From these shots, Bookchin creates a montage, which is the artwork. Movement, according to Deleuze, has “two facets, as inseparable as the inside and the outside, as the two sides of a coin: it is the relationship between parts and it is the state [affection] of the whole.”62 Bookchin’s use of two kinds of montage in her practice, simultaneous and sequential spatial, can help in clarifying these two facets.63 The simultaneous montage can be seen extensively in Mass Ornaments and Now he’s out in public (installation) where the
60 Deleuze, Cinema 1, 18. 61 Deleuze uses the French word plan for shots. While in English there is a distinction between the spatial shot (distance from the camera), and a temporal take (duration of the shot), in French plan is used to describe both, such as gros plan (close-up) and plan-sequence (sequence shot). For a detailed discussion see Ronald Bogue, Deleuze on Cinema (London: Routledge, 2003), 44, 45 and Colman, Deleuze and Cinema, 45. 62 Deleuze, Cinema 1, 19. 63 Natalie Bookchin, Portraits of the Multitude (Barcelona: La Virreina Centre de la Image, 2018), 12; Natalie Bookchin, “Re-editing the American Crisis: Natalie Bookchin’s Montaged Portraits,” interview by Angela Maiello. DVD booklet of Portraits of America (2017) accessed 2 February 2021, https://bookchin.net/wp-content/uploads/2018/06/Reediting-the-American-Crisis.pdf.
190
similarity of words and actions are highlighted. What is to be juxtaposed, which clips follow, where should a unison be placed – all this is arranged by Bookchin not in a traditional narration but a montage-like performance. The artwork is produced in the edit of the works where each vlog in Now he’s out in public is producing what can be called the ‘whole’ of the artwork. As the speakers are constantly shifting the topics of conversation, Now he’s out in public is determined by perceptual changes in totality (more on this below). However, the viewer is unable to absorb the entire installation from a single position or keep pace with the constant modifications. They are always trying to catch up with the fractured narrative. The sequential spatial montage can be recognised in Long Story Short, where the arrangement of individual frames gives an illusion of continuity and togetherness of the speakers. As demonstrated earlier, when a sequence plays out with multiple frames in Long Story Short, it appears that the speakers are aware of each other; they finish each other’s sentences, adding to what is being said (see fig. 38 in comparison with fig. 37). Several frames are arranged to highlight that speakers both listen and remain silent.64 The frames are akin to split screen, a demarcation of space that is occurring within a sequence of time.
Figure has been removed due to copyright restrictions.
Figure 38: Natalie Bookchin, Long Story Short, 2016, digital film, 45 mins. Two video stills.
64 Bookchin, Portraits of the Multitude, 13.
191
Deleuze declares montage “a composition, the assemblage [agencement] of movement-images as constituting an indirect image of time.”65 However, Bogue warns that putting it this way might lead us to think of the whole as a “unifying concept of the film (plot, theme, thesis, and so on),” the putting together of shots as montage and shots as, “actual manifestations of the unifying concept.”66 Such an interpretation would loosen Deleuze’s association of movement-images to the open whole of durée.67 In other words, the artwork can be considered a collection of frames and shots that are in relation to each other (relative movement creating a quantitative change) and in relation to something else that is indefinite and indeterminate (the absolute whole that changes qualitatively). By being attentive to the “internal relational movements” of the artwork,68 we can explicate the configuration of frames and shots in a particular way; this way the shot “traces a movement.”69 Caught inbetween the whole artwork and the discrete frame, the extracted duration of the vlog modifies the artwork, for example, by continuing the narrative or fracturing it, as it changes throughout the duration of the artwork as well as relates one frame to the other.70 Even though arguably the frames here are similar in their size and angle, variation is evident in the different sizes and perspectives of the images as arranged by screens in the gallery space.
By explaining the connections of the installation as frames and shots with the whole, I now turn towards the ways in which multiple temporalities of parts
65 Deleuze, Cinema 1, 30. 66 Bogue, Deleuze on Cinema, 48. 67 Durée is Bergson’s idea of time that is indeterminate. See Henri Bergson, “Matter and Memory,” and “Creative Evolution,” in Henri Bergson: Key Writings, ed. Keith Ansell-Pearson and John Ó Maoilearca (New York: Continuum, 2002); Gilles Deleuze, Bergsonism, trans. Hugh Tomlinson and Barbara Habberjam (New York: Zone Books, 1991); Bogue, Deleuze on Cinema, 14–21. 68 I borrow this phrase from Felicity Colman: Colman, Deleuze and Cinema, 46. 69 Deleuze, Cinema 1, 20. 70 Deleuze refers to shots having a unity that is entangled with the whole of the film in the way that it is related to other shots, as well as the displacement of frames that it determines. See Deleuze, Cinema 1, 27.
192
and rhythmic flows transform the whole that endures in these movements — or in other words, explain their gesturing.
To reiterate a previous point, movement can take place through the moving camera or can be induced through montage. These are two prominent means that cinema uses to show the dissociation of movement with bodies.71 However, unlike movement in cinema, which occurs through moving the camera (tracking, panning, etc.), the installation Now he’s out in public contains fixed framing. Instead of the movement of a camera, the lighting up of different screens at various intervals in the installation produces a new framing, parallel to the way a moving camera continuously generates new frames.72 Each new frame (executed as ‘shots’ by Bookchin), provides a different perspective on the issues of the work. Spatial movement occurs across temporalities that are nonsequential but contract and dilate the present towards both what has come before and what is about to come. This marks the installation’s movement as “indivisible qualitative multiplicity.”73 Looking at the techniques used by Bookchin, I want to suggest that such movement also “decomposes and recomposes” the installation.74 Considering frames, shots and montage in the artwork is helpful in understanding how time unfolds, how it “contracts and dilates, as movement takes on the power to slow down or accelerate.”75 It highlights that the incremental scale of arrangement in terms of frames, shots and the montage in the installation is also a performance of durations conducted by its screens, through which gesturing is made possible.
71 Bogue, Deleuze on Cinema, 46. 72 The moving camera reframing through cut and pan has been noted by Flaxman. Gregory Flaxman, “Out of Field: The Future of Film Studies,” Angelaki 17, no. 4 (2012): 130. 73 Bogue, Deleuze on Cinema, 45. 74 Deleuze, Cinema 1, 23. 75 Ibid., 24.
193
There are several ways in which the frames in Bookchin’s works enact a convergence and divergence of people, their collective socialities and public performance on-screen. Nevertheless, the similarity of frames, activities, voices and opinions coming together on the screens are indicative of heterogeneity. The multifarious spatial organisations enacted by frames are evident in these works. The multitude of spaces, both geographic and temporal, of the speakers in Now he’s out in public and dancers in Mass Ornament converge in various ways. The videos and vlogs uploaded from various parts of the world are grouped into similar categories and have identical keywords. The frames of the vlogs and video diaries in Now he’s out in public resemble each other: flat, frontal, talking heads, medium shot or medium close-up angle. These videos come together on the desktop screen(s) of Bookchin’s workspace while she also fragments them, editing to highlight a ‘similarity in difference.’ For example, what we see are similar opinions and biases, often edited congruously and as overlapping voices. At several points, instances of synchronicity arise when a number of screens in the installation relay the same phrases by speakers, such as “show your ID,” to emphasize a collective demand. However, just as the frames appear as points of unity, they also execute a multiplicity, which differences through the situatedness of the speakers; that is, the position of each speaker registers as specific to a social, political and cultural setting.
The similarity of shots in Bookchin’s artwork does not signal the same relations playing out in each instance. I am interested in how the multiplicity of frames engenders a relation with different temporal experiences as it elucidates the gesturality of screens. In Now he’s out in public (installation), this is prompted by Bookchin during the editing of the work. Both her editing technique and the arrangement of the screens in the installation work to create distinctions between the speakers. In the editing process, Bookchin
194
creates ‘slippages’ which are seen, heard and felt in the distributed appearance of the video.76 For instance, several screens display the video when the speakers recite the phrase “straight to the facts” in unison. While the phrase may indicate their quest for objectivity, it is followed by individuals voicing their different (yet similar in racist ideology) opinions. Referring to the black man in question, one white man remarks, “He’s no blacker than I am really…I mean if you look past skin tone,” followed up with opinions such as “I don’t know what race you are”, “No one knows anything about him”, and “Even his name is a mystery.” These slippages create affective voids in which something is missed between what is said and what is heard.
However, the different scales and durations in the installation version of Now he’s out in public is what works to produce temporal multiplicity. Temporality is nested, elongated and differently experienced via the aesthetics of the work. As Tim Barker explains, the interplay of “multiple temporal rhythms” of asynchronous and non-sequential time of the user, machine, software, and database changes our experience of time.77 By this, Barker does not mean the numerous ways in which time is experienced simply by a viewer but the “multiple variants of time” as the relation of composition and viewing the work.78 The layered temporalities of the work are performed both in and with the viewing of the installation. Here only fragments of the speakers’ original videos are used. Viewers are not always drawn to distinctions between people speaking but are more attentive to the content of the speech as they try to
76 Bookchin used the term slippages to describe the moments that emerge in the installation while speaking in LACE gallery video discussing Now he’s out in public. “Practice Session: Natalie Bookchin” LACE, Los Angeles, 2016, YouTube, accessed 8 February 2021, https://www.youtube.com/watch?app=desktop&v=rWi9UQZZCa8&list=UUKVWGnDf1Vm2tuB1awaj5Q w&index=11&t=0s. 77 Timothy Scott Barker, Time and the Digital: Connecting Technology, Aesthetics, and a Process Philosophy of Time (Chicago: University of Chicago Press, 2012), 14. 78 Ibid., 70.
195
figure out the person to whom the speaking refers, adding one speaker’s comment to the next comment. To this, the arrangement of the portions of the speakers’ videos also qualify the viewing-listening experience, as their differing pasts are relayed to the viewers’ present/presence. Frames here are generative of time, producing as coiling clusters of multiple durations: the entwined durations in which the artwork is viewed; the vlogs in their moments of recording; and the sequences as arranged by Bookchin, which themselves nest durations.
Furthermore, there are various techniques that Bookchin employs in the Now he’s out in public installation to stretch experiences of time. Pauses, bridging words such as ‘and’ and ‘but,’ and vocal expressions indicating hesitation, such as ‘hmm’ and ‘umm’ are highlighted and multiplied by juxtaposing and overlapping. In editing, Bookchin dissects the shaking voices, pauses, and uncertainty of the speakers by delaying their words, lengthening inbetweenness. This also places the edited video in contrasting temporal rhythm to the original vlogs. Moreover, the length during which viewers engage with the work, as well as the particular daily cycles and opening hours of the institution where the work may be displayed, are both open to external change.79
Now he’s out in public can be considered as a work that creates multiple temporalities through the gesturality of its multiple screens, which allow it to perform variable modes of framing. One of the ways in which these multiple temporalities are felt is in an intensification of framing, realised by both the making and viewing of the work. Multiple frames are also more than a
79 For example, public health situations can compel galleries to change access to artworks. In the current pandemic, several art institutions displayed exhibitions online. These steps are temporal, considering that they are currently made to feel necessary, when in fact this option could also be implemented as part of their routine functioning. Moreover, viewing the work online changes the experience of the viewer. A limited opening of the viewing spaces may also impact the viewing experience.
196
number; they are a composition and experience of varying temporalities mixing, clashing, changing and shifting altogether. In my focus on the temporal qualities of Now he’s out in public, I have developed an understanding of how this is produced in relation to the movement of and in frames. However, temporal qualities cannot only be understood by what is internal to frames. Unsurprisingly, image signification is not bound to the frame. The speakers in Now he’s out in public refer to African American men who are never seen in the installation — even their names are never heard, always remaining outside the frame. In explaining the indirectness of time in the movement- image and the release from subordination to space in the time-image, Deleuze clarifies,
The movement-image has not disappeared, but now exists only as the first dimension of an image that never stops growing in dimensions. We are not talking about dimensions of space, since the image maybe flat without depth, and through this very fact assumes all the more dimensions or powers which go beyond space.80
If the image goes beyond the frame, then thinking about the out-of-frame becomes imperative when articulating the gesturality of screens. Screens then are not only concerned with what is in the frame but also outside of it. Frames are not simply spaces that include and exclude images but are always open to something else. Michael Tawa argues that they are also “an extensive and excessive field that both delimits content and harbours its transgression — a setting that enables this mise en scène of appearance-disappearance to take place because of its openness to the unframes and the out-of-field.”81 Arguably, the on-screen or ‘in the frame’ and the off-screen are co- constitutive in postmedia. In other words, it is not the frame as such, but the
80 Deleuze, Cinema 2, 22. 81 Michael Tawa, Agencies of the Frame: Tectonic Strategies in Cinema and Architecture (London: Blackwell, 2010), 100.
197
out-of-frame-coming-in that screens gesture towards, which I will now elaborate upon.
OUT-OF-FRAME Framing, one can argue, is about what is excluded from the inside as much as it is about what is inside. The out-of-frame is not a mere exclusion but affects the inside qualitatively. This affecting and being affected as relation of outside-inside the frame is performed by postmedial screens. Through focusing on the out-of-frame, the (non)agency of screens as mere display is deposed. As discussed in chapter one, entities do not form separately but entangled agencies are separated through specific practices; that is, “agential cuts,” as described by Barad, enact the dissociation of entities. The frame is entangled with the out-of-frame and is only separated in the act of framing. Screens gesture the framing — a mediality assisting the frame and out-of- frame — which is challenged in postmedia due to the instability of the frame itself.
Scholarship derived mainly from screen studies and film philosophy concerning discussions of the out-of-frame can be broadly categorised as falling into relative and absolute notions of the out-of-frame. I address this literature in order to establish a spatial and temporal understanding of the out-of-frame, which has been steered through the Deleuzian concept of the cinematic out-of-field.82 However, digital images, framing and screens complicate this understanding within postmediality. Building on Deleuze’s concept of the out-of-field, I deliberately use the word frame to maintain their postmedial status. While frames continue relations with older media such as cinema, the interplay of frames with the out-of-frame as screen gesture is
82 Deleuze uses the term hors-champ, (‘out of field’) which can also be translated as ‘off-screen.’
198
significant in postmedia assemblages. Here, the relative out-of-frame is constituted by the contiguous relations of images and actions in the frame. On the other hand, the absolute out-of-frame is the Deleuzian out-of-field, constituting a relationship to heterogenous space-time, which itself makes up (the conditions of and conditions) postmediality.
The relative out-of-frame can be examined with respect to specific relations of the out-of-frame with images inside the frame. Off-screen and off-frame are some ways in which the relative out-of-frame can be configured. A thing, person or event not shown in the frame but only inferred, can be designated as being off-screen. This is a convention commonly used in cinema to hide or imply something or someone. In one of the most important commentaries in cinema studies on off-screen space, Noël Burch proffered six “segments” of off-screen space, which reside outside the confines of the frame.83 Out of these, four are located in the four cardinal directions with the frame at the centre, and the other two fall behind the set and the camera. Burch’s description of off-screen(s) is considered as adjacent to the frame, which merely acts as a demarcation.84 These off-screen spaces therefore fall in to the broader category of a relative out-of-frame, in which proximity to what is in the frame is maintained.85 The consequent understanding of the frame as a “centred space” by Burch was questioned by Pascal Bonitzer.86 He argued that in classical cinema, Burch’s off-screen spaces, such as entries, exits and the direction of an actor’s gaze, displaced the frame by taking attention away
83 Noël Burch, Theory of Film Practice [1969] (Princeton, NJ, Princeton University Press: 1981), 17. 84 Burch analyses how off-screen spaces are used by filmmakers to indicate other parts, people and events that are seen in consecutive shots, or are never shown but used as a reference, such as when only one part of the body is seen inside the frame. Ibid., 19–22. 85 Burch’s idea of off-screen considers spatial relativity of off-screen with the frame. He also gives two other categories of the off-screen space: imaginary and concrete. These too are ‘spaces’ in relation to the frame. Ibid., 21–24. 86 Pascal Bonitzer, “Off-Screen Space,” in Cahiers du Cinema Vol. 3, 1969–1972: The Politics of Representation ed. Nick Browne (London: Routledge, 1990), 294. (Originally published as Pascal Bonitzer, “Hors-Champ: Un éspace en défaut,” Cahiers du cinema, nos. 234–5 (1971–72), 15–26.)
199
from what is in it to what is outside it.87 The relative out-of-frame therefore ‘decentred’ the frame in the filmic world or in profilmic space; that is, the field in front of the camera, including parts of the set, which may or may not be seen in frame.
Under different conditions, off-screen also falls into the category of the relative out-of-frame, even if it is a different space to inside the frame, since it nevertheless does the work of prolonging what is inside the frame. Pre- cinema and early cinema screenings created an immersive space echoing or resonating with the scenario or mood of the film. Tom Gunning explains the significance of a train carriage and a ticket conductor in the pre-cinematic attraction of Hale’s Tours to create a cinematic experience.88 Miriam Hansen notes the live entertainment roots of cinema where the viewing spaces were curated for an integrated experience.89 Continuing this tradition, Ella Harris notes, are the likes of Secret Cinema, a contemporary pop-up cinema group. They utilise the physicality and performativity of the screening site to create an experience that is unique to film and viewing.90 In these discussions, the sites in which the works screen also make up the relative out-of-frame, which is conducive to viewing or interpreting images internal to the frame.
In the context of the cinematic out-of-frame, Deleuze’s analysis of the out-of- field is important because the out-of-frame cannot only be considered spatially but also opens onto the durational. Bogue notes that Burch’s off-
87 Ibid., 294, 295. 88 A popular experience in the early 20th century, Hale’s Tours (actually a chain of theatres) simulated train journeys where the passengers sat still while the images around them moved. Tom Gunning, “The Cinema of Attractions: Early Film, Its Spectator and the Avant-Garde,” in Wide Angle 8, no. 3, 4 (Autumn 1986): 65 89 Miriam Hansen, Babel and Babylon (London, Cambridge: Harvard University Press, 1991), 43; Also see Leon Zonn, ”Going to the Movies: The Filmic Site as Geographic Endeavor,” Aether 1 (2007): 63–67; Miriam Hansen, “Early Cinema, Late Cinema: Transformations of the Public Sphere,” in Viewing Positions, Ways of Seeing Film, ed. Linda Williams (New Brunswick, NJ: Rutgers University Press, 1997). 90 Ella Harris, "Exploring pop-up cinema and the city: Deleuzian encounters with secret cinema’s pop-up screening of The Third Man." Journal of Urban Cultural Studies 3, no. 1 (2016): 120.
200
screen spaces comprise the “relative out-of-field,” a spatial nesting of frames where a frame may be included in a larger frame, which is further included in a frame larger than the previous one, and so on. Eventually, this stretches out to infinity as a mise en abyme of framing unfolds.91 Hence the question of framing even as a spatial practice always implicitly points us toward indeterminacy. The relative out-of-frame touches on the “absolute out-of- field,” which is the open whole of durée.92 The concept of durée comes from Bergson’s single duration, which contracts and dilates to form space and time.93 It is the larger set that is always linked to the frame.94 The frame is never really ‘closed’ but can be thought of as being connected to the “universe or a plane [plan] of genuinely unlimited content.”95 Considering that frames always refer to an indeterminate whole is helpful in understanding the various immediate connections that the frame has, as well as those that are immanent and potential.
To be clear, both the relative and absolute out-of-frame operate together, although one can be more pronounced than the other in a particular film or artwork. But the absolute aspect of the out-of-frame is the opening of the frame, and subsequently the set, to a duration.96 This invitation to any potential space and time allows the insisting or subsisting of the absolute out- of-frame in a frame.97 The absolute out-of-field subsists when there are more connections between the frame and the relative out-of-frame. If the frame has lesser connections with the relative out-of-frame, the absolute out-of-frame insists, that is, the durée is experienced more strongly.98 The connection to the
91 Bogue, Deleuze on Cinema, 44. 92 Deleuze, Cinema 1, 16, 17 93 Bergson, Time and Free Will. 94 Deleuze, Cinema 1, 16. Set, if we recall from my introduction to this chapter, is used by Deleuze to define a frame. 95 Ibid. 96 Ibid. 97 Ibid. 98 Ibid.
201
durée is weaker, that is, subsisting, when the relative out-of-field is spatially seen. These connections can be established through editing. For example, if the characters are defined more spatially on screen, such as by using a range of angles, the connection to the relative out-of-field is greater, and the thread to the relation to the absolute outside is slimmer. Such relations of absolute out-of-field subsisting and insisting can be observed in the Google Maps assemblage that I discussed in chapter one. The user navigating with the help of Google Maps on their smartphone sees the changes in the map area on- screen as they move along. The persistence of relative out-of-frame with the framed map area is evident, as when the map area changes new information about the neighbourhood becomes available on-screen. The absolute out-of- frame subsists in this case, but if one zooms out of the map the relative out-of- frame becomes less important and the whole insists more.
Deleuze offers two more aspects of the out-of-field. Unlike subsisting and insisting, where one function can dominate, these two aspects are simultaneous: first, the actualisable relations that the frame has with other frames; and, second, its virtual relationship with the whole.99 In a common set-up in cinema where a scene has a number of shots of different sizes and angles, the virtual relation to the whole is attained through the sequence of images — its editing — which creates relationships between frames and thus signals a virtual whole. Deleuze says that this is nevertheless done by expanding and actualising relations, which is the first feature of the out-of- field;100 that is, the sequence of images in the edit connects the frame to the whole. Another more direct execution is where the relationship with other sequential frames is annulled, such as zooming out. This can be seen in Powers of Ten (1977) by Charles and Ray Eames, where the image zooms out moving
99 Ibid., 18. 100 Ibid.
202
from a picnic scene to the solar system and the ‘whole’ universe. Here the relation to the whole is established in and through the framed image itself.101
Yet how valid is the out-of-frame in a postmedial set-up that presents different conditions for the out-of-frame than cinema and its screenings? Deleuze was particularly concerned with a cinematographic perception in his two cinema books, which have contributed significantly to how concepts, practices and bodies in the wider world could also be thought through this same ‘lens’. Keeping in mind that the relationship between movement and time described by Deleuze was based on the analysis of a certain kind of image production and reception, it is imperative to reconsider in the contemporary context what relations hold between frames and the out-of- frame. With new filmmaking techniques such as motion capture and animation, can these concepts be re-evaluated in order to understand the instability of the out-of-frame in the postmedia context? Moreover, how are screens implicated in these enactments of qualitative multiple temporalities that seem to be a feature of postmedia assemblages?
If considered via a spatial paradigm, a relative out-of-frame may not quite actualise in digital and postmedia. While analysing the out-of-frame largely in digital cinema, and in relation to the image inside the frame, Gregory Flaxman declares, “the very era of the off-screen, which the cinema opens up, is now coming to a close.”102 Considering Flaxman’s analysis is based on the digital filmmaking practices of recording and editing films, he argues that the number of screens and cameras that are used in making a film such as Avatar (James Cameron, 2009) opens the dimensions of the framed image to infinity.103 This means that there is no out-of-frame since what is outside is
101 Ibid. 102 Flaxman, “Out of Field,” 127. 103 Ibid., 129.
203
“folded back into the image.”104 Flaxman reaffirms Deleuze’s proposition on electronic images that “the new images no longer have any outside (out-of- field), any more than they are internalised in a whole; rather they have a right side and a reverse, reversible and non-superimposable, like a power to turn back on themselves.”105 This means that digital images enact a relative out-of- frame that is constantly remodified and reorganised.
In a postmedial sense, the relative out-of-frame in Google Maps (an assemblage, which I examined in detail in chapter 1) collapses into an always being in frame on the screen. The zooming in and out of the map brings the off-screen into the frame in a continuous way. In the ‘globe view’ of Google Maps, the Earth is rendered in a spherical shape to indicate the planet as seen from space.106 From the microscopic close-up Street View, we can expand out to a macroscopic view of the planet. The relative out-of-frame (off-screen here) is always in the process of being wrapped back into the frame on the screen.
Similarly, although temporally faster, gathering the out-of-frame within the frame can be seen in tagging Facebook photos. Hovering on photos reveals the names of people tagged in the photos; clicking on them takes us to their profile pages. Here the photos bring the relative out-of-frame into the frame on the screen, but they also contract the past in the present instance of viewing, including the multiple temporalities of people tagged in the photograph, the time at which the photograph was taken, and its viewing converge within the frame.
104 Ibid., 127. 105 Deleuze, Cinema 2, 265. 106 This is a graphical view of the Earth in space. It has very little resemblance to the actual view of the Earth in space. For example, the shape of the Earth is that of an oblate spheroid, not a sphere. However, the sizes of countries are less distorted than in flat projections, where Greenland is often shown to be the same size as Africa.
204
Notably, the ongoingness of a folding back of the out-of-frame is performed by the screens. There are several examples of such shiftings of the out-of- frame into the frame in postmedia environments. Does this mean that Deleuze’s argument about the dissolution of the out-of-frame is valid in a postmedia context as well? Maybe we need to think an out-of-frame differently in postmedia because the image that is inside the frame is unlike the cinematic image. Yes, the postmedia image is a digital image, but its movement in the postmedia (imagistic) assemblage need to be considered carefully in relation to screens as active and performative assemblers.
The terms of the out-of-frame in postmedia are not the same as in cinema, where Deleuze originally conceptualised them. The action of dragging what lies beyond the frame into the frame is particularly stark in computer games where the player can move in the gaming environment; while moving, the image inside the frame is constantly changing as the relative out-of-frame is accessed. These games require the player to move the frame, through the mouse or the keyboard, to see what lies ahead or around them in order to move forward. As one redraws the frame displaced by the relative out-of- frame, new objects, new relations, and new connections are made which reconstitute and rearrange the existing ones. What complicates such an exchange is the necessary interplay of the relative out-of-frame with the image in the frame; the out-of-frame is always in the state of (be)coming into the frame. This also instigates a reworking of the relationship between the off-screen and the relative out-of-frame — a connection that had been collapsed as the same thing in prior cinema studies such as Burch’s.
When considering digital images, the relationship of frame(s) with out-of- frame(s) and screens has not been scrutinised enough. Morten Meldgaard follows this line of defining digital images in order to get to a Deleuzian
205
cinematic theory in a digital context. Digital images, Meldgaard states, are “plasmatic” and “informatic,” that is, they do not need a camera to capture them on a plate or film; they can be generated on a computer and are modifiable.107 So, if the image is not produced by the camera and the out-of- frame is already folded into the image as I have discussed above, the out-of- frame also has to be located not in terms of space and direction (relative out- of-frame) but temporally (absolute out-of-field) in relationship to the frame. Meldgaard hints at this understanding, stating,
The new image is plasmatic, owing to the fact that it can fuse ‘non- localizable’ temporal relations with the Cartesian space of the frame. The common space-time coordinates of the image will be deformed but only for a while. In a sense this is a resurrection of the ‘movement- image,’ but in a new dimension, since it is now able to account for the temporal dimensions that shattered classical cinema.108
While Meldgaard prompts the discussion in a productive direction, he does not describe what entails the temporality of the out-of-frame. I want to interrogate this in the postmedial context as this setting is composed of multi- medial temporalities characterised by movement, transformations and reconfigurations. Moreover, out-of-frames, gestured by screens in postmedia assemblages, produce an arrangement of time that changes qualitatively — a continuous change — which is what I will proceed to discuss now.
Although it is not as easily distinguishable, the off-screen and out-of-frame can be comprehended spatially in postmedia through the relationship of the screen and frame with images. Off-screen certainly means out-of-frame,
107 Morten Meldgaard, “Dimensions of the Out-of-field: A Contemporary Reading of Deleuzian Cinematics,” In TransVisuality: The Cultural Dimension of Visuality, vol. 1, ed. Tore Kristensen, Anders Michelsen, Frauke Wiegand (Liverpool: Liverpool University Press, 2013), 174, 181. 108 Ibid., 181-2.
206
something that is not seen, but out-of-frame does not necessarily mean off- screen. Images displayed on a screen can take up the entire screen space or only a part of the screen. For example, opening an image on the computer may not envelop the full screen space, but only a part of the screen. Opening the edit tool to manipulate the image will take place relatively out-of-frame but not off-screen. Of course, the edit toolbox is also a frame, but with respect to the boundary of the image, it is out-of-frame. In such a manner, the out-of-frame may not be off-screen. Other familiar everyday actions, such as scrolling through a website, require bringing the relative out-of-frame into the frame and onto the screen. Here too, the out-of-frame is not off-screen. However, unlike image editing discussed above, this is a more dynamic movement where the traversing of the out-of-frame into the frame destabilises framing as boundary-making. The out-of-frame becoming the frame displaces the frame; it does not merely confuse the frame and out-of-frame. The process of de-framing takes place along with the process of reframing. The out-of- frame is deterritorialised in relation to the frame and in certain conditions it is reterritorialised, for example, when moving through a game. Additionally, the absolute out-of-field as something that produces an opening onto duration, and the relative out-of-frame considered as predominantly governed by spatiality is not a division that sits comfortably for postmedia. Rather, I propose the out-of-frame as a process of engaging and generating inbetweenness and playing out through postmedia assemblages. My concern will be with what this inbetweenness generates and what the role of screens is in this generativity.
To recall the main point of the discussion on assemblages from chapter 1, assemblages are produced via temporary connections, which also means that disconnections are likewise made. An aspect of the multiplicity and heterogeneity of assemblages then involves the durational differences of such
207
unmaking and making. Yannis Hamilakis comments on a less discussed feature of assemblages, the “mnemonic/temporal,” to discern assertive sensorial and affective instances.109 This means that in the contingency and temporariness of the assemblage, only some temporalities are activated. Hamilakis asks about affective and sensorial experiences that could be generated in the assemblage. For my research, this opens significant questions about relations and affects with regard to screens, frames and the out-of- frame. I will discuss the affective responses of the temporal relations of screens in the next section. But before that, it is important to be attentive to the relations of the assemblage. Deleuze suggests assemblages require consideration of their direct connectivities but also, importantly, the condition and relations of inbetweenness that also contribute to the quality of relations.110 As he remarks, “it is not the elements or the sets which define the multiplicity. What defines it is the AND, as something which has its place between the elements or between the sets.”111 This inbetweenness can be taken as an affectivity experienced sensorially as suggested by Hamilakis, but I would additionally like to use this to think about the inbetweenness of different temporalities in postmedia assemblages.
The mutability of media in postmedia as discussed in chapter 1 complicates and augments the temporal complexity of the assemblage. Postmedia assemblages, with their variable elements, sensory modes of engagement and through their archival capacities, bring a multiplicity of temporal experiences that are both human and nonhuman. Abandoning the linear understanding of time as successive, time is an asynchronous and non-linear experience, unfolding in ways that resonate with Bergsonian duration. Since in postmedia
109 Yannis Hamilakis, “Sensorial Assemblages: Affect, Memory and Temporality in Assemblage Thinking,” Cambridge Archaeological Journal 27, no. 1 (2017): 172, 175. 110 Tim Ingold, The Life of Lines (London: Routledge, 2015), 147–53. 111 Deleuze and Parnet, Dialogues, 34.
208
the relative out-of-frame or the off-screen is constantly being pulled into the frame, the out-of-frame is not quite inside the frame but also not outside; it is inbetween. This inbetweenness of the out-of-frame threatens the stability of the frame as well as produces temporal disruption. The inbetween, we can recall, is not a space but “moving registers of tendencies.”112 Using the work Can You See Me Now? by Blast Theory, I will attend to changeable, amorphous and punctured temporalities. What does inbetweenness here do to screens, frames and other aspects of the assemblage?
MULTITUDINAL TEMPORALITIES IN AND OF OUT-OF-FRAME Can You See Me Now? (2003) is a location-based game by Blast Theory that combines and offsets online gaming action with gaming activity in physical space. It is a game where the players on the street — or runners — have to catch the online players. While the runners are physically present on location, the online players access a real time interactive rendition of the same locations in the form of an updating, moving graphics map. To login to the game, the online players register with the name they give upon answering the question “Is there someone you haven’t seen for a long time that you still think of?” The online players move around on the real time interactive map to dodge the runners. The physical runners carry a GPS device where they keep tabs on the location of the online players as they enter the game. The runners therefore negotiate their physical coordinates in relation to the online map. This way the x,y coordinate positions of all the players are known to each other (fig. 39).
112 Munster, “Into ‘inter’,” 59.
209
Figure has been removed due to copyright restrictions.
Figure 39: Blast Theory, Can You See Me Now?, 2003, location based mixed media game. Digital video still from documentation of the game played in Tokyo in 2005. While two sets of players are both moving in different kinds of spaces – on- screen and physical – the real time map is a kind of shared space in which they both view each other as dots moving around coordinates in a graphics schematic, where runners are marked in red and online players in white. The map visualisation available to the online players is three-dimensional, while the runners have a mobile computer device (designed before smartphones with large screen interfaces were common) with a two-dimensional visualisation of the location (fig. 40). All runners are also equipped with a walkie-talkie through which they can communicate with other runners, exchange possible tactics and provide instructions to catch the online players. The walkie-talkie communication is also relayed to the online players, potentially helping the latter in not getting caught. On the other hand, the online players have a chat window that enables them to converse with each other. To catch the online player, runners have to get within five meters to the coordinates of the online player, and record the name assigned by the online player together with a photograph of the physical street location.
210
Figure has been removed due to copyright restrictions.
Figure 40: Blast Theory, Can You See Me Now?, 2003, location based mixed media game. Digital video still from documentation of the game played in Tokyo in 2005. The gameplay, though deceivingly simple, involves a complex enactment in and through screens and frames based on the ways in which the postmedial assemblage entangles temporalities and relationalities between screens and spatialities. Can You See Me Now? is not only a game but a situation where the players have to cooperate with each other, build trust, and be aware of their social responsibilities.113 As a performative experience, this work mixes the physical with online participation.114 Blast Theory identifies the overall location for gaming as a hybrid city built from overlapping physical geography and digital codes.115 According to Maria Chatzichristodoulou, this hybrid city is in constant flux as the players rework its space-time relations every time when they interact or move around.116 However, I propose a
113 Duncan Rowland, Martin Flintham, Steve Benford, Nick Tandavanitj, Adam Drozd, and Rob Anastasi, “On the streets with Blast Theory and the MRL: ‘Can You See Me Now?’ and ‘Uncle Roy—All Around You,” in Proceedings from Mobile Entertainment: User-Centered Perspectives, ed. Karenza Moore, Jason Rutter (Manchester: ESRC Centre for Research on Innovation and Competition, 2004), 115–124, 122–123. 114 Piotr Woycicki, “A Critical Study of Physical Participation in Blast Theory’s Can You See Me Now?” International Journal of Performance Arts and Digital Media 10, no. 2 (2014): 198, https://doi.org//10.1080/14794713.2014.946286. 115 Blast Theory website, https://www.blasttheory.co.uk/projects/can-you-see-me-now. 116 Maria Chatzichristodoulou, “Blast Theory,” in British Theatre Companies 1995–2014: Mind the Gap, Kneehigh Theatre, Suspect Culture, Stan’s Cafe, Blast Theory, Punchdrunk, ed. Liz Tomlin (London: Bloomsbury, 2015), 231–254, 247; Maria Chatzichristodoulou, “When Presence & Absence Turn Into Pattern & Randomness: Can You See Me Now?” Leonardo Electronic Almanac 16, no. 4–5 (2009), https://www.leoalmanac.org/wp- content/uploads/2012/09/06_chatzichristodoulou.pdf.
211
different analysis, one that is attentive to the entanglement of screens, frames and human bodies and their multiple temporalities. How such intra-actions (and emergent production of agencies) take place through screenic gesturing of the frame and out-of-frame will also be explored.
The runners are constrained by the frame on the two-dimensional map they are viewing, and they have to keep moving in order to keep up with the online players. If they do not, the dot on the maps, or the position of online players, will move out-of-frame or what we might conventionally think of as off-screen. The frame keeps the dot on the screen, visible to the players. But as the frame keeps changing with the players’ movements, the screen enacts a gesture – it shifts the relative out-of-frame into the frame. This way there is no absolute out-of-field, as Deleuze suggested of cinema, as the out-of-frame is no longer pointing to the whole but is constantly becoming the whole. However, is this question purely a spatial infolding — the images being pulled from their out-of-frame in — or should we consider that the screen’s gesture in postmedia assemblages is also a temporal one? The screen, I argue, sustains the relation with the out-of-frame temporally.
In the postmedial set-up of Can You See Me Now? the frame ‘conquers’ what lies beyond it as it enfolds the outside. This can also mean that the frame acknowledges its boundaries, although it is always trying to extend them by exploring the potentialities of the other parts of the assemblage that it (re)connects to. Frames also participate in the transductive forces (re)articulating the human players and nonhuman computing devices and cityscapes. The activities in Can You See Me Now? take place in-the-frame and out-of-frame where the out-of-frame is no longer the cinematic space, but a dynamic open space of potentialities. If the relationships between the image and the frame are not the same as in cinema, then the residue of a Deleuzian
212
absolute out-of-frame has to be re-evaluated in postmedia based on the realignment of the relationship between frames and images in postmedia and inbetweenness, producing multi-temporalities. Together these give a constantly differing whole — a multiplicity — for the assemblage. Such ideas rest on the observation that although the out-of-field may be unstable and continuously changing in postmedia, it is not absent. The discussions below evaluating the enactments of screens, images, frames and out-of-frames are aimed at making the conditions of postmedia visible. In the process, I will also examine some smaller, paradoxical movements facilitated by the out-of-frame, such as instances where the out-of-frame resists being captured by the frame and where a dispersal of out-of-frames potentially generate crystal-images. These are images that become connected to their own screen images.117
The images on the screen of computers for the online players and the runners on their GPS device can be understood in two interconnected ways: as relational transactions and as actionable images. On-screen images for the runners and players are closer to diagrams than images that correspond to what and how we see. For both the players and runners, the screen images are visualised maps of different kinds, offering varied perspectives for the online players and the runners (the online player has a ‘window view’ while the runner has a ‘tabletop’ view). These kinds of images are not just seen but also move with and along the body. They can be understood as layouts that support the player and the runner in different ways. James J. Gibson points out that layouts show areas relative to each other and are afforded differently by creatures with varying perceptual systems.118 Here the layouts are different for each of the players. Moreover, for the runners, the contrasting changes of what they see on the GPS and the information that they are getting from their
117 Deleuze, Cinema 2, 69. 118 James J. Gibson, The Ecological Approach to Visual Perception (New York: Psychology Press, 1979), 307.
213
senses from being in the streets has to be constantly negotiated.119 Their GPS devices provide optical information about the online players, while moving around through the streets provides multiple different sensual and haptic information about the same space. Gibson talks about not perceiving the layout but its affordance.120 The runners have to be able to detect the affordances of the ground, other people moving around, parked vehicles and many other objects lying or moving on the roads, all the while manoeuvring around corners, squeezing through alleyways, and jumping across gaps. They are constantly (re)making the relations between their body and the environment they are in. Gibson says, “The awareness of the world and of one’s complementary relations to the world are not separable.”121 In the on- screen map the layout enacts a different affordance. As dots moving around on the map, the image is not prolonged through activity but is itself in action.122 Galloway proposes considering video games as actions which involve “active participation” by both the players and the machine.123 Deleuze’s action-image is no longer a cinematic image; instead, Galloway insists that it is the foundation of games where “materiality moves and restructures itself.”124 Following Galloway’s analysis, Harun Farocki has termed contemporary images commonly seen in games and mobile phone map applications as actionable images.125 These images can be acted upon, with their
119 I borrow this idea from an experiment by E.J. Gibson and R.D. Walk, illustrated by J.J. Gibson. It presented the contradiction of optical and haptic information in animals and babies. The experiment observed babies and animals as they crawled and walked on a glass floor. When it was covered from underneath, it gave the feeling of walking on solid ground; when the glass floor was not covered, it provided the babies and animals with the tactile feeling of the ground but visually the transparent glass gave a feeling of ungroundedness. Ibid., 156–157. 120 Ibid., 158. 121 Ibid., 141. 122 Maurizio Lazzarato, Videophilosophy: The Perception of Time in Post-Fordism, ed. and trans. Jay Hetrick (New York: Columbia University Press, 2019), 101. 123 Alexander R. Galloway, Gaming: Essays on Algorithmic Cultures (Minneapolis: University of Minnesota Press, 2006), 2. 124 Ibid., 3. 125 Harun Farocki proposed the actionable image in his lecture ‘Computer Animation Rules’ at IKKM, Weimar, 2014, after encountering Alexander R. Galloway’s actionable object in Gaming.
214
relations altering with each adjustment. The actionable images in Can You See Me Now? are formed by actions outside the frame.
The movement of players itself is of two kinds: one takes place physically, produced by the runners; the other occurs on-screen, and relates to the online players. The moving body of the online player in the graphics map is actualised by the runners in the act of chasing them. Moving the out-of-frame inside the frame means only a certain possibility is realised from all of the virtual potentialities. While tracking the dot of an online player, the runners are chasing an immediate past, as the dot has already moved before they catch up. However, since the out-of-frame continues to change, contingencies are altered as well. If the present is a contraction of the past, then the number of possibilities that can be executed increase at every instance. 126 Meanwhile, the actualisation that becomes the frame intra-acts — an entanglement of agencies where entities are not fully formed — with other elements of the assemblage to execute certain actions. These, too, are feeding into the changing multiplicity of potentialities. This way, the image on the screen for the players and the GPS device for the runners is not fixed, but in flux. Instead of a representation, this image is a “negotiation” between various connections.127 Or we could say that the image is a relationality.
In certain circumstances, the out-of-frame may not get pulled into the frame. The photograph taken by the runner in Can You See Me Now? captures the location where the runner has ‘spotted’ the online player.128 The photograph itself is banal and inconspicuous. I will analyse this in detail later, but for now
126 Deleuze, Cinema 2, 80. 127 Sandra Alvaro, “Complexity, Multi-perspectivism and Tracking: A Brief History of the Meaning of Image, from the Postmedia to the Postdigital,” Technoetic Art: A Journal of Speculative Research 11, no. 3 (2013): 205. 128 Interestingly, most of these photographs are angled towards the ground, as though the runners are marking the literal spot where the player would have been standing if they were playing in person.
215
I want to focus on the act of taking the photograph. When the runner is photographing the location, they are trying to frame a spot in any direction and with anything in the frame. In the action of photographing, the out-of- frame does not encroach; the frame is bounded and sealed in the photograph. This resistance of the out-of-frame to being pulled back inside the frame is also observed in the case where the GPS navigation system breaks down and the movement of the dot stops. During gameplay, the GPS was reported to malfunction between buildings and trees.129 When this takes place, the screen of the moving runner would not update the location; that is, the off- screen/relative out-of-frame does not come into the frame. In this case, the out-of-frame, for at least a short period of time, does not become a part of the frame. The GPS device is in the flow but not capturing it. How does this modify the temporal experience? Although parts of the GPS device continue to be occurring in sequential time, the frame, image, screen and the out-of- frame are suspended. This suspension gives rise to multiplicities.130
This assertion of a virtual image, in the sense that it opens up to a range of potentialities, could mean many things in the end: the online player, the location that is seen in the photograph, the name of the person that the online player remembers, or the object seen in the photograph. Hence the photograph encompasses many things at once while not specifying any. A possible delay in the GPS signal makes the runner move in a different way — they could be plotting their next move, and deliberately attempting to block the GPS signal so that the online player cannot see their location. In this way,
129 Rob Anastasi, Nick Tandavanitj, Martin Flintham, Andy Crabtree, Matt Adams, Ju Row-Farr, Jamie Iddon et al. "Can you see me now? a citywide mixed-reality gaming experience," in Proceedings of the Ubi-Comp (2002): 11. 130 The strategies of suspension as temporal experience can be explored using Alanna Thain’s redefinition of suspense. She describes suspense as “a technic for an attentive awareness to the minor form of difference that (re)constitutes a body in time, a feeling of futurity immediately impinging on the body’s stability and reopening it to intensive relationality.” (p 2) She proposes that suspense(ion) allows “the past experienced as the potential of the future.” Alanna Thain, Bodies in Suspense: Time and Affect in Cinema (Minneapolis: University of Minnesota Press, 2017), 84.
216
the runners could be anywhere for the online players. For the runners, a break in the signal can mean that the online player could be anywhere. In these instances, as a frame that is not being displaced, it takes on the features of a cinematic frame that is connected to the whole in the same way as the Deleuzian out-of-field functions.
Can You See Me Now? provides the online players and the runners with a different experience of space, as they occupy different kinds of environments.131 They have a different view of the city on their electronic and digital devices, which challenges their relationship with it.132 The mapping of the physical space onto the digital space where circulation of presence and absence of players has been well documented.133 However, instead of viewing this as a game where different spaces — online and physical spaces — both overlap and slip, I want to think of the gaming event through different temporalities. I will address movements that are not just spatial but are instead movements in/across time and the mutability of memory. The temporal modulations are of interest because they point towards what is peculiar to the postmedia assemblage. The runner is not in the same location as the player, but their temporalities are entangled in ways that compress, dilate and overlap in the gameplay. Data capture and updating happens in real time. The image on the screen evolves as connections are reconfigured and reproduced continuously in the assemblage. Moreover, the assemblage includes experiencing and observing agencies that are out of human time and scale. There are micro-temporalities where software and hardware function at their own speeds and degrees that may not be perceived by humans. However, they still strongly impact upon the player’s movements and actions,
131 Steve Benford, Andy Crabtree, Martin Flintham, Adam Drozd, Rob Anastasi, Mark Paxton, Nick Tandavanitj, Matt Adams, and Ju Row-Farr, “Can You See Me Now?,” in ACM Transactions on Computer- Human Interaction 13, no. 1 (2006): 108, https://doi.org/10.1145/1143518.1143522. 132 Rowland et al., “On the Streets with Blast Theory and the MRL,” 115-124, 123. 133 Chatzichristodoulou, “Blast Theory,” 231–254, 246–248.
217
modifying sensory and temporal experience.134 By paying attention to how relations and processes engage temporality, I also consider how the different entities in the assemblage participate in the making of durations, including real time. The constituents of the assemblage do not have a fixed identity but are processually made. As their capacities change, so do the relations.
In a postmedial assemblage, the machines have their own ongoing intensive movement. The apparatuses are contracting different kinds of matters, movement and signals that are also experienced by humans. There are various other possibilities that could be executed by the technological apparatus, but only one is actualised. In Can You See Me Now? the screen displays a flow of matter as the hardware and signaletic processes (software, GPS signals etc) modulate that flow. A number of temporalities and realities are flowing on the screen, off-screen, out-of-frame, and in the physical space, as well between the parts of the assemblage. In Can You See Me Now? the present of the runners and players is co-constituted through the “ever- expanding network and the unison of connected yet geographically dispersed movements in the present.”135 The postmedia assemblage creates a range of ways in which human and nonhuman entities are affected and are affecting each other. In Can You See Me Now? I focus on the out-of-frame, in order to explicate its production and affections in time.
The chasing of the dot (the GPS location of the online player) takes place through the screen of the GPS device of the runners. In the chase, time plays out differentially. For example, the movement of players is different from that of the dot’s movements on the screens. Moreover, the multiplicity of screens and frames are performing the displacement of the out-of-frame differently
134 Barker, Time and the Digital, 3. 135 Barker, Time and the Digital, 30.
218
for each player at separate times in the gameplay. The dot on the screen of the handheld computer device of the runners is creating another temporal experience, which may seem to follow the motion of online players but is subject to its own micro-durations and relays made possible by the assemblage of networked connections. As GPS signals and the internet are an important part of the assemblage, any interruption to them can cause temporal disparity for the players. As I have noted in the previous discussion, the runners utilise the deficiency of GPS signals around tall buildings, taking narrow paths closer to possible GPS inhibitors in order to stealthily catch up with the online players.136 Due to the delay in updating the position of the runners, the online players can be caught off-guard. Taking into account what this signaletic delay offers temporally, the online player might find the runner to be in the frame when they are out-of-frame, in a different space- time continuum than to what is immediately visible to the online player.
Since this game was last played in 2010, there have been substantial efficiencies achieved for GPS technology.137 The hardware has been enhanced, newer techniques of using more sensors and better GNSS (Global Navigation Satellite System) have contributed to greater accuracy and mitigation of multipath problems. How might the delays and breaks in the GPS signals have altered the experience of playing the game, or produce different durational experience of gameplay? With a GPS that now has more precision, would the out-of-frame continuously move into the frame rather than be deferred in the way I have suggested? Players may no longer have a
136 The Global Navigation Satellite Systems (GNSS) is degraded around tall buildings and trees due to the multipath effect, a phenomenon of the multiplication of signals from buildings and other surfaces. This process can be augmented by diffraction from small objects in the city such as traffic lights and road signs. See Shunsuke Miura, Shunsuke Kamijo, “GPS Error Correction by Multipath Adaptation,” International Journal of Intelligent Transportation System Research 13 (2015): 1–8, https://doi.org/10.1007/s13177-013-0073-9. 137 Some of the technical errors, such as the GPS issues, are covered in Benford et al., “Can You See Me Now?” 100–33, 110–11, 127–29.
219
convenient time to strategise due to GPS interruptions under contemporary conditions.
Moreover, the game location that was accessed by the online players was only an approximation of the actual location. As a schematic, the online map provides enough information to move around the turns and across the pavements, but it does not, for example, provide the steepness and narrowness of the road. These topographical features influence the conduct of the chase by the runners. Additionally, runners might be confronted by a crowded street while the online player navigated people-free roads on their screens. Both crowds and topography could slow down the runners.138 The physical spaces provided a different set of contingencies that are transmitted in the network creating and executing different possibilities. Moreover, moving in the city space introduced the history of the city as well. A city does not only bear the past in the form of monuments, but is wrapped up with the history of media, information infrastructure, and cultural and social networks. As Raymond Williams contends, these are “formed in the past, but… still active in the cultural process, not only and often not at all as an element of the past, but as an effective element of the present.”139
Up to this point, I have attempted to explicate the postmedial out-of-frame by analysing the work Can You See Me Now? by Blast Theory. I have argued for images as being actionable, where the relationship of the image in the frame is actualised by the virtual out-of-frame. Munster has described “distributed aesthetics” as those that depend on and prioritise temporalities over spatialities.140 This can help us in thinking about how these multiple
138 Ibid., 127–129. 139 Raymond Williams, Marxism and Literature (New York: Oxford University Press, 1977), 122. 140 Some of the driving factors behind this shift, as Munster elaborates, are the flow of global capital, social networks and information infrastructure. See Munster, Materializing New Media, 171, 172.
220
temporalities “become a set of temporal coordinates for digital aesthetic production.”141 Can the photograph that is produced at the end of gameplay in Can You See Me Now? affirm the temporality of screens in producing new kinds of digital images? Can we also consider not just the relationality of screens, frames and the out-of-frame, but also how these relations generate affective responses? To put it in another way, new affective responses and sensations are gestured in and through screens in postmedia. This will help in recognising how screens are co-composed in the postmedia assemblage.
The relationships in Can You See Me Now? are inflected with affect. Here I take my lead from Munster’s assertion that affect is
not reducible or located at the level of ‘the body.’ Nor is it a mode of knowing, ordering or representing this body… nor can affect be represented per se in aesthetic terms. Rather, affect occurs as a process of composition that is sustained through a relation between body and expression, representation, map and knowledge.142
This means that relations are experienced affectively. To discuss what I am calling affective temporalities, let us examine the difference in the experience of the online player to the moving body of the runner. The relationship between the body of the online player as one of controlling the movement of the white figure on the 3-D map on-screen falls into describing a relation of mere causality. Here the stationary body of the player ‘causes’ the movement of the figure in the 3-D map, which is then being chased by the runners. Instead, I want to think of how the different entities produce each other. Their becomings can be articulated as transversal. Colebrook explains that transversal becomings do not simply unfold progressively in time but change
141 Ibid., 173. 142 Ibid., 139, 140.
221
with every new encounter.143 Articulating gaps gestured by the screens and the out-of-frame in the assemblage can help in thinking about an inbetweenness that co-produces entities. The screens sustain affect for the online player throughout the game, developing out of an out-of-frame action. The experience of the online player moving through the graphical renderings of the space is unlike the runners on the ground, whose body physically moves through the city-space. However, the excitement of getting away, or the frustrated feeling of being ambushed by the runners could well have been felt by the online players. These online player experiences were not displayed in the figure on-screen. Here the body was known and sensed in a different way by the player than the runner. By that I do not mean that one of them was the winner, but the online player might register the experience without feeling it in their body. Munster explains that affect “arises relationally and is produced out of the difference” between the bodily sensations and the technical images of bodily rendering.144 The difference, continues Munster, registers as interfacings between the body and technology. Therefore, she calls this overall generation “informatic affect.”145 For the online player, these affects appeared with sensing the defeat in the game just as the graphic of the runner approaching them was being rendered.
When the runner gets close enough to the online player, they say the name that was entered by the online player when answering the question from the beginning: “Is there someone you haven’t seen for a long time that you still think of?” and take a photograph. This action of the runner taking the photograph of the online player, out-of-frame, makes the player physically present while maintaining their absence. The runner takes a photograph of
143 Claire Colebrook, Gilles Deleuze (London: Routledge, 2001), 37. 144 Munster, Materializing New Media, 142. 145 Ibid.
222
the location where the online player is, according to the GPS location. The photograph could be of anything — a shop, pavement, road, lamp post or something else (fig. 41). This physical location is irrelevant, since it is the dot on the screen that counts. What is inside the frame and how the framing is done does not matter because what is outside the frame is crucial here; the player, who could be in any location, could also even be in a different time zone. Therefore, the taking of the shot captures an overlap of times.
Figure has been removed due to copyright restrictions.
Figure 41: Blast Theory, Can You See Me Now?, 2003, location based mixed media game. Selection of stills from the game played in Tokyo in 2005.
Complicating this further is the name of a person, who is only involved with the game in and through the memory of the online player. The past returns both in the duration of the game as well as personally for the player when they remember someone. Who or what is being photographed then? Is the photograph that of the online player, the person whose name was entered by
223
the player, or the location? The photograph gathers up temporality itself, becoming an expression of a pure time-image or crystal-image, where different kinds of temporalities, multiple suggestions of time as different “circuits of time” coalesce.146 The name of the person is a recollection of the online player’s past, framed in the present of the photograph. Exactly whose photograph and durational experience are to be enacted in and by the multiple frames is indeterminate. Hence the temporality of the out-of-frame has to be recognised over the mimetic relationship between object or location and representation. The photograph is a collision of the time of its making, the runner’s time, and the lived present of the player, and also encompasses the person remembered. All of these meet in indeterminate duration. The photograph is not merely an absence of a body in space; it is the ‘after-image’ of an ever-expanding past and the forever passing present. In this sense, it is the out-of-frame that is photographed.
The time-image occurs where the image actualises so as to present a direct experience of time. This is not an actualisation of the virtual ‘whole,’ but the actual image becoming its own virtual image, “to the extent that there is no longer any linkage of the real with the imaginary, but indiscernibility of the two, a perpetual exchange.”147 Time is presented directly, and gives a sense of how duration splits between a past that is preserved and a present which is passing into the future.148 These “seeds of time,” present via a different configuration of screens, as I discuss below, highlight an experience of temporality only made possible through the relationality of screens. We will inquire, then, into a configuration of multiple screens in a gallery set-up that sees the gesturality of screens being performed in and out-of-frame(s).
146 Deleuze, Cinema 2, 80-82. 147 Deleuze, Cinema 2, 273. Emphasis in original. 148 Ibid., 274.
224
FRAMES IN CRYSTALLINE STATES I want to turn to the fluidity of frames and screens seen in Ragnar Kjartansson’s installation The Visitors (2012), which allows a postmedial multiplicity to be generated. This work is presented on nine screens (fig. 42). Eight of the screens show one musician each alone in a room. Each of them is playing a different musical instrument, however, they are all performing the same song. One screen displays the exterior of Rokeby Farms in upstate New York, where a few people are sitting on the porch. The scale of the installation is notable. Not only are there many screens, but also the screens are large, almost cinema-sized, and the images are framed to focus on the human figures. The size of the screens highlights details of the frame with an overall effect of drawing the audiences to the components of the framed image.
Figure has been removed due to copyright restrictions.
Figure 42: Ragnar Kjartansson, The Visitors, 2012, nine-channel video installation, 60 mins looped. Installation view at San Francisco Museum of Modern Art in 2017.
The same song is being played on guitar, piano, cello and accordion. For the main part, the musicians sing a refrain written by Ásdís Sif Gunnarsdóttir,
225
“Once again I fall into my feminine ways.” For the audience, they are playing like a band, but are located in separate spaces. Towards the end of a sixty- four minute one-take performance, the musicians leave their individual on- screen space to assemble together on the one screen, first in the piano room and then outdoors. As they come together, they cross each other’s frame- space, revealing that the multiple screens variously frame proximate though separated physical spaces. That the performers were playing in the same location at the same time makes the artist’s deployment of multiple screens an aesthetic strategy that might yield something for our understanding of the gesturality of screens advanced so far in this thesis.
Space and time are both simultaneous and yet multiple in The Visitors. The numerous frames and screens relate in/as (their) multiplicity. The framings bring together multiple durations into one space, sensed rhythmically but also as embodied variations of the individual musicians. Moreover, the multiplying of frames in this work becomes charged with new spatial relations when performers leave their place to enter another frame. As multiple frames cut into the arrangement of the actions and movement of the performers, the multiple screens enable relation across the differentiated place, embodied experience and rhythms, producing a new kind of multiplicity towards the end of the work and display on a single frame.
In some ways, The Visitors is the converse of space-time relations set up in Bookchin’s installations. The Visitors has a simultaneity of time expressed via different durations on the screens in one physical place that is the same yet spatially differentiated; whereas the speakers in Long Story Short were recorded in different times even though they share the same screen space, and through editing techniques appear to be in the same space. The gathering of frames and screens in relation to the on-screen performers/speakers and the
226
viewers is carried out differently as well. In The Visitors, screens scatter the physical space through frames that seemingly maintain a separation of space and time for the viewers. Here, viewers assemble the disparate frames by moving around the installation space — they literally gather up the out-of- frame around the different screens. Numerous screens in Now he’s out in public (installation) maintain the division between spaces but the audience brings them together as the speakers talk about the same person. However, this assimilation has to do with the content of the conversation, not the physical space per se. In The Visitors the audiences’ movements out-of-frame enact multiple temporalities in specific ways. Introducing Bergson’s inverse cone model of time, I will employ two aspects of Deleuze’s time-image to understand the multiplicities of frames and screens temporally: sheets of past, and peaks of present.149
Each screen in The Visitors is a crystalline image of time. As discussed in the last section, the crystal-image is where the actual and virtual are indistinguishable. The musician in each frame plays their own instrument and is connected to others by the viewer as they move through the exhibition space, assembling each layer of music — literally mixing the track. The audience experiences the music not only when they see a particular musician on screen but also while they hear and feel other instruments throughout the installation space. This relates to the “non-chronological coexistences or relations in the direct time-image.”150 Considering the musicians are playing together in the same space makes it even more difficult for the audience to distinguish them through sequential temporal relations. The musicians occupy all bar one of the frames, indicating that each is a protagonist as well
149 Ibid., 90. 150 This is Deleuze’s description of not being able to identify the exact meaning of ‘Rosebud,’ the last word spoken by Charles Foster Kane in Orson Welles’ Citizen Kane (1941). See Deleuze, Cinema 2, 111.
227
as part of a whole. Collectivity is notably present because there is not any one frame where the artwork as happening or as individuated can be pinpointed.
Here the crystal-image can be further understood via Deleuze’s commentary on Bergson’s temporal schema of the inverted cone. Evoking a non- chronological approach towards time, Deleuze says that the crystal reveals two “flows” of time: the past that is preserved, and the present that is passing.151 This past is a virtual past where all past events coexist; remembering is the act of actualisation. Memory always has a “seed” in the past, however virtual, otherwise there is no way to access it.152 If the present is preserved as a seed in the past (fig. 43), then the present moment contracts the past, inverting the cone.153 Amidst the concept that the past pre-exists and the past is infinitely contracted, there are “all the circles of the past constituting so many stretched or shrunk regions, strata and sheets: each region with its own characteristics, its ‘tones’, its ‘aspects’, its ‘singularities,’ its ‘shining points’ and its ‘dominant’ themes.”154 What may appear to be a past structured chronologically, instead exists as “sheets of past” existing together with “peaks of present.”155
151 Ibid., 98. 152 Ibid. 153 Bergson, Time and Free Will. 154 Deleuze, Cinema 2, 99. 155 Ibid., 101. The peaks of present can be elaborated as concurrent presents of “a present of past, a present of present and a present of future.” Ibid.
228
Figure 43: Bergson’s cone of memory, reproduced from Matter and and Memory. Here S is the present moment where the past contracts. Memories at AB and A’B’ and A”B” are different regions of the past (virtual circuits) that preserve the it.156
It is this temporal experience that is encountered by viewers of The Visitors. While moving through and between the screens, the audience is present when viewing a particular frame in front of them. That present also makes up the past, which is accessed again later when all the musicians come together. The viewers move from one peak/screen to another always in the present-passing. Each screen grasps them, the affective quality of the experience taking hold. Within the same event, the different presents of the musicians are like the peaks of the cone that “are implicated in the event, rolled up in the event, and thus simultaneous and inexplicable.”157 Towards the end of the performance, when all the musicians can be seen on the same screen together, the past, comprising all the musicians playing in the same house, is disclosed and made the virtuality that actualises in the new present. The past comes to unfold in the present.
The screens hold seeds of time, allowing the viewers to make temporal connections to the on-screen images. The viewers are not quite sure what the musicians are doing or even that are in close proximity with each other. The
156 Henri Bergson, Matter and Memory, trans. Nancy Margaret Paul and W. Scott Palmer (London: George Allen and Unwin, 1911), 210. 157 Ibid., 100.
229
musicians playing the instruments carry potentialities to be actualised. The musicians crossing each others’ frames and meeting in one location germinates the virtual and makes it an actual image. This is the crystal-image, where time is experienced in its direct bifurcations into past and present. It is difficult to distinguish between the actual image of the present and the virtual image of the past. Deleuze calls these two images that are “distinct yet discernible, and all the more discernible because distinct, because we do not know which is one and which is the other.”158 As Christoph Brunner suggests, here we register affect as a temporal quality:
In its affective force, the work immediately works on the way bodies move with and through it. The spatial folding is also a temporal folding of shared time, repetition and resonance between spaces, bodies and their relation to sensation. Sensation defines the zone of experience where a bodily capacity of sensing with its environment precedes a distinct perception of a body positioned in space.159
I contend that it is through the audience gathering and the out-of-frame gesturing that the affective is generated. Importantly, this interpolation of audience movement into the postmedial assemblage together with the out-of- framing gesture of screens makes The Visitors produce a qualitative multiplicity.
In moving through different kinds of postmedial assemblages, I have provided a temporal understanding of the gesturalities of screens. A rigorous canvassing of various practices analysed the multiple temporalities sustained,
158 Deleuze, Cinema 2, 81. 159 Christoph Brunner, “Affective Politics of Timing on Emergent Collectivity in Ragnar Kjartansson’s The Visitors,” in Timing of Affect: Epistemologies, Aesthetics and Politics, ed. Marie-Luise Angerer, Bernd Bösel and Michaela Ott (Berlin: Diaphanes, 2014), 245–262, 248.
230
experienced and transferred in postmedia assemblages. Overall, the chapter argued that the frame and out-of-frame are gestures of screens in postmedia where the temporalities of sustaining such mediality is scrutinised. Indeed, the result has been an extended, albeit enriching account. I examined the relationship between frames and screens in order to look beyond the ‘framing’ of images by screens. The multiplicity of screens and frames prevalent in postmedia make it imperative to extrapolate their meanings and enactments. I illustrated a qualitative understanding of these variegated temporalities of frames and other entities in postmedia assemblages. This is particularly seen in the various ways movements are generated in the editing and display of Bookchin’s work Now he’s out in public and everyone can see. But the already complex temporal experience becomes more complicated as the out-of-frame in postmedia influences the frame, or as I suggested, the screens (temporally) maintain the out-of-frame; the frame is unsteady as the out-of-frame is ‘becoming (the) frame’. Moreover, the frame and out-of-frame are co- constitutive and are not separate in the action of framing.
Looking beyond the spatial understanding of the out-of-frame, I investigated its temporal movements and hence the qualitative multiplicities generated. The multiple temporalities and the gesturing of screens was considered through the work Can You See Me Now? by Blast Theory. I examined multitudinal continuous temporalities traversing through movements on the screen and off the screen, along with the moving bodies and micro- temporalities of software and hardware that further intensify the relations or even intra-actions of screens, frames and out-of-frame(s). This, I argue, generates affective temporalities. In several instances here we are confronted with the crystallisation of time, where the frame and out-of-frame condense to generate a direct sense of pluri-potentialities of time.
231
In a different kind of congealing of temporalities, I turned towards Kjartansson’s The Visitors. Here my previous analysis on temporalities of frame and out-of-frame have to be accounted for together as the ‘whole’ is spatially fragmented and temporally dispersed in the recording and editing of the work. Then, in its display, the multiple frames coalesce these temporalities differently. Furthermore, the continuous out-of-frame temporalities are accumulated towards the end as the musicians leave their original frames/screens and gather up a different qualitative temporal experience on one screen.
232
4. << BLAN/CK SCREENS, ABSENT IMAGES >> SCREENS WITHOUT IMAGES PERFORMING RACE
The entanglement of images and screens in postmedia indicates a configuring of relations different to that in traditional photography and cinema. In Parallel I-IV (2010–2014) Harun Farocki focuses on the changes in image-making practices that have occurred with the use of computer animation tools. These videos are made almost entirely out of fragments of video games, in their making or playing, commenting on the visual aesthetics of the images in games and the assembling of the game world. In reference to these particular works, Farocki has been described as an archaeologist who excavates images to challenge their meanings.1
Figure has been removed due to copyright restrictions.
Figure 44: Harun Farocki, Parallel I, 2012, two-channel digital video, 16 mins looped. Digital video stills.
1 Andrea Lissoni in “Harun Farcoki — Cinema, Video Games and Finding the Detail, TateShots,” YouTube, Apr 15, 2016, https://www.youtube.com/watch?v=tERIscWmpSo.
233
Parallel I elaborates on the challenge of composing landscapes, particularly moving clouds, blowing winds and flowing water. The techniques utilised here go beyond illustrating the movement of water or objects implying the wind; they also account for changing textures and the interplay with other bodies.2 In figure 44, from the two-channel video Parallel I, shows water in the video games from the early 1980s and a more recent hyperreal representation of water.3 But what is less commented upon is that Farocki is also uncovering the dynamic relationship between images and screens. The images in Parallel I do not just signal the change of image-making practices but demonstrate the modulation of the relationship across screens and images that recognises other relations in the assemblage. The images on the screen indicate the larger technical framework. The so-called ‘better’ images, that is, the ones that seem to be more natural, also indicate a greater degree of power and control, including the ability of the illustrator to draw directly onto screens (mostly through hardware peripherals), the use of multiple software packages and the increased computational power available to render graphics. The viewer sees not simply a branch of a tree or tall grasses being moved by a gust of wind, instead they see the workings of a digital assemblage of game designers, software, hardware and the players that render various shapes and forms. This composes a world where the peculiarities of the relationship between images and screens are highlighted.
Parallel I-IV signals a change in computer graphic imaging practices and technologies, but more importantly it tracks the movement from cinematic images to a particular kind of built image, tracing the relations between the recording of images on camera and the drawing of images on a computer
2 Game physics, a subfield of game design, simulates the laws of physics in gameplay. 3 Techniques such as particle systems and fluid simulation/animation are used for different characteristics of water. For more discussion on a techno-historic perspective of water in video games, see Andrew Hutchison, “Making the Water Move: Techno-Historic Limits in the Game Aesthetics of Myst and Doom,” Game Studies 8, no. 1 (2008), http://gamestudies.org/0801/articles/hutch.
234
screen. Of course, the idea of composing an image is familiar; after all photography and cinema are a construct.4 However, Farocki indicates a different kind of creating. He asserts, “I’m just researching these strange new images which are somehow on the verge of competing with and defeating finally the cinematographic photographic image, the era of reproduction seems to be over more or less, and the era of construction of a new world seems to be somehow on the horizon or not on the horizon, it is already here.”5 For Thomas Elsaesser, Parallel I-IV is situated in the cinematic enquiry, and specifically questions the visuality created by methods of digital post-production.6 If this is the case, then arguably the camera has been replaced by the computer, or specifically the lens of the camera has been replaced by the on-screen computer camera. This is not to say that we don’t use cameras to record images — the relationship between images and camera of course still exists – but now more than ever, the adjunct relationship between on-screen production and images takes over and becomes more important and in some cases replaces the lens-based camera altogether. Taking this shift into account, I will problematise the relationship between screens and images, which I believe reveals aspects of the postmedial condition.
In this chapter, I am interested in a mode of encountering screens where they are without images. This mode offers an analysis of screens in conditions where elements, components and functionalities long associated with screens — in this case, images — are neither immediately present nor influencing a screen’s performance. Such a discussion evokes the question of whether there
4 Construct here refers to the fabrication of sets, acting, and posing in addition to building a story and narrative through camerawork and editing. 5 “Harun Farocki Interview, Galerie Thaddeus Ropac, Paris Maris, 2014,” YouTube, Jan 22, 2014, https://www.youtube.com/watch?v=c8YKKs0Pcx8. 6 Thomas Elsaesser, “Simulation and the Labour of Invisibility: Harun Farocki’s Life Manuals,” Animation 12, no. 3 (2017): 217, 227.
235
can be deliberations about medial screens that are without images. I am particularly concerned here with enactments of screens in postmedia assemblages where screens are without images. The phrase ‘without images’ in this context does not indicate a complete absence of images, but highlights instances where they do not dominate screen space. Images are not only invisible when they are not seen; they can be invisible when they are inactive, inoperative or non-functioning. This chapter closely analyses material- discursive practices of image-making, display and non-display that intra- actively constitute screens in specific conditions. At the same time, the discussions also initiate a broadening of the understanding of imaging by illustrating how it emerges in the intra-actions with and through screens and other parts of the postmedia assemblage. Images and screens are not disconnected, nor are they opposed to each other; their relationship and intra- actions are explicated and unfold through a postmedia assemblage.
The technique of chroma keying, increasingly used for a range of production settings in contemporary image cultures, illustrates a unique relationship between images and screens where images wait to be called on to screens. The chroma screen is the collective name given to the blue or green screens used in filmmaking, television and graphics during image capture for later compositing during post-production. In chroma keying, objects shot in front of the blue or green screens are separated from their backgrounds, so as to be superimposed on to another background. In the instances where chroma screens are recorded without images, empty apart from a blue or green colour field, the gestures of screens in these assemblages will be explored. In the entanglement of screens and images, if the images are delayed or are absent, then questions around how screens sustain relations and what their medial performances are become pertinent to this research. By considering chroma screens in and through material-discursive enactments, I demonstrate that the
236
green and blue colours of chroma screens can be considered as a cultural enactment of racial practices. Hence, the questions of what and how a gesture is performed by screens in the production of the contemporary image can now also be understood as not only medial or aesthetic or durational but also political.
I will begin with the image and screen framework in order to illustrate their entanglement, proposing the conditions where this entanglement is suspended. By introducing chroma screens, I analyse two works where chroma screens are deployed without images. Through Hsu Chia-Wei’s video installation Marshal Tie Jia - Turtle Island (2012), I analyse the intra-action of political and marginal histories in relation to green screens. This is not just an argument about the construction of the historical and the present moment; the intra-action indicates a making and unmaking of each temporality, a co- constitution where these temporalities become inseparable, and where one can be viewed through the other. I also discuss the use of chroma screens in Hito Steyerl’s How Not to be Seen: A Fucking Didactic Educational .MOV File (2013). Steyerl explores the politics of the visibility of images and proposes that chroma screens can be a way to hide from over-visibility by becoming a part of the image. Here, images are not in the foreground and nor are chroma screens in the background, rather they get mixed up and become inseparable. These performances of chroma screens also illuminate multiple dimensions of the relationships of contemporary screens with images. Finally, I take the image blankness of chroma screens as the condition for a performance of blackness in Sondra Perry’s work, Graft and Ash for a Three Monitor Workstation (2016) and in Sandra Mujinga’s Throwing Voice (2016). This opens up how racial practices and concepts have functioned in the blan/ckness of chroma screens — an entanglement of blankness with blackness — which, in Perry’s work particularly, I contend, becomes generative of a differencing.
237
IMAGE-SCREEN ENTANGLEMENT Before I analyse the enactment of and by screens in the absence of images, it is important to consider the entanglement of images and screens in order to throw some light on the emergence of both in and through certain practices. In chapter two I discussed how the entanglement of images, screens and the human body makes it difficult to ascertain the gesturing body. Here, however, I am concentrating on the entanglement of screens and images in specific instances where images are withheld from direct display on the chroma screen. Moreover, the discussion of entanglement in chapter one and two was framed in the immediacy of the gesturing body, while here the entanglement of images and screens is created by processes of delay and suspension. Of course, screens are entangled with several other things in the postmedia assemblage, but I concentrate here on one of their most important relationships – that with images.
The techniques that develop around the use of chroma screens highlight the manufacture of images as and for screens, thereby disclosing their entangled relationship differently. In Hsu Chia-Wei’s Marshal Tie Jia - Turtle Island (2012), a Min opera singer is singing in the temple of the Chinese frog deity, Marshal Tie Jia, on Matsu Islands in the East China Sea (fig. 45). This signals the location of a temple that was dismantled in the late 1940s and relocated to a neighbouring larger island. The Min opera is a favourite pastime of Marshal Tie Jia and is on the verge of becoming obsolete. As the singing proceeds, in one shot the crew and the parts of the film set are suddenly seen with the inclusion of a large green chroma screen. The viewer realises that the temple does not exist, but that the image of the temple has been ‘placed’ on the green screen (fig. 46). This cinematic technique, hors- champ, of revealing the off-frame entities that are otherwise hidden, is used powerfully to shatter the illusion of the temple as place for the viewer.
238
Processes and elements that are usually hidden, in other words are off- camera, become the profilmic, the part of the film that constitutes the film image. This technique is used by Hsu to reveal the set and break the idea of the illusion of its direct indexicality. Perhaps, here, the sharpest indication of the manufactured image is offered by the green chroma screen. In this particular kind of screen, the images are composed and compiled later in post-production using other screens (a process now mainly performed digitally). Moreover, with the images on the green screen, the reconstruction of the temple parallels the fabrication of the story in the video.
Figure has been removed due to copyright restrictions.
Figure 45: Hsu Chia-Wei, Marshal Tie Jia - Turtle Island, 2012, one-channel HD video, 6 mins 35 secs. Digital video still.
239
Figure has been removed due to copyright restrictions.
Figure 46: Hsu Chia-Wei, Marshal Tie Jia - Turtle Island, 2012, one-channel HD video, 6 mins 35 secs. Digital video still.
The intra-action of images and screens in the postmedia assemblage indicates that their agencies are co-constituted. As discussed in chapter one, agential intra-actions are stabilised in and through material-discursive practices. It is in and through these practices that objects, things and categories are enacted. The boundaries between images and screens as separate entities do not precede each other but emerge in and through specific material-discursive practices. Under such considerations, screen and image practices are important because a specific materialisation is produced enacting a media assemblage that enables its gathering in a particular way. For example, practices coming out of computer generated imagery (CGI), such as motion capture, overlay the data derived from actors with animation and graphics. Actors are filmed against single colour backgrounds of green or blue screens, as also seen in Marshal Tie Jia, to isolate their movement so that the data captured is more accurate. The digital material is distinct from the images simply recorded against a background. Indeed, the background disappears — suspended — as all becomes data and new images are inserted to become a
240
new background for the actors’ movement. The images and screens are separated in the recording and post-production work.
The relations between screens and images do not pre-exist but are instead enacted through chroma keying and other practices. At the same time, by using a concept of entanglement we can identify both screens and images as emergent via certain relations. By looking to specific practices we can register the becoming of screens, their screenness (the enactment of specific relations in a postmedia assemblage) and their connections with images and other entities in an assemblage. These questions become imperative particularly around conditions where the relationship between images and screens is challenged or suspended. To accomplish such an understanding of screens, I have proposed to closely analyse practices associated with chroma keying.
The performance of the image-screen relationship that can be observed in the practice of chroma keying is divergent from their relationship during display in film screenings. To explain the process with a few more details: chroma keying is a production technique where a single colour, commonly blue or green, is used as the background for a subject that is being recorded (film, video) or captured (photography). The background colour field of blue or green is then superimposed with other images through the process of keying and compositing.7 In the process of keying or keying out, whether using a hardware or software compositor or chroma keyer, the colour value of blue or green used in the background is made transparent.8 This keyed layer can be composited, that is, other visual elements can be added where the background colour once was (this technique is also commonly referred to as green screen
7 Jeff Foster, The Green Screen Handbook, Real-World Production Techniques (New York: Routledge, 2010), 23. 8 For a detailed technical explanation on different kind of mattes and compositors, see Foster, The Green Screen Handbook, 23–41; “Keying Introduction and Resources,” Adobe User Guide, accessed 2 February 2021, https://helpx.adobe.com/au/after-effects/using/keying.html.
241
or blue screen). In chroma keying the screens do not merely contain, display or project images as we see in most of the screens around us. Instead, the chroma screens are filled with images later; the recording of the screens, during filming or photographing happens without the images.9 Screens and images are connected as seen in display, projection and image-making tasks. There is a kind of delayed gratification, as chroma screens are a placeholder for images that are absent during the recording process. Moreover, during the post-production stage, they carry images that are created on other screens, such as the computer screen.
Practices utilising chroma keying methods perform distinctions and connections between screens and images, boundaries, and properties in numerous ways. In the case of cinema, the actors engage with the screens while imagining the images that will appear on them, and their actions impact the movement of the images that have not yet happened.10 In TV weather reports, the composited images appear almost immediately on chroma screens. During broadcasts and pre-recordings, the composited images appears to be behind the presenter, although they are performing in front of a blank chroma screen. This registers an immediacy of relation across screens and images. Typically, the weather presenter has a small monitor showing them the composited weather graphics that they now appear to be in front of. This helps in modulating their actions while they perform, and at the same time effecting change to images. In this way, chroma screens reveal images, screens and their relations, as well as entangle them. Entanglement in some
9 In the video work I don’t exist yet (2020) Susan Flock focusses on the ‘placeholder’ aspect of green screens. While the green screen objects in most films are spaces to be filled with computer generated imagery (CGI), in Flock’s work, a chroma green blob is the protagonist, and the video is narrated from its point of view through subtitles. 10 Certainly, acting is part of the film. Traditional filmmaking is a construction of a story and incidents in a narrative format. But there is a difference in setting up a performance without the profilmic, that is, creating a fictional world that is not in front of the camera but created in post-production. Editing is also a post- production process, but that is focussed on the flow of images and narrative structure of the work, not image production per se.
242
instances is enacted immediately, for example, in weather reports as just discussed. In other cases, the entanglement is not immediately seen, such as when chroma screens are used for complex compositing, as during recording the later images may not be evident. However, in both instances the possibility of recompositing the images is always there. Thus, while the screens and images are differentiated in the practice of compositing images, the relationship between chroma screens and images is not fixed to a particular set of images.
Chroma screens reveal another aspect of the relationship between screens and images. In the dissociation of images from screens, green/blue screens are recorded just as they are, plain and blank — screens without images, screens that wait for images, thus delaying them. This delay is actually not a deferral of relationship, but a new kind of connection that operates via gesturality. Screens, as I argued in chapter two, reveal the conditions and new configurations of mediality. As gestures, chroma screens become potential elements, existing in duration for the images that are to come; supporting them as their necessary conditioning elements when they do appear.11 Before images take (their) position on chroma screens, the gesturality of such screens is demonstrated in the ongoingness of the active waiting to become a place to ‘see’.
It is important to discuss the gesturalities of chroma screens here; that is, the relations the screens sustain while holding off images. Screens, as I have been arguing, are enacted in and through the relations of the postmedia assemblage. But if the entanglements of screens and images are not enacted immediately (as is the case with chroma screens) then what does this
11 Agamben argues that gestures do not arrive at a goal but “endure and support” the action. Agamben, “Notes on Gesture,” 56.
243
blankness of screens indicate? If images are suspended, then what is otherwise revealed? To investigate these questions, I propose to closely examine artists who foreground the emptiness of chroma screens in order to reveal affiliations and associations beyond that of images. In the process of this exposition, the starkness of the blue/green colour field gestures to what Agamben calls “pure and endless mediality.”12 But I also want to go a step further in this chapter. Such gestures cannot simply be understood within a media formalism. We will also see that the question of a screenic gesture made in colour – blue and green – will have social and cultural as well as aesthetic implications.
Taking another look at Marshal Tie Jia - Turtle Island discloses the political and marginal histories performed in and through its green chroma screens. The temple was initially erected on a tiny island in the Matsu Islands off the coast of Taiwan, before being relocated to a larger island nearby. In place of the temple, a bunker was constructed as Chiang Kai-Shek retreated to Taiwan from mainland China in the late 1940s. Moreover, the frog deity, Marshal Tie Jia, had been mythologically relocated to the islands from Wu-Yi mountain in mainland China, after the original temple was destroyed during the Cultural Revolution.13 The song in the video is taken from Min opera, a dying art. Marshal Tie Jia - Turtle Island, then collapses complex histories and stories and entangles them with the geopolitics of the region. The green screen in the video becomes a space for the intra-action of histories, factual and mythical. As the chroma screen composites the images of an imagined temple, it carries a performance of history as well. The chroma screen gestures towards the ongoingness of history and politics — examining both how and what is
12 Ibid., 53. 13 As explained by Hsu Chia-Wei on his website: “Marshal Tie Jia - Turtle Island,” Hsu Chia-Wei website, accessed 2 February 2021, http://hsuchiawei01.blogspot.com/search/label/Marshal%20Tie%20Jia%20- %20Turtle%20Island.
244
remembered. Here, composition entails the process of creation through histories and mythologies rather than just the image being composited on the chroma screen. We get a sense, through this work and the way it deploys a blank green screen, of a making of the present along with the past and how they come to be co-composed. Since the political is enfolded in the cultural, the truth remains historically ambiguous.14 Remembering, here, is not just what is spoken of and written but is also embedded in the geography.15 In Hsu’s work the chroma screens undertake technical and cultural work, gesturing towards relations outside the pure formalism of media assemblages.
Chroma screens can also become a technique of invisibility, where the images are not delayed but hide instead in the colour field, becoming part of an overall blankness. The images are then enacted in and through the blankness of chroma screens. Hito Steyerl’s How Not to be Seen: A Fucking Didactic Educational .MOV File (2013) explores various ways of hiding from the camera in plain sight. The video instructs viewers on how to escape being seen in a hyper-visible, visual set-up, optimised for surveillance and encouraging the creation and sharing of images. It details procedures that can be performed on and with the images to aid invisibility, such as camouflaging, and becoming smaller than a pixel, even while the screen supposedly functions to inflate visibility. The video is divided into five parts, each featuring a specific method: “make something invisible for the camera,” “be invisible in plain sight,” “become invisible by becoming a picture,” “be invisible by disappearing,” and “become invisible by merging into a world made of pictures.”16 Steyerl counteracts modes of visibility executed via the ubiquity of
14 Hsu, in Marcus Yee, “Makeshift Histories: Hsu Chia-Wei,” Asia Art Pacific 109, July/Aug 2018, http://artasiapacific.com/Magazine/109/MakeshiftHistories. 15 This is also realised in China’s fierce assertion of its territorial land and sea borders in recent years. This has arguably led to increased resistance by countries such as Taiwan, and their desire for independence from China. 16 Hito Steyerl, How Not to be Seen: A Fucking Didactic Educational .MOV File (2013).
245
postmedial screens by exploiting ways in which screens simultaneously conceal images and bodies. However, some humans, objects, processes, and events are invisible anyway. They are just not ‘seen’ or accounted. To put it another way, screens materialise images by hiding them.
Figure has been removed due to copyright restrictions.
Figure 47: Hito Steyerl, How Not to be Seen: A Fucking Didactic Educational .MOV File, 2013, one-channel HD video, 14 mins. Screenshot from digital video.
The blankness of chroma screens is performed differently in How Not to be Seen. More akin to an absorption of the images, the separation between images and screens is difficult to perform. Unlike the chroma screens of cinema and television that are used as containers, Steyerl exposes conditions where chroma screens hide people and objects by merging them with images. In part three of the video, the narrator tells us that some of the ways of disappearing are “camouflaging,” “concealing” and “cloaking.” The narrator lists them over visuals of Steyerl applying chroma green paint to her face. These coloured patches are then overlayed with other visuals or show the background itself (fig. 47). Steyerl’s face seems transparent, available for the ‘screening’ of images, and in the process disappearing as images are seen on and through the patches on her face. A few other ways to hide, states the
246
narrator, are to stay in “gated communities,” to be “fitted with an invisibility cloak,” to be “spam caught by filter” or by becoming a “disappearing person as the enemy of the state.” These are heard over visuals of an architectural animation walkthrough of a residential complex and shopping area. Noticeably, the phrase “being fitted with an invisibility cloak” overlaps with visuals of transparent people putting on cloaks that look very similar to burkas, the body covering garment for Muslim women (fig. 48). It is a covering that acts as a veil screening the body from attention.
Figure has been removed due to copyright restrictions.
Figure 48: Hito Steyerl, How Not to be Seen: A Fucking Didactic Educational .MOV File, 2013, one-channel HD video, 14 mins. Screenshot from digital video.
People in burkas, particularly in green chroma screens, are recurring figures throughout the work. This is an attempt to grasp the politics of who and how things are seen. Do we ‘screen’ things from our vision on purpose by making them background? Is one not ‘seen’ or is overlooked because we cannot identify them — that their facelessness conceals them in the burka — or does it make them conspicuous in a Western society? The tactics of visibility in the work also asks questions around our understanding of images through biases that are literally projected. The screens are spaces that hide and show, flattening spaces by hiding things and opening them up by adding more
247
images to them. On the other hand, if the conditions change then images can be seen and separated from chroma screens. For example, by changing the contrast levels, the body that tries to camouflage itself becomes visible once again. Hence, screens can become the agencies of observation, or the apparatus that determines certain conditions for the visibility of images.17
The intra-actions of screens and other entities in a postmedia assemblage execute differently in the practices of Hsu and Steyerl. This takes away the focus from the image-screen entanglements to other becomings of screens. However, the blankness of chroma screens in both the works does not highlight blankness as productive at the level of the chroma or colour itself. Also, why is there a general preference towards using green and blue in chroma screens? The idea of blankness along with the colour field is taken up by artist Sondra Perry in a compelling way. Blue and green screens are set-up not to contain or restrain blankness, but to perform and bring it in to existence through a gesturing. I will argue that the gesturality of screens sets up the potential for the colour (of the screens) to intra-act with the blackness of bodies in a certain way. In and through the cultural, material and discursive practices, I will demonstrate that the blankness of chroma screens in Perry’s work performs blackness. The material agency of the colour exclusions chroma screens enable in relation to cultures and discourses of race and racism produce and are produced by colonial and racial identities. The colour of chroma screens is at once an enactment of the colour of certain human skin tones and racism. To illustrate this argument, I will discuss the technical reasons and history of the use of blue and green colours in chroma screens and their relation to a history of racialised photographic and cinematographic practices. Admittedly, racism in media technologies is an
17 To be clear, here I refer to Karen Barad’s agential realist understanding of agencies of observation discussed in chapter one.
248
extensive sub-field, and my analysis will build on existing scholarship that highlights how racism at the level of visual media technologies is enacted. In doing so I want to make some specific points on the inability of material- discursive media practices to recognise black bodies. This will be diffracted through colonial and racial discourses, which provide a framework for analysing how the blan/ckness of chroma screens operates.
Figure has been removed due to copyright restrictions.
Figure 49: Sondra Perry, installation view of exhibition flesh out at Squeaky Wheel Film and Media Art Centre, Buffalo, New York, on view January 20 - May 6, 2017.
Sondra Perry makes installations, videos and performances that highlight their digitality. A significant aspect of Perry’s practice is her use of digital media to focus on race politics, especially in the American context. Chroma screens are important elements in her work, as seen in the solo exhibitions Resident Evil (2016) at The Kitchen, New York, and flesh out (2017) at the Squeaky Wheel Film and Media Art Centre, Buffalo (fig. 49) as well as in the installations of her workstation series. In the workstation series, Perry uses modified exercise machines with multiple monitors attached to them. The exercise machines are modified in ways that make them difficult to operate, such as by adding excessive weight or minimising friction. One work, Wet and
249
Wavy Looks—Typhoon coming on for a Three-Monitor Workstation (2016) uses a rowing machine filled with gel instead of water, giving it greater resistance and therefore being harder to use. I, however, will be discussing Graft and Ash for a Three Monitor Workstation (2016), a modified bicycle exercise machine with three screens (fig. 50).
Figure has been removed due to copyright restrictions.
Figure 50: Sondra Perry, Graft and Ash for a Three Monitor Workstation, 2016, video bicycle workstation, 9:05 mins. Installation view, The Kitchen New York, 2016.
The triptych nine-minute video in the work shows Perry’s animated head floating in front of a chroma blue backdrop intercut with YouTube clips on deliverance ministries, as well as a highly modified closeup of Perry’s skin (fig. 51). Her processed skin displays differently coloured patches, ranging from dark brown to light reddish brown; it is textured and seems to flow like a river.18 Perry describes this “flesh wall” as blackness which for her forms a
18 The processed skin is used by Perry in at least one other work, TK (Suspicious Glorious Absence) (2018). It is worth mentioning that these flowing movements of skin can be connected to Perry’s underlying interest in the Middle Passage and racial capitalism. In another work, Typhoon Coming On (2018), Perry uses a purple flowing river along with a digital manipulation of J.M.W. Turner’s The Slave Ship (1840), a painting depicting the Zong massacre of 1781, where the captain and crew of the British slave ship Zong threw up to 150 enslaved people overboard in order to obtain insurance compensation. See Arabella Stanger, “Bodily
250
transition space between human and a digital creature, and the labouring body and the machine.19
Figure has been removed due to copyright restrictions.
Figure 51: Sondra Perry, Graft and Ash for a Three Monitor Workstation, 2016, video bicycle workstation, 9:05 mins. Screenshot from digital video.
Perry’s digital avatar provides technical details about the exercise machine. It also states how the exercise machines keep the working body fit. The productive and labouring black body is called to our attention in many different ways here. Firstly, the avatar cites a study that shows high blood pressure ailments are common in black people who have sought and upheld equitable social relations but nonetheless faced discrimination. Ignorant of systemic inequality, these people blamed themselves, believing they were not good enough. Secondly, the black body is addressed in and through audio recordings of deliverance ministries, who hold that discrimination is the result of demons producing problems in people’s lives. In the video a pastor directs a question at the evil spirit possessing a woman, asking, “Who are you?” The
Wreckage, Economic Salvage and the Middle Passage,” Performance Research 24, no. 5 (2019): 11-20, https://doi.org//10.1080/13528165.2019.1671710. 19 “In Conversation: Sondra Perry & Hans Ulrich Obrist,” 15 December 2018, Luma Westbau, Zurich, accessed 2 February 2021, https://vimeo.com/307269119.
251
woman replies, “I’m a man like you…what is the problem?”20 Following another exchange, a kind of exorcism takes place, with screaming and shouting both from the pastor and the woman. As the demons are expelled and the woman is delivered from evil, the implication is that the black body might also be delivered from an unjust system. Thirdly, the black body is enunciated in its relationship with technology. Referring to itself as an inaccurate version of Perry, the avatar claims, “she [Perry] could not replicate her fatness in the software that was used to make us. Sondra’s body type was not an accessible pre-existing template.” This points to the existence of certain body types, largely thin, white women, who are desirable and most commonly marketed and available via software templates.21 What is raised here are not just issues of body types but what kinds of filters and predetermined templates already inhabit technologies and generate racism.
Figure has been removed due to copyright restrictions.
Figure 52: Sondra Perry, Graft and Ash for a Three Monitor Workstation, 2016, video bicycle workstation, 9:05 mins. Screenshot from digital video.
20 The part of the video component of Perry’s Graft and Ash for a Three Monitor Workstation, 2016. 21 Adwoa A. Afful and Rose Ricciardelli, “Shaping the Online Fat Acceptance Movement: Talking about Body Image and Beauty Standards,” Journal of Gender Studies 24, no. 4 (2015): 453-472, https://doi.org//10.1080/09589236.2015.1028523; Jane Arthurs, “Sex and the City and Consumer Culture: Remediating Postfeminist Drama,” Feminist Media Studies 3, no. 1 (2003): 83–98.
252
Only the audio of the deliverance ministry is heard as we watch a lattice frame with a ball of Perry’s skin rotating on top (fig. 52). The black human body is referenced by the skin rotating on top of the grid but there is a more powerful visual enactment of colour in its contrast with the blue chroma screen. Here the blue chroma screen is not set up to act as an unbiased or receptive field waiting blankly for images but is acknowledged to be a space entangled with racialised media and cinematographic and post-production practices. Perry’s chroma screen calls up the identity politics played out in contemporary America and questions the specific acts of violence against the black community by compositing a false narrative (individualised responsibility for racism, religious preaching and so on) similar to compositing images on chroma screens. Furthermore, Perry’s use of chroma screens is an attempt to think about practices of post-producing images. The blankness of chroma screens offers the possibilities that images can be produced in a space regardless of the space of production itself. However, that possibility has been restricted to certain type of images, primarily white American and Europeans, and the exclusion of coloured bodies. Typical methods used in filmmaking for lighting for black and brown skins in film and television productions continue to be problematic, as these bodies are either too dark or over-exposed by the implicit lighting and filmic models of the medial production space.22 The racialisation of technology amplifies through specific practices of filmmaking, one of which is chroma keying. When thought through in terms of blackness, these practices draw our
22 Only recently have filmmakers and camerapersons have started paying attention to appropriate lighting for black and brown actors. See Nadia Latif, “It’s Lit! How Film Finally Learned to Light Black Skin,” The Guardian, September 22, 2017, https://www.theguardian.com/film/2017/sep/21/its-lit-how-film-finally- learned-how-to-light-black-skin. Perry refers to the practice of warming up black and brown skin tones to make them more “palatable” for filming. See “Sondra Perry: Typhoon Coming On,” Serpentine Gallery, London, 2018, YouTube, accessed 2 February 2021, www.youtube.com/watch?v=Qunkb4piXGw.
253
attention to how black and brown bodies have been photographed in film and video.
The relation between blackness and the blue chroma screens that Perry uses also implies the agility of blackness, a movement and transformation where new imaginings can also take place. Like the blankness of chroma screens, blackness too can be reimagined. The agility of chroma screens lies in their post-productive capacities, where they have the potential to become any kind of space. Perry compares the live-ness and energy of chroma screens to that of black bodies. She argues that the agility of blackness makes it shift and move to create spaces where something different can occur.23 A comparison to the shapeshifting black body, a preoccupation of an Afro-futurist imaginary, can be made with the blue-ness of the chroma screen in Graft and Ash for a Three Monitor Workstation. Soyoung Yoon argues that the chroma blue here is a reference to space travel by being “an effective placeholder for the black void of outer space.”24
To be sure, these connections and conversations have to be examined as intra-actions where both chroma screens and racial practices are being reconfigured. The non-fixity of chroma screens is what makes them responsive. The gesturalities of chroma screens constitute the continuance of this openness to relations. Gesturalities do not merely signal movements in the assemblage, but also their continuance, as discussed in chapter two and three. In the video, Perry’s avatar and the deliverance ministries are placed on top of the blue chroma. The blue chroma screens intra-act with images of the deliverance that show the ‘possession’ of the black bodies with continuing
23 “Sondra Perry: Typhoon Coming On”; Hans Ulrich Obrist and Yana Peel, “Foreword,” in Sondra Perry, Typhoon Coming On, ed. Amira Gad, exhibition catalogue, Serpentine Galleries, London, March–May (London: Serpentine Galleries and Koenig Books, 2018), 15–17. 24 Soyoung Yoon, “Beware the Light: Figure versus Ground, White versus Black (Blue), or: Sondra Perry’s Blue Room and Technologies of Race,” Millennium Film Journal 65 (2017): 32.
254
discrimination of black bodies in technologies and filming practices. In discussing these intra-actions, I will start to explicate how the technological disposition against black bodies stems from photography and filming practices. Later, I will discuss that these are hooked with the difference of blackness to (European) whiteness. Blackness is then co-constituted in and through such material-discursive practices.
The next section briefly illustrates two ways in which technology and race have been diffracted. One is the enactment of racial bias through digital technologies. This includes technological objects such as colour film and more recent developments of racially biased classifications by artificial intelligence. The second extends the first to apprehend the agency of race by interrogating such systemic practices. This will allow us to build a framework for the emptiness and invisibility in which black bodies exist and how their mattering is performed via chroma screens.
Racial bias in technologies has been widely discussed in media and film studies scholarship, especially in the failure of a range of technology to recognise bodies of colour. The famous Shirley cards, introduced by Kodak in the mid 1950s, ensured that a white woman was the norm for adjusting colour balance in photography.25 The bias against using brown and black skin tones as the touchstone for colour balance in film stock was continued in the colour
25 As Lorna Roth writes, complaints by furniture and chocolate companies encouraged Kodak to be more sensitive to darker subjects. See Lorna Roth, “Looking at Shirley, the Ultimate Norm: Colour Balance, Image Technologies, and Cognitive Equity,” Canadian Journal of Communication 34, no. 1 (2009): 111–136. The racial bias of various technologies has also been accounted for in David Hankerson, Andrea R. Marshall, Jennifer Booker, Houda El Mimouni, Imani Walker and Jennifer A. Rode, “Does Technology Have Race?,” in Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (San Jose, USA 2016), https://doi.org//10.1145/2851581.2892578. Other practices, such as make-up, also contribute to racial bias. For a historical reading of these factors in film see, Richard Dyer, White (London, New York: Routledge, 2002); Cathy Greenhalgh, “Skin Tone and Faces: Cinematography Pedagogy which Foregrounds Inclusivity and Diversity in Teaching Lighting,” Cinematography in Progress, 2 July 2020, https://cinematographyinprogress.com/index.php/cito/article/view/45.
255
balancing techniques of digital cameras. While there are dual skin tone colour balancing systems, artificial light sources still make it difficult to photograph darker skin tones.26 If this systematic racism seemed disputable, then the bias in artificial intelligence and algorithms has certainly detected racial bias.27 Kate Crawford argues that artificial intelligence has a “white guy problem,” illustrating how algorithms perpetuates racism, such as Google’s classification of black people’s images as gorillas in 2015.28 Julia Agwin et al. explain the biases against black people in the software used across America to predict criminals.29 Similarly, Simone Browne highlights how racialised surveillance practices and policies have targeted and discriminated against particular groups of people.30
This disposition of technology that only allows certain bodies to become visible is termed “algorithmic visibility” by Daniela Agostinho.31 Analysing the visibility and invisibility of algorithmic practices in Perry’s work, Agostinho comments that for a black person’s body “the flesh is both bearer of subjection and source (code) of freedom.”32 She also finds that Perry uses chroma screens as a space where the requirements and aspects of the visibility
26 Sarah Lewis, “The Racial Bias Built into Photography,” New York Times, April 15, 2019, accessed 2 February 2021, https://www.nytimes.com/2019/04/25/lens/sarah-lewis-racial-bias-photography.html. 27 Clemens Apprich, Wendy Hui Kyong Chun, Florian Cramer and Hito Steyerl, Pattern Discrimination (Lüneburg: Meson Press, 2018); Joy Buolamwini, “Gender Shades: Intersectional Phenotypic and Demographic Evaluation of Face Datasets and Gender Classifiers” (Master diss., Massachusetts Institute of Technology,2017), https://dam-prod.media.mit.edu/x/2018/02/05/buolamwini-ms-17_WtMjoGY.pdf. 28 Kate Crawford, “A.I.’s White Guy Problem.” New York Times, June 25, 2016, accessed July 30, 2016. https://www.nytimes.com/2016/06/26/opinion/sunday/artificial-intelligences-white-guy-problem.html. 29 Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner, “Machine Bias: There’s Software Used Across the Country to Predict Future Criminals. And It’s Biased against Blacks,” ProPublica, May 23, 2016, accessed January 19, 2017, https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal- sentencing. 30 Browne shows that contemporary surveillance strategies are born out of slave surveillance practices. Simone Browne, Black Matters: On the Surveillance of Blackness (Durham, NC: Duke University Press, 2015), 16–17, 50. 31 Daniela Agustinho, “Chroma Key Dreams: Algorithmic Visibility, Fleshy Images and Scenes of Recognition,” Philosophy of Photography 9, no. 2 (2018): 141, https://doi.org//10.1386/pop.9.2.131_1. 32 Ibid., 148.
256
of race are re-evaluated.33 I agree that chroma screens provide a space where relations and conditions of visibility can be renegotiated. This is affirmed in how the blue chroma screens in Perry’s work direct us in thinking about ways of making race visible for surveillance and invisible for civil rights. However, I also contend that the chroma screens are not just spaces that mobilise and highlight racial injustices, but they also shift our attention to image-making practices that have sustained a racialised separation. Moreover, the blankness of chroma screens gesture blackness, sustaining it as a productive difference.
The idea of race as technology is addressed by both Wendy Hui Kyong Chun and Beth Coleman, where more agency is granted to how race is produced in and through techniques of framing, classifying and so on. Race has never been only cultural or biological, but it is always technological. This, Chun argues, has been evident in the practices of eugenics and segregation.34 Chun invokes Foucault’s proposition of race as a function of biopower: “Segregation and eugenics are thus examples of what Foucault has called modern racism, a racism fostered to allow states, which are supposedly dedicated to the social welfare of their populations, to exercise sovereign power— that is, to punish and destroy.”35
For Coleman, race as technology is no longer just an ideological category but also an aesthetic one, where “mutability of identity, reach of individual agency, and conditions of culture all influence each other.”36 Race is a technique, an instrument, according to Coleman, that can be used not only in
33 Ibid., 144. 34 Wendy Chun, “Introduction: Race and/as Technology; or, How to Do Things to Race,” Camera Obscura 24, no. 1 (70) (2009): 15. 35 Ibid., 15, 16. Kyla Schuller’s The Biopolitics of Feeling: Race, Sex, and Science in the Nineteenth Century (Durham, NC: Duke University Press, 2017) is a compelling study of the role of sentimental discourse (‘evoking’ feelings) in forging biopolitical power. Also see Race after the Internet, ed. Lisa Nakamura and Peter A. Chow- White (London; New York: Routledge, 2011). 36 Beth Coleman, “Race as Technology,” Camera Obscura 24, no. 1 (70) (2009): 180.
257
a dehumanised way but towards an affective understanding.37 Coleman’s separation of the agency of race from the agent (who she proposes should be “mobile”)38 is debatable, since they are not separate but co-emerge in their practices. Through dynamic intra-actions, agency is not a possession but instead an enactment. Race, then, is not an agentic property of human beings; rather, as I have argued throughout this thesis, it is produced by and produces humans in a problematic way.
Since agency is always open to intra-actions/changes, it is important to be accountable to the practices that constitute and produce particular bodies including racialised techniques and practices.39 Chun argues that race as technology “problematizes the usual modes of visualization and revelation, while at the same time making possible new modes of agency and causality.”40 This re-formulation of race is an important point to consider. Chun concludes,
Importantly, it [race as technology] displaces ontological questions of race — debates over what race really is and is not, focused on discerning the difference between ideology and truth — with ethical ones: what relations does race set up? The formulation of race as technology also opens up the possibility that, although the idea and the experience of race has been used for racist ends, the best way to fight racism might not be to deny the existence of race but to make race do different things.41
37 Ibid., 199. 38 Ibid., 200. 39 This is because, following agential realism as discussed in chapter one, agency is an intra-action of matter. Intra-acting matter produce particular bodies, objects and subjects in and through material-discursive practices. 40 Chun, “Introduction,” 28. 41 Ibid.
258
Taking a lead on the relations set-up by race, I am proposing that chroma screens perform these relations, and in specific instances they can be argued to set-up, arrange and configure relations between race and technology. But I want to take a step back to establish the tendencies of race in and through assemblages. Of course, this raises questions about the production of race and what it does. While these topics are significant, they are outside of the scope of this thesis. I will, however, briefly touch upon a few ideas in order to explicate an assemblagic understanding of race.
With the emergence of new materialism as an active field, the materiality of race as a category has also gathered interest. In pursuing a materialist ontology of race, Arun Saldanha considers race as the “irreducible and immanent heterogeneities” in a machinic assemblage.42 Thinking about race in ways informed by ideas of Deleuze and Guattari, he considers race in terms of bodies and their connections with things and places.43 Such interconnections do not determine race’s functions but provide it with capacities to connect further and evolve.44 Racial differences develop as bodies are arranged in particular ways.45 Saldanha considers the generative affordances of difference by “investigating racial difference in itself as it persists as a biocultural, biopolitical force amid other forces.”46 Race as assemblages of material, discourse and practice has also been discussed by Donald S. Moore, Jake Kosek and Anand Pandian.47 While I contend that
42 Arun Saldanha, “Reontologising Race: The Machinic Geography of Phenotype,” Environment and Planning D: Society and Space 24, no. 1 (2006): 19, https://doi.org/10.1068/d61j; Also see Arun Saldanha, Psychedelic White: Goa Trance and the Viscosity of Race (Minneapolis: University of Minnesota Press, 2007), 9. 43 Saldanha, “Reontologising Race,” 9–24. 44 Ibid., 19. 45 Saldanha, Psychedelic White, ix. 46 Arun Saldanha, “Bastard and Mixed-Blood are the Two Names of Race” in Deleuze and Race, ed. Arun Saldanha and Jason Michael Adams (Edinburgh: Edinburgh University Press, 2012), 8. For further discussion see Neel Ahuja, “Post-mortem on Race and Control,” in Control Culture: Foucault and Deleuze after Discipline, ed. Frida Beckman (Edinburgh: Edinburgh University Press, 2018), 40. 47 Donald S. Moore, Jake Kosek and Anand Pandian, “The Cultural Politics of Race and Nature: Terrains of Power and Practice”, in Race, Nature and the Politics of Difference, ed. Donald S. Moore, Jake Kosek and Anand Pandian (Durham, NC: Duke University Press, 2003), 1–70.
259
entanglements are not convincingly demonstrated by Saldanah, it is more important for this thesis to consider such propositions of the mattering of race in the postmedial context. Saldanha categorically strives for an ontological investigation, that is, the connections which provide a particular meaning to race. However, in this thesis, the connections and the conditions in which things become or are known are important. In other words, if race is open to connections, changes and modifications then we are able to also ask: what are the practices that materialise these intra-actions? In this chapter, in particular, we can inquire into how performances of blackness in and through colour (especially the colour of chroma screens) produce race through differencing.
My research attempts to move away from any preoccupation with the representation of blackness and race, which tends to dominate discussions around bias. I approach these instead as enactments that are explicitly questioned, as they are in Sondra Perry’s work. Referring to the machines of black bodies as an assemblage similar to the exercise machine in the installation, Perry’s avatar in Ash and Graft says, “We are a DIY, not all that representative thing which makes being a ‘being’ impossible or whatever.” This too is a call to shift away from representation as a paradigm for relating to black bodies. In the framework of this thesis, this signals an interest in how blackness relates to chroma screens to produce performative understandings of both; a knowing via the becoming of blackness and screens. In the blue/green colour of chroma screens blackness can be highlighted as becoming, enactment and performance. Perry’s practice articulates these questions and ascertains the production of blackness in a certain way.
I want to turn now to the relationship between blackness and blankness, a suggestion that emerged in black race theory, particularly Afro-pessimist
260
literature. Synthesising these ideas in relation to the blankness of chroma screens, I want to suggest blankness as a hinge that connects blackness and chroma screens. I will then look at practices that activate a relational understanding of how chroma screens operate through thinking colours themselves as relational, particularly blackness enmeshed with whiteness. These next two sections offer a productive understanding of blackness and/as blankness.
BLAN/CKNESS: CHROMA SCREEN PRACTICES AND BLUE-GREEN MATTERINGS I am concerned primarily with blackness in relation to blankness; the blankness which is also that of the screenspace – that which has not yet been filled with images. But this cannot be divorced from questions of blankness in knowing, or indeed of knowing that emerges through practices of seeing, given the occlusion of black bodies from histories and other ways of being and knowing the world. This very lack of knowing, blankness or ignorance is articulated in a different way by scholars of Afro-pessimism.
The concept of nothingness as posited in relation to black subjectivities, and also the delineation of blackness, has been discussed in numerous different ways by theorists such as Frank B Wilderson III, Sadiyya Hartman, Fred
Moten, Jared Sexton and Hortense Spillers. Thinking about blackness in relation to blankness raises the question of value. Blankness does not have to indicate that blackness has no value. As Wilderson notes, “the constituent elements of slavery are not exploitation and alienation but accumulation and fungibility.”48 In the affective interpretation offered by Hartman, the relation of blackness to blankness takes on a different meaning. According to
48 Frank B. Wilderson III, Red, White and Black: Cinema and the Structure of U.S. Antagonisms (Durham, NC: Duke University Press, 2003), 14.
261
Hartman, the fungibility of black bodies — their commodity-like “interchangeability and replaceability” — extends from being an exchange commodity to having unlimited abstract and metaphorical value in being easily replaced and rearranged, “an abstract and empty vessel vulnerable to the projection of others’ feelings, ideas, desires, and values.”49 In this way, Hartman sees a history and present in which the black body is made to perform any kind of labour while simultaneously indicating a nothingness, productive of adaptability and malleability. Moten carefully leads the discussion on black nothingness to challenge the idea of a “normative personhood”:
Perhaps nothingness insofar as it is irreducible to and not interchangeable with emptiness makes possible a recalibration of (what Wilderson calls ‘our black capacity’ to) desire that is not predicated on the constant oscillation between lack (always headed with a silent ‘b’) and (whiteness as) normativity.50
This “normative subjectivity” might also be reconfigured. The question of how social, economic, and political relations produce and maintain blackness is beyond the scope of this research. However, how blackness is performed, noticed and maintained in and through media practices definitely falls within its purview.
In the context of media technologies, Sean Cubitt argues that blackness describes the conditions of invisibility.51 Even though invisibility in this instance is not the same as the concepts of nothingness discussed above, these two have some historical and cultural equivalence. The colour black, Cubitt
49 Sadiyya V. Hartman, Scenes of Subjection: Terror, Slavery, and Self-Making in Nineteenth-Century America (New York; Oxford: Oxford University Press, 1997), 21. 50 Fred Moten, A Poetics of the Undercommons (Butte, MT; New York: Sputnik and Fizzle, 2016), 23. 51 Sean Cubitt, The Practice of Light: A Genealogy of Visual Technologies from Prints to Pixels (Cambridge, MA, MIT Press, 2014), 21.
262
argues, is considered to be an absence of light.52 Although black absorbs all visible spectrums of light and reflects nothing, it is still regarded as an absence of colour. In the same sense, white reflects all the light and is perceived as presence in opposition to an absence of light in blackness. So ‘black = absorption of light = nothingness’ is the ocular invisibility of anything in blackness and ‘white = reflection of light = presence’ is the visibility of things. In this triangulation of black, white and light, blackness is reduced to invisibility: “The history as well as the practice of black — and of light — is full of gaps, of tricks to make us ‘see’ an invisibility that otherwise refuses visibility or attempts to fill light-based media with darkness.”53 These ‘tricks,’ in Cubitt’s assessment, include the adjustment in contrast levels in the period of black-and-white television.54 These high-contrast images increased the level of blackness as well as whiteness, deepening their difference and suppressing tonalities. Blackness is highlighted in relation to whiteness here and is contrasted to its surrounding tonal spectrum. Arguably, it is the nothingness of blackness that then becomes more visible.
In fact, this relation between blackness and whiteness is one of entanglement. It is not reducible to either black or white; it is not a separation between black and white, but an entwinement. How blackness comes to be made visible in this togetherness can be illustrated in Denise Ferreira da Silva’s notion of blacklight. Ultraviolet light, also known as blacklight, is used by da Silva to activate a mode of thinking about blackness. Ultraviolet light is invisible to humans but is seen as a glowing fluorescence when reflected by various materials. Blacklight then reveals particular matter without separating it, albeit highlighting the entanglement. But more importantly, it destroys the
52 Ibid., 43. 53 Ibid., 41. 54 Ibid., 39.
263
thing that is exposed to it, as seen in its potential for altering DNA. For Ferreira da Silva, this means that blacklight activates the process “of separating form (the code, the formula, the algorithm, or the principle) and matter (content, or that of which something is composed)…once released from blacklight, matter becomes available for something that can be termed a recoding.”55 Here the entanglement of blackness and whiteness is recognised, while blackness and whiteness can also be recomposed through material- discursive practices. Blackness can be viewed under certain conditions as Ferreira da Silva argues; I propose that these conditions are created by chroma screens, specifically as Perry deploys them. To be sure, chroma screens offer a separation between the foreground and the background. But when used in the blank-black relation found in Graft and Ash for a Three Monitor Workstation, chroma screens, as elements of the assemblage, enact a productive difference. I have previously discussed chroma screens as material-discursive practices that produce agential cuts — boundary-making practices of intra-active agencies – and as such, they show up the boundaries of blackness. I will return to this idea of entanglement and a productive blankness of blackness a little later.
Thus, the blankness of blackness is recognised in the entanglement of blackness with whiteness. Material-discursive media practices enact blackness as blankness simultaneously to its relationship with whiteness. It is through certain technical elements such as chroma screens that these processes come to ‘light.’ To trace blackness in and as chroma screens through blankness, gives blankness the function of being a conjunctive hinge in the entangled relations of media practices and colour. I borrow the understanding of hinge from Kathryn Yusoff, who uses it to describe the
55 Denise Ferreira da Silva, “In the Raw,” e-flux 93, 2018, https://www.e-flux.com/journal/93/215795/in-the- raw.
264
intersection of matter and race, for example, where she calls the “concept of inhuman” a hinge in geology and humanism. She argues, “It is a hinge that establishes an extractive axis in both subjective and geologic (or planetary) life.”56 Employing this understanding, blankness as a hinge creates an axis for both aligning and unaligning chroma screens and blackness. Chroma screens are entwined with media practices that accommodate a particular kind of body and selectively (mis)recognise race. These practices continue to be part of contemporary living conditions and of postmediality. How the colour of chroma screens (blue or green) contributes to this enactment of blackness will now be examined.
The technique of chroma keying provides a noteworthy way in which the blan/ckness of screens can be observed. Chroma screens actualise and perform chromatic relations that exist between blackness, blueness and greenness, and the racialised practices of cinematography and photography. Blackness and chroma screens are entwined sociotechnically in ways that constitute and perform each other.
The history of chroma keying lies in film compositing techniques used at the advent of cinema. Compositing in the simplest terms is the process of combining two or more image sources. Digital compositing is the integration of two or more image sources through digital modification to produce a combined form.57 The practice of compositing was widely used in photography and then in motion pictures.58 One of the earliest composite photographic images was made by Oscar G. Rejlander in 1857, where he
56 Kathryn Yusoff, A Billion Black Anthropocenes or None (Minneapolis: University of Minnesota Press, 2018), 16. 57 Ron Brinkmann, The Art and Science of Digital Compositing (San Diego: Morgan Kaufmann, 1999), 2. 58 The backgrounds of photographs were manipulated by creating combination prints, where different negatives were exposed on to photographic paper resulting in an image where subjects were placed in front of backgrounds that were different to the ones they were initially shot in.
265
combined thirty-two different glass negatives (fig. 53) to produce The Two Ways of Life.59 Optical printers in cinema were invented to allow multiple pieces of footage to be combined to create one print.60 Special effects inventor and engineer Petro Vlahos patented the sodium vapour compositing process in 1963.61 This technique used sodium (yellow) light and a white background for filming. Vlahos can also be credited with pioneering the blue screen technique, which was derived from his earlier Colour Difference Travelling Matte System. Contemporary digital compositing, achieved through computers and software, began as the equivalent of optical compositing in cinema.62 Digital image mattes or masks are generated for a seamless combination of elements from multiple images. The process of generating a matte for compositing is referred to as ‘matte extraction,’ ‘pulling a matte’ or ‘keying.’63
59 Ibid., 4. 60 Disney’s The Three Caballeros (1944) was one of the first films to use a crude form of the optical printer. See Foster, The Green Screen Handbook, 6. 61 In the 1950s the British film industry was also developing a similar method to that of Vlahos, although completely independently. The film Mary Poppins (1965) notably used the sodium vapour process. Ibid., 12. 62 Brinkmann, The Art and Science of Digital Compositing, 6. 63 Brinkmann explains the possible confusion that can arise from the terms matte and mask because they are used as both noun and verb, that is, they refer to both the image that has been matted as well as the technique. Ibid., 79. Foster notes that matte difference and chroma keying are different process. Even though chroma keying is generally used to refer to the whole process of matte compositing, technically speaking it is only the process of switching off certain pixels (green or blue of the background). See Foster, The Green Screen Handbook, 23. However, for this research I am referring to the entire hardware and software process of matting and compositing as chroma keying. Brinkmann differentiates chroma keying from the colour difference method, which involves numerous steps of matte extraction, colour correction and image combination. However, in several other studies, chroma keying subsumes the colour difference method. Even the keying/matting machines and software systems such as Ultimatte and Primatte that employ the colour difference methods brand themselves as keying technologies. It can be argued that in both the chroma-keying and colour difference methods, the keying process (through which the mattes are produced) is the same. As a technique of matte extraction both use the same technical principles. See Brinkmann, The Art and Science of Digital Compositing, 81, 84.
266
Figure has been removed due to copyright restrictions.
Figure 53: Oscar G. Rejlander, The Two Ways of Life, 1857, printed in 1920s, carbon print, 40.6 x 76.2 cm, The MET, https://www.metmuseum.org/art/collection/search/294822.
Chroma keying generally refers to both the generation of matte and compositing; that is, the separation of the object from a blue or green screen background and inserting a new background.64 Depending on the hardware and software, there are different techniques of keying. For example, in luminance-based or luma-keying, the matte extraction of the object depends on the difference in brightness between the background and foreground. Chroma keying fixes the background pixels within a chosen range of colour or hue. In some cases, such as the standard video signals, not only the hue but also the luminance and saturation can be specified.65 The separation of the object in the foreground from the background is critical in chroma keying. Among the most common issues in chroma keying are colour separation, lighting and colour spill. The colour of the background should not be present in the object in the foreground as this may conflict with the keying.66 Consistent lighting is another requirement, as it effects the intensity of the
64 Brinkmann, The Art and Science of Digital Compositing, 79. 65 Ibid., 82. 66 Moshe Ben-Ezra, “Segmentation with Invisible Keying Signal” in IEEE Computer Society Conference on Computer Vision and Pattern Recognition 1 (2000): 32, https://doi.org//10.1109/CVPR.2000.855795.
267
colour in the keying. Flat lighting ensures that the chrominance of the background is regular. Moreover, colour spill, where the colour of chroma screens is reflected by parts of the object in the foreground, also has to be taken into account.67
Compositing through keying is based on the relationships of colours, and then managed by complex algorithmic systems that give better control of those relationships. The composite image can be understood as a combination that adds two images by subtracting them from their original position. However, technically the process of extracting the foreground from the background in chroma keying (and colour difference method) elucidates the relationship between black and white. To illustrate this better, let us go through the process discussed by Ron Brinkmann briefly.68 In the process of keying an image of a subject with a blue chroma screen background, all of the colour blue will be replaced by green. This means that where the value of blue is greater than green, the pixels will turn black (green and blue produces black). Due to this, the blue background turns black (fig. 54), and other pixels in the subject image where blue is more than green will also change colour.69 Brinkmann clarifies that this step includes a lot of other parameters, and also depends on how much blue has to be removed as the substitution of colours may also cause differences in intensity of the subject image in the foreground. After this step, a matte image is extracted by subtracting blue from the red or green pixels. This renders the matte image or object black and the background white (also called inverted matte). Next, the intended
67 Ibid., 2. Also, Alvy Ray Smith and James F Blinn have introduced a more ‘mathematical’ solution by adding another background in addition to the blue screen. This “multi-background” technique cannot work with actors but takes care of uneven lighting and colour spilling on the object. See Alvy Ray Smith and James F. Blinn, “Blue Screen Matting,” in SIGGRAPH Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (New York: Association for Computing Machinery, 1996), 259–268, https://doi.org/10.1145/237170.237263. 68 A more detailed version, drawing on Vlahos’ patents, can be found in Smith and Blinn, “Blue Screen Matting,” 259–68. 69 Brinkmann, The Art and Science of Digital Compositing, 84, 237.
268
background is placed onto the black foreground (fig. 54) and finally re-masks the subject image in the foreground.70 Brinkmann warns that while this process may seem simple, in practice the colours are never consistent or even. The whites and blacks are usually different shades of grey, which then have to be adjusted by tweaking the values of the colour channels. The process is made easier if the background colours are pure; hence the use of red, blue or green in chroma keying.71 This means that chroma keying and/or the colour difference method are based on values of colour understood through the relations that colours have to each other on a spectrum.72 In additive and subtractive relations with other colours, the blue or green background assists the production of a composite image.
Figure has been removed due to copyright restrictions.
Figure 54: Chroma keying on blue screen through the process of colour difference. Digital image, https://fxmakingof.wordpress.com/2014/01/21/the-1940s-an-overview-of-special- effects/.
70 Ibid., 85. 71 Ibid. Although digital technologies have made compositing with any colour much easier. As I discuss later, the importance of using green or blue depends on other non-technical practices. 72 Nevertheless, colours are relational since they are part of the visible light spectrum and cannot be entirely fixed or demarcated. A colour is not light reflected at one fixed wavelength but rather a range of wavelengths. Blue that is reddish is different when the same blue is placed alongside white. The colour of an object is also contingent on the source of light, its luminescence and the degree of saturation. For an in-depth discussion on this see Stephen Neale, Cinema and Technology: Sound, Image, Colour (Bloomington, IN: Indiana University Press, 1985), 110. Moreover, the colour of the object is conditional upon whomever is viewing the object, since two humans hardly ever perceive colours in the exact same way. Chares A. Riley, Colour Codes: Modern Theories of Colour in Philosophy, Painting and Architecture, Literature, Music, and Psychology (Hanover, NH: University Press of New England, 1995), 2.
269
This discussion on the technical aspects of chroma screens and colour relations lays the ground for thinking about how techniques that emerge from technical elements engage with discursive practices in what Barad terms the active matterings of matter.73 Although in Perry’s Ash and Graft colour is compositional, it is also racial and political, all at once. The agentic capacity of chroma screens matter (to) black bodies. The colours blue and green — the most widely used colours in chroma screens — have a specific mode of gesturing, that is, they perform blackness so that it is maintained in specific ways in a postmedia assemblage.
The most important consideration in deciding to use either blue or green chroma screens depends on what is being shot and the background that will be composited behind it. For a sharp separation the colours in the background should be distinct to those in the foreground. This can be challenging when it comes to the colour red, another colour other than blue and green that is advised, since it does not present a high contrast to the colours of human skin tones or objects in the foreground and therefore can interfere with them during keying.74 Traditionally blue has been used in cinema since the 1920s as the choice of the background colour for chroma screens. The rise of green screens is credited to the growth in use of video and improved graphics in late 1960s and 1970s (NBC USA being the first one to use it in 1950s).75 Some have argued that green is preferable as blue is a more common colour in clothing or objects in the foreground.76
73 Barad, Meeting the Universe, 151-2. 74 Red chroma keying is used is very specific shoots, for example when filming spacecrafts. Brinkmann, The Art and Science of Digital Compositing, 20. 75 Robert Henson, Weather on the Air (Boston, MA: American Meteorological Society, 2010), 101-3. 76 This is a popular notion in online discussions. See “Difference between green screen and blue screen” on https://www.camberwellstudios.co.uk/blog/film-production-tips/whats-the-difference-between-green-screen- vs-a-blue-screen.html, and https://www.gearshift.tv/faqs/what-s-the-difference-between-a-green-screen-and- a-blue-screen-.cfm.
270
However, there are other technical reasons for the increasing use of green chroma screens for video and digital work. First, is the availability of the colour green. The green chroma screen colour is a particular shade (354C in the Pantone colour scheme), which became more common only after developments in paint colour technology (Day-Glo paints) in the 1960s.77 Secondly, it is relatively easier to do the lighting set-up for green chroma screens than for blue. In the colour spectrum, green is a colour of high luminosity, so it requires less light to produce an evenly lit chroma screen background. Although less lights help control the colour spill on the objects in the foreground, the green spill is also brighter on the object in the foreground, which is difficult to manage. On the other hand, blue is also ‘noisier’; that is, it is grainier than the green channel.78 This is a point to consider in both film stocks and video as they might already be grainy. The grain of film stock may be enhanced with the use of blue chroma.79 In video, increasing the sharpness of the image can make the images grainier with the use of a blue chroma background. Thirdly, most video and digital cameras contain the Bayer pattern filters in their image sensors. The Bayer photo mosaic pattern, modelled according to the human eye’s sensitivity to colours, has the same number of green photo sensors as blue and red sensors combined, making them more sensitive to the colour green.80
77 Brinkmann, The Art and Science of Digital Compositing, 219. 78 Ibid., 220. 79 Special film stocks are produced just for chroma screen recording, such as Kodak’s SFX 200T. 80 The Bayer photo mosaic pattern is widely used in digital camera sensors and is modelled on the human eye. The human eye has two types of receptors: rods, which detect light intensity; and cones, which detect colours. Most humans have three kinds of cone cells, labelled S, M and L: S cone cells detect small wavelengths that fall in and around the blue spectrum; M cones detect medium wavelengths around the colour green; and L cones detect long wavelengths around the red spectrum. As far as the number of cones go, most humans have more green colour detecting cones than blue and red (sometimes the number is put at 50% M cones and 25% each for S and L cones). Since most humans have more cones that detect green, the Bayer sensors also have 50% green detection sensors, and 25% each for blue and red sensors. While Bayer pattern is the most popular one in commercial cameras, researchers have come up with other patterns for better light efficiency, see Keigo Hirakawa and Patrick J. Wolfe, "Spatio-Spectral Color Filter Array Design for Optimal Image Recovery," IEEE Transactions on Image Processing 17, no.10 (Oct. 2008): 1876-1890, https://doi.org//10.1109/TIP.2008.2002164.
271
This short discussion on technical ‘greenness’ takes into account the materiality of the green colour of chroma screens. But a materiality encompassing the geographical, colonial and political intra-actions of the colour will help us to think about the differential enactment of black bodies in and through green chroma screens. Here specific material-discursive practices such as the Anthropocene and surveillance technologies can articulate the green connection between race and colour of chroma screens.
One way in which green is associated with race is through its association with the Earth and geography. The relation of the colour green to black bodies can be further understood as a part of extractive colonial practices and histories. In discussing ideas around the relationship between geology and race, Yusoff challenges what she calls “the racial blindness of the Anthropocene.”81 Her interest in an entangled logic and politics of the extraction of bodies and minerals as racialised fossil narratives. She specifically focusses on the ways the Anthropocene has failed to deal with black bodies in its contemporary debates. She argues that the ecological concerns with regard to the Anthropocene do not take into account the violence of imperialism and colonialism, which for centuries was enacted on and through brown and black bodies.82 The literal enslavement of black bodies and the tearing of resources from their land, upon which colonial trade and empire were forged, has made both land and bodies into properties. Instead of regarding black bodies as human they have become inert matter, devoid of human agency, and instead part of geological matter.83 Yusoff builds on the work of others such as Hartman and the concept of fungibility, which I discussed earlier in this chapter.
81 Yusoff, A Billion Black Anthropocenes or None, xiii. 82 Ibid. 83 Ibid., 4.
272
Figure has been removed due to copyright restrictions.
Figure 55: Sandra Mujinga, Throwing Voices, 2016, three lexan plates, perlon wires, HD video installation, 44:08 mins. Installation view at Oslo Kunstforening, Norway, 2016.
The obfuscation of black bodies in extractive geological politics takes a diametrically opposite form in surveillance technology, where black bodies are made conspicuous. The linking of geological race formation and technological surveys of black bodies in and through green chroma screens is explored in the work of Sandra Mujinga. Mujinga works across sound, video, performance and installation art, using green chroma screen in numerous ways to work through questions of representation, surveillance, masking and obscuring. Sometimes the green chroma screen is used as the colour of the walls as seen in Nokturnal Kinship 1–3 (2018) or as the colour of the light of the exhibition space in her solo exhibition SONW: Shadow of New Worlds (2020), and also as a background to videos such as Throwing Voice (2016), and Catching Up (2017). In Throwing Voices (fig. 55) the video is projected through three polycarbonate sheets in an upright position, curling from its sides and held together with wire (fig. 56). In the video, Mujinga herself poses against a green screen. Her face is contorted, and her clothes permeate the green of the chroma screen at places, giving her body an ‘under construction’ look. The video is accompanied with the audio from a make-up tutorial by black females
273
on how to highlight different parts of the face. It seems as though the black body itself is a work in progress.
Figure has been removed due to copyright restrictions.
Figure 56: Sandra Mujinga, Throwing Voices, 2016, three lexan plates, perlon wires, HD video installation, 44:08 mins. Screenshot from digital video.
The work resonates with Steyerl’s use of green screens in recognising the politics of visibility of chroma screens, although Mujinga aligns this technological visibility specifically with the colour of black skin. Considering the “emptiness” of blackness, she places her work in the visibility/invisibility duality of black bodies, but also examines what it means to be in darkness.84 Affirming her views on the use of green screens, she says, “In my videos when you see a black background, it’s green screen… It allows me to host ideas and alternative spaces. Green is ultimately Black.”85 Here the green chroma screen is equated with black bodies that are invisible and hidden, often just merging into the background. However, the green chroma screen in Throwing Voices embodies much more than the dualities of visibility/invisibility and absence/presence. Even though these dualities are equated with the
84 “Sandra Mujinga at Bergen Kunsthall,” Contemporary Art Daily, January 14, 2020, https://contemporaryartdaily.com/2020/01/sandra-mujinga-at-bergen-kunsthall. 85 Excerpts from “There’s Nothing Black About This,” an interview by Olamiju Fajemisin with Sandra Mujinga, in “Vleeshal - Sandra Mujinga: Midnight,” e-fllux, March 7 2020, https://www.e- flux.com/announcements/318278/sandra-mujingamidnight.
274
positioning of black bodies, Mujinga folds visibility with invisibility and highlights their co-presence. It is only when a black body is invisible that its appearance is out of the ordinary and hence more visible. In that appearance, its invisibility also shines through. The blackness executed in the material- discursive practices of green chroma screens in Throwing Voices is a specific entanglement.
I turn now to another instance in the blue-ness of chroma screens. One of the reasons for using blue and green for chroma screens is that they are the colours furthest away from human skin tones. The human skin has yellow to red tones which contrasts well with blue and green.86 Brinkmann notes that of the colour of hair and skin is a deciding factor in the use of either a blue or green screen. He states that some people argue that darker skin tones have more blue than whiter ones, but is quick to brush these arguments aside by stating that such a theory “has never been rigorously tested.”87 On the other hand, Foster explains why blonde hair might key better against a blue screen since a green background makes the hair appear too red.88 However, he concedes that it depends on the colours used in the foreground image. Nonetheless, a simple test shows that blonde hair is extremely challenging to work with in green screen but works better on a blue background. The tendencies towards yellow to red undertones are that of white skin as commonly observed by cinematographers. Black and brown skin mostly have blue undertones.89 A blue background, therefore, absorbs black tones and making it more difficult to separate out.
86 Brinkmann, The Art and Science of Digital Compositing, 219; Andrey Lebrov, “Greenscreen vs Bluescreen: When and Why”, YouTube, 28 September 2019, https://www.youtube.com/watch?v=ava4Z3sJMLk&t=105s. 87 Brinkmann, The Art and Science of Digital Compositing, 220. 88 Jeff Foster, The Green Screen Handbook, 19. 89 The unique skin tone of every individual interacts differently in various kinds of light conditions. Cinematographers consider the skin colour of the actor, along with make-up and costumes for specific outcomes. Dyer, White, 82–103. Cybel Martin, “The Art of Lighting Dark Skin for Film and HD,” Shadow and Act, 4 February, 2014, https://shadowandact.com/the-art-of-lighting-dark-skin-for-film-and-hd
275
Sondra Perry, cognisant of all these relational colour combinations, actually correlates chroma key blue with blackness. After all blue is the colour of blackness in so many ways. From the colour of skin, to the journey across the blue waters of the Atlantic; from blues music to the paintings of Ellsworth Kelly, blue accompanies black, coalescing and relating with and to it. In spite of these relations, blue chroma screens are not suited to the black skin tone for image compositing. In a way, blue chroma screens enact a refusal to acknowledge black bodies as separate since the colour blue is more active in black skin. If the colour for a chroma screen background is based on the use of colour that is furthest away from the skin tone, Perry asks the pertinent question, whose skin tone are we talking about?90 Drawing a connection specifically with the blue colour of chroma screens and black, she states that the blue chroma key absorbs shadows much better than the green chroma key, and hence “approximates blackness.”91 In this way, she positions blue chroma screens in an ambiguous and unstable but active space. The instability of the blue chroma screens is not a representation of blackness but of its performance in and through material-discursive practices, some of which I have explicated here. Here the screen is not just hiding and revealing images, but instead intra-acting with history, race, laws and nation states as much as the discursive practices of medial representation and technological development. Blackness engages with and co-constitutes these ways of arrangements in and of the world.
In an agential realist framework, detecting and evaluating blackness is inseparable from the material-discursive phenomena of chroma screens. Since
90 “Sondra Perry: Typhoon Coming On,” 91 Interview by Tamar Clarke-Brown, “Adrift in the Chroma Key Blues: A Chat with Sondra Perry on Black Radicality + Things that are Yet to Happen in ‘Typhoon coming on,’” AQNB, May 1, 2018, https://www.aqnb.com/2018/05/01/adrift-in-the-chroma-key-blues-a-chat-with-sondra-perry-on-black- radicality- things-that-are-yet-to-happen-in-typhoon-coming-on.
276
assessments and modes of evaluation are inseparable from what is being evaluated, blackness is co-constitutive of the (blue/green) colour of chroma screens. Moreover, matter and meaning exist in the act of evaluation.92 The meaning and productive value of blackness in a dominant practice of postmedial post-production lies with the performance of chroma screens. The value of blackness and chroma screens is only given in and through their intra-active performances. Barad places a lot of importance on the act of measuring, which we can see is also taking place through the technical processes of colour-matting. But it is more important to understand how things are measured and the ethics at play than simply what a measurement is; the how of the measuring constitutes what is revealed.93 Meanwhile, blackness has been compared to whiteness and the difference between them bestowed as per white standards. According to Ferreira da Silva, blackness is determinate for the Hegelian and Kantian traditions as these assume human difference is based on a universal benchmark, which, for them, is White Europeanness.94 But as I have shown through this chapter, blackness is not predetermined but caught up with — among a range of other relations and things — a powerful imaging and medial technology, co-constituted in, with, and through chroma screens.
Might chroma screens then perform an excess of blackness? Such excess, on the one hand, according to Ferreira da Silva, legitimises racial violence against blacks.95 Yet this excess, which seems to be a “disruptive force,” is simultaneously the creative capacity of blackness. The blankness of chroma screens can in this way be understood as a performance of excessive
92 Barad, Meeting the Universe Halfway, 37. 93 Ibid., 197. 94 That is why blackness is different. See Denise Ferreira da Silva, “1 (life) ÷ 0 (blackness) = ∞ − ∞ or ∞ / ∞: On Matter Beyond the Equation of Value,” e-flux 79, 2017, https://www.e-flux.com/journal/79/94686/1-life- 0-blackness-or-on-matter-beyond-the-equation-of-value. 95 Ibid.
277
blackness. In the entanglement of blackness and whiteness, blackness is executed as a productive difference that also maintains shifting ontological and semantic boundaries. Visibility, the opposite of blankness, and transparency are not the only demands. Although, of course, this requires us to likewise evaluate why invisibility should, at least to the extent of opacity, be required. As Édouard Glissant has argued, transparency is a grasping at something being brought closer to the self, and thus reducing the thing that is being grasped.96 I will discuss how opacity is not in conflict with visibility but constitutes a different kind of comprehension of blackness.
BLAN/CK OPACITY Glissant’s call for a “right to opacity” is a demand to unthink difference in a reductive way. He says:
Difference itself can contrive to reduce things to the Transparent. If we examine the process of ‘understanding’ people and ideas from the perspective of Western thought, we discover that its basis is this requirement for transparency. In order to understand and thus accept you, I have to measure your solidity with the ideal scale providing me with grounds to make comparisons and, perhaps, judgements.97
In the light of the above statement, the claims of visibility of blackness in and through chroma screens that I have argued until now conflicts with Glissant’s argument to allow blackness to remain unrecognised and opaque. However, Glissant is discussing a particular kind of transparency that is demanded by Western thought. This refers back to the earlier discussion by Ferreira da Silva, regarding making blackness and whiteness accord to the standards of
96 Édouard Glissant, Poetics of Relation, trans. Betsy Wing (Ann Arbor: University of Michigan Press, 2010), 191. 97 Ibid., 189, 190.
278
white Europeans. Elsewhere Glissant talks about grasping and bringing a thing closer to oneself as not recognising the thing. He explains:
In this version of understanding the verb to grasp contains the movement of hands that grab their surroundings and bring them back to themselves. A gesture of enclosure if not appropriation. Let our understanding prefer the gesture of giving-on-and-with that opens finally on totality.98
To understand something one needs to get closer to it, not bring it closer. The (in)visibility of blackness can be thought of in terms of Glissant’s right to opacity, where blackness is not to be understood/grasped but as a movement towards.
His idea of the right to opacity is an acknowledgement of not being able to fully know or grasp. Western thought, according to Glissant, insists on transparency and knowing, in other words being able to separate and establish differences. But Glissant says that accepting the difference should not mean reducing the difference under the pretext of understanding it. Instead he proposes,
Agree not merely to the right to difference, but carrying this further, agree also to the right to opacity that is not enclosure within an impenetrable autarchy but subsistence within an irreducible singularity. Opacities can coexist and converge, weaving fabrics.99
For Glissant, opacity is not closing down or separation into the categories of ‘self’ and ‘other,’ it is rather being open to divergences of many opacities. He
98 Ibid., 191, 192.
99 Ibid., 190.
279
talks about ‘weaving’ – an intertwining without any boundaries. For him, difference is maintained in multiple opacities.
Although Ferreira da Silva also conceives of this kind of singular multiplicity, she arrives at it in a slightly different way. She insists on thinking about difference in ways that avoid separability, determinacy and sequentiality, arguing that things should be seen as entangled.100 She suggests the term nonlocality for such an imagining, as entanglement does not exist in space and time for her.101 Ferreira da Silva advocates nonlocalisation, then as having actual and virtual existence:
What nonlocality exposes is a more complex reality in which everything has both actual (spacetime) and a virtual (nonlocal) existence. If so, then why not conceive of human existence in the same manner? Why not assume that beyond their physical (bodily and geographic) conditions of existence, in their fundamental constitution, at the subatomic level, humans exist entangled with everything else (animate and in-animate) in the universe.102
The nonlocality Ferreira da Silva proposes is a coexistence of everything, “a singular expression.”103 In this light, the chroma screens in Graft and Ash for a Three Monitor Workstation are an instance where their blankness is a kind of nonlocalisation. Blankness is not restricted to the installation, the exhibition space or the space occupied by the exercise machine but moves us to feel entangled with it. When sitting on the exercise machine to watch the blue screens in Graft and Ash, different chroma screens come together: the chroma screens where special effect images are rendered, that also allow racialised
100 Denise Ferreira da Silva, “On Difference without Separability,” in Incerteza Viva, 32 Bienal de São Paulo (São Paulo: Fundacion Bienal de São Paulo, 2016), 63, 64. 101 Ibid., 64. 102 Ibid., 64, 65. 103 Ferreira da Silva borrows from Leibniz’s notion of singularity. Ferreira da Silva, “On Difference without Separability,” 65.
280
bodies to be performed; screens whose materiality is dependent upon extractive racial geopolitics and where the ghosts of the past (deliverance ministry) might be expelled. The expulsion of the ghost is only heard over a blank-black and black-blue chroma screen.
Perhaps the ghosts that are discharged from the bodies of black people could be spirits that have been lost to racial injustices, now trying to find a way back into the world through the blank chroma screens. The differencing of the blankness of chroma screens are where both the actual (space-time) and virtual (nonlocal) exist.104 To recall the words of Perry’s animated avatar, who hints at some of the virtualities of blankness, “We are a DIY, not all that representative thing, which makes being a ‘being’ impossible or whatever.” It is unclear if she is talking about black bodies or the exercise machine. Moreover, depending on an audience member’s engagement with the exercise machine, the blankness of chroma screens provides different experiences. If an audience member uses the exercise machine, even if the machine resists such use, they are akin to the black bodies of which the avatar speaks, believing the world is just but still facing racism. With that exercise machine, the audience member experiences a becoming of a kind of blackness.
The analysis of chroma screens in postmedia assemblages, thus far, demonstrates a racial gesturality of screens. In an attempt to move away from images and their representation on screens, this chapter aimed instead to describe particular becomings of ‘screens without images’ emerging from practices of image-making and display. I analysed works where chroma screens are without images, that is, where images are not composited on to them. The emptiness on/of screens here is understood as an openness to form
104 Ibid., 64.
281
and reform relations with parts of the assemblage. Moreover, the blankness of screens presents a gesturality that can be apprehended politically, not just aesthetically. In Hsu Chia-Wei’s Marshal Tie Jia - Turtle Island (2012), I analysed the intra-action of political, geographical and marginal histories, both factual and mythical, as gestured by chroma screens. In Hito Steyerl’s How Not to be Seen: A Fucking Didactic Educational. MOV File (2013), I examined the various relations of concealment between images and screens enacted in and through chroma screens. In both of these works, the chroma screens hide and reveal connections, and sustain them in and through their blankness.
My main argument regarding racialised practices can be read in my discussion of Perry’s Graft and Ash for a Three Monitor Workstation (2016). I discussed material-discursive practices that enact racial bias in photography, cinema and digital technologies in order to explore the relationship between blankness and blackness. By considering chroma screens in and through material-discursive intra-action, I demonstrated that the green and blue of chroma screens can be considered as a cultural enactment of racialised medial practices. In the continued obfuscation of black bodies, chroma screens in Perry’s work provide, instead, a productive understanding of blackness that generates it as difference. The blank chroma screens in Perry’s work highlight blankness as productive at the level of the chroma or colour itself. Moreover, blackness itself is co-constituted in, with and through chroma screens to produce a performative understanding of both. Gestures here are considered to be made in colour — not just the blue or green colour of chroma screens but of blackness itself. In the process of this thinking of colour, screens and bodies, I have also attempted to illustrate that entanglements such as that of screens and images, and blackness and whiteness, do not simply involve connection. Rather, they are woven in a way that their dependency on each other extends beyond the merely influencing or being associated with each
282
other. Their coalescing defies identification of each of the constituents as pre- given and discrete. They are co-produced in and through various material- discursive practices and processes that undo these in novel ways.
283
POSTSCRIPT I started this thesis by proposing that we deliberate upon the agency of screens in postmedia to make sense of its dynamicity. Such vigour has been abundantly observed in the multiple roles and enactments of screens in the postmedia condition. Developing screenness as a performative notion of the agency of screens, I have examined the gesturality of screens in and through which they configure and maintain mediality in a postmedia assemblage. Taking on specific gestures of postmedial screens, which I have sought out in multiplicities of frames and in imageless but affective screens, I considered how such screenic enactments relate and rearrange the relations of a postmedia assemblage.
The discussion in the second half of this thesis provided in-depth analysis of at least two ways in which screens gesture. Of course, these are not the only gestures of screens nor are the out-of-frame and the gesture of suspending images from screens always enacted in the same way in all postmedia assemblages. It is only fitting to conclude this thesis by pointing to how these particular gestures might also work together in a postmedia assemblage, as well as to speculate about other potential gestures by screens. Hence, I want to consider how both the out-of-frame and blankness could be considered as a collective gesturality that allows us to approach postmediality from a combinatorial perspective. Simultaneously, I want to provide an opening towards thinking about how the omnipresence of screens advances via gestures and enactments rather than taking such ubiquity for granted. Towards these ends, I will briefly examine the gesturalities of screen(s) in Afghan War Diary (2010) by Matthieu Cherubini. This discussion will also allow me to gather a medial, aesthetic and political comprehension of screens in one location/instance. I want to deepen the insights of my performative
284
understanding of screens by explicitly considering the agency, constraints and exclusions of material-discursive practices seen in the particular kind of screen(s) in this work. As I have proposed throughout this thesis, screens are not objects or things but a collection of practices that entangle with numerous material structures and discursivities. Screens, then, in Afghan War Diary materialise via the intra-actions of a range of material-discursive practices including those of gameplay, geospatial visual correspondence, internet connectivity and access, war strategy and historical political allegiances.
Figure has been removed due to copyright restrictions.
Figure 57: Matthieu Cherubini, Afghan War Diary, 2010, online website. Screenshot from documentation of the website, http://mchrbn.net/afghan-war-diary/.
Afghan War Diary is an online work presenting three different aerial shots on a three-dimensional geospatial map of locations in Afghanistan (fig.57). 1 Forming a triptych, each area in the frame relates the deaths of players in a shooting game to actual killings during US military activity in Afghanistan between January 2004 and December 2009. There are three main operations in this work: one is the real-time death of a player occurring in the video game Counter-Strike (2000–ongoing), an online multiplayer first-person
1 The website first used the Google Earth and switched to Cesium platform, a 3-D geospatial application, in 2018.
285
shooting game, structured around terrorist and counterterrorist attacks. The Afghan War Diary website connects to one of the servers of the game to acquire logs of player killings (referred to as frags). Second, the real-time gaming deaths of players prompts a connection to the Wikileaks document “Afghan War Diary,” which details at least 75,000 US military events that took place in Afghanistan between 2004 and 2009.2 The location of one of these events from the Wikileaks database is extracted. The third operation is the geolocation of that event on the three-dimensional geospatial map. All three frames on the screen show different geopositions in Afghanistan, thereby referring to separate US military events. Each time a player in Counter-Strike is killed, a new US military event in Wikileaks website is accessed. On the Afghan War Diary website, the displayed location manoeuvres to a new area — the one garnered from the Wikileaks military event — in a way typical of contemporary map navigation, zooming in and out of landscapes while traversing them. In this way, the locations on the three frames are indirectly linked to a player’s death in Counter-Strike.
The work is an assemblage connecting to servers that host Counter-Strike in order to tap into the real-time deaths of players, the Wikileaks files database and the three-dimensional geospatial imagery (from Google Earth or Cesium). We can recall from my discussions on mobile phone navigation using Google Maps in chapter one and Blast Theory’s Can You See Me Now? in chapter three that in the complexity of location search screens and other medial entities emerge through specific material-discursive practices. These can include crowdsourcing practices and smaller components, such as the gyroscope. In Afghan War Diary, the postmedial assemblage emerges via an intricate exchange of data and location, which takes place across multiple
2 The Wikileaks database, which has made public Iraq and Afghan war reports that are otherwise classified information, is downloaded on the website server which makes the accessibility prompt.
286
platforms, media devices and assembles various networked organisations, governments and corporations. The work interweaves three databases in a manner similar to contemporary surveillance systems, which gather information from various data sources;3 albeit here the three databases are cited on one screen (the website) with three frames.
The gesturalities of screens developed in this thesis play out in Afghan War Diary, demonstrating how the assemblage sustains connections that are generated through the dynamic operativity and redistribution of the assemblage by its screens. As the players in the online game die, the scenes in the triptych keep changing. On each of the images in the three frames, the name of a player who has “killed someone” is juxtaposed with the date and time of the US military operation and number of deaths that took place in an actual wartime incident (fig. 58). No violence, killings or bodies are actually displayed on-screen. As gameplay, such actions occur on other (players’) screens. These out-of-frames from the game, nonetheless, come on-screen and register in the frames of Afghan War Diary’s viewer, as with every player’s death, one of the three frames traverses to a new location using geospatial capability of the three-dimensional map. The fictitious deaths in the game trigger the locations of actual military actions and frame those on the screen by showing those areas on the three-dimensional map. There is not just one out-of-frame — that of the game players — but also the out-of-frames of the Wikileaks database and the actual killings in Afghanistan. The frames in the work then have multiple out-of-frames that are moving into the frames.
3 Kyle Weise, “Control Room,” in Military Vision (Melbourne: Screen Space, 2013), 30.
287
Figure has been removed due to copyright restrictions.
Figure 58: Matthieu Cherubini, Afghan War Diary, 2010, online website. Focussed screenshot from documentation of the website, http://mchrbn.net/afghan-war-diary/.
The viewer does not have any control over the movement of the video. Rather, the out-of-frame conditions the frames on-screen. The screen does not simply display but becomes the medial space in which the multiplicity of relations between the out-of-frames and in-the-frame is sustained. The gestures of the screen in this work cultivate the connection between numerous game players that are out-of-frame with the interior of the frames in the work, sustain a relation to the out-of-frame of the US military event with the interior of the geospatial frames in the work, which continues the association of the out-of-frame of the game player and the out-of-frame of the US military action. These out-of-frames are not just referenced spatially in the geolocation images, but the images on-screen temporally prehend the
288
past.4 The multiple temporalities of the several out-of-frames are sustained on the screen observed by viewers. This out-of-frame gesturing by the screen generates relations between the present (real-time game play), the past (war and its violence) and a kind of immediate future — the travels of how these might converge in a ‘blank’ future of the three-dimensional map. Here the multiplicity of frames, actions and temporalities converge on the screen yet do not quite meet and resolve into one. As a result, we are left with gaps and slippages through which a kind of affective expressivity emerges. This space and time on-screen give emergent form to sensing a war that also endures.
Comprehending Afghan War Diary as ‘without images’ is to understand the stark absence of images of actual death or violence. The geospatial images on the website seem to be empty, unaffected landscapes, which hide the atrocities of war. Here the absence of images and the emptiness of screens are different from that of chroma screens, but there is nevertheless a void of visual content where we might expect to see it depicted. Each time a death in the game occurs, we do not see it, nor do we see attacks that happened in gameplay or actual events actioned by the US military. The violence is visually missing but nonetheless enacted in and through the landscape, changing each time a player dies. While the screen performs the imperialist war logic of the US, its violence is not seen but only gestured towards each time a game death occurs. This also speaks to the violence that is sustained by actual bodies of people — especially the many Afghani civilians killed in the
4 While I do not analyse Alfred North Whitehead’s philosophy in this thesis, his concept of prehension may be used to think about how screens might be capable of sustaining relations of prehension between the out- of-frame and the in-frame. Here, I am referring to Timothy Barker’s work on the ways in which humans and digital entities prehend each other to produce “visible outcomes” while working through “a network of imperceptible processes.” Barker, Time and the Digital, 53–4.
289
Afghan war — who are also never seen.5 The screen, then, marks the long duration of deaths, both real and artificial, in this work.
Amid this gesturing, in and through which images and views are arranged, there are other significant components of the assemblage that are facilitated and reconfigured by its screen. Here the screen performs the dual function of hiding and revealing. It obscures the computation of the machines and networks that connect to the game server, Wikileaks files and geospatial visual renderings, among several other technical connections. While Afghan War Diary also brings into focus the algorithmic and database logics that operate to bring the images into relation across the frames, their continuity across each separate feed and network — from online game to geospatial visuals — is facilitated by the screen. The screen of the work is orchestrating the exchange between Counter-Strike gameplaying via screens every time a player dies. This enactment produces a spatial and temporal movement as the US military action database is activated and location images are displayed. While the screen maintains the assemblage’s opacity, the traversing of locations as empty imagescapes also reveals the automated manoeuvres of military drones that are especially characteristic of strikes undertaken across the duration of the Afghan War.6
5 The US government has not kept a count of the civilian deaths from airstrikes by US military in Afghanistan (or in Iraq or Syria). Moreover, civilian deaths are seen to add to the hostility from locals rather than acknowledging the loss of human life. Thomas Gregory, “Potential Lives, Impossible Deaths,” International Feminist Journal of Politics 14, no. 3 (2012): 327–347, https://doi.org//10.1080/14616742.2012.659851. 6 Since the US conducted its first drone strike in Afghanistan in 2002, their number has consistently increased. For instance, according to the 2012 report from the Bureau of Investigative Journalism, between 2008 and October 31 2012, 1,015 drone strikes had been conducted in Afghanistan by the Coalition forces. Between January 2015 and December 2019, the minimum number of drone strikes was reported as 13,067. Chris Woods and Alice Ross, “Revealed: US and Britain Launched 1,200 Drone Strikes in Recent Wars,” Bureau of Investigative Journalism, December 4, 2012, https://www.thebureauinvestigates.com/stories/2012-12- 04/revealed-us-and-britain-launched-1-200-drone-strikes-in-recent-wars; Bureau of Investigative Journalism, “Drone Warfare,” https://www.thebureauinvestigates.com/projects/drone-war. For an overview of the legal and ethical problems posed by drones, see Michael J. Boyle, “The Legal and Ethical Implications of Drone Warfare,” International Journal of Human Rights 19, no. 2 (2015): 105–126, https://doi.org//10.1080/13642987.2014.991210.
290
These revelations in and through screenic enactments are gleaned through the intra-actions of material-discursive practices and the inbetweenness of screens. A few instances of this can be seen in the military research antecedents of geomapping and drone technologies; and Counter-Strike is part of the shooting-game culture encouraged by the US military for combat training when soldiers are not on active duty.7 In this way, the history and materiality of the multiple screens in Afghan War Diary — such as those viewed by the viewer, and the out-of-frame screens used in the gameplay — are understood to be related to viewing habits and social spaces, data processing and surveillance practices. The gestures of these digital postmedial screens can then be extrapolated to that of the folding screen device, used to divide rooms for privacy and protection from wind or fire. Revisiting these screens that I first mentioned in the introduction to this thesis (fig. 3), we can bring in to focus now not the panel/screens themselves but the gaps between these, through which one may be able to glimpse something on the other side. This too is how the postmedial screen unfolds its networked operations and algorithmic processes in the operations and processes of Afghan War Diary.
Reading Cherubini’s work through a simultaneous performance of screen gesturalities allows us to attend to the ways in which mediality is produced in the ways a postmedia assemblage performs, configures and reconfigures relations. This does not mean that these are the only gestures in this work. In Afghan War Diary, the automated processes of going through server logs of the game and Wikileaks database can produce glitches. These may not necessarily stop the work operating but can generate errors. For example, all three frames may traverse the same points in the Wikileaks incident logs and
7 Scott N. Romaniuk and Tobias Burgers, “How the US Military is using ‘Violent, Chaotic, Beautiful’ Video Games to Train Soldiers,” The Conversation, March 7, 2017, https://theconversation.com/how-the-us-military- is-using-violent-chaotic-beautiful-video-games-to-train-soldiers-73826.
291
hence, the same location is displayed on-screen in all three frames. This kind of glitch or malfunctioning — when something continues to operate but functions differently to what it was intended or expected — can also be considered as a kind of screenic gesture. In this case the screen sustains mediality via a kind of stuttering or repetition, in order to sustain the relations to algorithmic processes and networks that are the obfuscated causes of such ‘malfunctioning.’8
In another consideration, an affective experience of violence is also accomplished through screen gestures. As discussed above, the movement to a location on the three-dimensional map takes place when a Counter-Strike player is killed. It is in that change of geolocation — the sweeping blurry movement while moving across to a new location – a prepatory gesture is produced in which violence is anticipated. In that inbetweenness where the movement on-screen on the map is being calibrated to show a different location, the viewer prepares for the violence that will be confirmed through on-screen images. This violence is being transmitted across players in Counter- Strike as they continue to kill each other and the Wikileaks files keep calling up the actual war incidents. But the violence is never brought on-screen as such, instead the movement across the screen space makes an affective space and duration for it.
8 There is limited scope for expansion here, except to briefly state that in this instance the error cannot be considered in the same way the glitch is used artistically to display (mis)transmission. On this occasion in Afghan War Diary, the screen is not ‘displaying’ glitches in the sense of breaking down the image on-screen — as in glitch aesthetics, where the dysfunctionality of a system is exhibited via images and/or sounds as a clear deviation. I am instead considering a scenario where the same image (location) is replayed on all three frames at the same time. Here I am arguing that the screen continues the mediality of the assemblage but does not break or ‘glitch’ it. Glitch is different to an error, where “an error might produce a glitch but might not lead to a perceivable malfunction of a system.” Olga Goriunova and Alexei Shulgin, “Glitch,” in Software Studies: A Lexicon, ed. Matthew Fuller (Cambridge, MA: MIT Press, 2008), 111. Also see Rosa Menkman, “Glitch Studies Manifesto,” Video Vortex Reader II: Moving Images Beyond YouTube, ed. Geert Lovink and Rachel Somers Miles (Amsterdam: Institute of Network Cultures, 2011), 336–347. My understanding of error here is informed by Mark Nunes who takes error as a “failure” but also characterises it as a potential. Arguably the screen here is enacting that potential. Mark Nunes, “Error, Noise, and Potential: The Outside of Purpose,” in Error: Glitch, Noise, and Jam in New Media Cultures, ed. Mark Nunes (London: Continuum 2011), 14.
292
In suggesting the possibility of other kinds of screen gestures here we can broaden our understanding of what other possible enactments screens might be capable of and how these undertake the work of both sustaining and reconfiguring medial relations. In concluding this thesis, I want to continue to think of the various ‘doings’ of screens, since their increasing activities point us towards new gestures in future media assemblages. My main proposition has been to highlight a co-emergence of screens and postmedia that recognises the mediality of screens as an inbetween sphere of operation. Understanding the agency of screens as differential enactments in and through specific image-making and display practices has drawn attention to the way screens are ‘doings’; that is, they are enactments. This can inform how we approach future screens but can also help us think about how we configure and explore our ongoing medial experiences.
In many ways, when it comes to virtual and mixed reality projects, screens have been positioned as a hurdle to an encompassing experience. Some, such as Manovich, argue that immersive practices actively work towards eliminating screens.9 However, I would argue that virtual reality does not do away with screens; rather, screens materialise differently, deploying gestures through which presence, subjectivity and embodiment come to the fore. Another direction in which screens are heading is toward integration with the human body and its movements and capacities. This is starting to emerge in the domain of smartwatches where various small body movements, such as muscle contractions and hand motion, can be used to measure bodily
9 Manovich, “An Archaeology of a Computer Screen.”
293
functions and even perform various tasks on-screen.10 In such cases the movements of the wearer’s body will converge with and on screens as a mode of maintaining relations in the assemblage. Here human gestures will move completely away from meaning making as displayed in early cinema to become fully subsumed in the sphere of mediality. The gestures that humanity had lost according to Agamben, along with the traces of gestures captured by Muybridge, will be retrieved and fully ‘on display’ on-screen.11
As this thesis has persistently argued, an exact description or typology of screens does not fit the postmedial condition. Instead, I have sought to offer a performative account of the becomings of screens in postmedia. I have offered ways to think about the agency of screens through the relations that specific practices materialise in an assemblage. I have drawn attention to where this agency is highlighted and foregrounded — in the artworks that multiply, experiment with and give full range to the gestures of postmedial screens. Screens gesture in ways that continue and/or rearrange mediality. By staying with screens but moving beyond their display, we can attend not simply to the postmedia condition but rather to the conditions of postmedia that allow such movements and open-ended exchanges between and across widely differing media.
10 Yu Zhang, Tao Gu, Chu Luo, Vassilis Kostakos and Aruna Seneviratne, “FinDroidHR: Smartwatch Gesture Input with Optical Heartrate Monitor,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, article 56, 2018, 2, https://doi.org//10.1145/3191788. Hongyi Wen, Julian Ramos Rojas and Anind Dey. (2016). “Serendipity: Finger Gesture Recognition using an Off-the-Shelf Smartwatch,” Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 2016, 3847–3851, https://doi.org//10.1145/2858036.2858466. 11 Agamben, “Notes on Gesture,” 56.
294
BIBLIOGRAPHY
Adajania, Nancy. “Darkness is What Light Will Never Be: Shilpa Gupta’s Experiments with Truth.” In Shilpa Gupta, edited by Nancy Adajania. Munich: Prestel Verlag, 2010.
Adobe User Guide. “Keying Introduction and Resources.” Accessed 2 February 2021, https://helpx.adobe.com/au/after-effects/using/keying.html.
Afful, Adwoa A., and Rose Ricciardelli. “Shaping the Online Fat Acceptance Movement: Talking about Body Image and Beauty Standards.” Journal of Gender Studies 24, no. 4 (2015): 453-472. https://doi.org//10.1080/09589236.2015.1028523.
Agamben, Giorgio. “Kommerell, or On Gesture.” [1991] In Potentialities. Collected Essays in Philosophy, translated by Daniel Heller-Roazen, 77-85. Stanford, CA: Stanford University Press, 2000.
———. “Notes on Gesture.” In Means Without End: Notes on Politics, translated by Vincenzo Binetti and Cesare Casarino, 49-60. Minneapolis: University of Minnesota Press, 2000.
———. “Difference and Repetition: On Guy Debord’s films.” [1995] In Guy Debord and the Situationist International: Texts and Documents, edited by T. McDonough, 313-319. Cambridge, MA: MIT Press, 2002.
———. “The Author as Gesture.” In Profanations, translated by Jeff Fort, 61-72. New York: Zone Books, 2007.
———. Homo Sacer: Sovereign Power and Bare Life. Stanford, CA: Stanford University Press, 1998.
———. State of Exception. Translated by Kevin Attell. Chicago: University of Chicago Press, 2005.
Agustinho, Daniela. “Chroma Key Dreams: Algorithmic Visibility, Fleshy Images and Scenes of Recognition.” Philosophy of Photography 9, no. 2 (2018): 131-55. https://doi.org//10.1386/pop.9.2.131_1.
Ahuja, Neel. “Post-mortem on Race and Control,” in Control Culture: Foucault and Deleuze after Discipline, edited by Frida Beckman, 34-43. Edinburgh: Edinburgh University Press, 2018.
Alaimo, Stacy. Bodily Natures: Science, Environment, and the Material Self. Bloomington: Indiana University Press, 2010.
295
Alvaro, Sandra. “Complexity, Multi-perspectivism and Tracking: A Brief History of the Meaning of Image, from the Postmedia to the Postdigital.” Technoetic Art: A Journal of Speculative Research 11, no. 3 (2013): 199-207.
Anastasi, Rob, Nick Tandavanitj, Martin Flintham, Andy Crabtree, Matt Adams, Ju Row-Farr, Jamie Iddon. "Can you see me now? a citywide mixed-reality gaming experience." Proceedings of the Ubi-Comp (2002).
Angwin, Julia, Jeff Larson, Surya Mattu, and Lauren Kirchner. “Machine Bias: There’s Software Used Across the Country to Predict Future Criminals. And It’s Biased against Blacks.” ProPublica, May 23 (2016): 139-59. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal- sentencing.
Ann Doane, Mary. “Temporality, Storage, Legibility: Freud, Marey, and the Cinema.” Critical Inquiry 22, no. 2 (1996): 313–343.
Apprich, Clemens, Josephine Berry Slater, Anthony Iles and Oliver Lerone Schultz, ed. Provocative Alloys: A Post-Media Anthology. London, Lüneburg: Mute and Post-Media Lab, 2013.
Apprich, Clemens, Wendy Hui Kyong Chun, Florian Cramer and Hito Steyerl. Pattern Discrimination. Lüneburg: Meson Press, 2018.
Arthurs, Jane. “Sex and the City and Consumer Culture: Remediating Postfeminist Drama.” Feminist Media Studies 3, no. 1 (2003): 83–98.
Ash, James. “Emerging Spatialities of the Screen: Video Games and the Reconfiguration of Spatial Awareness.” Environment and Planning A 41 (2009): 2105-24.
Audouze, Françoise. “Leroi-Gourhan, a Philosopher of Technique and Evolution.” Journal of Archaeological Research 10, no. 4 (2002): 277-306.
Aumont, Jacques. Montage Eisenstein. Translated by Lee Hildreth, Constance Penley and Andrew Ross. London: BFI Publishing and Indiana University Press, 1987.
Avezzù, Giorgio. “The Deep Time of the Screen, and its Forgotten Etymology.” Journal of Aesthetics & Culture 11, no. 1 (2019). https://doi.org/10.1080/20004214.2019.1610296.
Balázs, Béla. Béla Balázs: Early Film Theory. Edited by Erica Carter. Translated by Rodney Livingstone. Oxford: Berghahn Books, 2010.
Balme, Christopher B. “Intermediality: Rethinking the Relationship between Theatre and Media.” TheWis+, no. 4 (2006): 1-18. https://epub.ub.uni- muenchen.de/13098/1/Balme_13098.pdf.
Barad, Karen. “Agential Realism. Feminist Interventions in Understanding Scientific Practices.” In The Science Reader, edited by Mario Biagioli, 1 – 11. New York, London: Routledge, 1999.
296
———. “Posthumanist Performativity: Toward an Understanding of How Matter Comes to Matter.” Signs 28, no. 3 (2003): 801-831.
———. Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning. Durham, NC: Duke University Press, 2007.
———. “Interview with Karen Barad.” Rick Dolphijn and Iris van der Tuin. In New Materialisms: Interviews and Cartographies, edited by Rick Dolphijn and Iris van der Tuin, 48-70. London: Open Humanities Press, 2012.
———. “Karen Barad: Intra-actions.” Interview by Adam Kleinman. Mousse 34 (2012): 76-81.
Barandiaran, Xabier, Ezequiel Di Paolo, and Marieke Rohde. “Defining Agency: Individuality, Normativity, Asymmetry, and Spatio-Temporality in Action.” Adaptive Behavior 17, no. 5 (2009): 367–386.
Baranski, Andrzej. Anna-Katrina Shedletsky, Kuldeep P. Lonkar, Serhan Isikman, Stephen Brian Lynch, Colin M. Ely, Christopher Werner, Erik de Jong, and Samuel B. Weiss. Motion and gesture input from a wearable device. US Patent: US20160091980A1, filed February 6, 2015, and issued, November 26, https://patents.google.com/patent/US20160091980A1/en.
Barasch, Moshe. “Gesture.” Grove Art Online, 2003. https://doi.org/10.1093/gao/9781884446054.article.T031819.
Barker, Timothy Scott. Time and the Digital: Connecting Technology, Aesthetics, and a Process Philosophy of Time. Chicago: University of Chicago Press, 2012.
Barr, Charles. “Cinemascope: Before and After.” Film Quarterly 16, no. 4 (1963): 4–24.
Barthes, Roland. The Responsibility of Forms: Critical Essays on Music, Art and Representation. Oxford: Blackwell, 1986.
Baudry, Jean-Louis. “Ideological Effects of the Basic Cinematographic Apparatus.” Film Quarterly 28, no. 2 (1974): 39–47.
Belting, Hans. “Image, Medium, Body: A New Approach to Iconology.” Critical Inquiry 31 (2005): 302-319.
———. An Anthropology of Images: Picture, Medium, Body. Translated by Thomas Dunlap. Princeton: Princeton University Press, 2011.
Belton, John. Widescreen Cinema. Cambridge, MA: Harvard University Press, 1992.
Ben-Ezra, Moshe. “Segmentation with Invisible Keying Signal.” IEEE Computer Society Conference on Computer Vision and Pattern Recognition 1(2000): 32-37. https://doi.org//10.1109/CVPR.2000.855795.
297
Benford, Steve, Andy Crabtree, Martin Flintham, Adam Drozd, Rob Anastasi, Mark Paxton, Nick Tandavanitj, Matt Adams, and Ju Row-Farr. “Can You See Me Now?” ACM Transactions on Computer-Human Interaction 13, no. 1 (2006): 100-133. https://doi.org/10.1145/1143518.1143522.
Benjamin, Walter. Illuminations. Edited by Hannah Arendt. Translated by Harry Zohn. New York: Schocken, 1968.
———. The Arcades Project. Translated by Howard Eiland and Kevin McLaughlin. Cambridge, London: Harvard University Press, 1999.
Bennett, Jane. Vibrant Matter: A Political Ecology of Things. Durham, NC: Duke University Press, 2010.
Bennett, Jill. “Aesthetics of Intermediality.” In Art History 30, 3 (2007): 432-450.
Berger, John. Ways of Seeing. London: Penguin Books, 1972.
Bergson, Henri. Henri Bergson: Key Writings. Edited by Keith Ansell-Pearson and John Ó Maoilearca. New York: Continuum, 2002.
———. Matter and Memory. Translated by Nancy Margaret Paul and W. Scott Palmer. London: George Allen and Unwin, 1911.
———. Time and Free Will: An Essay on the Immediate Data of Consciousness. Translated by F. L. Pogson (London: Allen & Unwin, 1950).
Berry, Chris. “The New Gestural Cinema. Yang Fudong and the Gallery Film.” Film Quarterly 67, no. 3 (2014): 17-29.
Bitti, Pio Enrico, and Isabella Poggi, “Symbolic Nonverbal Behavior: Talking Through Gestures.” In Fundamentals of Nonverbal Behavior, edited by R.S. Feldman and B. Rime, 433–456. New York: Cambridge University Press, 1991.
Bizzocchi, Jim. “The Fragmented Frame: The Poetics of the Split-Screen.” 2009, 2, http://web.mit.edu/comm-forum/legacy/mit6/papers/Bizzocchi.pdf.
Blast Theory, Can You See Me Now?. Location based mixed media. 2003. https://www.blasttheory.co.uk/projects/can-you-see-me-now/.
Bolter, Jay David, and Richard Grusin, “Immediacy, Hypermediacy, and Remediation.” in Remediation: Understanding New Media, ed. Jay David Bolter and Richard Grusin, 20–51. Cambridge, MA: MIT Press, 2000.
———. Remediation: Understanding New Media. Cambridge, Mass.: MIT Press, 2000.
Bonitzer, Pascal. “Off-Screen Space.” In Cahiers du Cinema Vol. 3, 1969–1972: The Politics of Representation, edited by Nick Browne. London: Routledge, 1990. [Originally published as Pascal Bonitzer, “Hors-Champ: Un e ́space en de ́faut.” Cahiers du cinema, nos. 234–5 (1971–72), 15–26.]
298
Bookchin, Natalie, and Blake Stimson. “Out in Public: Natalie Bookchin in Conversation with Blake Stimson.” In Video Vortex Reader II: Moving Images Beyond YouTube, edited by Geert Lovink and Rachel Somers Miles, 306-318. Amsterdam: Institute of Network Cultures, 2011. https://rhizome.org/editorial/2011/mar.
———. LACE gallery video discussing Now he’s out in public. “Practice Session: Natalie Bookchin” LACE, Los Angeles, 2016, YouTube, accessed February 8, 2021, https://www.youtube.com/watch?app=desktop&v=rWi9UQZZCa8&list=UUKVWG nDf1Vm2tuB1awaj5Qw&index=11&t=0s.
———. “Re-editing the American Crisis: Natalie Bookchin’s Montaged Portraits.” Interview by Angela Maiello. From DVD booklet of Portraits of America. Accessed 2 February 2021. https://bookchin.net/wp-content/uploads/2018/06/Reediting-the- American-Crisis.pdf.
———. Portraits of the Multitude. Barcelona: La Virreina Centre de la Image, 2018.
Bordwell, David. On the History of Film Style. Cambridge, Mass.: Harvard University Press, 1997.
Bourdieu, Pierre. Outline of a Theory of Practice. Cambridge, UK: Cambridge University Press, 1977.
Bourriaud, Nicolas. The Radicant. New York: Lukas & Sternberg, 2009.
Boyd, Danah, and Kate Crawford. “Critical Questions for Big Data.” Information, Communication & Society 15, no. 5 (2012): 662–679. https://doi.org//10.1080/1369118X.2012.678878.
Boyle, Michael J. “The Legal and Ethical Implications of Drone Warfare,” International Journal of Human Rights 19, no. 2 (2015): 105–126. https://doi.org//10.1080/13642987.2014.991210.
Braidotti, Rosi. The Posthuman. Cambridge: Polity Press, 2013.
Bremner, Paul, Anthony G. Pipe, Chris Melhuish, Mike Fraser, and Sriram Subramanian. “The Effects of Robot-Performed Co-Verbal Gesture on Listener Behaviour.” Proceedings of 11th IEEE-RAS International Conference on Humanoid Robots 11 (2011): 458– 465.
Brinkmann, Ron. The Art and Science of Digital Compositing. San Diego: Morgan Kaufmann, 1999.
Browne, Simone. Black Matters: On the Surveillance of Blackness. Durham, NC: Duke University Press, 2015.
Bruhn, Jørgen. The Intermediality of Narrative Literature. Medialities Matter. London: Palgrave MacMillan, 2016.
299
Brunner, Christoph. “Affective Politics of Timing on Emergent Collectivity in Ragnar Kjartansson’s The Visitors.” In Timing of Affect: Epistemologies, Aesthetics and Politics, ed. Marie-Luise Angerer, Bernd Bösel and Michaela Ott, 255-262. Berlin: Diaphanes, 2014.
Bruno, Giuliana. Surface: Matters of Aesthetics, Materiality and Media. Chicago, University of Chicago, 2014.
Buchanan, Ian. “Assemblage Theory, or, the Future of an Illusion.” Deleuze Studies 11, no. 3 (2017): 457-74.
Buckley, Craig, Rüdiger Campe and Francesco Casetti, eds. Screen Genealogies: From Optical Device to Environmental Medium. Amsterdam: Amsterdam University Press, 2019. https://doi.org//10.5117/9789463729000.
Bulgakowa, Oksana, and Dietmar Hochmuth. The Factory of Gestures: Body Language in Film. DVD. Potemkin Press, 2008.
———. The Factory of Gestures: Body Language in Film. 7-channel installation, 2008.
Buolamwini, Joy. “Gender Shades: Intersectional Phenotypic and Demographic Evaluation of Face Datasets and Gender Classifiers.” Master diss, Massachusetts Institute of Technology, 2017.
Burch, Noël. Theory of Film Practice [1969]. Princeton, NJ, Princeton University Press: 1981.
Bureau of Investigative Journalism. “Drone Warfare.” Interactive statistics generator. https://www.thebureauinvestigates.com/projects/drone-war.
Butler, Alison. “A Deictic Turn: Space and Location in Contemporary Gallery Film and Video Installation.” Screen 51, no. 4 (2010): 305-23.
Butler, Judith. “Gender as Performance: An Interview with Judith Butler.” Radical Philosophy 67 (1994): 32–9.
———. Bodies That Matter: On the Discursive Limits of ‘Sex’. New York: Routledge, 1993.
Carroll, Noël. Theorising the Moving Image. New York: Cambridge University Press, 1996.
Casetti, Francesco. “Back to the Motherland: The Film Theatre in the Postmedia Age.” Screen 52, no. 1 (2011): 1–12.
———. “What Is a Screen Nowadays?.” In Public Space, Media Space, edited by Chris Berry, Janet Harbord, and Rachel O. Moore, 16–40. New York: Palgrave Macmillan, 2013.
Castle, Terry. "Phantasmagoria: Spectral Technology and the Metaphorics of Modern Reverie." Critical Inquiry 15, no. 1 (1988): 26-61. http://www.jstor.org/stable/1343603.
300
Cavell, Stanley. The World Viewed: The Reflections on the Ontology of Cinema. New York: Viking Press, 1971.
Cetina, Karin Knorr, and Urs Bruegger. “Traders’ Engagement with Markets: a Postsocial Relationship.” Theory, Culture & Society 19, no. 5–6 (2002): 161–85.
Chang, Yu-Ching, and Chi-Min Hsieh. “Filmic Framing in Video Games: a Comparative Analysis of Screen Space Design.” Multimedia Tools and Applications 77, no. 6 (2018): 6531–6554. https://doi.org/10.1007/s11042-017-4564-6.
Chatzichristodoulou, Maria. “Blast Theory.” In British Theatre Companies 1995–2014: Mind the Gap, Kneehigh Theatre, Suspect Culture, Stan’s Cafe, Blast Theory, Punchdrunk, edited by Liz Tomlin, 231–254. London: Bloomsbury, 2015.
———. “When Presence & Absence Turn into Pattern & Randomness: Can You See Me Now?.” Leonardo Electronic Almanac 16, no. 4–5 (2009). https://www.leoalmanac.org/wp-content/uploads/2012/09/06_chatzichristodoulou.pdf.
Chippendale, Paul. “Towards Automatic Body Language Annotation.” Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition (FGR06), 487–92. Southampton, UK (2006). http://doi.org//10.1109/FGR.2006.105.
Chun, Wendy. “Introduction: Race and/as Technology; or, How to Do Things to Race.” Camera Obscura 24, no. 1 (70) (2009): 7-35.
Clüver, Claus. “Intermedia and Interart Studies.” In Changing Borders: Contemporary Positions in Intermediality. Edited by Jens Arvidson, Mikael Askander, Jørgen Bruhn, Heidrun Führer, 19-37. Lund: Intermedia Studies Press, 2007.
Colebrook, Claire. Gilles Deleuze. London: Routledge, 2001.
Coleman, Beth. “Race as Technology.” Camera Obscura 24, no. 1 (70) (2009): 177-207.
Coleman, Rebecca. “Pricing Bodies: A Feminist New Materialist Approach to the Relations Between the Economic and Socio-cultural.” Distinktion 19, no.2 (2018): 230–248. https://doi.org//10.1080/1600910X.2018.1495658.
Colman, Felicity. “Agency.” New Materialism Almanac, May 17, 2018. http://newmaterialism.eu/almanac/a/agency.html.
———. Deleuze and Cinema: The Film Concepts. London: Bloomsbury, 2011.
Colomina, Beatriz. “Enclosed by Images: The Eameses Multimedia Architecture.” in Grey Room 2 (2001): 6–29.
Coole, Diana, and Samantha Frost, ed, New Materialisms: Ontology, Agency, and Politics. Durham, NC: Duke University Press, 2010.
301
Coole, Diana. “Agentic Capacities and Capacious Historical Materialism: Thinking with New Materialisms in the Political Sciences.” Millennium 41, no. 3 (June 2013): 451– 69. https://doi.org/10.1177/0305829813481006.
Cossar, Harper. “The Shape of New Media: Screen Space, Aspect Ratios, and Digitextuality.” Journal of Film & Video 61, no. 4 (2009): 3-16.
Costa, Sandra, Hagen Lehmann, Kerstin Dautenhahn, Ben Robins, and Filomena Soares. “Using a Humanoid Robot to Elicit Body Awareness and Appropriate Physical Interaction in Children with Autism.” International Journal of Social Robotics 7, no. 2 (2015): 265–278.
Crary, Jonathan. Techniques of the Observer: On Visions and Modernity in the Nineteenth Century. Cambridge, MA: MIT Press, 1992.
Crawford, Kate. “A.I.’s White Guy Problem.” New York Times, June 25, 2016. https://www.nytimes.com/2016/06/26/opinion/sunday/artificial-intelligences-white- guy-problem.html.
Cresswell, Tim. On the Move. Mobility in the Modern Western World. New York: Routledge, 2006.
Cubitt, Sean. “Current Screens.” In Imagery in the 21st Century, edited by Oliver Grau with Thomas Veigl, 21-35. Cambridge, MA: MIT Press, 2011.
———. “Large Screens, Third Screens, Virtuality, and Innovation.” In The Oxford Handbook of Sound and Image in Digital Media, edited by Carol Vernallis, Amy Herzog and John Richardson, 77-8. Oxford: Oxford University Press, 2013. https://doi.org//10.1093/oxfordhb/9780199757640.013.023.
———. The Practice of Light: A Genealogy of Visual Technologies from Prints to Pixels. Cambridge, MA: MIT Press, 2014.
Daguerre, Louis-Jacques-Mandé. “Description of the Process of Painting and Lighting in…Pictures of the Diorama (1839).” In The Screen Media Reader: Culture, Theory, Practice, edited by Stephen Monteiro, 163–166. New York: Bloomsbury, 2016.
DeCordova, Richard. “From Lumière to Pathé: The Breakup of Perspectival Space.” In Early Cinema: Space, Frame, Narrative, edited by Thomas Elsaesser, 76-85. London: BFI, 1990).
DeLanda, Manuel. Assemblage Theory. Edinburgh: Edinburgh University Press, 2016.
Deleuze, Gilles, and Claire Parnet. Dialogues. London: Athlone Press, 1987.
Deleuze, Gilles, and Félix Guattari. A Thousand Plateaus: Capitalism and Schizophrenia. Translated by Brian Massumi. London: Bloomsbury, 2017.
302
Deleuze, Gilles. “The Brain Is the Screen.” Translated Marie Therese Guirgis. In The Brain Is the Screen: Deleuze and the Philosophy of Cinema, edited by Gregory Flaxman, 365-369. Minneapolis: University of Minnesota Press, 2000.
———. Bergsonism, translated by Hugh Tomlinson and Barbara Habberjam. New York: Zone Books, 1991.
———. Cinema 1: The Movement-Image. Translated by Hugh Tomlinson and Barbara Habberjam. Minneapolis: University of Minnesota Press, 1986.
———. Cinema 2: The Time-Image. Translated by Hugh Tomlinson and Robert Galeta. Minneapolis: University of Minnesota Press, 1989.
———. Francis Bacon: The Logic of Sensation. London: Continuum, 2004.
Design Boom, “Simon Weckert Causes a ‘Virtual Traffic Jam’ with 99 Smartphones and a Handcart,” March 3, 2020, https://www.designboom.com/art/simon-weckert-virtual- traffic-jam-99-smartphones-handcart-02-03-2020.
Didi-Huberman, Georges. The Surviving Image: Phantoms of Time and Time of Phantoms: Aby Warburg’s History of Art. Translated by Harvey L. Mendelson. University Park, PA: Pennsylvania State University Press, 2017.
———. ATLAS: How to Carry the World on One’s Back?. Madrid: Museo Reina Sofía, November 2010–March 2011.
Dijck, José van. “Datafication, Dataism and Dataveillance: Big Data between Scientific Paradigm and Ideology.” Surveillance and Society 12, no. 2 (2014): 197–208.
Doane, Mary Ann. “The Indexical and the Concept of Medium Specificity.” Differences 18, no. 1 (2007): 128–52. https://doi.org/10.1215/10407391-2006-025.
———. “The Indexical and the Concept of Medium Specificity.” Differences 18, no. 1 (2007): 128–52. https://doi.org/10.1215/10407391-2006-025.
Dyer, Richard. White. London, New York: Routledge, 2002
Eder, Sebastian. “This Reality Does Not Exist,” Frankfurter Allgemeine, March 2, 2020, https://www.faz.net/aktuell/stil/trends-nischen/berliner-kuenstler-simon-weckert-hat- google-maps-ausgetrickst-16615421.html.
Efron, David. Gesture and Environment. The Hague: Mouton, 1972.
Eibl-Eibesfeldt, Irenäus. “Similarities and Differences Between Cultures in Expressive Movements.” In Nonverbal Communication, edited by R.A. Hinde, 297–314. London: Cambridge University Press, London, 1972.
Eikenes, Jon Olav H., and Andrew Morrison. “Navimation: Exploring Time, Space & Motion in the Design of Screen-based Interfaces.” International Journal of Design 4, no 1 (2010): 1–16.
303
Eisenstein, Sergei M. Selected Works, Volume 1 Writings, 1922-34. Translated and edited by Richard Taylor. London: BFI Publishing and Indiana University Press, 1988.
———. Selected Works, Volume II, Towards a Theory of Montage. Edited by Michael Glenny and Richard Taylor. Translated by Michael Glenny. London: BFI Publishing and Indiana University Press, 1991.
———. Neravnodushnaia priroda. Tom 1. Chuvstvo kino. Edited by Naum Kleiman. Moskva, 2004.
Elleström, Lars. “The Modalities of Media: A Model for Understanding Intermedial Relations.” In Media Borders, Multimodality and Intermediality, edited by Lars Elleström, 11-48. London: Palgrave Macmillan, 2010.
Elsaesser, Thomas. “Simulation and the Labour of Invisibility: Harun Farocki’s Life Manuals.” Animation 12, no. 3 (2017): 214-29.
———. “The ‘Return’ of 3-D: On Some of the Logics and Genealogies of the Image in the Twenty-First Century.” Critical Inquiry 39 (2013): 217–46.
Farman, Jason. Mobile Interface Theory: Embodied Space and Locative Media. London, New York: Routledge, 2012.
Farocki, Harun. “Harun Farocki Interview | Galerie Thaddeus Ropac | Paris Maris, 2014.” January 22, 2014. https://www.youtube.com/watch?v=c8YKKs0Pcx8.
Fazi, M. Beatrice, and Matthew Fuller. “Computational Aesthetics.” In Companion to Digital Art, edited by Christiane Paul, 281-96. Hoboken, NJ: John Wiley & Sons, 2016.
Ferreira da Silva, Denise. “1 (life) ÷ 0 (blackness) = ∞ − ∞ or ∞ / ∞: On Matter Beyond the Equation of Value.” e-flux 79 (2017). https://www.e-flux.com/journal/79/94686/1-life- 0-blackness-or-on-matter-beyond-the-equation-of-value.
———. “In the Raw.” e-flux 93, (2018). https://www.e-flux.com/journal/93/215795/in-the- raw.
———. “On Difference without Separability.” Incerteza Viva, 32 Bienal de São Paulo. São Paulo: Fundacion Bienal de São Paulo, 2016.
Flaxman, Gregory. “Out of Field: The Future of Film Studies.” Angelaki 17, no. 4 (2012): 119-137.
Flusser, Vilém. Gestures. Translated by Nancy Ann Roth. Minneapolis: University of Minnesota, 2014.
Forrest, Nicholas. “Interview: John Gerrard on his ‘Slippery’ Sims at Thomas Dane Gallery.” Blouin Artinfo, February 16, 2015. http://uk.blouinartinfo.com/news/story/1103413/interview-john-gerrard-on-his- slippery-sims-at-thomas-dane (site discontinued).
304
Forster, Kurt. “Aby Warburg: His Study of Ritual and Art on Two Continents.” October 77 (Summer 1996): 5-24.
Foster, Jeff. The Green Screen Handbook, Real-World Production Techniques. New York: Routledge, 2010.
Foucault, Michel. Discipline and Punish. The Birth of the Prison. Translated by Alan Sheridan. New York: Vintage Books, 1995.
Fowler, Catherine. “Into the Light: Re-Considering Off-Frame and Off-Screen Space in Gallery Films.” New Review of Film and Television Studies 6, no. 3 (2008): 253–267. http://doi.org//10.1080/17400300802418578.
Frank, Allegra. “Everything We Know about Nintendo Switch (Update).” Polygon, March 2, 2017. https://www.polygon.com/2017/1/13/14241898/nintendo-switch-details.
Fried. Michael. Art and Objecthood: Essays and Reviews. [1966] Chicago: University of Chicago Press, 1998.
Friedberg, Anne. The Virtual Window, From Alberti to Microsoft. Cambridge, MA: MIT Press, 2006.
Frost, Samantha. “The Implications of the New Materialisms for Feminist Epistemology.” In Feminist Epistemology and Philosophy of Science: Power in Knowledge, edited by Heidi E. Grasswick, 69–83. Dordrecht; New York: Springer, 2011.
Fuller, Matthew. Media Ecologies: Materialist Energies in Art and Technoculture. Cambridge, MA: MIT Press, 2007.
Gadney, Guy. “Broadband Futures.” LAMP, Australian Film Television and Radio School, September 7, 2005, podcast. https://storylabs.us/lamp/tag/guy-gadney-guy/.
Galloway, Alexander R. Gaming: Essays on Algorithmic Cultures. Minneapolis: University of Minnesota Press, 2006.
———. The Interface Effect. London: Polity, 2012.
Gawne, Luren. “Emoji Aren’t Ruining Language: They’re A Natural Substitute for Gesture ���.” The Conversation, July 5 2019. https://theconversation.com/emoji- arent-ruining-language-theyre-a-natural-substitute-for-gesture-118689.
Gernsheim, Helmut. L.J.M. Daguerre: The History of the Diorama and the Daguerrotype. New York: Dover, 1968.
Gerrard, John. Interview with Simon Groom at the Scottish National Gallery of Modern Art. In John Gerrard. Madrid: Ivorypress, 2011.
Gibson, James J. The Ecological Approach to Visual Perception. New York: Psychology Press, 1979.
305
Gidal, Peter. “The Theory and Definition of Structural/Materialist Film.” In Structural Film Anthology, edited by Peter Gidal, 1-21. London: British Film Institute, 1978.
Glissant, Édouard. Poetics of Relation. Translated by Betsy Wing. Ann Arbor: University of Michigan Press, 2010.
Goriunova, Olga, and Alexei Shulgin. “Glitch.” In Software Studies: A Lexicon, edited by Matthew Fuller, 110-119. Cambridge, MA: MIT Press, 2008.
Greenberg, Clement. “Avant-Garde and Kitsch.” Partisan Review 6 (Fall 1939): 34-49.
———. “Towards a Newer Laocoön.” Partisan Review 7, no. 4 (1940): 296–310.
Greenhalgh, Cathy. “Skin Tone and Faces: Cinematography Pedagogy which Foregrounds Inclusivity and Diversity in Teaching Lighting.” Cinematography in Progress, July 2, 2020, https://cinematographyinprogress.com.
Gregg, Melissa, and Gregory J. Seigworth, ed. The Affect Theory Reader. Durham, NC: Duke University Press, 2010.
———. “An Inventory of Shimmers.” In The Affect Theory Reader, edited by Melissa Gregg and Gregory J. Seigworth, 1-25. Durham, NC London: Duke University Press, 2010.
Gregory, Thomas. “Potential Lives, Impossible Deaths.” International Feminist Journal of Politics 14, no. 3 (2012): 327–347. https://doi.org//10.1080/14616742.2012.659851.
Griffiths, Alison. Shivers Down Your Spine. Cinema, Museum and the Immersive View. New York: Columbia University Press: 2008.
Grusin, Richard. “Radical Mediation.” Critical Inquiry 42 (2015): 124-148.
———. The Nonhuman Turn. Minneapolis: University of Minnesota Press, 2015.
Guattari, Félix. Soft Subversions: Texts and Interviews 1977–1985. Edited by Sylvere Lotringer. Translated by Chet Wiener and Emily Wittman, 291–306. Los Angeles: Semiotext(e), 2009.
Gunning, Tom. “The Cinema of Attraction: Early Film, Its Spectator and the Avant- Garde,” Wide Angle 8, no. 3-4 (1985): 63-70.
Hamilakis, Yannis. “Sensorial Assemblages: Affect, Memory and Temporality in Assemblage Thinking.” Cambridge Archaeological Journal 27, no. 1 (2017): 169-182.
Hankerson, David, Andrea R. Marshall, Jennifer Booker, Houda El Mimouni, Imani Walker and Jennifer A. Rode. “Does Technology Have Race?.” In CHI EA’16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, 473-486. San Jose, USA 2016. https://doi.org//10.1145/2851581.2892578.
306
Hansen, Miriam. “Early Cinema, Late Cinema: Transformations of the Public Sphere.” In Viewing Positions, Ways of Seeing Film, edited by Linda Williams, 134-52. New Brunswick, NJ: Rutgers University Press, 1997.
———. Babel and Babylon. London, Cambridge: Harvard University Press, 1991.
Haraway, Donna. "Situated Knowledges: The Science Question in Feminism as a Site of Discourse on the Privilege of Partial Perspective." Feminist Studies 14, no. 3 (1988): 575-99.
———. “Manifesto for Cyborgs: Science, Technology, and Socialist Feminism in the 1980s." Socialist Review 80 (1985): 65-108.
Harris, Ella. "Exploring pop-up cinema and the city: Deleuzian encounters with secret cinema’s pop-up screening of The Third Man." Journal of Urban Cultural Studies 3, no. 1 (2016): 113-132.
Harrison, Simon. “The Production Line as a Context for Low Metaphoricity: Exploring Links Between Gestures, Iconicity, and Artefacts on a Factory Shop Floor.” In Metaphor in Specialist Discourse, edited by J. Berenike Herrmann and Tony Berber Sardinha, 131–61. Amsterdam: John Benjamins Publishing, 2015.
Hartman, Sadiyya V. Scenes of Subjection: Terror, Slavery, and Self-Making in Nineteenth- Century America. New York; Oxford: Oxford University Press, 1997.
Hemment, Drew. “Locative Arts.” Leonardo 39, no. 4 (2006): 348-355.
Henson, Robert. Weather on the Air. Boston, MA: American Meteorological Society, 2010.
Higgins, Dick with Hannah Higgins. “Intermedia.” [1965, 1981] Leonardo 34, no. 1 (2001): 49-54.
Hirakawa, Keigo, and Patrick J. Wolfe. "Spatio-Spectral Color Filter Array Design for Optimal Image Recovery." IEEE Transactions on Image Processing 17, no.10 (Oct. 2008): 1876-1890, https://doi.org//10.1109/TIP.2008.2002164.
Hird, Myra J. “Review: Feminist Engagements with Matter.” Feminist Studies 35, no. 2 (2009): 329–346.
Hofer, Sonya. “Screenness in Experimental Electronica Performances.” Music and the Moving Image 10, no. 2 (2017): 16–32.
Hsu, Chia-Wei. “Marshal Tie Jia - Turtle Island.” Video excerpt, accessed 2 February 2021. http://hsuchiawei01.blogspot.com/search/label/Marshal%20Tie%20Jia%20- %20Turtle%20Island.
Camberwell Studio Production. Difference between blue screen and green screen. https://www.camberwellstudios.co.uk/blog/film-production-tips/whats-the-difference- between-green-screen-vs-a-blue-screen.html.
307
Gear Shift TV. Difference between blue screen and green screen. https://www.gearshift.tv/faqs/what-s-the-difference-between-a-green-screen-and-a- blue-screen-.cfm.
Huang, Chien-Ming, and Bilge Mutlu. “Modeling and Evaluating Narrative Gestures for Humanlike Robots.” Robotics: Science and Systems (2013): 57-64. https://doi.org//10.15607/RSS.2013.IX.026.
Huhtamo, Erkki. “Elements of Screenology: Toward an Archaeology of the Screen.” ICONICS 7 (2004): 31–82.
———. “Screen Tests: Why Do We Need an Archaeology of the Screen?” Cinema Journal 51, no. 2 (2012): 144–48, https://doi.org//10.1353/cj.2012.0011.
———. “Screenology; or, Media Archaeology of the Screen.” In The Screen Media Reader: Culture, Theory, Practice, edited by Stephen Monteiro, 77 – 111. New York: Bloomsbury, 2016.
———. “The Four Practices? Challenges for an Archaeology of the Screen.” In Screens: From Materiality to Spectatorship — a Historical and Theoretical Reassessment, edited by Dominique Chateau and José Moure, 116–124. Amsterdam: Amsterdam University Press, 2016.
Hutchison, Andrew. “Making the Water Move: Techno-Historic Limits in the Game Aesthetics of Myst and Doom.” Game Studies 8, no. 1 (2008). http://gamestudies.org/0801/articles/hutch.
Ingold, Tim. The Life of Lines. London: Routledge, 2015.
Introna, Lucas D., and Fernando M. Ilharco. “On the Meaning of Screens: Towards a Phenomenological Account of Screennes.” Human Studies 29, no. 1 (2006): 57–76.
———. “The Ontological Screening of Contemporary Life: A Phenomenological Analysis of Screens.” European Journal of Information Systems 13, no. 3 (2004): 221–234. https://doi.org//10.1057/palgrave.ejis.3000503.
Iurato, Giuseppe. “On the Historical Evolution of Gyroscopic Instrumentation: A Very Brief Account.” HAL archives, 2015. https://hal.archives-ouvertes.fr/hal-01136829.
Jenkins, Henry. Convergence Culture: Where Old and New Media Collide. New York, London: New York University Press, 2006.
Johnson, Christopher D. “About the Mnemosyne Atlas.” Cornell University Library. https://warburg.library.cornell.edu/about.
———. Memory, Metaphor, and Aby Warburg’s Atlas of Images. Ithaca, New York: Cornell University Press, 2012.
Karam, Maria, and M.C. Schraefel. “A Taxonomy of Gestures in Human Computer Interactions.” Technical Report ECSTR-IAM05-009, Electronics and Computer
308
Science, University of Southampton, 2005. http://eprints.soton.ac.uk/id/eprint/261149.
Kashmere, Brett. “Underground Film, Into the Light: Two Sides of the Projected Image in American Art, 1945-1975.” Synoptique 8, (2008). http://www.synoptique.ca/core/en/articles/kashmere_underground.
Kendon, Adam. Gesture: Visible Action as Utterance. Cambridge, MA: Cambridge University Press, 2004.
Kendrick, James. “Aspect Ratios and Joe Six-Packs: Home Theater Enthusiasts Battle to Legitimize the DVD Experience.” Velvet Light Trap 56 (2005): 58–70.
King, Geoff, and Tanya Krzywinska. “Film Studies and Digital Games.” In Understanding Digital Games, edited by Jo Bryce and Jason Rutter, 112-128. London: Sage, 2006.
Kingston, John D., Bruno D. Marino, and Andrew Hill. "Isotopic Evidence for Neogene Hominid Paleoenvironments in the Kenya Rift Valley." Science 264, no. 5161 (1994): 955-59. http://www.jstor.org/stable/2883754.
Kitchin, Rob. The Data Revolution: Big Data, Open Data, Data Infrastructures and Their Consequences. London: Sage, 2014.
———. ‘A Voyage on the North Sea’: Art in the Age of the Post-Medium Condition. London: Thames & Hudson, 1999.
Krauss, Rosalind. “Reinventing the Medium.” Critical Inquiry 25, no. 2 (1999): 289–305.
———. Under Blue Cup. Cambridge MA: MIT Press, 2011.
Kristeva, Julia. “Gesture: Practice or Communication?.” [1968] Translated by Jonathan Benthall. In The Body Reader: Social Aspects of the Human Body, edited by Ted Polhemus. Harmondsworth: Penguin Books, 1978.
Krueger, Myron W. Thomas Gionfriddo and Katrin Hinrichsen. “VIDEOPLACE—An Artificial Reality.” ACM SIGCHI Bulletin 16, no. 4 (1985): 35–40.
LaFrance, Marianne, and Clara Mayo, “Cultural Aspects of Nonverbal Communication.” International Journal of Intercultural Relations 2, no. 1 (1978): 71–89.
Langer, Susanne. Feeling and Form: A Theory of Art. New York: Charles Scribner’s Sons, 1953.
Langford, Michelle. Allegorical Images: Tableau, Time and Gesture in the Cinema of Werner Schroeter. Bristol: Intellect, 2006.
Latif, Nadia. “It’s Lit! How Film Finally Learned to Light Black Skin” The Guardian, September 22, 2017. https://www.theguardian.com/film/2017/sep/21/its-lit-how-film- finally-learned-how-to-light-black-skin.
309
Latour, Bruno. Politics of Nature: How to Bring the Sciences into Democracy. Translated by Catherine Porter. Cambridge: Harvard University Press, 2004.
Laurel, Brenda. Computer as Theatre. Reading, Mass.: Addison-Wesley Publications, 1991.
Lauretis, Teresa de, and Stephen Heath, eds., The Cinematic Apparatus. New York: St Martin’s Press, 1980.
Lazzarato, Maurizio. Videophilosophy: The Perception of Time in Post-Fordism. Edited and translated by Jay Hetrick. New York: Columbia University Press, 2019.
Lebrov, Andrey. “Greenscreen vs Bluescreen: When and Why.” YouTube, September 28, 2019, https://www.youtube.com/watch?v=ava4Z3sJMLk&t=105s.
Leighton, Tanya, ed. Art and the Moving Image: A Critical Reader. London, Tate Publishing, 2008.
Lemke, Thomas. “An Alternative Model of Politics? Prospects and Problems of Jane Bennett’s Vital Materialism.” Theory, Culture & Society 35 no. 6 (2015): 31–54.
Leonardi, P.M. “Materiality, Sociomateriality, and Sociotechnical Systems: What Do these Terms Mean? How are they Different? Do We Need Them?” In Materiality and Organizing: Social Interaction in a Technological World, edited by P.M. Leonardi, B.A. Nardi, and J. Kallinikos, 25–48. Oxford: Oxford University Press, 2012.
Leroi-Gourhan, Andre. Gesture and Speech. [1964] Translated by Anna Bustock Berger. Cambridge MA, London: MIT Press, 1993.
Lewis, Sarah. “The Racial Bias Built into Photography.” New York Times, April 15, 2019. https://www.nytimes.com/2019/04/25/lens/sarah-lewis-racial-bias-photography.html.
Lindsay, Vachel. The Art of the Moving Picture. New York: Macmillan, 1915.
Lippit, Akira Mizuta. “Digesture: Gesture and Inscription in Experimental Cinema.” In Migrations of Gesture, edited by Carrie Noland and Sally Ann Ness, 113-132. Minneapolis: University of Minnesota Press, 2008.
———. “Phenomenologies of the Surface: Radiation-Body-Image.” Qui Parle 9, no. 2 (1996): 31–50.
Lissoni, Andrea. “Harun Farcoki — Cinema, Video Games and Finding the Detail, TateShots.” YouTube, April 15, 2016. https://www.youtube.com/watch?v=tERIscWmpSo.
Lombard, Matthew, and Theresa Ditton, “At the Heart of It All.” Journal of Computer Mediated Communication 3, no. 2 (1997). https://doi.org//10.1111/j.1083- 6101.1997.tb00072.x.
Lord, Susan, and Janine Marchessault, eds. Fluid Screens, Expanded Cinema. Toronto: Toronto: University of Toronto Press, 2007.
310
Lozano-Hemmer, Rafael. Body Movies, Relational Architecture 6. 2001-2008. https://www.lozano-hemmer.com/body_movies.php.
Lütticken, Sven. “From Media to Mythology.” New Left Review 6, no. Nov-Dec (2000): 134–43.
Manning, Erin. Relationscapes: Movement, Art, Philosophy. Cambridge, MA: MIT Press, 2009.
Manovich, Lev. “An Archeology of a Computer Screen.” Kunstforum International 132 (1995): 124–35.
———. “Post-Media Aesthetics.” In (Dis)LOCATIONS, edited by Astrid Sommer. Karlsruhe: ZKM, Centre for Art and Media/ Centre for Interactive Cinema Research, University of New South Wales, 2001. http://manovich.net/index.php/projects/post- media-aesthetics.
———. “Towards an Archaeology of the Computer Screen.” In Cain, Abel or Cable? The Screen Arts in the Digital Age, edited by Thomas Elsaesser and Kay Hoffmann, 27–43. Amsterdam: Amsterdam University Press, 1998.
———. The Language of New Media. Cambridge, MA: MIT Press, 2001.
Marks, Laura. Touch: Sensuous Theory and Multisensory Media. Minneapolis: University of Minnesota Press, 2002.
Martin, Cybel. “The Art of Lighting Dark Skin for Film and HD.” Shadow and Act, February 4, 2014. https://shadowandact.com/the-art-of-lighting-dark-skin-for-film- and-hd.
Massumi, Brian. “The Thinking-Feeling of What Happens.” Inflexions 1 (2008): 1–40.
———. “Translator’s Foreword: Pleasures of Philosophy” to A Thousand Plateaus: Capitalism and Schizophrenia by Gilles Deleuze and Félix Guattari, vii-xiv. Translated by Brian Massumi. London: Bloomsbury, 2017.
———. Parables for the Virtual: Movement, Affect, Sensation. Durham and London: Duke University Press, 2002.
———. Semblance and Event: Activist Philosophy and the Occurrent Arts. Cambridge, MA: MIT Press, 2011.
McCloud, Scott. Understanding Comics. Northampton, MA: Kitchen Sink Press, 1993.
McLuhan, Marshall. Understanding Media: The Extensions of Man. London: Routledge & Kegan Paul, 1964.
McNeill, David. Gesture and Thought. Chicago: University of Chicago Press, 2008.
———. Hand and Mind: What Gestures Reveal about Thought. Chicago: University of Chicago Press, 1992.
311
McQuire, Scott, Meredith Martin, and Sabine Niederer, ed. Urban Screens Reader. Amsterdam: Institute of Network Cultures, 2009.
Meldgaard, Morten. “Dimensions of the Out-of-field: A Contemporary Reading of Deleuzian Cinematics.” In TransVisuality: The Cultural Dimension of Visuality, vol. 1, edited by Tore Kristensen, Anders Michelsen, Frauke Wiegand, 174-184. Liverpool: Liverpool University Press, 2013.
Menkman, Rosa. “Glitch Studies Manifesto.” In Video Vortex Reader II: Moving Images Beyond YouTube, edited by Geert Lovink and Rachel Somers Miles, 336–347. Amsterdam: Institute of Network Cultures, 2011.
Merleau-Ponty, Maurice. Phenomenology of Perception. Translated by Colin Smith. London: Routledge, 2004.
Metz, Christian. Psychoanalysis and Cinema: The Imaginary Signifier. London: Macmillan, 1982.
Michaud, Philippe-Alain. Aby Warburg and the Image in Motion. Translated by Sophie Hawkes. New York: Zone Books, 2004.
Mitchell, W.J.T. “Screening Nature (and the Nature of the Screen).” New Review of Film and Television Studies 13, no. 3 (2015): 231-46. https://doi.org//10.1080/17400309.2015.1058141.
———. What do Pictures Want? The Lives and Loves of Images. Chicago: University of Chicago Press, 2004.
Miura, Shunsuke, and Shunsuke Kamijo. “GPS Error Correction by Multipath Adaptation.” International Journal of Intelligent Transportation System Research 13 (2015): 1–8. https://doi.org/10.1007/s13177-013-0073-9.
Mol, Annemarie. The Body Multiple: Ontology in Medical Practice. Durham, NC: Duke University Press, 2002.
Mondada, Lorenza. “Video Recording Practices and the Reflexive Constitution of the Interactional Order: Some Systematic Uses of the Split-Screen Technique.” Human Studies 32 (2009): 67–99. https://doi.org//10.1007/s10746-009-9110-8.
Mondloch, Kate. “Be Here (and There) Now: The Spatial Dynamics of Screen-Reliant Installation Art.” Art Journal 66, no. 3 (2007): 20–33.
———. Screens: Viewing Media Installation Art. Minneapolis: University of Minnesota Press, 2010.
Monteiro, Stephen, ed. Screen Media Reader: Culture, Theory, Practice. London: Bloomsbury, 2017.
———. “Introduction: Reflecting on the Screen.” In The Screen Media Reader: Culture, Theory, Practice, edited by Stephen Monteiro, 1-11. London: Bloomsbury, 2017.
312
Moore, Donald S., Jake Kosek and Anand Pandian. “The Cultural Politics of Race and Nature: Terrains of Power and Practice.” In Race, Nature and the Politics of Difference, edited by Donald S. Moore, Jake Kosek and Anand Pandian, 1-70. Durham, NC: Duke University Press, 2003).
Moten, Fred. A Poetics of the Undercommons. Butte, MT, New York: Sputnik and Fizzle, 2016.
Mujinga, Sandra, SONW: Shadow of New Worlds. Bergen: Bergen Kunsthall. November 22, 2019 – January 19, 2020.
———. “Sandra Mujinga at Bergen Kunsthall.” Contemporary Art Daily, January 14, 2020. https://contemporaryartdaily.com/2020/01/sandra-mujinga-at-bergen-kunsthall.
———. “There’s Nothing Black About This.” Interview by Olamiju Fajemisin, e-flux, March 7, 2020. https://www.e-flux.com/announcements/318278/sandra- mujingamidnight/.
Müller, Jürgen E. “Intermediality and Media Historiography in the Digital Era.” Film and Media Studies 2 (2010): 15–38.
———. “Intermediality Revisited: Some Reflections about Basic Principles of this Axe de pertinence.” In Media Borders, Multimodality and Intermediality. Edited by Lars Elleström, 237-52. New York: Palgrave Macmillan, 2010.
Mulvey, Laura. “Visual Pleasure and Narrative Cinema.” Screen 16, no. 3 (1975): 6–18. https://doi.org/https://doi.org/10.1093/screen/16.3.6.
Mumford, Lewis. Technics and Civilization. London: Routledge and Kegan Paul, 1955.
Munster, Anna. Materializing New Media: Embodiment in Information Aesthetics. Lebanon, NH: Dartmouth College Press, 2006.
———. “Into ‘inter’: The Between in Interacting.” Rivista di estetica 63 (2016): 56-67.
Musser, Charles. “Toward a History of Screen Practice.” Quarterly Review of Film Studies 9, no. 1 (1984): 59–69.
Muybridge, Eadweard. The Human Figure in Motion: Eadweard Muybridge 1830–1904. New York: Dover Publications, 1955.
Nail, Thomas. “What Is an Assemblage?” SubStance 46, no. 1 (2017): 21–37. https://doi.org/10.3368/ss.46.1.21.
Nakamura, Lisa, and Peter A. Chow-White, ed. Race after the Internet. London, New York: Routledge, 2011.
Narain, Sashank, Amirali Sanatinia and Guevara Noubir. “Single-stroke Language- Agnostic Keylogging using Stereo-Microphones and Domain Specific Machine Learning.” WiSec ’14, Proceedings of the 2014 ACM Conference on Security and Privacy in
313
Wireless & Mobile Networks, 201–212. Oxford, UK, 2014. https://doi.org/10.1145/2627393.2627417.
Neale, Stephen. Cinema and Technology: Sound, Image, Colour. Bloomington: Indiana University Press, 1985.
Noland, Carrie. “Introduction.” In Migrations of Gesture. Edited by Carrie Noland and Sally Ann Ness, ix-xxviii. Minneapolis: University of Minnesota Press, 2008.
Nowell-Smith, Geoffrey. “Eisenstein on Montage.” in Selected Works, Vol II: Towards a Theory of Montage by Sergei M. Eisenstein, edited by Michael Glenny and Richard Taylor, translated by Michael Glenny, xiii-xvi. London: BFI Publishing, 1991.
Nunes, Mark. “Error, Noise, and Potential: The Outside of Purpose.” In Error: Glitch, Noise, and Jam in New Media Cultures, edited by Mark Nunes, 3-23. London: Continuum 2011.
Obrist, Hans Ulrich, and Yana Peel. “Foreword.” In Sondra Perry, Typhoon Coming On, edited by Amira Gad, 15–17. Serpentine Galleries, London, March–May, 2018. Exhibition catalogue.
Olenina, Ana Hedberg, and Irina Schulzki. “Mediating Gesture in Theory and Practice.” Apparatus. Film, Media and Digital Cultures in Central and Eastern Europe 5 (2017). http://dx.doi.org/10.17892/app.2017.0005.100.
Orlikowski, Wanda J. “Sociomaterial Practices: Exploring Technology at Work.” Organization Studies 28 (2007): 1435–1448.
Orlikowski, Wanda J., and Susan V. Scott, “Exploring Material-Discursive Practices.” Journal of Management Studies 52, no. 5 (2015): 697–705. https://doi.org/10.1111/joms.12114.
Paech, Joachim. “Intermediate Images.” Acta Univ. Sapientiae, Film and Media Studies 9 (2014): 31–49.
Pallasmaa, Juhani. The Architecture of Image: The Existential Space in Cinema. Helsinki: Rakennustieto, 2007.
Paul, Christian. “Mediations of Light: Screens as Information Surfaces.” In Digital Light. Edited by Sean Cubitt, Daniel Palmer and Nathaniel Tkacz, 179-192. London: Open Humanities Press, 2015.
Perry, Sondra. Resident Evil. New York: The Kitchen. November 2 – December 10, 2016.
———. Graft and Ash for a Three Monitor Workstation. Video bicycle workstation. 9:05 mins video. 2016.
———. flesh out. Buffalo: Squeaky Wheel Film and Media Art Centre. January 20 – May 6, 2017.
314
———. Typhoon Coming On. London: Serpentine Gallery. March 6 – May 20, 2018.
———. “Adrift in the Chroma Key Blues: A Chat with Sondra Perry on Black Radicality + Things that are Yet to Happen in ‘Typhoon coming on’.” Interview by Tamar Clarke-Brown. AQNB, May 1, 2018. https://www.aqnb.com/2018/05/01/adrift-in-the- chroma-key-blues-a-chat-with-sondra-perry-on-black-radicality- things-that-are-yet- to-happen-in-typhoon-coming-on.
———. “In Conversation: Sondra Perry & Hans Ulrich Obrist.” December 15, 2018, Luma Westbau, Zurich. Accessed 2 February 2021. https://vimeo.com/307269119.
———. “Sondra Perry: Typhoon Coming On.” Serpentine Gallery, London, 2018, YouTube. Accessed 2 February 2021. www.youtube.com/watch?v=Qunkb4piXGw.
Peters, John Durham. “Projection and Protection: On the Deep Optical and Ballistical Intersections of Screens.” Presented at Genealogies of the Excessive Screen, Yale University Art Gallery lecture series, February 16, 2017.
———. “The Charge of a Light Barricade: Optics and Ballistics in the Ambiguous Being of Screens.” in Screen Genealogies: From Optical Device to Environmental Medium, edited by Craig Buckley, Rüdiger Campe, and Francesco Casetti, 215-235. Amsterdam: Amsterdam University Press, 2019. https://doi.org//10.5117/9789463729000.
Peterson, Michael. “Apple Patents Method to Control Watch with Hand Gestures and Movements.” Apple Toolbox, May 1, 2018. https://appletoolbox.com/2018/05/apple- patents-method-to-control-watch-with-hand-gestures-movements.
Pethő, Ágnes, ed. Film in the Post-Media Age. Newcastle upon Tyne: Cambridge Scholars Publishing, 2012.
———. “The Garden of Intermedial Delights: Cinematic ‘adaptations’ of Bosch, from Modernism to the Postmedia Age.” Screen 55, no. 4 (2014): 471–89. https://doi.org/10.1093/screen/hju036.
Phillips, John. “Agencement/Assemblage,” Theory, Culture & Society 23, no. 2–3 (2006): 108–109.
Pickering, Andrew. "After Representation: Science Studies in the Performative Idiom." Talk at the Philosophy of Science Association Annual Meeting, New Orleans, 1994.
———. The Mangle of Practice: Time, Agency, and Science. Chicago: University of Chicago Press, 1995.
Pisters, Patricia. “Image as Gesture: Notes on Aernout Mik’s Communitas and the Modern Political Film.” Journal for Cultural Research 19, NO. 1 (2015): 69–81. https://doi.org//10.1080/14797585.2014.920186.
———. The Neuro-Image: A Deleuzian Film-Philosophy of Digital Screen Culture. Stanford: Stanford University Press, 2012.
315
Quaranta, Domenico. Beyond New Media Art. Brescia: Link Editions, 2013.
Raban, W. “Reflexivity and Expanded Cinema: A Cinema of Transgression?” In Expanded Cinema: Art Performance and Film, edited by Steven Ball, 98–107. London: Tate Publishing, 2011.
Rajewsky, Irina O. “Intermediality, Intertextuality, and Remediation: A Literary Perspective on Intermediality.” Intermédialités 6 (2005): 43–64.
Raley, Rita. “Dataveillance and Counterveillance.” In Raw Data is an Oxymoron, edited by Lisa. Gitelman, 121–146. Cambridge, MA: MIT Press, 2013.
Rees, A.L. “Expanded Cinema and Narrative: A Troubled History.” In Expanded Cinema: Art Performance and Film, edited by Steven Ball, 148–156. London: Tate Publishing, 2011.
Reiser, Martin, and Andrea Zapp, eds., New Screen Media: Cinema/art/narrative. London: British Film Institute, 2002.
Riley, Chares A. Colour Codes: Modern Theories of Colour in Philosophy, Painting and Architecture, Literature, Music, and Psychology. Hanover, NH: University Press of New England, 1995.
Rockwell, John. “An Extravaganza by John Cage Due.” New York Times, May 3, 1975. https://www.nytimes.com/1975/05/03/archives/an-extravaganza-by-john-cage-due- 4hour-multimedia-concert-set-in.html.
Rodowick, David N. The Virtual Life of Film. Cambridge, Mass.; London, England: Harvard University Press, 2007.
Romaniuk, Scott N., and Tobias Burgers. “How the US Military is using ‘Violent, Chaotic, Beautiful’ Video Games to Train Soldiers.” The Conversation, March 7, 2017. https://theconversation.com/how-the-us-military-is-using-violent-chaotic-beautiful- video-games-to-train-soldiers-73826.
Rossell, Deac. “Chronophotography in the Context of Moving Pictures.” Early Popular Visual Culture 11, no. 1 (2013):10–27. https://doi.org//10.1080/17460654.2012.756651.
Roth, Lorna. “Looking at Shirley, the Ultimate Norm: Colour Balance, Image Technologies, and Cognitive Equity” Canadian Journal of Communication 34, no. 1 (2009): 111–136.
Rouse, Joseph. How Scientific Practices Matter: Reclaiming Philosophical Naturalism. Chicago: University of Chicago Press, 2002.
Rowland, Duncan, Martin Flintham, Steve Benford, Nick Tandavanitj, Adam Drozd, and Rob Anastasi. “On the streets with Blast Theory and the MRL: ‘Can You See Me Now?’ and ‘Uncle Roy—All Around You’.” In Mobile Entertainment: User-centred Perspectives, 25-27 March 2004, edited by Karenza Moore, Jason Rutter, 115-124. Manchester: ESRC Centre for Research on Innovation and Competition, 2004.
316
Rubinstein, Daniel and Sluis, Katrina. “A Life More Photographic.” Photographies 1, no. 1 (2008): 9-28.
Sæther, Susanne Ø., and Synne T. Bull, ed. Screen Space Reconfigured (Amsterdam: Amsterdam University Press, 2020. https://doi.org//10.5117/9789089649928.
Saldanha, Arun. “Reontologising Race: The Machinic Geography of Phenotype.” Environment and Planning D: Society and Space 24, no. 1 (2006): 9-24. https://doi.org/10.1068/d61j.
———. Psychedelic White: Goa Trance and the Viscosity of Race. Minneapolis: University of Minnesota Press, 2007.
———. “Bastard and Mixed-Blood are the Two Names of Race.” In Deleuze and Race. Edited Arun Saldanha and Jason Michael Adams, 6-34. Edinburgh: Edinburgh University Press, 2012.
Salter, Chris. “Disturbance, Translation, Enculturation: Necessary Research in New Media, Technology, and the Senses.” Visual Anthropology Review 34, no. 1 (2018): 87– 97.
Sauvagnargues, Anne. Artmachines: Deleuze, Guattari, Simondon. Translated by Suzanne Verderber and Eugene W Holland. Edinburgh: Edinburgh University Press, 2016. https://doi.org/10.3366/j.ctt1bh2jvt.
Schivelbusch, Wolfgang. Disenchanted Night: The Industrialization of Light in the Nineteenth Century. Berkeley: University of California Press, 1995.
Schlosser, Markus. “Agency.” Stanford Encyclopedia of Philosophy, August 10, 2015. https://plato.stanford.edu/entries/agency.
Schneider, Rebecca. “New Materialisms and Performance Studies.” The Drama Review 59, no. 4 (2015): 7-17.
Schoon, Ben. “Google Maps ‘hack’ uses 99 Smartphones to Create Virtual Traffic Jams.” 9To5Google, February 4, 2020. https://9to5google.com/2020/02/04/google-maps-hack- virtual-traffic-jam.
Schüll, Natasha Dow. “Data for Life: Wearable Technology and the Design of Self-care.” BioSocieties (2016): 1–17.
Schuller, Kyla. The Biopolitics of Feeling: Race, Sex, and Science in the Nineteenth Century. Durham, NC: Duke University Press, 2017.
Schulzki, Irina. “The Underlying Gesture: Towards the Notion of Gesture in Jean d’Udine and Sergei Eisenstein.” In From Sensation to Synaesthesia in Film and New Media, edited by Rossella Catanese, Francesca Scotto Lavina and Valentina Valente, 102- 115. Newcastle upon Tyne: Cambridge Scholars Publishing 2019.
Shaviro, Steven. Post-Cinematic Affect. Zero Books, 2010.
317
Shaw, Jeffrey and Peter Weibel, ed. Future Cinema: The Cinematic Imaginary after Film. Cambridge, London: MIT Press, 2003.
Shaw, Jeffrey. “Movies After Film – The Digitally Expanded Cinema.” In The New Screen Media: Cinema/Art/Narrative, edited by Martin Rieser and Andrea Zapp, 268-75. London: British Film Institute, 2002.
Sheldon, Rebekah. “Form/Matter/Chora: Object-Oriented Ontology and Feminist New Materialism.” In The Nonhuman Turn, edited by Richard Grusin, 193-222. Minneapolis: University of Minnesota Press, 2015.
Smith, Alvy Ray, and James F. Blinn. “Blue Screen Matting.” In SIGGRAPH Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, 259–268. New York: Association for Computing Machinery, 1996. https://doi.org/10.1145/237170.237263.
Sobchack, Vivian. “Comprehending Screens: A Meditation in Medias Res.” Rivista Di Estetica 55 (2014): 87–101. https://doi.org/10.4000/estetica.959.
———. The Address of the Eye: A Phenomenology of Film Experience. Princeton, NJ: Princeton University Press, 1992.
Solnit, Rebecca. Motion Studies: Time, Space and Eadweard Muybridge. London: Bloomsbury, 2003.
Somaini, Antonio. “Cinema as ‘Dynamic Mummification,’ History as Montage: Eisenstein’s Media Archaeology.” In Sergei M. Eisenstein: Notes for a General History of Cinema, edited by Naum Kleiman and Antonio Somaini, 19-105. Amsterdam: Amsterdam University Press, 2016.
———. “The Screen as ‘Battleground’: Eisenstein’s ‘Dynamic Square’ and the Plasticity of the Projection Format.” In Format Matters: Standards, Practices, and Politics in Media Cultures, edited by Marek Jancovic, Axel Volmar and Alexandra Schneider, 219–236. Lüneburg: Meson Press, 2020.
Spielmann, Yvonne. “Intermedia in Electronic Images.” Leonardo 34, no. 1 (2001): 55–61. https://doi.org/10.1162/002409401300052523.
Stadler, Jane, and Kelly McWilliam. Screen Media: Analysing Film and Television. Sydney: Allen & Unwin, 2009. https://doi.org/10.4324/9781003117230.
Stanger, Arabella. “Bodily Wreckage, Economic Salvage and the Middle Passage.” Performance Research 24, no. 5 (2019): 11-20. https://doi.org//10.1080/13528165.2019.1671710.
Stern, Daniel. Forms of Vitality. Oxford: Oxford University Press, 2010.
Stern, Lesley. “Ghosting: The Performance and Migration of Cinematic Gesture, Focussing on Hou Hsiao-Hsien’s Good Men, Good Women.” In Migrations of Gesture,
318
edited by Carrie Noland and Sally Ann Ness, 185-215. Minneapolis: University of Minnesota Press, 2008.
Stewart, Jessica. “Artist Creates Virtual Traffic Jam on Google Maps with a Wagon Full of 99 Smartphones.” My Modern Met, February 17, 2020. https://mymodernmet.com/google-map-hacks-simon-weckert.
Steyerl, Hito. The Wretched of the Screen. Berlin: Sternberg Press, 2012.
Steyerl, Hito. How Not to be Seen: A Fucking Didactic Educational .MOV File. 14 mins, video. 2013.
Strauven, Wanda. “The Archaeology of the Touch Screen.” Maske und Kothurn 58, no. 4 (2012): 73–74. https://doi.org/10.7788/muk-2012-0405.
———. “The Screenic Image: Between Verticality and Horizontality, Viewing and Touching, Displaying and Playing.” In Screens: From Materiality to Spectatorship — a Historical and Theoretical Reassessment, edited by Dominique Chateau and José Moure, 143-156. Amsterdam: Amsterdam University Press, 2016.
Struppek, Mirjam. “Urban Screens — The Urbane Potential of Public Screens for Interaction.” Intelligent Agent 6, no. 2 (2006): 16-24. http://www.intelligentagent.com/archive/Vol6_No2_interactive_city_struppek.htm.
Styhre, Alexander. “Screenness and Organizing: Sociomaterial Practices in Mediated Worlds.” VINE 43, no. 1 (2013): 4-21. http://doi.org//10.1108/03055721311302115.
Tarkovsky, Andrey. Sculpting in Time. Translated by Kitty Hunter-Blair. Austin, TX: University of Texas Press, 1986.
Tawa, Michael. Agencies of the Frame: Tectonic Strategies in Cinema and Architecture. London: Blackwell, 2010.
Thain, Alanna. Bodies in Suspense: Time and Affect in Cinema. Minneapolis: University of Minnesota Press, 2017.
Thielmann, Tristan. “‘You have Reached your Destination!’ Position, Positioning and Superpositioning of Space through Car Navigation Systems.” Social Geography 2, no. 1 (2007): 63–75.
———. “Located Media and Mediated Localities: An Introduction to Media Geography.” Aether 5 (2010): 1–17.
Thrift, Nigel. “Beyond Mediation: Three New Material Registers and Their Consequences.” In Materiality, ed. Daniel Miller, 231-56. Durham, NC: Duke University Press, 2005.
Till, Chris. “Exercise as Labour: Quantified Self and the Transformation of Exercise into Labour.” Societies 4, no. 3 (2014): 446–462.
319
Tong, Ng Suat. “An Open Debate about Closure.” The Comics Journal 111 (1999): 77–79.
Tortajada, Maria. “Marey and the Synthesis of Movement: The Reconstruction of a Concept.” In Cine-Dispositives: Essays in Epistemology Across Media, edited by Maria Tortajada and François Albera, 93–114. Amsterdam: Amsterdam University Press, 2015.
Trodd, Tamara, ed. Screen/Space. The Projected Image in Contemporary Art. Manchester: Manchester University Press, 2011.
Truelove, Verity, Oscar Oviedo-Trespalacios, James Freeman, and Jeremy Davey. “Sanctions or Crashes? A Mixed-Method Study of Factors Influencing General and Concealed Mobile Phone Use while Driving.” Safety Science 135 (2021). https://doi.org/10.1016/j.ssci.2020.105119.
Tuters, Marc, and Kazys Varnelis. “Beyond Locative Media. Giving Shape to the Internet of Things.” Leonardo 39, no. 4 (2006): 357–363.
Väliaho, Pasi. Biopolitical Screens: Image, Power, and the Neoliberal Brain. Cambridge, MA; London: MIT Press.
Veljović, Ljiljana. “History and Present of Gyroscope Models and Vector Rotators.” Scientific Technical Review 60, no. 3–4 (2010): 101-110.
Verhoeff, Nanna, and Heidi Rae Cooley. “Performativity/Expressivity: The Mobile Microscreen and its Subjects.” In Technē/Technology: Researching Cinema and Media Technologies – Their Development, Use, and Impact, edited by Annie van den Oever, 207- 16. Amsterdam: Amsterdam University Press, 2014.
Verhoeff, Nanna, Jean Dubois, Dave Colangelo, and Claude Fortin. “Disrupting the City: Using Urban Screens to Remediate Public Space.” presented at Inter-Society for the Electronic Art (ISEA), Vancouver, Canada, August 2015.
Verhoeff, Nanna. Mobile Screens. The Visual Regime of Navigation. Amsterdam: Amsterdam University Press, 2012. https://library.oapen.org/handle/20.500.12657/34549.
Videodrome. Directed by David Cronenberg, 1983, 87 mins, Canadian Film Development Corporation.
Vos, Eric. “The Eternal Network: Mail Art, Intermedia Semiotics, Interarts Studies.” In Interart Poetics: Essays on the Interrelations of the Arts and Media, edited by Ulla-Britta Lagerroth, Hans Lund and Erik Hedling, 325-36. Amsterdam: Rodopi, 1997.
Walley, Jonathan. “Modes of Film Practice in the Avant-garde.” In Art and the Moving Image: A Critical Reader, edited by Tanya Leighton, 182–199. London: Tate Publishing, 2008.
Warfield, Katie. “Making the Cut: An Agential Realist Examination of Selfies and Touch.” Social Media + Society 2, no. 2 (2016): 1–10. https://doi.org//10.1177/2056305116641706.
320
Weibel, Peter. “The Post-Media Condition.” [2005] Mute, March 2012. https://www.metamute.org/editorial/lab/post-media-condition.
———. “Narrated Theory: Multiple Projection and Multiple Narration (Past and Future).” In The New Screen Media: Cinema/Art/Narrative, edited by Martin Rieser and Andrea Zapp, 42-53. London: British Film Institute, 2002.
Weise, Kyle. “Control Room.” In Military Vision, 23-33.Melbourne: Screen Space, 2013.
Wen, Hongyi, Julian Ramos Rojas and Anind Dey. “Serendipity: Finger Gesture Recognition using an Off-the-Shelf Smartwatch.” Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 2016, 3847–3851. https://doi.org//10.1145/2858036.2858466.
White, Randall. Introduction to Gesture and Speech [1964], by Andre Leroi-Gourhan, xiii- xxii. Translated by Anna Bustock Berger. Cambridge MA, London: MIT Press, 1993.
Wilde, Danielle. “A New Performativity: Wearables and Body-Devices.” In conference proceedings of Re:live Media Art History, 184-190. Melbourne, 2009. http://www.mat.ucsb.edu/Publications/burbano_MAH2009.pdf.
Wilderson III, Frank B. Red, White and Black: Cinema and the Structure of U.S. Antagonisms. Durham, NC: Duke University Press, 2003.
Williams, Raymond. Marxism and Literature. New York: Oxford University Press.
———. Television: Technology and Cultural Form. London: Collins, 1974.
Winthereik, Brit Ross, Peter A. Lutz, Lucy Suchman, and Helen Verran, ed. “Attending to Screens and Screenness.” STS Encounters 4, no. 2 (2011).
Woods, Chris, and Alice Ross. “Revealed: US and Britain Launched 1,200 Drone Strikes in Recent Wars.” Bureau of Investigative Journalism, December 4, 2012. https://www.thebureauinvestigates.com/stories/2012-12-04/revealed-us-and-britain- launched-1-200-drone-strikes-in-recent-wars.
Woycicki, Piotr. “A Critical Study of Physical Participation in Blast Theory’s Can You See Me Now?.” International Journal of Performance Arts and Digital Media 10, no. 2 (2014): 193-204. https://doi.org/10.1080/14794713.2014.946286.
Yee, Marcus. “Makeshift Histories: Hsu Chia-Wei.” Asia Art Pacific 109, July/Aug 2018. http://artasiapacific.com/Magazine/109/MakeshiftHistories.
Yoon, Soyoung. “Beware the Light: Figure versus Ground, White versus Black (Blue), or: Sondra Perry’s Blue Room and Technologies of Race.” Millennium Film Journal 65 (2017): 30-37.
Youngblood, Gene. Expanded Cinema. New York: Dutton, 1970.
321
Yusoff, Kathryn. A Billion Black Anthropocenes or None. Minneapolis: University of Minnesota Press, 2018.
Zapp, Andrea, and Martin Rieser, ed. New Screen Media: Cinema/Art/Narrative. London: British Film Institute, 2002.
Zhang, Yu, Tao Gu, Chu Luo, Vassilis Kostakos and Aruna Seneviratne. “FinDroidHR: Smartwatch Gesture Input with Optical Heartrate Monitor.” In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, article 56, 2018. https://doi.org//10.1145/3191788.
Zonn, Leon. “Going to the Movies: The Filmic Site as Geographic Endeavor.” Aether 1 (2007): 63–67.
322