Haptic Media Scenes
Total Page:16
File Type:pdf, Size:1020Kb
Haptic Media Scenes Elisabeth Nesheim Thesis for the degree of Philosophiae Doctor (PhD) University of Bergen, Norway 2020 Haptic Media Scenes Elisabeth Nesheim ThesisAvhandling for the for degree graden of philosophiaePhilosophiae doctorDoctor (ph.d (PhD). ) atved the Universitetet University of i BergenBergen Date of defense:2017 28.08.2020 Dato for disputas: 1111 © Copyright Elisabeth Nesheim The material in this publication is covered by the provisions of the Copyright Act. Year: 2020 Title: Haptic Media Scenes Name: Elisabeth Nesheim Print: Skipnes Kommunikasjon / University of Bergen SCIENTIFIC ENVIRONMENT This dissertation is the result of research conducted at the University of Bergen (UIB), Humanities Faculty, Department of Linguistic, Literary, and Aesthetic Studies between August 2012 and March 2020, as part of a PhD research fellowship in Digital Culture. The funded doctoral program is two-fold, where 75% of the contract is assigned individual research to develop the doctoral thesis. Teaching and program specific research activities constitute 25% of the fellowship. Courses taught include DIKULT103, 104, 208, 250, and 302, in the areas of digital art, electronic literature, media aesthetics, philosophy, and digital humanities. My main supervisor is Scott Rettberg, professor of Digital Culture at UIB, and my co-supervisor is Jill Walker Rettberg, professor of Digital Culture at UIB. I have been part of two research groups throughout the fellowship: the Digital Culture Research group and the Bergen Electronic Literature research group (BEL), and been partaking in workshops, conferences, and exhibitions organized within the program. In addition, important meetings and conversations have taken place at conferences, workshops, and festivals, such as Transmediale, Piksel, Ars Electronica, Medialab Prado, the TBLR PhD seminar, and the E&T Education and Technology summer school. 4 ACKNOWLEDGMENTS Writing a doctoral thesis is an extensive process, with many twists and turns. A lot of people have influenced my direction and drive, and some mentions are in order. First and foremost, I want to extend my deepest gratitude towards Scott Rettberg, supervisor extraordinaire, for encouraging me to take a PhD in the first place. For your generosity, including me in your scientific network, introducing me to relevant artists and scholars, as a peer. For your patience, insights, and your feedback. Thank you. To Jill Walker Rettberg, for your sharp and precise mind, strong ethics, and caring nature. For seeing me as more than a PhD candidate with a research topic, as a person, and a mother. You are an inspiration. To Daniel Apollon, for our many talks and broadening my perspective, introducing relevant and important topics, themes and problems, many which have found their way into the thesis. For promoting the notion of the scientific author, as a person on top of her theoretical findings and discussions. I am ever grateful to Gisle Frøysland who has been formative in my thinking about technology and art. Your insights from the FLOSS-movement and artistic practice are ever relevant and working with Piksel opened the door to the rich and vibrant European media art world. On this note, I also want to extend a thanks to Marco Donnarumma for challenging my concept of interactivity and interface, for teaching me Xth sense, and introducing me to biomusical art. I am indebted to the numerous talks on the topic of haptics, embodiment, interactivity, interface, extension, augmentations, and art, with Michelle Teran, Johnny Holmvåg, Rolf Guescini, Per Martinsen, Luciana Gattass, Nick Montfort, Servando Barreiro, Judd Morrisey, Espen Gangvik, Talan Memmott, Anne Sofia Karhio, Laetitia Le Chatton, Sylvia Duerr, Øli Parker, Bruno Fractale, Danja Vasiliev, Taras Mashtalir, Álvaro Seiça, Patricia Tomaszek, Kathi Inman Behrens, and Daniela Arriado. And my colleagues and peers at the University of Bergen, Daniel Jung, Henning Skarbø, and Anders Gullestad. A particular thanks to Rob Wittig, for our ACKNOWLEDGEMENTS talks (and walks) throughout the development and writing of the master thesis, and eventually during the formulation of the PhD project, which have had a great impact on my thinking, particularly the notion of play and interface. To Elisabeth Døskeland, for allowing me to heal, and get back on track. To Claire, my French guide, for your structured and practical mind helping me take the last important steps towards the end. You have been invaluable. I also need to send a heartwarming thanks to all my friends, who have been involved in this process from beginning to the end. To Kristine for love and USB. To Thomas for your energy and concern, my big brother from another mother. To Paola and Silje for our shared talks and cheering me on. To Michael Zepf for science fiction insights and letting me be a ninja. To Margrethe Norheim for your generosity, your love and support. Last, but not least, my family. To my awesome mother, Annlaug Nesheim, for your continuous faith and constant encouragement coupled with strict advice. For understanding the nature of writing and the academic realm. For your heart, soul and mind. I love you. To Anders, dearest brother, for sharing the curious mind, the interest of science and technology, for challenging my thinking bringing in the mindset and practice of the electronic engineer. For ModoSol and technical play. You are forever in my heart. To my love, Stig, for letting me be me, for your support and encouragement, giving me room and time for writing and playing, for generously adding to my pool of knowledge. Now a new chapter begins! And finally, to my two loving daughters, Anna & Sunniva, born during this period of research, for shining bright, offering inspiration and easy access to the realm of love, laughter, and exploration, as a healthy break from abstract thinking. Intelligenz Bureau. 6 ABSTRACT The aim of this thesis is to apply new media phenomenological and enactive embodied cognition approaches to explain the role of haptic sensitivity and communication in personal computer environments for productivity. Prior theory has given little attention to the role of haptic senses in influencing cognitive processes, and do not frame the richness of haptic communication in interaction design—as haptic interactivity in HCI has historically tended to be designed and analyzed from a perspective on communication as transmissions, sending and receiving haptic signals. The haptic sense may not only mediate contact confirmation and affirmation, but also rich semiotic and affective messages—yet this is a strong contrast between this inherent ability of haptic perception, and current day support for such haptic communication interfaces. I therefore ask: How do the haptic senses (touch and proprioception) impact our cognitive faculty when mediated through digital and sensor technologies? How may these insights be employed in interface design to facilitate rich haptic communication? To answer these questions, I use theoretical close readings that embrace two research fields, new media phenomenology and enactive embodied cognition. The theoretical discussion is supported by neuroscientific evidence, and tested empirically through case studies centered on digital art. I use these insights to develop the concept of the haptic figura, an analytical tool to frame the communicative qualities of haptic media. The concept gauges rich machine- mediated haptic interactivity and communication in systems with a material solution supporting active haptic perception, and the mediation of semiotic and affective messages that are understood and felt. As such the concept may function as a design tool for developers, but also for media critics evaluating haptic media. The tool is used to frame a discussion on opportunities and shortcomings of haptic interfaces for productivity, differentiating between media systems for the hand and the full body. ABSTRACT The significance of this investigation is demonstrating that haptic communication is an underutilized element in personal computer environments for productivity and providing an analytical framework for a more nuanced understanding of haptic communication as enabling the mediation of a range of semiotic and affective messages, beyond notification and confirmation interactivity. 8 TABLE OF CONTENTS SCIENTIFIC ENVIRONMENT 3 ACKNOWLEDGMENTS 5 ABSTRACT 7 TABLE OF CONTENTS 9 INTRODUCTION 17 Topic and background 18 Outcome and research questions 20 Theoretical framework and research terminology 21 Main findings 22 Thesis chapter overview 23 LITERATURE REVIEW: Haptics in Interactive Scenarios 27 Haptic sensitivity: from touch to haptic perception 27 From touch to haptic 28 Haptic vision and visual art 29 Somesthetic senses: touch, proprioception and haptic sensing 30 Linking touch and proprioception 31 The haptic sense in human-computer interaction 32 From haptic to haptics: understanding haptic media 34 1 SCREEN MIRAGE: Oculus Prime and the Haptic Turn 37 1.1 Oculus Prime 38 Origin of ocularcentrism: distance and surface as an instrument for thought 39 Cybernetics and visual perception 42 The machine eye 45 Second sight 46 Automated vision 47 Screen essentialism 48 1.2 The Multiple Screen 49 The Material Screen 50 Perception of material 52 The natural field of view and the rectangle 54 The expanding field of view 59 TABLE OF CONTENTS Beyond the field of view 60 Sum of the parts 63 Screen Metaphors 64 Windows, Frames, Virtual, and Screen 64 The mirror and the mirage 66 1.3 Screen Mirage: