Art in noise: an embodied simulation account of cinematic sound design Mark Shelton Ward A thesis submitted in fulfilment of the requirements for the degree of Doctor of Philosophy Supervisor: Professor Ross Gibson Faculty of Arts and Design University of Canberra Australian Capital Territory, Australia July 2017 Emendations completed 21 April 2020 Abstract Sound and moving pictures share over a century of entwined history yet the dominant visual narrative accounts of cinematic sound lack explanatory power. This thesis responds by developing a novel conceptual framework using the paradigm of embodied cognition. An embodied cognition approach provides a means to analyse the functional architecture, meaning-making capacities, and aesthetics of cinematic media in ways not afforded by conventional media theory. The framework deploys embodied simulation theory (Gallese 2005) to emphasise the affective aspects of Feeling of Body (Wojciehowski & Gallese 2011) and puts into play the concept of embodied meaning (Johnson, M 2007b) which asserts all human meaning and abstract conceptual thinking have basis in our sensorimotor interactions with the world. In the context of cinematic media, embodied simulation theory suggests the stylistic crafting of sensory-perceptual content not only elicits affect but shapes cognition. Specifically, a four-tiered conceptual framework is advanced using embodied simulation theory to contain, integrate and structure three extant models for the elicitation or representation of affect at discrete levels of description. A strength of the four-tiered framework is a highly granular and scalable capacity to describe how cinematic sound may produce embodied meaning. To illustrate the explanatory power of an embodied simulation account of cinematic media, I examine the design strategies used in my own cinematic sound work – specifically, Jane Campion’s In the Cut (2003) – and the work of other sound designers. Lastly, the implications of an embodied simulation account of cinematic media are discussed in relation to investigating the body-mind nexus, bridging the cognitive sciences and Cultural Studies, and informing institutions involved in media education and cultural policy. i | ©2017 Mark Shelton Ward. PhD Thesis, Faculty of Arts and Design, University of Canberra. To Prue, without you I would have stumbled, and never got up. v | ©2017 Mark Shelton Ward. PhD Thesis, Faculty of Arts and Design, University of Canberra. Acknowledgements My deepest gratitude goes to my principal supervisor, Professor Ross Gibson, whose patience allowed me to explore disciplines far flung from my native Film Studies. Without his encouragement and quiet intelligence I would have abandoned my quixotic quest a very long time ago. Research is a solitary and oftentimes isolating pursuit; but along the way I met many people whose wisdom and generosity I encountered at just the right time. Space does not permit me to acknowledge all their names, but some demand it. Deserving of special mention are Ed Tan (who showed me how a humanities scholar might use science to reveal hidden truths), and the late Nico Frijda (whom I met all-too-briefly but whose candid advice encouraged me greatly). Another deserving of special mention is the late Andrew Plain, a big-hearted man and a gifted sound designer who allowed me space to make some noise. To my teachers of media, I acknowledge a great debt. To the late Bill Constable, who opened a door and pushed me through it. To Noel Bolden, Barry Fernandes and Noelig Quinn, my shrewd shepherds while a student at AFTRS in the 1980s, always ready to help, but only when really needed. And to Kari Hanet, who first opened my eyes to the provocations of film theory. I also thank my own students of media, especially those during my time as Head of Sound at AFTRS (2006–2009), who unwittingly put my ideas to the acid test. There is nothing quite like the push-and-shove of industry to sort the theoretical wheat from the chaff, and many of the thorny issues which motivate this research I encountered during my early experiences in industry. And so I am grateful for the time I shared with David White, Victor Gentile, and Liam Egan, my colleagues in the founding of Counterpoint Sound and Oracle Pictures; together we grappled with just vii | ©2017 Mark Shelton Ward. PhD Thesis, Faculty of Arts and Design, University of Canberra. what is going on when communicating feelings and thoughts to an audience through cinematic imagery. I also acknowledge the skills of professional editor Deborah White who provided proofreading services according to the university-endorsed ‘Guidelines for editing research theses’ prescribed by the Institute for Professional Editors (IPEd). Closer to home, I thank my precious daughters, Xanthe and Aislin, for whom this research has occupied an inordinate amount of their young lives. It seems the writing of a doctoral thesis is always a long and lonely endeavour, but it took me years to map the cross-disciplinary terrain and hike my way through it, getting lost and found along the way. This research has greatly preoccupied my thoughts, sometimes unhealthily so, making me a distracted father, and for this I forever apologise. Finally, and most, most importantly, I thank with deep love and gratitude my friend and partner-in-life, Prue. Without her strength and unshrinking support my efforts would have come to nothing. viii | ©2017 Mark Shelton Ward. PhD Thesis, Faculty of Arts and Design, University of Canberra. Table of Contents Abstract i Certificate of Authorship of Thesis iii Dedication v Acknowledgements vii Table of Contents ix Table of Figures xiii Glossary xvii Chapter 1. Introduction 1 1.1 Initial research question and interdisciplinary context 1.2 Motivation for research 1.3 Statement of problem and framing the research question 1.4 Research aims and objectives 1.5 Premises, claims, and hypotheses 1.5.1 Premises 1.5.2 Claims 1.5.3 Hypotheses 1.6 Approach 1.7 Method 1.8 Delimitations to research 1.9 Chapter outline 1.10 Slipping straitjackets: an unconventional thesis structure 1.11 Definition of key terms Chapter 2. Film Studies and the Status of Sound 19 2.1 Sound and/or Film Studies? 2.1.1 Introducing the task of theorising cinematic sound 2.1.2 Problems and opportunities for soundtrack scholarship 2.1.3 Sound qua sound (not merely music) ix | ©2017 Mark Shelton Ward. PhD Thesis, Faculty of Arts and Design, University of Canberra. 2.2 Sound design: history and definition of an ambiguous term 2.2.1 An alternate historiography of sound and cinema 2.2.2 Sound design: history of an ambiguous term 2.2.3 Sound design: definition of an ambiguous term 2.3 Extant literature theorising film-sound 2.3.1 Types of extant film-sound literature 2.3.2 Early sound and cinema (1895–1919) 2.3.3 Classical film-sound theory (1917–1960) 2.3.4 Post-classical film-sound theory (1960–present) 2.4 Framing the gap in Film Studies – beckoning fields of innovation Chapter 3. Again, What is Cinema? 85 3.1 The status of sound in conventional Film Studies 3.2 Praxis and tacit knowledge 3.3 A working definition of cinematic media 3.4 Multimodal integration and cinematic imagery 3.5 Sound design and embodied meaning 3.6 Perceptual design and narrative design 3.7 Proto-narrative and the cinematic 3.8 Feeling of Body Chapter 4. Embodied Cognition, Affect, and Meaning 117 4.1 Embodied cognition 4.2 Emotion science 4.2.1 Competing theories of emotion 4.2.2 ‘Primacy of affect’ versus ‘primacy of cognition’ 4.3 Embodied cognition and affect 4.4 Friction at the nexus of scientific and cultural theory 4.5 Summary: embodied cognition, affect, and meaning Chapter 5. Embodied Simulation and Cinematic Media 133 5.1 Embodied simulation theory 5.2 Function of audition within embodied simulation 5.3 Embodied simulation and cinematic media x | ©2017 Mark Shelton Ward. PhD Thesis, Faculty of Arts and Design, University of Canberra. 5.3.1 Embodied simulation theory and art 5.3.2 Feeling of Body before Theory of Mind 5.4 Cinematic sound and the induction of affect 5.4.1 BRECVEMA framework 5.4.2 Mood-cue approach 5.4.3 Emotional communication theory 5.5 Summary: embodied simulation and cinematic media Chapter 6. The Body, Consciousness, and Cinematic Media 155 6.1 Consciousness, self, and narrative 6.2 Explicating consciousness 6.2.1 Approaches to theorising consciousness 6.2.2 Types of consciousness – Phenomenal and access consciousness 6.2.3 Types of consciousness – Core and extended consciousness 6.2.4 Types of consciousness – Embodied and narrative self 6.3 Narrative consciousness as animal–culture interface 6.4 Summary: consciousness and cinematic media Chapter 7. Biological Basis of Cinematic Deep Structure 173 7.1 Cinematic design strategies 7.1.1 Perceptual design and narrative design 7.2 Deep structure of cinematic media 7.2.1 Audiovisual fusion 7.2.2 Audiovisual metaphor 7.2.3 Proto-narrative 7.2.3.1 Definition and function of proto-narrative 7.2.3.2 Proto-narrative as experiential template 7.2.3.3 Music video as proto-narrative 7.2.3.4 Evolution, proto-narrative and narrative 7.3 Body as basis for knowledge 7.4 In the Cut (2003): an embodied simulation approach 7.4.1 Cinematic imagery, image schemas, and conceptual thought 7.4.2 Affective acoustics as embodied spatial schema 7.4.3 Affective zones and cognitive overflow xi | ©2017 Mark Shelton Ward. PhD Thesis, Faculty of Arts and Design, University of Canberra. 7.4.4 Feeling of Body and the construction of character 7.5 Summary: a biological basis for cinematic feeling and