DISEMBODIED PERFORMANCE Abstraction of Representation in Live Theater PPPETER AAALEXANDER TTTORPEY Bachelor of the Arts in Media Arts University of Arizona, 2003 Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning, in partial fulfillment of the requirements for the degree of Master of Science in Media Arts and Sciences at the Massachusetts Institute of Technology September 2009 © 2009 Massachusetts Institute of Technology. All rights reserved. Author: PETER ALEXANDER TORPEY Program in Media Arts and Sciences 7 August 2009 Certified by: TOD MACHOVER Professor of Music and Media Program in Media Arts and Sciences Thesis Supervisor Accepted by: DEB ROY Chair, Academic Program in Media Arts and Sciences DISEMBODIED PERFORMANCE Abstraction of Representation in Live Theater PPPETER AAALEXANDER TTTORPEY Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning, on 7 August 2009, in partial fulfillment of the requirements for the degree of Master of Science in Media Arts and Sciences at the Massachusetts Institute of Technology A B S T R A C T Early in Tod Machover’s opera Death and the Powers , the main character, Simon Powers, is subsumed into a technological environment of his own creation. The theatrical set comes alive in the form of robotic, visual, and sonic elements that allow the actor to extend his range and influence across the stage in unique and dynamic ways. The environment must compellingly assume the behavior and expression of the absent Simon. This thesis presents a new approach called Disembodied Performance that adapts ideas from affective psychology, cognitive science, and the theatrical tradition to create a framework for thinking about the translation of stage presence. An implementation of a system informed by this methodology is demonstrated. In order to distill the essence of this character, we recover performance parameters in real-time from physiological sensors, voice, and vision systems. This system allows the offstage actor to express emotion and interact with others onstage. The Disembodied Performance approach takes a new direction in augmented performance by employing a non- representational abstraction of a human presence that fully translates a character into an environment. The technique and theory presented also have broad-reaching applications outside of theater for personal expression, telepresence, and storytelling. Thesis Supervisor: TOD MACHOVER Professor of Music and Media Program in Media Arts and Sciences Massachusetts Institute of Technology DISEMBODIED PERFORMANCE Abstraction of Representation in Live Theater PPPETER AAALEXANDER TTTORPEY The following person served as a reader for this thesis: Thesis Reader DAVID SMALL Associate Professor of Media Arts and Sciences Program in Media Arts and Sciences Massachusetts Institute of Technology DISEMBODIED PERFORMANCE Abstraction of Representation in Live Theater PPPETER AAALEXANDER TTTORPEY The following person served as a reader for this thesis: Thesis Reader WHITMAN A. RICHARDS Professor Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology DISEMBODIED PERFORMANCE Abstraction of Representation in Live Theater PPPETER AAALEXANDER TTTORPEY The following person served as a reader for this thesis: Thesis Reader ALEX MCDOWELL , RDI Production Designer DreamWorks Animation SKG, Inc. ACKNOWLEDGMENTS TOD MACHOVER As my advisor for the past two years, you have provided me with numer ous inspirational ideas, opportunities, and experiences for which I am greatly indebted. I very much look forward to our continued collaboration and learning from you. DAVID SMALL I wish to thank my thesis readers for their support. Your inputs and WHITMAN RICHARDS experience have been most valuable while formulating the approaches and ALEX MCDOWELL ideas presented in this thesis. I have benefitted from your wisdom and surely will continue to do so. DIANE PAULUS Thank you for taking the time to participate in this research or for offering JAMES MADDALENA essential guidance. I look forward to working with each of you in the CYNTHIA BREAZEAL future, be it for Powers or other projects. ROSALIND PICARD MARK & NAOMI TORPEY Throughout my life, you’ve always been there to support me in innumerable ways. From you, I’ve learned the most important lessons in life, especially that I should never to settle for anything less than my dreams. ELEANOR OLVEY Thank you, my dear longtime friend. You’ve always been there when I needed you and have always managed to encourage me, in spite of myself. ELENA JESSOP To my friend, colleague, and collaborator, I offer my thanks for your ideas, your contributions, and your company. NOAH FEEHAN I am grateful to my Opera of the Future colleagues for their kind ANITA LILLIE friendship, assistance, and for the way in which they have challenged me ANDY CAVATORTA explicitly or by example to better my work and myself. NINA YOUNG WEI DONG ADAM BOULANGER CRAIG LEWISTON 11 BOB HSIUNG To those who have worked so tirelessly producing Death and the Powers and DEBBIE GIFFORD to the Opera of the Future administrative assistants, you have made much ARIANE MARTINS possible and life a little easier. You have my gratitude. TAYA LEARY KIRSTEN BROOKS KRISTIN GALLAS JIHYON KIM BEN BLOOMBERG To date, 30 undergraduates have worked for Opera of the Future on Death MICHAEL MILLER and the Powers . Their intellect, expertise, humor, and labor have been SIMONE OVSEY invaluable to the opera and several have contributed directly to the work JASON KU presented in this thesis, so I heartily salute my Intrepid UROPs. OTHER INTREPID UROP S DEBORAH FREY I believe that who I am is the product of the experiences and people I’ve WAYNE MONTIETH met throughout my life. With that in mind, I recognize these educators and JEFF SIMPSON staff from elementary school through college who have had a significant JUSTINE WEAVER impact on the way I think and view the world in a way that is essential to ROBERT BLACK the research I pursue in this thesis and beyond. SUZANNE BRAYER STUART REGES JEFF IMIG JANICE DEWEY 12 This thesis is dedicated to my grandfather MEYER LEVIN 1911–2008 I have learned much from Meyer. He taught me to take nothing for granted, to build and create, and to always use the right tool for the job… even if I have to invent it myself. 13 14 C O N T E N T S Acknowledgments ................................................................................... 11 Contents ................................................................................................. 15 List of Figures .......................................................................................... 17 1 Introduction .................................................................................. 21 1.1 Motivation.................................................................................. 22 1.1.1 Death and the Powers ................................................................ 22 1.1.2 Disembodiment ....................................................................... 23 1.2 Framework and Objectives.......................................................... 24 1.3 A Caveat or Two ......................................................................... 25 1.4 A Note on Terminology .............................................................. 26 1.5 Thesis Structure .......................................................................... 26 2 Background ................................................................................... 29 2.1 Augmented Performance ............................................................. 29 2.1.1 Media on the Stage ................................................................... 30 2.1.2 Dance and Visualization ........................................................... 32 2.1.3 Hyperinstruments .................................................................... 34 2.1.4 Color Music ............................................................................. 36 2.1.5 Tele-Installation ....................................................................... 37 2.2 Synaesthesia ................................................................................ 38 2.3 Modeling Affect .......................................................................... 41 2.4 Inference ..................................................................................... 43 2.4.1 Representation Mapping through Reified Inference .................. 46 2.4.2 An Example of Reified Inference in Application ....................... 48 2.5 Mapping ..................................................................................... 50 2.6 Distinction from Related Work ................................................... 52 3 Death and the Powers .................................................................... 55 3.1 Genesis ....................................................................................... 56 3.2 Synopsis...................................................................................... 57 3.3 Production Design ...................................................................... 60 3.3.1 Sound in Space......................................................................... 63 15 3.3.2 The Chandelier ........................................................................ 64 3.3.3 Operabots ................................................................................ 66 3.3.4 Walls .......................................................................................