Signature Redacted
Total Page:16
File Type:pdf, Size:1020Kb
Massively Multiplayer Operas: Interactive Systems for Collaborative Musical Narrative David Su Bachelor of Arts, Columbia University, 2014 Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning, in partial fulfillment of the requirements for the degree of Master of Science in Media Arts and Sciences at the Massachusetts Institute of Technology June 2019 @Massachusetts Institute of Technology, 2019. All rights reserved. Signature redacted Author: David Su / Program in Media Arts and Sciences May ;0, 2019 Signature redacted Certified by: Tod Machover Muriel R. Cooper Professor of Music and Media Program in Media Arts and Sciences ThesistSupervisor Signature redacted Accepted by: MASSACHUSETTS INSTITUTE Tod Machover / OFTECHNOLOGY > Academic Head LUL 2 6 2019 Program in Media Arts and Sciences C0 LIBRARIES 77 Massachusetts Avenue Cambridge, MA 02139 MITLibraries http://Iibraries.mit.edu/ask DISCLAIMER NOTICE Due to the condition of the original material, there are unavoidable flaws in this reproduction. We have made every effort possible to provide you with the best copy available. Thank you. The images contained in this document are of the best quality available. Massively Multiplayer Operas: Interactive Systems for Collaborative Musical Narrative David Su Submitted to the Program in Media Arts and Sciences, School of Architecture and Planning on May 10, 2019 in partial fulfillment of the requirements for the degree of Master of Science in Media Arts and Sciences Abstract Music, narrative, and social interaction have long been intertwined. The objective of this thesis is to create a platform, designed for interactive multiplayer operas, that explores the potential for technology-enabled systems to facilitate creativity through expression, the emotional affordances of musical storytelling, and the spatiotemporal boundaries of copresence. A variety of design experiments for collaborative musical narrative are implemented and evaluated. The work also introduces a real-time lyri- cal conversation system, with user interfaces that allow for simultaneous musical and narrative expression with a high degree of granularity. These experiences are encap- sulated by an overarching lyrical multiplayer narrative opera platform. This project seeks to provide a novel means of creating and understanding multi-user, interactive music systems in which users participate in active and collaborative music-making in conjunction with narrative engagement. Thesis Supervisor: Tod Machover Muriel R. Cooper Professor of Music and Media Program in Media Arts and Sciences Massively Multiplayer Operas: Interactive Systems for Collaborative Musical Narrative David Su The following person served advisor for this thesis: /Signature redacted (2 od Machover Muriel R. Coope r Professor of Music and Media Program in Med ia Arts and Sciences Massively Multiplayer Operas: Interactive Systems for Collaborative Musical Narrative David Su The following person served as a reader for this thesis: Signature redacted Thesis Reader: Zach Lieberman Adjunct Associate Professor of Media Arts and Sciences Program in Media Arts and Sciences Co-Founder School for Poetic Computation Massively Multiplayer Operas: Interactive Systems for Collaborative Musical Narrative David Su The following person served as a reader for this thesis: Signature redacted Thesis Reader: Alex Rigopulos Co-Founder and Chairman Harmonix Music Systems Acknowledgments I am deeply thankful to the following people for their support and inspiration: My advisor, Tod Machover, for his extraordinary mentorship, direction, and en- couragement over the past two years. My readers, Zach Lieberman and Alex Rigopulos, for their invaluable guidance and feedback throughout the thesis process. My girlfriend, Dominique Star, for all her wisdom, creativity, and patience. My parents, Richard Su and Vicki Zhang, for their unceasing faith and support. Joseph Bonavia, for his assistance with interface design and prototyping. The Opera of the Future group: Ben Bloomberg, Priscilla Capistrano, Sizi Chen, Charles Holbrow, R6becca Kleinberger, Hane Lee, Nicole L'Huillier, Alexandra Rieger, Nikhil Singh, and Akito van Troyer, for a truly amazing two years with an incredible family. Professors Rosalind Picard, Pattie Maes, Scot Osterweil, Vijay Iyer, Mark Feld- meier, Frederick Harris, and John Harbison for everything they taught me. Friends and collaborators at the MIT Media Lab, including Tomas Vega Galvez, Adam Haar Horowitz, Thomas Sanchez Lengeling, Nick Ouejian, Vik Parthiban, and Emily Salvador, for providing a creative environment like no other. Melissa Plasse, Magnus Ferguson, Kate Burrows, Alexandria Petteruti, Christo- pher Pegram, Josh Shpak, and Laurie Lee, for being there during tough times. Mochi Star-Su, for the moral support. Contents 1 Introduction 8 2 Background 10 2.1 Related W ork ... ....... ....... .... ....... 10 2.1.1 Interactive Operas ... ........... .... ... 10 2.1.2 Playfully Collaborative Music Making . .. ... .... 12 2.1.3 Choice-Based Narrative Systems .... .. ... .... 14 2.2 Design Principles ..... ....... ....... ....... 15 2.2.1 Conversations, Expressivity, and Granularity ....... 15 2.2.2 Scenes, Consistency, and Persistence .. .. ....... 17 2.2.3 Ethical Considerations .... .... .... ....... 18 3 Design Experiments 20 3.1 Seasons Change Together ... ..... .... .. ....... 20 3.1.1 Words .. .... ..... .... .... .. ....... 21 3.1.2 Textures . .... .... .... .... ... ....... 25 3.1.3 Patterns ... .... .... .... .... ....... 25 3.1.4 Playtesting Feedback and Discussion .... ....... 27 3.2 A Real-Time Lyrical Conversation System . .... .. ..... 29 3.2.1 User Interface Design ... ... .. 29 3.2.2 Audio Implementation . .... ... 31 3.2.3 Playtesting Feedback and Discussion . 33 3.3 Evergreen Blues Suite ... ... .. ... 35 3.3.1 All the World's a Stage . .. ... 37 3.3.2 Evergreen Blues .. ... ... ... 42 3.3.3 Look at the Audience . ... .... 49 3.3.4 Playtesting Feedback and Discussion . 55 4 Towards a Lyrical Multiplayer N arrative Opera Platform 58 4.1 Architecture Overview..... .... 58 4.2 The LMNOP API . ..... .... 59 4.2.1 M usic .... ..... .... 60 4.2.2 Dialogue .... ... ... 63 4.2.3 Data . ... .... .. ... 63 4.2.4 Scene .... ..... .... 66 5 Conclusions and Future Work 70 Bibliography 74 7 1 Introduction Almost every culture in the world has a tradition of multiple people creating music together, from gamelan to gagaku to Gregorian chant. Most cultures also have traditions of musical storytelling through the combination of music and words, such as that of the yuefu (fd) in China, the griot in West Africa, and the bush ballad in Australia. Perhaps the most all-encompassing musical nar- rative form in Western culture is the opera. When people attend an opera, they engage in an experience involving relationships between the audience members and performers, between the performance and musical composition, and be- tween the musical elements and underlying narrative. The joy of interacting with other humans through musical creation as well as the emotional resonance of musical storytelling both appear to be universal. While the networking capabilities of the Internet have allowed us to par- ticipate in social interactions on a whole new scale, advances in recording and streaming technology have not achieved the same results with regards to our collective engagement with music. One of the most common means of expe- riencing music today involving a single person listening to static, linear sound files, mediated by a streaming service's algorithm that caters towards disparate songs in an effort to maintain short-term attention. In essence, the predominant music experience in the 21st century is isolated, passive, and ephemeral. This thesis aims to rectify such a situation by providing a social, interac- tive, and persistent musical experience through the creation of a platform for massively multiplayer operas; that is, interactive musical narrative experiences that retain statefulness between scenes, that are playable by many simultaneous participants, and that foster collaboration and expression. The remainder of this thesis is organized as follows. Chapter 2, Background, 8 presents an overview of existing artwork and literature in the realms of inter- active operas, playfully collaborative music-making, and choice-based narrative systems. The chapter then presents design principles of expressivity and gran- ularity in the context of musical conversations as well as persistence and con- sistency across scenes on a narrative level. Chapter 3, Design Experiments, discusses the creation and playtesting of three multiplayer interactive operatic experiences: Seasons Change Together (an interactive song construction expe- rience), a real-time lyrical conversation system, and the Evergreen Blues Suite (a suite of three interactive songs). Chapter 4, Towards a Lyrical Multiplayer Narrative Opera Platform, describes the architecture and implementation of a platform allowing for the creation of such multiplayer opera experiences. Fi- nally, chapter 5, Conclusions and Future Work, synthesizes the conclusions of this thesis in addition to presenting potential future research directions. It is the goal that this thesis will push the boundaries of the operatic form as well as open up new possibilities for the creation and enjoyment of music, narrative, and social interaction. 9 2 Background In order to create a platform for collaborative interactive musical narrative, the choices and