A Flexible Platform for Tangible Graphic Scores by Simon Alexander-Adams Bachelor of Fine Arts, University of Michigan School of Music Theatre and Dance, 2014 Thesis Submitted in Partial Fulfillment of the Requirements of the Degree of Master of Arts, Media Arts School of Music, Theatre & Dance University of Michigan / J Approvedby: Michael Gurevich, Thesis Director Assistant Professor, Performing Arts Technology, School of Music Theatre and Dance Sile O'Modhrain, Thesis Committee Member, Associate Professor, Performing Arts Technology School of Music Theatre and Dance Stephen Rush, Thesis Committee Member, Professor, Performing Arts Technology Christopher McNamara, Thesis Committee Member, Lecturer, Screen Arts and Cultures College of Literature, Science, and the Arts Date Degree Conferred: May 2015 ABSTRACT This thesis describes a system for composing and performing tangible graphic scores. A tangible score is simultaneously interface and composition in that musical instruction is encoded into the body of the instrument itself. The finished device utilizes modular textured surfaces in conjunction with interactive compositional structures, exploring aspects of indeterminacy, simulated physics systems, and the concept of utilizing traces of previous performances as the basis for future compositions. This document begins by examining pertinent historical work, and related research to the topic, and discusses the conceptual, aesthetic and motivational factors that influenced the compositional decisions. The development of the system is documented, outlining design considerations, and stepping through initial ideas and iterations, up to the final implementation of the system. Then, the compositional process is examined for three separate systems of compositions, followed by a description of a performance given with the system in the Digital Media Commons Video Studio. Lastly, the paper critiques the successes, shortcomings, limitations and possibilities for future development of the system. ii © Copyright 2015 by Simon Alexander-Adams All Rights Reserved iii ACKNOWLEDGEMENTS Michael Gurevich for your guidance and advising with this thesis, and for introducing me to the world of interaction design. Working with you has significantly influenced my practice of art making and the way I work with technology. Thesis committee members, Stephen Rush, Sile O’Modrain, Christopher McNamara for your insightful comments about this project, and support throughout my undergraduate and graduate work at the University of Michigan. The Digital Media Commons Video Studio staff for providing an excellent facility for the final performance of this thesis. Especially Jeff Alder and Jacques Mersereau, for their deep concern for the quality of the student productions taking place in the Duderstadt Video Studio Carlos Garcia, not only for enabling this thesis through access to equipment and workspace, but also serving as a source of advice from the technical to artistic. The Makerworks staff, for providing a wonderful facility for fabrication and making things. Eric Sheffield and Connor Barry, for being awesome fellow grad students, and providing an open environment for sharing ideas and providing feedback throughout the design process. The Performing Arts Technology department for being such an amazing community of artists, creators, musicians, thinkers and collaborators. It would be impossible to name everyone who has made my time at the University of Michigan the great experience that it was. iv TABLE OF CONTENTS ABSTRACT ...................................................................................................................................................... ii ACKNOWLEDGEMENTS ................................................................................................................................ iv TABLE OF CONTENTS ..................................................................................................................................... v LIST OF FIGURES .......................................................................................................................................... vii 1. INTRODUCTION ..................................................................................................................................... 1 2. BACKGROUND ....................................................................................................................................... 3 2.1 Graphic Scores and Nontraditional Notation ................................................................................ 3 2.1.1 Conventional Graphic Notation ............................................................................................ 3 2.1.2 Dynamic and Interactive Scores ............................................................................................ 4 2.1.3 Tangible Scores ..................................................................................................................... 6 2.2 Interfaces for Musical Expression ................................................................................................. 7 2.2.1 Tangible User Interfaces ....................................................................................................... 8 2.2.2 Tabletop Interfaces ............................................................................................................... 9 2.3 Conceptual Influence .................................................................................................................. 10 2.3.1 Composing with Indeterminacy .......................................................................................... 10 2.3.2 Lines and Traces .................................................................................................................. 10 2.3.3 Noise and Glitch Aesthetics ................................................................................................ 11 3. DESIGNING A FLEXIABLE PLATFORM FOR TANGIBLE GRAPHIC SCORE ............................................... 13 3.1 Design Considerations................................................................................................................. 13 3.2 Initial Ideas and Iterations .......................................................................................................... 15 3.2.1 Disks and Colored Light ....................................................................................................... 15 3.2.2 Textured Modular Surfaces ................................................................................................. 16 3.2.3 Final Direction ..................................................................................................................... 18 Hardware Design ..................................................................................................................................... 18 3.3 Textured Plates ........................................................................................................................... 23 3.4 Analyzing Camera Data ............................................................................................................... 27 3.5 Sound Generation ....................................................................................................................... 28 3.6 Software Communication ........................................................................................................... 29 4. THE COMPOSITIONAL PROCESS .......................................................................................................... 30 4.1 Composing with History .............................................................................................................. 31 4.1.1 Capturing the Aggregate ..................................................................................................... 31 v 4.1.2 Recording Traces ................................................................................................................. 34 4.2 Snakes ......................................................................................................................................... 36 4.3 Particles ....................................................................................................................................... 41 4.4 Nodes .......................................................................................................................................... 45 5. VIDEO STUDIO PERFORMANCE ........................................................................................................... 52 6. DISCUSSION AND CRITICISM ............................................................................................................... 54 6.1 Reflections on the Performance ................................................................................................. 54 6.2 Technical and Practical Issues ..................................................................................................... 54 6.3 Future Directions ........................................................................................................................ 55 7. BIBLIOGRAPHY .................................................................................................................................... 57 APPENDIX A – snake attractors python script ............................................................................................. 61 APPENDIX B – snake sequencer python script ...........................................................................................