TOWARD INTELLIGENT MUSICAL INSTRUMENTS: NEW WIRELESS MODULAR GESTURAL CONTROL INTERFACES By CHESTER JAMES UDELL III A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2012 1 © 2012 Chester James Udell III 2 To my wife 3 ACKNOWLEDGMENTS I thank Dr. James Paul Sain for his continual guidance through this long journey. I also thank Dr. Karl Gugel, who had the sense of adventure to entertain a crazy composition student’s desire to pursue electrical engineering. Furthermore, I thank all of the professors who invested considerable time and effort in me including Dr. Paul Richards, Dr. Paul Koonce, Dr. Silvio Dos Santos, and Dr. Welson Tremura. You all have made a profound impact on my career and on my outlook of music, technology, and life. I am also obliged to Dr. A. Antonio Arroyo and Dr. Arthur Jennings for their advice and input during my doctoral study and dissertation. And finally, I would not have made it this far if not for the continual support and wisdom of my wife. I love you, Monique. 4 TABLE OF CONTENTS page ACKNOWLEDGMENTS .................................................................................................. 4 LIST OF FIGURES .......................................................................................................... 7 LIST OF OBJECTS ......................................................................................................... 8 LIST OF ABBREVIATIONS ............................................................................................. 9 ABSTRACT ................................................................................................................... 11 CHAPTER 1 INTRODUCTION .................................................................................................... 13 Current Issues for Augmented Music Instruments .................................................. 15 Proposed Solutions ................................................................................................. 17 Modular ............................................................................................................ 17 Reversible ........................................................................................................ 17 Non-invasive ..................................................................................................... 18 Reconfigurable ................................................................................................. 18 Limitations and Scope............................................................................................. 19 2 HISTORIC TRAJECTORY ...................................................................................... 22 Anatomy of a Musical Instrument ............................................................................ 22 Musical Instruments after Electricity........................................................................ 23 Early Electronic Instruments (The Primacy of the Keyboard) ........................... 24 Music for Loudspeakers ................................................................................... 28 Computing, From Laboratory to Stage ............................................................. 29 MIDI and Other Serial Communication Protocols ............................................. 32 3 ELECTRIC VERSUS ELECTRONIC MUSIC INSTRUMENTS TODAY .................. 34 Electric VS Electronic.............................................................................................. 34 Instrument Taxonomy ............................................................................................. 35 Alternative Controllers............................................................................................. 36 Augmented Instruments .......................................................................................... 39 Augmentations of the Trombone ............................................................................. 44 4 ON MUSICAL GESTURE ....................................................................................... 49 Sound, Motion, & Effort ........................................................................................... 50 Gesture............................................................................................................. 52 Musical Gesture ............................................................................................... 53 5 Instrumental Gesture ........................................................................................ 54 Spectro-morphology ............................................................................................... 55 Gesture & Paralanguage ........................................................................................ 57 5 INTERFACING WITH THE ACOUSMATIC. ........................................................... 58 Sensor Interfaces .................................................................................................... 58 Related Work: Wireless Sensor Interfaces ............................................................. 61 Current Trends ........................................................................................................ 62 Single-Serving Innovations ............................................................................... 63 Transparency ................................................................................................... 65 Accessibility ...................................................................................................... 65 The Dysfunctions of MIDI ........................................................................................ 66 6 EMOTION: RECONFIGURABLE MODULAR WIRELESS SENSOR INTERFACES FOR MUSICAL INSTRUMENTS ..................................................... 70 Overall Design Philosophy ...................................................................................... 70 Sensor Nodes ......................................................................................................... 72 Node Design Considerations ............................................................................ 74 Node Types ...................................................................................................... 75 The MCU ................................................................................................................ 76 Addressing Protocol ................................................................................................ 76 Radio Specifications ............................................................................................... 77 Receiver Hub .......................................................................................................... 79 Hub Design Considerations .................................................................................... 80 Software Client ....................................................................................................... 81 Data Input ......................................................................................................... 81 Data Processing ............................................................................................... 84 Data Mapping ................................................................................................... 85 Implementation: Augmented Trombone Using eMotion .......................................... 86 7 CONCLUSIONS AND FUTURE DIRECTIONS ...................................................... 89 Broader Impacts ..................................................................................................... 90 Future Directions .................................................................................................... 90 Conclusions ............................................................................................................ 94 APPENDIX: MUSICAL SCORE: CAPOEIRISTA FOR FLUTE, BERIMBAU, AND LIVE ELECTRONICS .................................................................................................... 95 LIST OF REFERENCES ............................................................................................. 107 BIOGRAPHICAL SKETCH .......................................................................................... 113 6 LIST OF FIGURES Figure page 3-1 An illustration of where in the process electricity is introduced for electric instruments. ........................................................................................................ 34 3-2 An illustration of where in the process electricity is introduced for electronic instruments ......................................................................................................... 35 3-3 Based on Miranda-Wanderley’s 2006 text on Taxonomy of DMI Types. ............ 36 5-1 Market-available wired sensor interfaces and their specifications as of 2011. ... 60 5-2 Market-available wireless sensor interfaces and specifications as of 2011. ....... 62 6-1 System Overview ................................................................................................ 71 6-2 Sensor Nodes ..................................................................................................... 72 6-3 A comparison of various market-available wireless data transceivers. ............... 79 6-4 Software client workflow. ...................................................................................