Hydraulophones: Acoustic Musical Instruments and Expressive User Interfaces
Total Page:16
File Type:pdf, Size:1020Kb
Hydraulophones: Acoustic musical instruments and expressive user interfaces by Ryan E. Janzen A thesis submitted in conformity with the requirements for the degree of Master of Applied Science Graduate Department of Electrical and Computer Engineering University of Toronto Copyright c 2008 by Ryan E. Janzen Abstract Hydraulophones: Acoustic musical instruments and expressive user interfaces Ryan E. Janzen Master of Applied Science Graduate Department of Electrical and Computer Engineering University of Toronto 2008 Fluid flow creates an expansive range of acoustic possibilities, particularly in the case of water, which has unique turbulence and vortex shedding properties as compared with the air of ordinary wind instruments. Sound from water flow is explained with reference to a new class of musical instruments, hydraulophones, in which oscillation originates directly from matter in its liquid state. Several hydraulophones which were realized in practical form are described. A unique user-interface consisting of a row of water jets is presented, in terms of its expressiveness, tactility, responsiveness to derivatives and inte- grals of displacement, and in terms of the direct physical interaction between a user and the physical process of sound production. Signal processing algorithms are introduced, which extract further information from turbulent water flow, for industrial applications as well as musical applications. Version v200809 ii The guidance, collaboration, knowledge, creativity, boldness, and camaraderie of Pro- fessor Steve Mann is greatly appreciated. The work of Dr. James Fung, Chris Aimone, Raymond Lo, Mark Post, Mike Hung, Ahmed Sharifi, Fabian Wauthier, and James Meier, in terms of previous and current contributions, is acknowledged and tributed. Recent hydraulophone installations in pub- lic spaces were designed in collaboration with Steve Mann and Chris Aimone. Concert performances were presented in collaboration with Steve Mann, Dr. John Derksen, Dr. Roger Mantie, John Kameel Farah, Nick Storring, Chris Aimone, Ariel Garten, Noah Mintz, Laura Bolt, and Eyal Katz. Demonstrations and concerts were done with the cooperation of: • Galapagos Art Space, New York • New York University • DGI-Byen, Copenhagen, Denmark • Nuit Blanche, Toronto • Luminato Festival, Toronto • Harbourfront Centre, Toronto • Music Gallery, Toronto • The Power Plant Contemporary Art Gallery, Toronto • Ontario Science Centre, Toronto • University of Toronto Faculty of Music • Hart House, Toronto • Hart House Symphonic Band • Knox College, Toronto • Dundas Square, Toronto • CITY-TV, Toronto • CBC Radio iii • Danish Radio • OM Reunion Festival • Brampton Independent Arts Festival • Kensington Market Pedestrian Sundays, Toronto • Grange Park, Toronto • Baldwin Street BIA Festival, Toronto • International Computer Music Conference 2007 • New Interfaces for Musical Expression Conference 2007 • IEEE International Conference on Multimedia and Expo 2006 • ACM Multimedia 2007 • University of Augsburg, Germany The support of the Canada Council for the Arts, Ontario Arts Council, Toronto Arts Council, TELUS, and Ontario Centres of Excellence is acknowledged. iv Contents 1 Introduction: Musical Instruments in All States... 1 1.1 Classification of musical instruments as a context for hydraulophone . 1 1.2 Hydraulophone, and a physics-based organology . 3 1.3 States of matter, and musical terminology . 4 1.4 Early hydraulophones . 5 1.5 Contributions of this research work . 9 2 Is sound ever produced purely from liquid? 12 2.1 ......................................... 12 2.2 Acoustic oscillation in water already in existence . 13 2.3 Use of liquid in existing musical instruments, where liquid is not the pri- mary medium of sound generation . 14 2.4 Organological purity, and the state-of-matter of the sound source . 16 3 Sound Production from Liquid 18 3.1 Turbulence . 18 3.2 Turbulent Spectra . 19 3.3 Producing turbulence on purpose . 21 3.4 Karman vortex street . 22 3.5 Strouhal number . 23 3.6 Intentional introduction of Karman vortex street . 23 v 3.7 Background: Underwater acoustics . 24 4 Signal Processing of Fluid Dynamics Signals 27 4.1 Introduction . 27 4.2 Acoustic pickups to listen to turbulent sound . 27 4.2.1 Types of acoustic pickups . 28 4.2.2 Custom-built hydrophones . 29 4.2.3 Spatio-temporal uncertainty . 30 4.3 Detection and estimation of fluid flow based on sound alone . 32 4.3.1 Listening to water flow in hydraulophones . 35 4.4 Flow sensor using spectral-division least-squares . 35 4.4.1 Differentiating between hot and cold water . 37 4.4.2 Differentiating between flow rates . 37 4.5 Height of a water jet: Simple method to evaluate flow rate . 38 4.5.1 Theoretical analysis of water jet height . 39 4.5.2 Application . 41 4.6 Filterbanks . 42 4.7 Summary . 45 5 Properties and Applications of Modern Hydraulophones 46 5.1 Development of modern hydraulophones during the course of this research 46 5.2 Compositions and Performances . 47 5.3 Poiseuille Embouchure . 57 5.4 Absement and Presement . 61 5.5 Hydraulophone installations . 64 6 Fluid User-Interfaces 69 6.1 Fluid expressivity . 69 6.2 Multi-modal feedback . 70 vi 6.3 Self-cleaning keyboard . 72 6.4 On the ability of a fluid user-interface's mouths to repel foreign objects . 74 6.4.1 Theoretical analysis: Drag on a Sphere . 75 6.4.2 Levitation of a spherical object . 77 6.4.3 Small contaminants . 77 6.4.4 Large contaminants . 78 6.4.5 Evaluating jet flow . 79 6.4.6 Ontario Science Centre South Hydraulophone: Summary of data . 80 6.4.7 Summary . 81 6.5 Water jets as pixels: Water fountains as both sensors and displays . 82 6.5.1 Overview . 82 6.5.2 Water jets as interactive media . 82 6.5.3 Bidirectionality from flow control . 83 6.5.4 Sensory consistency, by valve design . 83 6.5.5 Application . 85 6.5.6 Programmatic sequence of the educational game . 86 6.5.7 Distinguishing sequential notes, and implications for level of difficulty 86 6.5.8 Contributions of the author (bidirectional fluid user-interfaces) . 89 6.6 Computer Vision for fluid user-interfaces . 90 6.7 Summary . 90 7 Conclusion 91 Bibliography 92 vii Chapter 1 Introduction: Musical Instruments in All States of Matter 1.1 Classification of musical instruments as a context for hydraulophone Sound produced by liquid matter is a physical phenomenon which has interesting applica- tions in engineering, science and music. As liquid sound phenomena can, for one, be used in musical instruments, the instruments and phenomena are perhaps best understood by first examining their context within the realm of existing musical instruments1. Traditionally, musical instruments have often been grouped into three categories: strings, wind and percussion. In 1703, S´ebastiende Brossard did precisely that [16] when he classified instruments in his Dictionnaire de Musique. Since then, various researchers have devised their own classification schemes to account for instruments newly discovered among cultures worldwide. In 1914, Erich von Hornbostel and Curt Sachs modified previous classification schemes, creating four main categories in an attempt to include all 1In this thesis, ideas on musical instrument organology follow the train of thought presented by S. Mann in [14] and later by Mann and Janzen in [16]. 1 Chapter 1. Introduction: Musical Instruments in All States... 2 conceivable instruments: idiophones (vibrations from the \substance of the instrument itself, owing to its solidity and elasticity ... without requiring stretched membranes or strings"), membranophones (having stretched membranes), chordophones (having long strings), and aerophones (using wind) [7, p.169]. As well, Francis Galpin (1910) and Sachs (1940) each added a category to account for instruments which made use of electricity [7, p.176][26]. Sachs used the name \elec- trophones" [26]. Among present-day organologists and ethnomusicologists, there is general agreement that this classification scheme should depend only on how sound is initially produced in the instrument [16]. For example, the fifth category, \electrophones", is reserved for instruments such as the Theremin, ondes-Martenot, and the modern synthesizer, which actually generate the sound signal source itself [16] electrically, regardless of whether electricity is used in some other fashion. An example of using electricity in some other fashion lies in a pipe organ. Even if there is an electric motor to pump wind into the organ, rather than a hand-operated blower, the pipe organ is still an aerophone. As well, it is still an aerophone regardless of whether the valves delivering wind to the pipes are mechanically, electrically, pneumatically, or hydraulically actuated [16]. The electric guitar serves as another example: Even if an effects pedal is added in order to post-process the audio signal, the electric guitar remains a chordophone. In describing the operation of an instrument, one could distinguish between how an instrument is controlled (i.e. that which comes before the initial sound production mechanism), how an instrument's sound is post-processed (that which comes after the sound production mechanism), and the physics of the sound production itself [16]. The physical state-of-matter of sound production played a role in a newer classification system proposed by Andre Shaeffner in 1932 [16], which he claimed was \exhaustive, potentially covering all real and conceivable instruments" [7, p176]. In the Schaeffner Chapter 1. Introduction: Musical Instruments in All States... 3 classification system, sound production in solid versus