AN INVESTIGATION OF AUDIO SIGNAL-DRIVEN SOUND SYNTHESIS WITH A FOCUS ON ITS USE FOR BOWED STRINGED SYNTHESISERS by CORNELIUS POEPEL A thesis submitted to the University of Birmingham for the degree of DOCTOR OF PHILOSOPHY Department of Music The University of Birmingham June 2011 University of Birmingham Research Archive e-theses repository This unpublished thesis/dissertation is copyright of the author and/or third parties. The intellectual property rights of the author or third parties in respect of this work are as defined by The Copyright Designs and Patents Act 1988 or as modified by any successor legislation. Any use made of information contained in this thesis/dissertation must be in accordance with that legislation and must be properly acknowledged. Further distribution or reproduction in any format is prohibited without the permission of the copyright holder. Contents 1 Introduction 1 2 Motivation 10 2.1 Personal Experiences . 10 2.2 Using Potentials . 14 2.3 Human-Computer Interaction in Music . 15 3 Design of Interactive Digital Audio Systems 21 3.1 From Idea to Application . 24 3.2 System Requirements . 26 3.3 Formalisation . 30 3.4 Specification . 32 3.5 Abstraction . 33 3.6 Models . 36 3.7 Formal Specification . 39 3.8 Verification . 41 4 Related Research 44 4.1 Research Field . 44 i CONTENTS ii 4.1.1 Two Directions in the Design of Musical Interfaces . 46 4.1.2 Controlling a Synthesis Engine . 47 4.1.3 Building an Interface for a Trained Performer . 48 4.2 Computer-Based Stringed Instruments . 49 4.3 Using Sensors . 52 4.3.1 Sensor-Bow . 52 4.3.2 SuperPolm . 53 4.3.3 Funny Fiddle . 56 4.3.4 BoSSA . 57 4.3.5 Hyperbow . 60 4.3.6 vBow . 62 4.3.7 IRCAM Bow . 65 4.4 Using Sensors and the Audio Signal . 67 4.4.1 Celletto . 67 4.4.2 Bowed Stringed Hyperinstruments . 68 4.4.3 Sbass . 71 4.4.4 Digital Stradivarius Project . 72 4.4.5 Eviolin . 76 4.4.6 Overtone Violin . 80 4.4.7 Augmented Cello . 81 4.5 Using the Audio Signal . 83 4.5.1 Max Mathews' Electric Violin . 83 4.5.2 Meta Viola . 84 4.5.3 Audio-Driven Timbre Generator . 85 4.5.4 Voice-Driven Synthesis . 87 CONTENTS iii 4.6 Adaptive Audio Effects . 89 4.6.1 Mu-Tron . 89 4.6.2 Adaptive Digital Audio Effects . 90 4.7 Discussion of Related Research . 92 4.7.1 Aims and Requirements . 93 4.7.2 Formalisation . 95 4.7.3 Specification and Measurement . 96 4.7.4 Mechanical Models of the Performer . 100 4.7.5 Playing or Controlling . 101 4.7.6 Views on Musical Instruments . 105 4.7.7 Conclusion and Questions . 110 5 Approaches to Digital Sound Synthesis 112 5.1 The Conventional Parameter-Driven Approach . 114 5.1.1 Historical Foundations . 117 5.1.2 Formalisation Strategies . 124 5.1.3 Problems and Common Improvements . 127 5.2 Performer and Instrument . 130 5.2.1 Performers' Actions . 131 5.2.2 Gesture . 133 5.2.3 Measurement of Performers' Actions . 135 5.2.4 Openness, Transparency and Intimacy . 143 5.3 String Players' Requirements . 147 5.3.1 String Players' Actions . 149 5.3.2 Conclusions for Requirements . 157 CONTENTS iv 5.4 A Signal-Driven Approach to Sound Synthesis . 163 5.4.1 The String as Information-Carrier . 164 5.4.2 Basic Architecture . 165 5.5 Hypothesis . 169 5.5.1 Definition of Terms . 171 5.5.2 Method . 172 5.5.3 Artistic and Scientific Approaches . 173 6 Implementation 175 6.1 Soft- and Hardware . 176 6.2 Signal-Driven Synthesis Methods . 178 6.2.1 Selection of Synthesis Methods . 178 6.2.2 Subtractive Synthesis . 182 6.2.3 Simple FM Synthesis . 184 6.2.4 Multiple Carrier FM Synthesis . 187 6.2.5 Multiple Modulator FM Synthesis . 189 6.2.6 Feedback FM Synthesis . 192 6.2.7 Phase Modulation Synthesis . 194 6.2.8 Combinations in FM Synthesis . 197 6.2.9 Granulation . 204 6.3 Signal-Driven Implementations by Other Researchers . 206 6.3.1 Self Modulation . 207 6.3.2 Adaptive FM Synthesis . 208 6.4 Sensor Systems . 209 6.4.1 Bodyless Instrument and Audio Signal Transducer . 210 CONTENTS v 6.4.2 Signal Parameter Extraction . 215 6.5 Signal-Driven Synthesis: Technical Observations . 217 6.5.1 Reduction of Latency . 218 6.5.2 Signal-Dependent Sound Disturbances . 219 6.5.3 Amplitude Range as a Timbre Parameter . 219 6.6 Design of Sounds Using Signal-Driven Methods . 220 6.6.1 FM Synthesis . 222 6.6.2 Subtractive Synthesis . 223 6.6.3 Granulation . 224 6.7 A Different View of Sound Synthesis . 225 7 Evaluation 230 7.1 Personal Experiences . 231 7.2 String Players Testing the Instrument . 236 7.3 Empirical Study with String Players . 242 7.3.1 Question . 244 7.3.2 Premises . 244 7.3.3 Design of the Study . 246 7.3.4 Procedure . 250 7.3.5 Analysis . 251 7.3.6 Results . 252 7.4 Empirical Study with Listeners . 257 7.4.1 Question . 257 7.4.2 Premises . 258 7.4.3 Design of Study . 258 CONTENTS vi 7.4.4 Procedure . 265 7.4.5 Analysis . 266 7.4.6 Results . 273 7.5 Compositions and Performances . 276 7.6 User Group hot strings SIG . 282 7.7 Hypothesis and Results . 289 7.7.1 Validity of Comparison . 290 7.7.2 Discussion of Results . 292 8 Future Work 295 A Materials: Performers' Study 302 A.1 Questionnaire . 302 B Materials: Listeners' Study 313 B.1 Questionnaire . 313 B.2 Results . 334 C Code 342 D Bow Tracking System 349 E DVD and Audio CDs 354 E.1 Audio CDs . 354 E.2 DVD . 357 References 359 List of Figures 3.1 Levels and transformation in the development process. 25 3.2 Verification relating to different levels. 41 4.1 EMS AKS Synthi A. 47 4.2 Number of research papers. 51 4.3 Bow and body of the SuperPolm. 55 4.4 vBow architecture. 63 4.5 IRCAM bow and augmented violin. 66 4.6 Traditional violin with sensors and Marching Cello. 75 4.7 Overtone violin. 82 5.1 Basic principle of Parameter-Driven Sound Synthesis. 115 5.2 Example of unit generators of MUSIC V. 119 5.3 Common areas of improvement. 129 5.4 Abstraction of playing parameters. 137 5.5 Model of playing parameters. 138 5.6 Ideal architecture of ASDSS. 165 5.7 Basic architecture of ASDSS. 169 vii LIST OF FIGURES viii 6.1 Signal-driven subtractive synthesis. 183 6.2 Parameter-driven simple FM synthesis. 185 6.3 Signal-driven simple FM synthesis. 186 6.4 Signal-driven multiple carrier FM synthesis. 188 6.5 Signal-driven multiple modulator FM synthesis. 191 6.6 Signal-driven feedback FM synthesis. 193 6.7 Signal-driven simple PM synthesis. 195 6.8 Signal-driven multiple modulator PM synthesis. 196 6.9 Operator matrix of.