UNIVERSIDADE DE COIMBRA MASTER THESIS Concatenative Synthesis Applied to Rhythm Author: Supervisors: Francisco MONTEIRO Dr. Amílcar CARDOSO Dr. Fernando PERDIGÃO Dr. Pedro MARTINS A thesis submitted in fulfillment of the requirements for the degree of Mestrado Integrado em Engenheria Eletrotécnica e Computadores in the Departmento de Engenheria Eletrotécnica e Computadores February 19, 2019 iii Declaration of Authorship I, Francisco MONTEIRO, declare that this thesis titled, “Concatenative Synthesis Applied to Rhythm” and the work presented in it are my own. I confirm that: • Where any part of this thesis has previously been submitted for a degree or any other qualification at this University or any other institution, this has been clearly stated. • Where I have consulted the published work of others, this is always clearly attributed. • Where I have quoted from the work of others, the source is always given. With the exception of such quotations, this thesis is entirely my own work. • I have acknowledged all main sources of help. • Where the thesis is based on work done by myself jointly with others, I have made clear exactly what was done by others and what I have contributed my- self. Signed: Date: v “When I’m writing, sometimes it gets to that place where I feel like the piece is writing itself and I’m trying not to get in the way.” John Zorn vii UNIVERSIDADE DE COIMBRA Abstract Faculdade de Ciências e Tecnologia Departmento de Engenheria Eletrotécnica e Computadores Mestrado Integrado em Engenheria Eletrotécnica e Computadores Concatenative Synthesis Applied to Rhythm by Francisco MONTEIRO Music and Technology have a long symbiotic history together and became even more related in recent times, when technology is so present in our everyday lives. Technological breakthroughs have greatly impacted the way many tasks are carried, including the music composition process. The digital era allowed the access to large databases of music and sounds on digital format and has changed the way we cre- ate music. There is an ongoing need to develop new tools and new ways to explore these new possibilities. Different forms of sound synthesis have surfaced over the years, including Granular and Concatenative Synthesis, which offered the ability to use these unlimited sound databases. Alongside with these, developments in the Music Information Retrieval field opened doors for sound interpretation, improv- ing the indexing ability of digital files. We have seen Concatenative Synthesis being applied in the past for speech synthesis and for music composition, but few applica- tions focused on live generation of rhythms. We propose a Concatenative Synthesizer oriented towards rhythmic composi- tion, which has the ability to create its own drum kit by interpreting an input drum audio signal. The system then recreates the input through different ways, includ- ing an variation algorithm based on Euclidean Rhythms. It was implemented in the programming language Max/MSP and the extension Max For Live, in order to make it usable in the DAW environment. We have also created a basic interface to interact with the user. In the end, we present a user-based evaluation and discuss the possible future developments. ix Acknowledgements To my family, the unwavering pillar of my life. To all of them, a big acknowledge- ment for the unconditional love. Julia for for the comprehension and eternal pa- tience. Diogo for always being there, even when I’m the worst. Mário for the eternal support and for being my biggest music influence. Arménio, Graciema, Emília and Adriano. All my uncles, aunts and cousins. You are everything to me. To my supervisors, Amílcar Cardoso, Pedro Martins and Fernando Perdigão for believing in my project, for their constant support, for always leaving the door open for me and bringing me down to earth when I needed. To all my friends. Antonio for all the help in this entire project and for the end- less discussions about music. Filipe for the tips, the design help and the being the greatest person on the planet. Bicá, Craveiro, Gonçalo, Martins, João, Joana, Ritas, Maria, Francisco, Bernardo, Miguel, Ivan, Gabriel, António and many many more, way too many to mention. Seriously, thank you all. To all my friends, colleagues and mentors from Radio Universidade de Coimbra who taught and still teach me everyday so much about music. A shoutout to the Magia Negra crew. To all those who participated in the tests. xi Contents Declaration of Authorship iii Abstract vii Acknowledgements ix 1 Introduction1 1.1 Context.....................................1 1.2 Goals......................................2 1.3 Motivation...................................2 1.4 Structure....................................3 2 State of the Art5 2.1 Automatic Composition...........................5 2.1.1 Algorithmic Composition......................6 2.1.2 Computer-Aided Algorithmic Composition...........8 2.2 Concatenative Synthesis...........................8 2.3 Music Information Retrieval......................... 11 2.3.1 Drum Transcription......................... 14 2.4 Conclusion................................... 16 3 Methodology 17 3.1 Conceptualization............................... 17 3.2 Objectives................................... 17 3.3 Process..................................... 18 3.4 Task Planning................................. 18 4 System Description 21 4.1 Definition................................... 21 4.2 Tools...................................... 22 4.2.1 Ableton Live............................. 22 4.2.2 Max/MSP............................... 23 4.2.3 Max For Live............................. 24 4.3 Implementation................................ 25 4.3.1 Unit Definition............................ 25 4.3.2 The Sampler: chucker~ ........................ 27 4.3.3 Segment Analysis........................... 29 4.4 Concatenative Synthesizer.......................... 32 4.4.1 Main Sequence............................ 32 4.4.2 Euclidean Rhythms.......................... 33 4.4.3 Concatenation Process........................ 35 4.5 Interface.................................... 37 4.6 Conclusion................................... 38 xii 5 Tests and Evaluation 39 5.1 Testing Method................................ 39 5.2 Questions................................... 40 5.3 Answers.................................... 40 6 Conclusions and Future Work 45 6.1 Future Work.................................. 46 Bibliography 49 A Max/MSP Patchers 53 A.1 Chucker Arguments............................. 53 A.2 Analysis.................................... 54 A.3 Synthesis.................................... 54 B Sequence Database Creation 59 B.1 Matlab Scripts................................. 59 B.1.1 main.m ................................. 59 B.1.2 ExtractDrum.m ............................ 60 B.1.3 WriteLine.m .............................. 61 B.1.4 LineCount.m .............................. 61 B.2 Drum Articulation.............................. 62 xiii List of Figures 2.1 Brian Eno’s Scape ...............................7 2.2 CataRT interface................................9 2.3 Eargram interface............................... 10 2.4 RhythmCAT interface............................ 11 2.5 Illustration of traditional drum instrument sounds............ 15 3.1 Task Planning................................. 19 4.1 Device Diagram................................ 21 4.2 Ableton Live 9 interface........................... 22 4.3 Ableton Temporal Control Bar........................ 22 4.4 MAX/MSP Final Patch............................ 23 4.5 Interface Patcher............................... 24 4.6 Excerpt of a Music Score........................... 25 4.7 Roland Tr-909................................. 26 4.8 Unquantized Drum Break Segmentation.................. 27 4.9 Quantized Drum Break Segmentation................... 28 4.10 Spectral Representation of a Drum Sound................. 29 4.11 Diagram representing the Analysis Process................ 30 4.12 Patcher: AnalysisSegm............................ 31 4.13 Fixed Segmentation Problem illustration.................. 32 4.14 Euclidean Rhythm Geometric Representation............... 34 4.15 Patcher: Concatenation Process....................... 35 4.16 Spectral Analysis of Sample Length Correction.............. 36 4.17 Max For Live Device Interface........................ 37 5.1 Test Form.................................... 41 5.2 Answers 1, 2, 3................................. 42 5.3 Answers 4, 5, 6................................. 43 A.1 Patcher: DrumChuckers........................... 53 A.2 Patcher: LDT kNN.............................. 54 A.6 Patcher: loadfile................................ 54 A.3 Patcher: FilterOutGhost........................... 55 A.4 Patcher: FindIsolated............................. 56 A.5 Patcher: findDrumLength.......................... 57 A.7 Patcher: LengthCorrection.......................... 58 xv List of Tables 2.1 MIR Subfields................................. 13 2.2 Drum Transcription Tasks.......................... 13 4.1 Example of chucker~ list based communication.............. 28 B.1 Drum Articulation Example......................... 62 xvii List of Abbreviations ADC Analog (to) Digital Conversion ADSR Attack Decay Sustain Release BFCC Bark (-) Frequency Cepstrum Coefficients BPM Beats Per Minute CAAC Computer (-) Aided Algorithmic Composition DAW Digital Audio Workstation DSP Digital Signal Processing GUI Graphical User Interface tools HMM Hidden Markov Model LVT Live Drum Transcription