Automatic Chord-Scale Type Detection Using Chroma Features
Total Page:16
File Type:pdf, Size:1020Kb
Master thesis on Sound and Music Computing Universitat Pompeu Fabra Automatic Chord-Scale Type Detection using Chroma Features Emir Demirel Supervisor: Baris Bozkurt July 2018 Master thesis on Sound and Music Computing Universitat Pompeu Fabra Automatic Chord-Scale Type Detection using Chroma Features Emir Demirel Supervisor: Baris Bozkurt July 2018 Contents 1 Introduction 1 1.1 Motivation . 2 1.2 Conceptual Background . 4 1.3 On-line Music Courses . 6 1.4 MusicCritic . 8 2 State of the Art 10 2.1 Review of Chord-Scale Detection Methods . 10 2.2 Review on Improvisation Assessment Metrics . 14 3 Datasets 15 3.1 The Chord-Scale Dataset . 15 3.2 Data from MusicCritic . 17 4 Feature Extraction 20 4.1 Preprocessing . 20 4.2 Chroma Feature Extraction . 24 4.3 Reference Frequency Determination for Pitch-Class Mapping . 27 4.4 Post-Processing . 27 5 Classification 32 5.1 Template Based Additive Likelihood Estimation . 32 5.2 Support Vector Classification . 34 6 Experiments 39 6.1 Evaluation on the Chord-Scale Dataset . 39 6.2 Evaluation of Student Performances using MusicCritic framework . 41 7 Results 46 7.1 Evaluation on the Chord-Scale Dataset . 46 7.2 Evaluation on Student Performances in the Scale Exercise . 53 8 Discussions 58 9 Conclusion 60 List of Figures 62 List of Tables 64 Bibliography 65 Acknowledgement There are many people that I would like to express my gratitudes. It has been a great experience to be involved in the research done in Music Technology Group (MTG) of Universitat Pompeu Fabra (UPF) in Barcelona. From the first day to the last, I have been constantly in the process of learning. Thank you all MTG people for making this research possible. First, I would like to thank Xavier Serra for providing me with the opportunity to work and do research in MTG, and moreover being a mentor for me since the first day I came to Barcelona. I would like to thank Baris Bozkurt, my supervisor, for all of his technical and personal support throughout my masters studies, and being so patient for all my questions throughout my research. I was very lucky to be able to work within TECSOME project with such great people, who were great inspirations for me in my research. I would like to thank Vsevolod Eremenko, Oriol Romani , Rong Gong, Rafael Caro Repetto, Blazej Kotowski for all of their great ideas and support. Thank you all the researchers in MTG, for their valuable conversations, enthusiasm in music and research, and more so, for being true inspirations. I have learned so much valuable information and gained a priceless insight in Music and Informatics thanks to the top notch academic staff of UPF, especially within the Sound and Music Computing (SMC) masters program. Thank you Perfecto Herrera, Agustin Martorell, Sergi Jorda, Rafael Ramirez, Enric Gine, Alastair Porter, Dimitri Bogdanov, Frederic Font, Davinia Hernandez, and again, Xavier Serra and Baris Bozkurt. I would like to thank to the administration staff of MTG, Cristina Garrido and Sonia Espi Fernandez for their support and assistance. I would also like to specially thank to all of the fellow SMC masters students for being companions along this journey. It would have been impossible to come to Barcelona and continue my career as a researcher without the support of family, my mother Nevin Demirel, my father Mustafa Demirel and my sister, Gozde Demirel Akan. Their full support kept me going and motivated even in the most challenging times. Special thanks to Toprak Barut and Hikmet Altunbaslier, two brilliant musicians from Ankara, Turkey, who have made the computational part of my project possible. Also I would like to thank all my fellow musician friends from Ankara for all of their vision and understanding of music. Thank you Gurkan Uysal, my first music mentor, who has sparkled this passion of mine for music. Your initial guidance for music has always been invaluable for me. Abstract There has been great effort in designing data-driven applications which exploit com- putational methods to solve musically related problems in the research field of Music Information Retrieval. However, the current state of music modeling needs to be expanded in consideration with musical domain knowledge, as well as the human perception and cognition of music. In this thesis work, we study and evaluate differ- ent computational methods to carry out a "modal analysis" for Jazz improvisation performances by modeling the concept of "chord-scales". The Chord-Scale The- ory is a theoretical concept that explains the relationship between the harmonic context of a musical piece and possible scale types to be used in improvisation. This work proposes different computational approaches to detect (or recognize) the chord-scale type present in the target Jazz solo, given the harmonic context. The experiments are conducted on two different datasets which are created within the course of this work. One of the datasets is made publicly available. To achieve the task of chord-scale type detection, we exploit a rule-based and a supervised learn- ing method. The rule-based approach is developed in order to reveal possibilities for computational modeling of chord-scales. In the supervised learning algorithm, Support Vector Machines are chosen as classifiers. The classification of audio data is performed using chroma features. Furthermore, we conduct a case study on user (student) performance using "MusicCritic", which is a novel framework for auto- matic student performance assessment. This work has its value for conducting one of the first research on the numeric representations of chord-scales in improvised solo performances and pointing out several possible directions for exploring some of the core elements behind Jazz improvisation. Keywords: computational musicology; music information retrieval; chord-scale de- tection; chroma features; supervised learning; machine learning; Jazz improvisation Chapter 1 Introduction Being able to play an instrument and improvise with it requires high musical knowl- edge and vast amount of training. Most skilled and experienced Jazz improvisers exploit the musical concept called "scales" in their solos. In general, scale infor- mation gives a reflection of the harmonic structure of a musical piece. Therefore, learning how to play scales is a crucial step for both improving improvisational skills and having a stronger sense about musical harmony. In this work, we present a novel approach to detect or estimate the chord-scale type of a musical performance using both machine learning and rule-based methods that exploit tonal features. Moreover, our study examines the performance of other existing methods for scale estimation methods within the context of our research. By doing so, we aim to highlight the potential of proposed approach for automatic chord-scale detection task. We have created an open-source dataset to conduct our experiments, which consists of improvisation performances with manually annotated chord-scales. For demonstration of a use case of our technology, we exploit MusicCritic, which is a novel framework for automatic musical performance assessment. The student impro- visation performances obtained via MusicCritic are computationally analyzed and automatically assessed based on musical heuristics. 1 2 Chapter 1. Introduction 1.1 Motivation There has been great effort in the last few decades in designing computational mod- els to solve problems within the research field of Music Information Retrieval (MIR), using data-driven techniques like machine learning, deep learning, etc. However, the current state of music modeling needs to be expanded with a focus in music theo- retical (domain) knowledge, in consideration with human perception and cognition of music. A focus in theoretical knowledge to model "chord-scales" would help us discover and understand some of the core elements behind Jazz improvisation. In this thesis work, we study an essential theoretical concept in Jazz improvisation, the chord-scales, for the computational analysis of improvisation performances. Figure (1) Three dimensions of research in Music Information Retrieval There are numerous sources to get proper training and education to be able to understand and produce music and learn how to play an instrument and to be able to improvise with it. The availability of today’s on-line education tools provide a way to enhance this training process. Although these tools are useful for self-training, there is a space for improvement in terms of the effectiveness of the education output. First of all, the process of learning relies heavily on feedback on student’s performance. There has been a paradigm shift in the education system in the last passing decades 1.1. Motivation 3 due to the increasing number of on-line education platforms. With reference to the data collected by Class Central (https://www.classcentral.com/ ), the total number of students who enrolled in at least one on-line course has already exceed 58 million in 2016. Over 700 universities around the world have released free on-line courses. These on-line education platforms are able to reach a very large audience of students in various fields with the availability of Massive Open Online Courses (MOOCs) [1]. The absence of an automatic feedback mechanism requires human experts to grade the musical performances for the student, which could be highly time consuming in the context of MOOCs. The work of this paper aims to create and develop a system that could be used for providing automatic feedback to the students for improvisation or any other chord-scale related exercises within the on-line music learning context. This study also intends to touch upon the significance of scale-based analysis for the overall harmonic analysis of a musical excerpt. Chord-scales reflect many im- portant aspects regarding the harmonic content of a musical performance. Hence, the information regarding the concept of chord-scales may reveal new directions in computational musicology.