A Knowledge-Based Approach to Computational Analysis of Melodic Intonation in Indian Art Music
Total Page:16
File Type:pdf, Size:1020Kb
Towards a multimodal knowledge base for Indian art music: A case study with melodic intonation Gopala Krishna Koduri TESI DOCTORAL UPF / 2016 Director de la tesi Dr. Xavier Serra i Casals Music Technology Group Department of Information and Communication Technologies Copyright © Gopala Krishna Koduri 2016 http://compmusic.upf.edu/phd-thesis-gkoduri Licensed under Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 You are free to share – to copy and redistribute the material in any medium or format under the following conditions: • Attribution – You must give appropriate credit, provide a link to the li- cense, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. • Noncommercial – You may not use this work for commercial purposes. • No Derivatives – If you remix, transform, or build upon the material, you may not distribute the modified material. The doctoral defense was held on …………………………………….. at the Universitat Pompeu Fabra and scored as …………………………………………………………………. Dr. Xavier Serra Casals Thesis Supervisor Universitat Pompeu Fabra, Barcelona Dr. Anja Volk Thesis Committee Member Utrecht University, Netherlands Dr. Baris Bozkurt Thesis Committee Member Koç University, Turkey Dr. George Fazekas Thesis Committee Member Queen Mary University of London, UK To Amma. vi This thesis has been carried out between Oct. 2011 and Oct. 2016 attheMu- sic Technology Group (MTG) of Universitat Pompeu Fabra (UPF) in Barcelona (Spain), supervised by Dr. Xavier Serra Casals. The work in ch. 3 has been con- ducted in collaboration with Dr. Preeti Rao (IIT-B, Mumbai, India). The work in ch. 5 and part of ch. 6 has been conducted in collaboration with Dr. Joan Serrà (Telefónica R&D, Barcelona, Spain). The work in ch. 7 has been carried out with help from the CompMusic team, mainly Vignesh Ishwar, Ajay Srinivasamurthy and Hema murthy. The work in ch. 8 has been conducted in collaboration with Sertan Senturk in the CompMusic team. This work has been supported by the Dept. of Information and Communication Technologies (DTIC) PhD fellowship (2011-16), Universitat Pompeu Fabra and the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583). Acknowledgements IIIT-H (International Institute of Information Technology - Hyderabad), my alma mater, had a lasting impact on my thought processes and life goals. More im- portantly, my experiences during the time there helped me realize the relevance of inclusive growth and development to Indian society, which is etched with di- versity in almost every aspect of life. My further academic pursuit is greatly influenced by this. I’m deeply thankful to everyone who has been part of that journey, especially Prof. Bipin Indurkhya who encouraged us to pursue our in- terests in a stress-free environment and provided us ample opportunities to excel, and Pranav Kumar Vasishta, who with his constructive discourses has shed much light on engaging topics ranging from individual freedom to societal structures. It was during my internship with Preeti Rao in IIT-B (Indian Institute of Technol- ogy - Bombay) during 2010, that I really found camaraderie in my academic work. I’m very thankful to her for providing that opportunity. It was also there that I first met Xavier Serrra who was there to present the work done at MTG and an introduction to CompMusic project. It is also there that I first met Sankalp Gu- lati, whose guidance in melodic-analysis had been immensely helpful when I was just getting started. After the internship that Xavier offered me at MTG, itwas crystal clear that the objectives of CompMusic project completely aligned with whatever I planned after my stint at IIIT-H. My stay at MTG has been a thoroughly rewarding experience to say the least. To have worked with/alongside the most brilliant minds in the MIR domain is humbling. Xavier’s exceptional foresight and prowess in planning are something that I will always reflect on as a constant source of learning. Without his guid- ance and support, this work would not have been the same. I’m deeply grateful to him for this opportunity which was provided with plentiful freedom and en- vii viii couragement. I thank Joan Serrà who guided my work on intonation description during the initial years. His attention to detail and academic rigor are inspiring. Thanks to Emilia Gómez for letting me audit her course on basics of signalpro- cessing which later helped me greatly in my work. Thanks to Hendrik Purwins who always had been very approachable and the goto guide for anything related to machine learning. The CompMusic project has brought together an amazing group of people from around the world, and it is fortunate for having been part of it. They have not only been my colleagues at work, but also the social circle outside work who have lent every help in calling Barcelona my home. Special thanks to (in no particular order) Ajay Srinivasamurthy, Sankalp Gulati, Sertan Senturk, Vignesh Ishwar, Kaustuv Kanti Ganguli, Marius Miron, Swapnil Gupta, Mohamed Sordo, Alastair Porter, Sergio Oramas, Rafael Caro Rapetto, Rong Gong, Frederic Font, Georgi Dzhambazov, Andrés Ferraro, Vinutha Prasad, Shuo Zhang, Shrey Dutta, Padi Sarala, Hasan Sercan Atlı, Burak Uyar, Joe Cheri Ross, Sridharan Sankaran, Ak- shay Anantapadmanabhan, Jom Kuriakose, Jilt Sebastian and Amruta Vidwans. I had an opportunity to interact with almost everyone at MTG which helped me in one way or another. Thanks to Srikanth Cherla, Dmitry Bogdanov, Juan José Bosch, Oscar Mayor, Panos Papiotis, Agustín Martorell, Sergio Giraldo, Nadine Kroher, Martí Umbert, Sebastián Mealla, Zacharias Vamvakousis, Julián Urbano, Jose Zapata, Justin Salamon and Álvaro Sarasúa. I thank Cristina Garrido, Alba Rosado and Sonia Espí for making my day-to-day life at MTG a lot easier. Also thanks to Lydia García, Jana Safrankova, Vanessa Jimenez and all other university staff for helping me wade through the administrative formalities with ease. Spe- cial thanks to my multilingual colleagues, Frederic Font and Rafael Caro Rapetto, for helping me with Catalan and Spanish translations of the abstract. I thank Hema murthy, Preeti rao, T. M. Krishna, Suvarnalata Rao, N. Ramanathan and M. Subramanian for their feedback and help in several important parts of the thesis. I thank all my co-authors and collaborators for their contributions. I thank the (anonymous or otherwise) reviewers of several publications for their detailed feedback which helped me constantly improve this work. I also thank the members of the thesis committee for offering their invaluable time and feedback. Barcelona is definitely not the same without the friends I made in this vibrant city. For all the treks, tours, dinners, cultural exchanges, volleyball and mafia games, many thanks to Ratheesh, Manoj, Shefali, Jordi, Indira, Laia, Alex, Srini- bas, Praveen, Geordie, Princy, Kalpani, Windhya, Ferdinand, Pallabi, Neus and Waqas. Warmest gratitude to Eva, the most empathetic person I have ever known. To a part-time introvert like me, it is just magic how she makes friends in a jiffy ix and make them feel comfortable as if they have known each other for their whole life! Finally, none of this would have been possible but for the iron will of my mother who religiously sacrificed anything and everything for my sister’s and my ed- ucation. For somebody who is barely exposed to the world outside our home, I’m always puzzled by her mental stamina. Nothing can possibly express my gratitude to her, and my father who firmly stood by her. Thanks to the more- courageous-more-patient version of me - my sister, and the wonderful being she hooked me up with - my wife, for bearing and handling my sudden disappear- ances in the hours of the need during these years. Gopala Krishna Koduri 2nd November 2016 Abstract This thesis is a result of our research efforts in building a multi-modal knowledge- base for the specific case of Carnatic music. Besides making use of metadata and symbolic notations, we process natural language text and audio data to ex- tract culturally relevant and musically meaningful information and structuring it with formal knowledge representations. This process broadly consists of two parts. In the first part, we analyze the audio recordings for intonation descrip- tion of pitches used in the performances. We conduct a thorough survey and evaluation of the previously proposed pitch distribution based approaches on a common dataset, outlining their merits and limitations. We propose a new data model to describe pitches to overcome the shortcomings identified. This expands the perspective of the note model in-vogue to cater to the conceptualization of melodic space in Carnatic music. We put forward three different approaches to retrieve compact description of pitches used in a given recording employing our data model. We qualitatively evaluate our approaches comparing the representa- tions of pitched obtained from our approach with those from a manually labeled dataset, showing that our data model and approaches have resulted in represen- tations that are very similar to the latter. Further, in a raaga classification task on the largest Carnatic music dataset so far, two of our approaches are shown to outperform the state-of-the-art by a statistically significant margin. In the second part, we develop knowledge representations for various concepts in Carnatic music, with a particular emphasis on the melodic framework. We discuss the limitations of the current semantic web technologies in expressing the order in sequential data that curtails the application of logical inference. We present our use of rule languages to overcome this limitation to a certain extent. We then use open information extraction systems to retrieve concepts, entities and their relationships from natural language text concerning Carnatic music.