Phrase-based Rāga Recognition using Vector Space Modeling Sankalp Gulati*, Joan Serrà^, Vignesh Ishwar*, Sertan Şentürk* and Xavier Serra* *Music Technology Group, Universitat Pompeu Fabra, Barcelona, Spain ^Telefonica Research, Barcelona, Spain Te 41st IEEE International Conference on Acoustics, Speech and Signal Processing Shanghai, China, 2016 Indian Art Music Indian Art Music Indian subcontinent: India, Pakistan, Bangladesh, Srilanka, Nepal and Bhutan Indian Art Music Indian Art Music: Hindustani music Indian Art Music: Carnatic music Rāga: melodic framework q Grammar for improvisation and composition Automatic Rāga Recognition Yaman Training corpus Shankarabharnam Todī Darbari Kalyan Bageśrī Kambhojī Hamsadhwani Des Harikambhoji Kirvani Rāga recognition Atana System Behag Kapi Begada Rāga label Rāga Characterization: Svaras 10 30 time (s) Rāga Characterization: Svaras 10 30 time (s) Rāga Characterization: Svaras 10 30 time (s) 1 Lower Sa Higher Sa 0.8 Tonic middle Sa 0.6 0.4 Normalized salience 0.2 0 -12050 100 150 0 200 250 120300 Frequency (bins), 1 bin = 10 Cents, Ref = 55 Hz Frequency (cents), fref = tonic frequency q P. Chordia and S. Şentürk, “Joint recognition of raag and tonic in North Indian music,” Computer Music Journal, vol. 37, no. 3, pp. 82–98, 2013. q G. K. Koduri, S. Gulati, P. Rao, and X. Serra, “Rāga recognition based on pitch distribution methods,” Journal of New Music Research, vol. 41, no. 4, pp. 337–350, 2012. Rāga Characterization: Intonation 10 30 time (s) 1 Lower Sa Higher Sa 0.8 Tonic middle Sa 0.6 0.4 Normalized salience 0.2 0 -12050 100 150 0 200 250 120300 Frequency (bins), 1 bin = 10 Cents, Ref = 55 Hz Frequency (cents), fref = tonic frequency Rāga Characterization: Intonation 10 30 time (s) 1 Lower Sa Higher Sa 0.8 Tonic middle Sa 0.6 0.4 Normalized salience 0.2 0 -12050 100 150 0 200 250 120300 Frequency (bins), 1 bin = 10 Cents, Ref = 55 Hz Frequency (cents), fref = tonic frequency q G.K.Koduri,V.Ishwar,J.Serrà,andX.Serra,“Intonation analysis of rāgas in Carnatic music,” Journal of New Music Research, vol. 43, no. 1, pp. 72–93, 2014. q H. G. Ranjani, S. Arthi, and T. V. Sreenivas, “Carnatic music analysis: Shadja, swara identifcation and raga verifcation in alapana using stochastic models,” in IEEE WASPAA, 2011, pp. 29–32. Rāga Characterization: Ārōh-Avrōh 10 30 time (s) q Ascending-descending svara pattern; melodic progression Rāga Characterization: Ārōh-Avrōh 10 30 time (s) Melodic Progression Templates N-gram Distribution Hidden Markov Model q S. Shetty and K. K. Achary, “Raga mining of indian music by extracting arohana-avarohana pattern,” Int. Journal of Recent Trends in Engineering, vol. 1, no. 1, pp. 362–366, 2009. q V. Kumar, H Pandya, and C. V. Jawahar, “Identifying ragas in indian music,” in 22nd Int. Conf. on Pattern Recognition (ICPR), 2014, pp. 767–772. q P. V. Rajkumar, K. P. Saishankar, and M. John, “Identifcation of Carnatic raagas using hidden markov models,” in IEEE 9th Int. Symposium on Applied Machine Intelligence and Informatics (SAMI), 2011, pp. 107–110. Rāga Characterization: Melodic motifs 10 30 time (s) Rāga Characterization: Melodic motifs 10 30 time (s) Rāga A Rāga B Rāga C q R. Sridhar and T. V. Geetha, “Raga identifcation of carnatic music for music information retrieval,” International Journal of Recent Trends in Engineering, vol. 1, no. 1, pp. 571–574, 2009. q S. Dutta, S. PV Krishnaraj, and H. A. Murthy, “Raga verifcation in carnatic music using longest common segment set,” in Int. Soc. for Music Information Retrieval Conf. (ISMIR), pp. 605-611,2015 Goal q Automatic rāga recognition Yaman Training corpus Shankarabharnam Todī Darbari Kalyan Bageśrī Kambhojī Hamsadhwani Des Harikambhoji Kirvani Rāga recognition Atana System Behag Kapi Begada Rāga label Goal q Automatic rāga recognition Yaman Training corpus Shankarabharnam Todī Darbari 10 30 Kalyan 10 30 Bageśrī 10 10 30time (s) 30 Kambhojī time (s) 10 time (s) 30 10Hamsadhwani10 3030 time (s) 1010 Des 3030 10 time1010 (s) 30 Harikambhoji 30time30 time (s) (s) 10 30 1010 timetime (s) (s) 3030 10 time (s) timetime (s) (s) 30 1010 Kirvani 3030 Rāga recognitiontime (s) Atana timetime (s) (s) 10 time (s) 30 1010 timetime (s) (s) 3030 10 System 30 1010 Behag timetime (s) (s) 3030 10 time (s) 30 1010 Kapi 3030 time (s) Begada timetime (s) (s) 10 time (s) 30 1010 timetime (s) (s) 3030 10 time (s) 30 1010 timetime (s) (s) 3030 10 time (s) 30 1010 timetime (s) (s) 3030 R10āga label time (s) 30 1010 timetime (s) (s) 3030 10 time (s) 30 1010 timetime (s) (s) 3030 time (s) 1010 timetime (s) (s) 3030 1010 timetime (s) (s) 3030 1010 timetime (s) (s) 3030 timetime (s) (s) Block Diagram: Proposed Approach Carnatic music collection Melodic Pattern Discovery Data Intra-recording Inter-recording Processing Pattern Discovery Pattern Detection Melodic Pattern Clustering Pattern Network Similarity Community Generation Thresholding Detection Feature Extraction Vocabulary Term-frequency Feature Extraction Feature Extraction Normalization Feature Matrix Block Diagram: Pattern Discovery Carnatic music collection Melodic Pattern Discovery Data Intra-recording Inter-recording Processing Pattern Discovery Pattern Detection Melodic Pattern Clustering Pattern Network Similarity Community Generation Thresholding Detection Feature Extraction Vocabulary Term-frequency Feature Extraction Feature Extraction Normalization Feature Matrix Block Diagram: Pattern Clustering Carnatic music collection Melodic Pattern Discovery Data Intra-recording Inter-recording Processing Pattern Discovery Pattern Detection Melodic Pattern Clustering Pattern Network Similarity Community Generation Thresholding Detection Feature Extraction Vocabulary Term-frequency Feature Extraction Feature Extraction Normalization Feature Matrix Block Diagram: Feature Extraction Carnatic music collection Melodic Pattern Discovery Data Intra-recording Inter-recording Processing Pattern Discovery Pattern Detection Melodic Pattern Clustering Pattern Network Similarity Community Generation Thresholding Detection Feature Extraction Vocabulary Term-frequency Feature Extraction Feature Extraction Normalization Feature Matrix Block Diagram: Pattern Discovery Carnatic music collection Melodic Pattern Discovery Data Intra-recording Inter-recording Processing Pattern Discovery Pattern Detection Melodic Pattern Clustering Pattern Network Similarity Community Generation Thresholding Detection Feature Extraction Vocabulary Term-frequency Feature Extraction Feature Extraction Normalization Feature Matrix Block Diagram: Pattern Discovery Carnatic music collection Melodic Pattern Discovery Data Intra-recording Inter-recording Processing Pattern Discovery Pattern Detection Melodic Pattern Clustering Pattern Network Similarity Community S. Gulati, J. SerràGeneration, V. Ishwar , andThresholding X. Serra, “Mining melodicDetection patterns in large audio collections of Indian art music,” in Int. Conf. on Signal Image Feature Extraction Technology & Internet Based Systems - MIRA, Marrakesh, Morocco, 2014, pp. 264–271.Vocabulary Term-frequency Feature Extraction Feature Extraction Normalization Feature Matrix Proposed Approach: Pattern Discovery Data Intra-recording Inter-recording Processing Pattern Discovery Pattern Detection -Predominant pitch estimation -Downsampling -Hz to Cents -Tonic normalization -Brute-force segmentation -Uniform Time-scaling -Segment filtering Flat Non-flat Proposed Approach: Pattern Discovery Data Intra-recording Inter-recording Processing Pattern Discovery Pattern Detection Recording 1 Recording 2 Seed Patterns Recording N Proposed Approach: Pattern Discovery Data Intra-recording Inter-recording Processing Pattern Discovery Pattern Detection Inter-recording search Rank-refinement Block Diagram: Pattern Clustering Carnatic music collection Melodic Pattern Discovery Data Intra-recording Inter-recording Processing Pattern Discovery Pattern Detection Melodic Pattern Clustering Pattern Network Similarity Community Generation Thresholding Detection Feature Extraction Vocabulary Term-frequency Feature Extraction Feature Extraction Normalization Feature Matrix Proposed Approach: Pattern Clustering Pattern Network Similarity Community Generation Thresholding Detection Undirectional q M. EJ Newman, “Te structure and function of complex networks,” Society for Industrial and Applied Mathematics (SIAM) review, vol. 45, no. 2, pp. 167–256, 2003. Proposed Approach: Pattern Clustering Pattern Network Similarity Community Generation Thresholding Detection * Ts q M. EJ Newman, “Te structure and function of complex networks,” Society for Industrial and Applied Mathematics (SIAM) review, vol. 45, no. 2, pp. 167–256, 2003. q S. Maslov and K. Sneppen, “Specifcity and stability in topology of protein networks,” Science, vol. 296, no. 5569, pp. 910– 913, 2002. Proposed Approach: Pattern Clustering Pattern Network Similarity Community Generation Thresholding Detection q V. D. Blondel, J. L. Guillaume, R. Lambiotte, and E. Lefebvre, “Fast unfolding of communities in large networks,” Journal of Statistical Mechanics: Teory and Experiment, vol. 2008, no. 10, pp. P10008, 2008. q S Fortunato, “Community detection in graphs,” Physics Reports, vol. 486, no. 3, pp. 75–174, 2010. Proposed Approach: Pattern Clustering Pattern Network Similarity Community Generation Thresholding Detection q V. D. Blondel, J. L. Guillaume, R. Lambiotte, and E. Lefebvre, “Fast unfolding of communities in large networks,” Journal of Statistical Mechanics: Teory and Experiment, vol. 2008, no. 10, pp. P10008, 2008. q S Fortunato,