Overview of Research at IITB Computational studies on Hindustani music

4th CompMusic Seminar IIT Bombay, May 2014

Preeti Rao Department of Electrical Engineering I.I.T. Bombay

1 The growing access to great music…

Is largely due to the creation of digital music archives by “digitization” of the vast stores of broadcast, concert and studio recordings.

Digitization refers to conversion of the sound to sampled waveforms that can be rendered on any digital device. Associated material such as information, photographs, images, and writings about the music add further value.

2 The growing access to great music…

Is largely due to the creation of digital music archives by “digitization” of the vast available stores of broadcast, concert and studio recordings.

Digitization refers to conversion of the sound to sampled waveforms that can be rendered on any digital device. Associated material such as information, photographs, images, and writings about the music add further value.

Our work

Information

audio waveform

3 Music CD cover information… (YouTube has even less!)

4 Some goals of the project • Automatic “tagging” of audio by genre, style, raga, tala and other discovered descriptors and relationships

• Automatic creation of “navigation layer” for performance audio recordings

• Facilitating discovery of musically relevant objects such as melodic and rhythmic motifs (recurring patterns)

• Building tools that facilitate musicological research on performance practices

5 Some goals of the project

• Automatic “tagging” of audio by genre, style, raga, tala and other discovered descriptors and relationships

• Automatic creation of “navigation layer” for concert audio

• Facilitating search for musically relevant objects such as melodic and rhythmic motifs

• Building tools that facilitate musicological research on performance practices

Common to all the above: Need for a music signal representation (aka features) and similarity measure (for classification)

6 Hindustani music descriptors/tags

• Artiste (instrument), accompanists

• Genre (dhrupad, khyal, tarana…), sub-genre (gharana)

• Concert structure and sections with timing – Bada khyal, Chhota khyal • Bandish: alap, vistaar, taan

• Raga, Tala, Laya of major sections

• Composition (bandish, identified by mukhda)

7 Hindustani music descriptors/tags

• Artiste (instrument), accompanists

• Genre (dhrupad, khyal, tarana…), sub-genre (gharana)

• Concert structure and sections with timing – Bada khyal, Chhota khyal • Bandish: alap, vistaar, taan

• Raga, Tala, Laya of major sections

• Composition (bandish, identified by mukhda)

The question Can the descriptions be obtained by audio content analysis and possibly enhanced with contextual semantic information?

8 Research challenges

• The audio signal processing is challenging due to the mixture of several instruments, strong diversity in the characteristics and the highly time-varying nature.

• Characterizing the major musical dimensions: melody and rhythm in a manner relevant to the genre.

• Modeling musicological knowledge and other implicit knowledge that listeners and musicians bring.

9 Kishori Amonkar: Deshkar, Gaud-Sarang Raga Deshkar

Raga Gaud Sarang

10 original resynthesized Raga Deshkar

Raga Gaud Sarang

11 Raga Deshkar

P P P P G G G G G G R S R R S R S S R S S D R S

Raga Gaud Sarang

R G S S S S S S S R D D S R D D D 12 Rhythmic structure Dhrupad alap, raga Yaman, Uday Bhawalkar

13 Concert segmentation Dhrupad alap, raga Yaman, Uday Bhawalkar

alap jod jhala 14 “Similarity” matrix and boundary detection function

15 Concert segmentation Khyal, raga Bhupali, Rashid Khan

Bada khyal in Madhya laya,Tintal; Chhota khyal in Drut laya,Tintal Rhythmogram of Bhoopali, RK Bada Khyal in Madhya laya,Tintal Chota khyal in Drut laya,Tintal

16 Concert segmentation Khyal, raga Bhupali, Rashid Khan

Bada khyal in Madhya laya,Tintal; Chhota khyal in Drut laya,Tintal Rhythmogram of Bhoopali, RK Bada Khyal in Madhya laya,Tintal Chota khyal in Drut laya,Tintal

alap bada khyal chhota khyal bandish vistar sargam tan bandish tan 17 Melodic motif (mukhda) detection

Kishori Amonkar, Deshkar, Tintal Bandish: Piya Jaag 18 Melodic motif (mukhda) detection

Kishori Amonkar, Deshkar, Tintal Bandish: Piya Jaag 19 Within-concert variability of motif

20 Within-concert variability across concerts Intra-phrase class distance distribution

Kafi

21 Svara intonation: raga characteristics from pitch distribution

Pt. Vidhyadhar Vyas Marwa and Puriya (share the same swaras)

22 Style (genre) classification

Raga Todi: Malini Rajurkar

Raga Subhapanthuvarali: M S Subhalaxmi 23 Ontology for Indian music

Learning metadata (textual) from forums by using NLP techniques to learn relationships between entities.

• Augment audio-based music ontology for Indian music with information extracted from online music forums to achieve superior retrieval systems for Indian music.

• Challenges: text sources are unstructured, ungrammatical…

24 Summary

• Melody and rhythm representations that are musicologically informed can be useful in the description of music recordings and for measuring musical similarity.

• Music knowledge can be vital. Modeling knowledge sources however is a challenge.

25 Thank you

Department of Electrical Engineering , IIT Bombay 26