Presenting: Ariel Tangi YiYair RhtRechter Supervisor: Gidi Nave

29.6.2011 2

• Most of the people can recognize artists, what are the features that makes it possible? • How can we organize artists based on sound? 3

• Musicology research • Create dynamic playlists suitable with the user’s musical preferences 4

Music Library Musical map

Feature Mapping and Training Extraction Classification 5

• Eac h song diiddivide d into 3 segments of 30 second

RAP/ ROCK POP JAZZ CLASSIC ELECTRONIC HIPHIP--HOPHOP

Red Hot Britney Ella Infected Metallica Chili OutKast Bach Spears Fitzgera ld MhMushroom Peppers

Jimi Frank The Black Eyed The Doors Madonna Vivaldi DJ Tiesto Hendrix Sinatra Peas

Dream Sarah LTE Spice Girls Snoop Dogg Beethoven Armin Van Buuren Theater Vaughan

Symphony Queen Shakira 50 Cent Chopin Prodigy X Guns N Coldplay Eminem Pendulum Roses

Pink Floyd Nirvana Jamiroquai David Guetta

Leonard Pearl Jam ChCohen

Aerosmith 6

• Features represent Timbre, Genre and Mood ¾MFCC

MFCC Average MFCC Variance MFCC #1 MFCC #1 ¾ MFCC #2 MFCC #2 Spectral analysis MFCC #3 MFCC #3 MFCC #4 MFCC #4 MFCC #5 MFCC #5 MFCC #6 MFCC #6 MFCC #7 MFCC #7 MFCC #8 MFCC #8 ¾Tempo MFCC #9 MFCC #9 MFCC #10 MFCC #10 MFCC #11 MFCC #11 MFCC #12 MFCC #12 MFCC #13 MFCC #13 • Extracted using MIR Toolbox Spectral Average Spectral Variance

Centroid Centroid Spread Spread Skeness Skeness Flatness Flatness Low energy Low energy Roll-off Roll-off Zero-Cross Zero-Cross Irregularity Irregularity Brightness Brightness

Tempo 7

• Common tool in MIR and speech recognition • Approximates the human auditory system

DCT Spectrum Log Scaling (Discrete Cosine (Fourier Transform) Transform) 8

• First 3 moments of the spectrum • Spectral Roll-off frequency

• SlSpectral Flatness –noiiisiness of the spectrum 9

• Represent the data in an orthogonal basis in which the variance is maximized • Better separation and computability improvement • Finds the best seppgarating features ¾MFCC, Flatness, 3rd Moment 10

Music Library Musical map Feature Mapping and Training Extraction Classification 11

• 2-Dimensional node grid • UidUnsupervised traiiining process • Useful for high dimensional input visualization • Preserves topological properties Features ... … … … … … … … … … … … … … … … … … … 12

• Choose a segment randomly

• Winner determination d 2 dist() x, mi =−∑( xjjj mijj) λ j j=1 13

• Closest node and neighborhood adaptation

mtiicii( +=1) mt( ) + h( t)⎣⎦⎡⎤ xt( ) − mt( ) 14

• Starting with a large learning rate, gradually decreased to facilitate convergence

⎛⎞rr− 2 ht= α ( )exp⎜⎟− ci ci ⎜⎟2 2 t ⎝⎠σ ( ) 15

Music Library Musical map Feature Mapping and Training Extraction Classification 16

• Histogram maps for each artist

EiEminem Chop in GNRGuns N Roses Ella Fit zgerald 17

Madonna Eminem Leonard Cohen Britney Spears DJ Tiesto Spice Girls Cent 50 Armin Van Buuren DidGDavid Guetta Snoop Dogg

Jamiroquai Infected Mushroom

Shakira Dream Theater

Symphony X Nirvana Frank Sinatra Aerosmith Pearl Jam Ella Fitzgerald Metallica

Liquid Tension Experiment Jimi Hendrix Sarah Vaughan Queen Red Hot Chili Peppers Pendulum Bach Pink Floyd The Black Eyed Peas Guns N Roses Chopin Prodigy Coldplay Beethoven OutKast

Vivaldi The Doors 18

Madonna Eminem Leonard Cohen Britney Spears DJ Tiesto Spice Girls Cent 50 Armin Van Buuren DidGDavid Guetta Snoop Dogg Pop/Electronic Rap Infected Mushroom Jamiroquai

Shakira Dream Theater

Symphony X Nirvana Frank Sinatra Aerosmith Rock Pearl Jam Ella Fitzgerald Metallica

Liquid Tension Experiment Jazz Jimi Hendrix Sarah Vaughan Queen Red Hot Chili Peppers Pendulum Bach Pink Floyd The Black Eyed Peas Guns N Roses Classical Chopin Hip‐Hop Prodigy Coldplay Beethoven OutKast

Vivaldi The Doors 19

• New (untrained) song from each album • Find best matching node in the map • Classification based on artist’ s histograms

New Song 20

Artist Classification rate as a function of the number of artists 100

50x50 Map

90 20x20 Map

80

70 tion rate (%) aa 60 47%

Classific 50

40

38.6% 30 0 5 10 15 20 25 30 35 Number of Artists 21

• Classifying unknown artists:

Mozart Billy Holiday Beatles KanyeWest 2Pac Pantera Alice in Chains Astral Projection 22

Classical Jazz

Rock Hip‐Hop Pop/Electronic

Rap 23

• PCA + SOM for representing music similarities • We manage to implement approach for music organiiization • 50.7% artist classification rate • 78.5% genre classification rate 24

• Further study for features selection • Automatic playlist creation • Similar artists suggestions • Adapting the algorithm to other fields i.e. painter classification 25