Topic

Chromagram & Cesptrogram

Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio The Spectrogram spectrogram(y,1024,512,1024,fs,'yaxis');

•A series of short term DFTs •Typically just displays the magnitudes of X from 0 Hz to Nyquist rate

Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio

• Stevens, Volkmann and Newmann (1937)

• A scale of pitches judged by listeners to be equidistant.

• The reference point: – 1000 mels = 1000 Hz at 40 dB SPL

• Below 500Hz mel ~=

• Above 1000 Hz mel ~= log(hertz)

From: Appleton and Perera, eds., The Development and Practice of Electronic Music, Prentice-Hall, 1975, p. 56; after Stevens and Bryan Pardo, 2008, Northwestern University EECS 352: MachineDavis, HearingPerception of Music and Audio Mel Filter Bank

• Filters spaced equally in the log of the . • Mels are (more or less) related to frequency by… æöf f mel =+2595log10 ç÷ 1 èø700 • Edge of each filter = center frequency of adjacent filter • Typically, 40 filters are used

Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio Source-Filter Model

Source Signal Filter Output Signal x(t) h(t) y(t)

x(t)*h(t) = y(t)

Convolution Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio The • Filtering is – in the time domain – A product in the • What if we want to make it an addition operation? Yk[ ] = Xk[ ]× Hk[ ] ||||||Yk[ ] = Xk[ ] × Hk[ ] log( |Yk[ ] |) =+ log( | Xk[ ] |) log( | Hk[ ] |)

Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio The Cepstrum • Filtering is – Convolution in the time domain – A product in the frequency domain • What if we want to make it an addition operation? • They do this by defining the cepstrum.

−1 Cepx (q) = Z (log | X (z) |)

A frequency representation

Quefrency The Inverse Z transform (general case of the Inverse Discrete )

Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio What is the Cepstrum for?

• Invented for finding echoes (aftershocks) in seismograph data.

• If something is useful for finding echoes, it is useful for finding impulse response functions

• …which makes it useful for finding filter coefficients.

Let’s look at an example…

Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio Some terms

• Spectrum • Cepstrum • Spectrogram • Cepstrogram • Frequency • Quefrency • Filtering • Liftering

Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio The Cepstrum • Gives information about rate of change in the different quefrency bands. • Popular representation for speech and music • Distinguishing FILTER from the SIGNAL – Some quefrencies represent the filter (what instrument), others represent the signal (what pitch) • For these applications, the spectrum is usually first transformed to Mel Frequency bands. • Result: Mel Frequency Cepstral Coefficients (MFCC)

Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio Making a Mel Freq Cepstrogram

Signal in Frequency Sample jth window index number sn() |()|Sk xn() Sliding j |DFT| j Mel filter Window bank

c j ()m logc (m ) ( j ) Cepj () i DCT Mel filter index Quefrency index Here DCT = Discrete Cosine Transform

Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio Let’s have a look!

• (Go to bassoon/tuba demo)

Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio Equal Temperament

• Octave is a relationship by power of 2. • There are 12 half-steps in an octave

number of half-steps from the reference pitch n frequency of 12 desired pitch ff= 2 ref frequency of the reference pitch Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio Spiral Pitch representation

Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio Chroma: Many to one • Chroma = log2(freq) – floor(log2(freq)) • Chroma periodic in range 0 to (almost) 1 • Chroma map on to pitch classes

900 0.0

800 800 Hz 700

600

500 0.75 CHROMA 0.25 400 frequency 400 Hz 300

200 200 Hz 100 100 Hz 0 0 50 100 150 200 250 300 350 0.5 time

Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio Making a Chromagram

• Decide how to quantize (bin) the chroma range. – 12 pitch classes? 120 bins? Equal temperment? • Make a spectrogram • For each time-step in the spectrogram – find the chroma for each frequency from 0 to N/2 – Sum the amplitude of all with the same chroma bin • (Some chromagrams also add in the energy from the odd ) – Place that value in the chroma bin

Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio Series

• Approximate notated pitch for the harmonics () of a frequency

f 2f 3f 4f 5f 6f 7f 8f 9f 10f 11f 12f

C C G C E G Bb C D E F# G

EECS 352: Machine Perception of Music and Audio Bryan Pardo 2008 A fancier chromagram

• For complex (like the bassoon example from class) you might want to consider adding up energy from more harmonics than just the octaves (1f, 2f, 4f…etc). • Try taking the energy from the 3rd, 5th and 7th harmonics as well.

Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio Chromagram of Clarinet

C C#2 D D#4 E

F 6 F#

G 8 G#

A10 A# B12 100 200 300 400 500 600 700 800 900

Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio Chromagram of Clarinet

Bryan Pardo, 2008, Northwestern University EECS 352: Machine Perception of Music and Audio