Hearing & Music (Chapter 10-11a)
Lecture 18
Jonathan Pillow Sensation & Perception (PSY 345 / NEU 325) Spring 2019
1 Harmonics • Objects tend to vibrate at multiple “resonant frequencies” (integer multiples of some fundamental frequency) • most vibrations die down, but some persist because their wavelength is reinforced by the object’s physical properties • Auditory system acutely sensitive to harmonics
Example: guitar string
Fundamental F1 2nd harmonic F2 3rd harmonic F3 (1st harmonic) (2 x F1) (3 x F1)
2 Many sounds, including voices, are harmonic
3 If the fundamental of a harmonic sound is removed, listeners will still hear its pitch
demo:
https://oup-arc.com/access/content/sensation-and-perception-5e-student-resources/ sensation-and-perception-5e-activity-10-2?previousFilter=tag_chapter-10
• only 3 harmonics are needed
5 Only three harmonics are needed to hear a missing fundamental
2
• all harmonics are aligned at the 3 fundamental freq. • fundamental could therefore be 4 conveyed by temporal code (“phase locking”)
2+3+4
6 Only three harmonics are needed to hear a missing fundamental
2
• Could also be conveyed by 3 “pattern matching” of the place code on the cochlea 4
2+3+4
7 Complex Sounds
Timbre: Psychological sensation by which a listener can judge that two sounds with the same fundamental loudness and pitch are dissimilar • conveyed by harmonics and other frequencies • Perception of timbre depends on context in which sound is heard
https://oup-arc.com/access/content/sensation-and-perception-5e-student-resources/ Timbre demo: sensation-and-perception-5e-activity-10-3?previousFilter=tag_chapter-10
8 Figure 10.20 Timbre
9 Auditory Scene Analysis
What happens in natural situations? • Acoustic environment can be a busy place with multiple sound sources • How does the auditory system sort out these sources?
! Source segregation - processing an auditory scene consisting of multiple sound sources into its separate sources
10 Waveforms from all sounds are summed into a single waveform arriving at the ears
11 Cocktail party effect
• We can “select out” and attend to one conversation even when many are present simultaneously
• first documented by Colin Cherry, 1953
12 Cocktail party effect
Cherry’s findings: • Same voice speaking, Presented to Both ears ⇒ Very Difficult • Same voice speaking, Separate ears ⇒ Easy
13 Cocktail party effect
However, subjects: • couldn’t identify a single phrase from non-attended ear • couldn’t say for sure if it was English • didn’t notice a change to German • didn’t notice speech being played backward • Did notice: change from male to female speaker
14 Cocktail party effect
• Suggests we can easily use spatial, timing, and spectral cues to separate sound streams, but cannot attend to multiple sound streams at the same time!
15 Continuity and Restoration Effects How do we know that listeners hear sounds as continuous? • Principle of good continuation: in spite of interruptions, one can still “hear” a sound • Experiments (e.g., Kluender and Jenison, 1992) suggest that missing sounds are restored and encoded in the brain as if they were actually present!
16 Continuity Effects
17 Also true for speech: Adding noise can improve comprehension
original speech
speech w/ gaps
gaps filled by noise
18 Brain automatically fills in sound that is missing due to noise
gap filled by noise (cough)
speech with a gap
Q: Can you tell which phoneme is missing?
19 Brain automatically fills in sound that is missing due to noise
gap filled by noise (cough)
speech with a gap
Q: Can you tell which phoneme is missing?
20 Continuity and Restoration Effects in Music
Beat-box tutorial:
http://www.youtube.com/watch?v=8D7hCqGm0X0
21 Yanny vs. Laurel
https://www.youtube.com/watch?time_continue=11&v=7X_WvGAhMlQ
22 Summary
• Interaural timing differences (ITD) • Interaural level differences (ILD) • MSO, LSO • cone of confusion • head-related transfer function (HRTF) • harmonics • missing fundamental • timbre • auditory scene analysis • cocktail party effect • continuity and restoration effects
23 11 Music and Speech Perception
24 Music
• Universal: found across cultures • Also distinctive of cultures • Can have strong emotional effects • Has strong physiological effects
• Pythagoras: Numbers and music intervals • Strong link between music and mathematics
25 The sounds of music (i.e. “pitch”) extend across a frequency range from about 25 to 4200 Hz
• 7 octaves
• hard to identify octaves above this limit
26 Musical Intervals Octave: The interval between two sound frequencies having a ratio of 2:1
• Example: Middle C (C4) has a fundamental frequency of 261.6 Hz; • notes that are one octave from middle C:
! 130.8 Hz (C3) (below)
! 523.2 Hz (C5) (above)
27 Helix showing two characteristics of musical pitch:
1) tone height - increasing with frequency
2) tone chroma - which note within a scale (determines “feel” of note)
28 Helix showing two characteristics of musical pitch:
Octave determines pitch more strongly than frequency
• C3 (130.8 Hz) sounds more similar to C4 (261.6
Hz) than to E3 (164.8 Hz) • There is more to musical pitch than just frequency!
29 Chords G - B - D G Major Chord, several heights
Certain combinations are considered pleasing, or “consonant”
Varies across cultures, but tends to focus on even ratios of frequencies (F1): “fifth” - 2:3 “major third” - 4:5
30 Dissonant: Less elegant ratios of note frequencies
e.g., diminished fifth or “tritone”: 7:5 (six semitones)
Disapproved of since the middle ages: The name diabolus in musica ("the Devil in music") has been applied to the interval from at least the early 18th century. Johann Joseph Fux cites the phrase in his seminal 1725 work Gradus ad Parnassum. Georg Telemann in 1733 notes, "'mi against fa', which the ancients called 'Satan in music'.”
Because of that original symbolic association with the devil and its avoidance, this interval came to be heard in Western cultural convention as suggesting an "evil" connotative meaning in music. Today the interval continues to suggest an "oppressive", "scary", or "evil" sound.
31 • Melody: An arrangement of notes or chords in succession • Examples: “Twinkle, Twinkle, Little Star” or “Baa Baa Black Sheep” • defined by relationship between notes (i.e., relative pitch, not absolute pitch)
• Key: refers to the reference or starting note (Same melody can be played starting from a different note)
32 Absolute vs. Relative pitch
Most people have relative pitch - they can identify the interval from one note to another (eg, an octave or a fifth). absolute pitch - very rare ability (1 / 10,000 in US) to identify or create a musical note without any external reference
• more prevalent among people who grew up in east Asia (influence of tonal languages?) • may also have genetic component (currently unknown) • musical training during early critical period seems to have influence • more common in people with autism / Aspberger’s • may make it difficult to play music transposed to different key
33 Emotional content
Emotional content of music is:
1. psychological, not physical: - same set of notes can sound “happy” or “sad” depending on where you start (C major vs. A minor scale)
2. culture-dependent: - emotional coloration of sounds (even consonant and dissonant intervals) are not the same across cultures
34 Cultural differences
• Some relationships between notes, such as octaves (2:1), are universal; • fifth (3:2) is next most common interval (harmonic series is a common inspiration for scales)
However, cultures divide up the octave quite differently: • Western music: 12 semi-tone (equal-tempered) scale (but “blue note” in jazz/blues falls in between ) 7-note (eg major) scales for composition: do-re-mi-fa-so-la-ti • Arabic music: 24 quarter-tones • Japan, China, India many others: 5-note (pentatonic) scale • Indonesian pentatonic scale: pelog
https://en.wikipedia.org/wiki/Pelog
35