BINAURAL HEARING and INTELLIGIBILITY in AUDITORY DISPLAYS
______Durand R. Begault
Human Factors Research & Technology Division NASA Ames Research Center Moffett Field, California 1. Binaural hearing phenomena
2. Newly developed auditory displays that exploit spatial hearing for improving
-speech intelligibility -alarm intelligibility in aviation applications Physical characteristics of sound and perceived attributes • Frequency (perceived pitch) • Intensity (loudness) • Spectral content (timbre) • FIS, plus binaural differences (localization) Physical characteristics of sound and perceived attributes • Frequency (perceived pitch) • Intensity (loudness) • Spectral content (timbre) • FIS, plus binaural differences (localization)
** All characteristics are important in the identification and discrimination of auditory signals and for speech intelligibility in communication contexts Two important functions of the binaural hearing system • Localization (lateral and 3-dimensional) • Binaural release from masking: Echo supression, room perception Psychologically Model of the binaural hearing system - driven Binaural hearing (localization; signal separation & detection): forming spatial auditory events from acoustical (bottom-up) and psychological driven
- (top-down) inputs Acoustic signal Psychologically Model of the binaural hearing system - driven Binaural hearing (localization; signal separation & detection) driven -
Acoustic signal Filtering of acoustic signal by pinnae, ear canal Psychologically Model of the binaural hearing system - driven Binaural hearing (localization; signal separation & detection) driven - Filtering by inner ear; frequency-specific neuron firings
Acoustic signal Filtering of acoustic signal by pinnae, ear canal Psychologically Model of the binaural hearing system - driven Binaural hearing (localization; signal separation & detection)
Physiological evaluation of interaural timing and level differences driven - Filtering by inner ear; frequency-specific neuron firings
Acoustic signal Filtering of acoustic signal by pinnae, ear canal Psychologically Model of the binaural hearing system Multi-sensory information; cognition - driven Binaural hearing (localization; signal separation & detection)
Physiological evaluation of interaural timing and level differences driven driven - - Filtering by inner ear; frequency-specific neuron firings Acoustic signal Acoustic signal Filtering of acoustic signal by pinnae, ear canal. Lateral localization of auditory images
“Duplex” theory of localization
• ILD (interaural level difference) • ITD (interaural time difference) Lateral spatial image shift
• ILD (interaural level difference) caused by head shadow of wavelengths > 1.5 kHz Level difference (dB) Level difference (dB) Lateral image shift
• ITD (interaural time difference) Head-related transfer function cues (HRTFs) provide cues for front-back discrimination and elevation 10 Right 30°, elevated
0
-10 Right 90°, ear level Basis of 3-D audio
-20 Right 120, below signal processing in auditory displays
Log Magnitude (dB) -30
-40
-50 100 2000 4000 6000 8000 10000 12000 14000 16000 Frequency
45°, 0°
135°, 0° Vibration source (internal, external) Sound sources can be ‘felt’ Ground, Structure response and ‘seen’ as well as heard
Airborne Walls, Chairs, Walls, sound Windows, Tables, Windows, objects floor plants Head- 3-D audio mounted hearing feeling seeing display visual display
Expectation Inter-modal coordination Identification Experience-adaptation
Response: qualitative assessment Performance metric Applications of spatial sound for improving intelligibility in auditory displays Using binaural hearing advantage for separating multiple auditory “streams” (simultaneous sources) 3-D communication system patented, developed for NASA-KSC Speech Intelligibility advantage compared to one-ear listening
8.0 Full frequency bandwidth
6.0
4.0 Advantage (dB) 2.0
Telephone bandwidth
0.0
Azimuth of spatialized signal (mean of left & right sides) Hearing loss for target users: 64 active commercial airline pilots
100%
75%
Question 1 ("Told by a doctor")
50% Question 2 ("Personally suspect...)
General Population 25% Percentage of "yes" answers 0% 35-44 (11) 45-54 (23) 55-64 (30)
Age range of pilots (no. of subjects) Audiogram data summary for 20 active commercial pilots (age range 35-64; not corrected for presbycusis)
Audiogram data 100
75
% > 25 dB HL 50 % > 20 dB HL
25 % of 20 pilots evaluated
0
Frequency (kHz) Use of auditory icons (AI) and left-right spatialization for information redundancy, situational awareness of actions of crew (CRM) and haptic feedback substitution
Oil Pressure
Identify Which Throttle (Lft. or Rgt)...... ___ Check Oil Press...... Color___, Value___
30-90 Retard Throttle...... ___ Retarded “Page-through” "Page Through" 0-30 Engine Shutdown...... O.K. & “switch”auditory icon AIs for touch(For selecting screen menu pages from checklisttabs)
"Click" auditory icon (For selecting items NASA ARC advanced cab that change orientation Disc. Menu simulator Rgt FUEL Lft within the menu display) Eng Eng “Mechanical latch”
Shut Off 30.2 Shut Off Valve Totalizer Valve AIs for actions
F. Press 28 APU F. Press 28 "Latch" auditory icon corresponding(For actions that to correspond Auto Auto electrical, fuel, Man Man to changes in the aircraft Off Off electrical, hydraulic, or LOW LOW hydraulic systems P P high high P P engine systems) Fuel Heat 15.1 Lbs 15.1 Lbs ISO VALVES Head up auditory display for TCAS
3-D audio alert 60 45 30 15 15 30 45 60
60 50 40 30 20 10 0 10 20 30 40 50 60 Visual field of view
CAPTAIN'S COMMON FIRST SCREEN SCREEN OFFICER'S SCREEN Application of 3-D audio head-up display for Traffic Collision Avoidance System (TCAS II) investigated. Target acquisition times can decrease from 0.5 – 2.2 sec. Mean target acquisition Mean target acquisition times times (4.7 vs. 2.5 s) and (2.63 vs. 2.13 s) and standard deviations for first standard deviations for TCAS experiment. The 3-D audio second TCAS experiment. The cues were exaggerated in azimuth relative 3-D audio cues were not exaggerated, and to the visual target, and no elevation cues there were three categories of elevation were supplied. cues.
7 5
6 4 5
QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture. 4 3
3 2
2 1 1
0 0 Monotic Audio 3-D Audio head-down head-up No Map Display No Map Display map display 3-D audio display Head-up auditory display with head-up visual display
Reduction in taxi time: Advantage of 3-D audio 60
40
20
0
-20
-40 Time (sec): mean of 3 routes
-60 1 2 3 4 5 6 7 8 9 10 11 12 Crew Spatially-modulated auditory alerts In an auditory display, how to insure that an alarm is audible?
-“Common sense” engineering approach: make the alarm a lot louder than the background noise for wide-area coverage
Fire alarm and horn from ca. 1933 In an auditory display, how to insure that an alarm is audible?
-ISO 7731 (“Danger signals Signal for work places-Auditory danger signals”) specifies signal to be >= 13 dB Noise re masked threshold in a 1/3 octave band (0.3-3.0 kHz)
-Recipe for “startle effect”, high overall SPLs, and potentially low performance in a high-stress environment Current approach -Improve detection of an alarm (signal) against ambient sound (noise) using signal processing techniques other than level increase Requirement / Caveat -Technique should apply to currently-used alarms (to avoid “relearning” semantic content of new auditory signals). Technique -Three methods addressed in patent application (pending) for accomplishing this. QuickTime™ and a TIFF (LZW) decompressor are needed to see this picture. Alarm (basic stimulus) 737-300 alarm: Two successive square waves (preceding verbal “wind sheer” alert)
200 Hz 764 Hz
300 ms 300 ms Stimuli x 2 R ear
0 Hz jitter 1.6 Hz jitter 3.3 Hz jitter Amplitude L ear
Time
Summed L+R RMS levels equivalent for all stimuli; but jittered stimuli have + 5 dB peaks re unjittered due to HRTF. Results (1) Headphone with jittered signal has 13.4 dB advantage over monaural loudspeaker (existing condition on aircraft), partly due to attenuation of noise by headphone
-10 0 -2 -15 -4 -6 -20 -8 -10 -25 -12
Attenuation dB .... -14 -30...... -16 -18 -35 -20 Threshold re noise level (dB) 2000 2500 3150 4000 5000 6300 8000 10000 12500 16000 Octave band Center Frequency 1.6 Hz 3.3 Hz trajectory trajectory (existing) Headphone, Headphone, Loudspeaker Headphone attenuation Results Sennheiser HD 480 vented Results (2) Headphone with jittered signal has significant (p < .000) 7.8 dB advantage over headphone without jittered signal. No significant difference between 1.6 and 3.3 Hz modulation.
Condition 4, 5, 6 (headphones)
0 9 8 -5 7 6 -10 5 4 -15 "Spatial unmasking" 3
dB advantage 2 Peak level due to 90 -20 deg. HRTF 1
Threshold re noise level (dB) 0 -25 1 0.0 Hz 1.6 Hz 3.3 Hz Trajectory velocity results source of unmasking (?) Conclusions
A new approach to designing alerts for auditory displays in high-stress interfaces: use of spatial modulation for improved detection.
Headphones + spatial modulation lower threshold by 13.4 dB.
Spatial modulation lowers threshold by 7.8 dB. 5 dB is due to HRTF interaural level difference if instantaneous (peak) level differences are assumed. This amount is reduced as a function of longer temporal integration periods. Remaining advantage is due to time varying interaural cross-correlation. • IMMEDIATE SITUATIONAL AWARENESS (WITH HEADS-UP ADVANTAGE) BINAURAL LOCALIZATION • ALTERNATIVE or REDUNDANT DISPLAY for VISUALLY-ACQUIRED INFORMATION + • INTELLIGIBILITY IMPROVEMENT THE ''COCKTAIL Binaural release from masking PARTY'' EFFECT • DISCRIMINATION and SELECTIVE + ATTENTION IMPROVEMENT ACTIVE NOISE HEARING CONSERVATION CANCELLATION
Benefits: increased = aviation safety & efficiency