The Fusion and Layering of Noise and Tone: Implications for in African Instruments Author(s): Cornelia Fales and Stephen McAdams Source: Leonardo Music Journal, Vol. 4 (1994), pp. 69-77 Published by: The MIT Press Stable URL: http://www.jstor.org/stable/1513183 Accessed: 03/06/2010 10:44

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at http://www.jstor.org/action/showPublisher?publisherCode=mitpress.

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected].

The MIT Press is collaborating with JSTOR to digitize, preserve and extend access to Leonardo Music Journal.

http://www.jstor.org 4

A B S T R A C T SOUNDING THE MIND

Since their earliest explora- The Fusion and Layering of Noise tions of Africanmusic, Western re- searchers have noted a fascination on the part of traditionalmusicians and Tone: Implicatons for Timbre for noise as a timbralelement. The authors present the results of per- ceptual and acoustic investigations in African Instruments of the fusion and "layeringnof noise and tone. These results have implicationsfor pitch and timbre in both traditionaland non-traditional, acoustic and synthesized music. CorneliaFales The results define possible percep- tual relations between noise and and StephenMcAdams tone and reveal that the construc- tion of noise devices should follow relativelyprecise acoustic rules in- volving the , the band- width and the level of the noise relative to those of the tone. The results also exemplifythe fusion of From theirfirst contact with African music, West- wardestablishing the parameters twoextremely different , ern ethnomusicologistshave remarked on the predilectionof of noise and tone that influence withimplications for the blendingof the perceptual relation between instrumentaltimbres in an orches- Africanmusicians to "layer"(or superimpose) musical and tralsethng. The expenments non-musicalsound until the distinctionbetween them is lost them. We consldered three pos- shouldbe of interesttocomposers [1]. In particular,it has been observed that African music sible relationsbetween noise and whosynthesize mixtures of noise showsa special fascinationwith 'Cnoise''-thetransformation tone: (1) fusion (the noise and andperiodicsoundandforwhom of ordinary,mundane into the substance of music. tone are perceptuallyintegrated the controlof suchmixtures rb mains problemabc. And while the manufactureof classicalinstruments and the lnto a slngle event, both performancepractice of Westernmusicians has aimed toward contributing to the perceptual reducingthe amountof extraneousnoise producedby an in- qualityof the event), (2) laymng strument,African musicians augment the naturalnoise po- (the two components are perceptuallysegregated into dis- tential of their instrumentby attachingnoise-makers such as iinct percepts,each with its own perceptualqualities) and (3) rattling seeds or bottle caps on which the vibrationsof the maskingoftone by noise (the more intense noise "coversup" main resonatoroperate. The effect is a complicatedlayering the tone that can thus no longer be heard andhas no percep- of sound, rich in aperiodiccomplexity. tual effect on the noise component). Layering is distin- One of the motivationsfor our study was the realization guished from fusion and maskingin that a layeredtone can that the combination of noise and musical elements, tradi- be heard separatelyfrom the noise. Whilea tone that is fused tionallydescribed by ethnomusicologistsas "layering,"actu- with a noise cannot be heard separately,it still contributesto ally takes at least two perceptualforms. Either the sound is the timbralquality of the noise; a masked tone has no such trulylayered, and listenershear twoor more perceptuallydiF effect, because it has been perceptually"eradicated." These tinct sounds concurrently,or the physicallysuperimposed three perceptualphenomena appear to depend on variations sounds are perceptuallyfused, so that listenershear a single of three acousticparameters: (1) the relativeintensity levels sound a blend of the two sounds neither of which is iden- of the two components, (2) the bandwidthof the noise and tifiable as the primar-yor the superimposedsound. A more (3) the center frequency of the noise band relative to the commonlyrecognized example of this distinction is the dif- tone frequency.Since informalpilot tests showedthat, in ad- ference betweena voice speakingthrough or in the presence dition to these parameters,the perceptualrelation between of noise (layerednoise and voice) and a hoarse voice speak- ing (noisyvoice). The directionof our researchhas involvedboth perceptual Fig. 1. Intensity series: Means (filled circles) and standard devia- experimentand acousticanalyses. Like manypsychoacoustic tions (SD) (vertical bars indicate i 1 SD) of responses for each in- studies, the first (experimental)part of our investigationin- tensity difference between tone and noise for high register stimuli volved the use of artificialstimuli, reduced to controllable with a 100-Hz band of noise centered on a 400-Hz tone. acoustic variables. To complement this part, an acoustic INTENSllY SERIES,high register analysisof real African instruments in which noise plays a prominent part was essential to verify the relevance of our 6- experimental results to sounds actuallyproduced by those

'e instruments.In particular,our experimentswere geared to- g:

V)

3 Cornelia Fales (ethnomusicologist), Institut de Recherche et Coordination Acoustique/Musique (IRCAM), 1 place Stravinsky, F-75004 Paris, France. 2 Stephen McAdams (experimental psychologist), Laboratoire de Psychologie Experimentale (CNRS), Universite Rene Descartes, EPHE, 28 rue Serpente, F-75006 Paris, France. Manuscript solicited by Kathryn Vaughn. -2s -1s -s s 1S 25 Received 11 May l994. Difference in noise and tone level (dB)

LEONARDO MUSIC JOURNAL, Vol. 4, pp. 69-77, 1994 K)1995 ISAST 69 - -t | g

INTENSll Y SERIES,low register North Americans.Since the aim of this . . . . . part of the work was to establish some Fig. 2. Intensity se- basic psychoacousticthresholds and be- 6" " * _ ' ries: Means (filled cause psychoacousticphenomena at this circles) and stan- level are generallythought to be a func- dard deviations tion of the human hearing apparatus (SD) (vertical bars (whichis essentiallythe same for all hu- * indicate + 1 SD) of 3 tC * responses for each man beings), we considered the results intensity difference obtained from this study to reflect uni- between tone and versal properties of auditory percep- noise for low regis tion althoughthis assumptionwarrants 1 ter stimuli with a verificationin future research.From an , 100-Hz band of , . . . ethnomusicologicalpoint of view, how- -25 -15 -5 5 15 25 noise centered on a ever,it is noteworthythat informalpilot Differenceinnoise and tone 1e Level (dB) l,OOO-Hztone. testswith these subjectsrevealed that the notion of fusion between dissimilartim- breswas a difficultconcept to graspintel- noise and tone was greatly affected by -in which the noise center fre- lectuallyand, even more so, perceptually. the pitch register of the two elements, cywas tested relative to the tone Initially,subjects were askedto rate on a each of the three parameterswere var- Lency-the tone varied as described continuum the "degreeof fusion"dem- ied for stimuliconstructed around both in high and low registers, and the onstratedby a seriesof stimuli,but it be- high and low registers (centered at r frequency of the noise was offset came evident that the subjectswere un- 1,000 Hz and 400 Hz, respectively). the tone by up to 200 Hz in the low clear as to the definition of the terand up to 400 for the high reg- perceptualphenomenon that they were The noise in this last series had a askedto judge. Therefore,as an alterna- PERCEPrUAL RESEARCH bandwidth of 100 Hz. tive, the subjectswere askedwhether the General Meffiod Wa given trial in one of the series, a acoustic components comprisedone or The experiments were organized in /noise stimulus was presented two sounds,since fusion transformsmul- three series, each testing the effect of a times in succession. Each stimulus tiple elements into a single unitaryper- different acoustic parameteron tone/ 00msec in duration and the onsets cept. Subjects were asked, in other noise fusion. In the Intensity Series, ffsets of the noise and tone compo- words, to rate the degree to which the pure tones (sinusoidal waveform) in ,were synchronous. The tone was tone could be heard separatelyfrom the high and low registers were combined nted at a mean level of 57 dB SPL noise in each trial. with band-filteredwhite noise having a ingrandomly by + 9 dB around this Twenty-three listeners (all but one bandwidthof 100 Hz and a center fre- on a trial-to-trial basis), and the had had some musical training; none quencycoinciding with the frequencyof level was determined with respect were professional musicians) were in- the tone. For this series, the intensity islevel for each trial. structed to decide whether they heard level of the tone variedrandomly within e perceptual research was per- the tone separately from the noise or a given range, while the noise level was edat Institut de llecherche et Coor- not and to rate their certaintyby select- offset from the tone level by specific in- ionAcoustique/Musique (IRGAM), ing one of six buttons as follows: (1) crements.For the BandwidthSeries, this sic research institute in Paris. We "Toneheard separately:I am sure";(2) procedure was repeated for several fore were initially constrained in "Tone heard separately: I am fairly noise bandwidths ranging from 50 to steners we could readily test, being sure";(3) 'Woneheard separately:I am 200 Hz. For the Center Frequency Se- ed primarily to Europeans and not sure"; (4) "ToneNOT heard sepa-

Fig. 3. Bandwidthseries: Means (filled circles) for response catego- Fig. 4. Bandwidthseries: Means (filled circles) for response catego- ries 3 and 4 (indicatingfusion) for each bandwidthas a function of ries 3 and 4 (indicatingfusion) for each bandwidthas a function of level difference between tone and noise for high register.Vertical level diSerence between tone and noise for low register.Vertical axis shows amplitudedifference between tone and noise; horizon- axis shows amplitudedifference between tone and noise; horizon- - tal axis indicatesbandwidth. tal axis indicatesbandwidth. BANDWIDTHSERIES, high register BANDWIDTHSERIES, low register 6 | | | | | l | r o o 6 | t | | | l s 4

s o 4 o * m . z 2 \ 4< wx _ aJ ,, > Cd 0 2- Ct r aJ o aJ @ V *C ,)l 0 \ A R-2 .H

aJ -2 \ aJ a; _ -4 - --r \ -4 CG aJ -6v - 50 75 lO0 125 150 175 200 sO 7s lQ0 125 lS0 175 200 Noise bandwidth(Hz) Noise bandwidth (Hz)

70 Fales and Mc24dams,The Fusion and Layering of Noise and Tone ' O./ ...... a X ...... s = X Z ......

BANDWIDTH SERIES, high register BANDWIDTH SERIES, low register

6S 6 s sz S- n 4 e 'e

i 3^

2 G 2 \s

1-

- - - w w l l - w - | - w w w w | - - | - - | v -240 -40 160 360 -120 -20 80 180 Offset of noise centerfrequency from tone frequency Offsetof noisecenter frequency from tone frequency Fig. 5. CenterFrequency Series: Means (filled circles) of responses Fig. 6. CenterFrequency Series: Means (filled circles) of responses for high registerstimuli in which center frequencyof noise band- for low registerstimuli in which center frequencyof noise band- width is offset from tone frequencyby up to 400 Hz. Verticalaxis width is offset from tone frequencyby up to 200 Hz. Verticalaxis shows response, horizontalaxis shows offset of noisocenter fro shows response, horizontalaxis shows offset of noise-centerfre- quencyfrom tone. quencyfrom tone. rately:I am not sure"; (5) 4'ToneNOT the high and low registers differs most were averagedacross subjectsand plot- heard separately:I am fairly sure";(6) dramatically on the high intensity-differ- ted in Figs 3 and 4 for the low and high '4Tone NOT heard separately: I am ence end of the curve. For the low regis- registers,respectively. This comparison sure." We call these judgments "tone ter, when the noise exceeds the tone by shows that the difference in level be- separationratings." For the final Center more than about 4 dB, the tone begins to tween tone and noise at the fusion FrequencySeries, listeners were alerted, lose audibility;when the noise intensity is thresholddecreases slightly for both the in addition,that the range of perceptual less than the tone by about 15 dB, the high and low registersas the bandwidth differences might be changed, since tone separates and the sound is layered. increases from 50 to 75 and then re- there might no longer be a condition in Based on intermediate ratings, a Elrstes- mains relatively constant for larger which the tone was not heard at all, as timation of the necessary intensity rela- bandwidths. occurredin the firsttwo series of trials. tion between noise and tone, therefore, puts the fusion region between -7 and Center FrequencySeries IntensitySeries -1 dB, noise relative to tone. For the high In this experimental series, a constant Tone separationratings were made for register, fusion appears to occur when the bandwidth( 100 Hz) and noise-intensity 11 intensity differences between tone intensity level of the noise is between -3 level (65 dB with respect to the tone and noise varyingfrom -25 to +25 dB in and +4 dB, relative to the tone. There level) were maintained, while varying 5dB increments.Each conditionwas re- would appear, therefore, to be a differ- the difference between the center fre- peated five times for each listener. ence between high and low registers, fu- quency of the noise and the tone fre- These responses were then averaged sion occurring at a slightly higher inten- quency from -200 to +200 Hz in incre- across listeners and repetitions. The sity difference (about 3 dB) for the high ments of 20 Hz for the low registerand mean responses and their standardde- register than for the low register. from -400 to +400 Hz in increments of viations (a measure of variation across 40 Hz for the high register.It is impor- responses for a given condition) for BandwidthSeries tant to note that for the stimuli in this each intensity difference are shown in Tone separation ratings were made for series, there was no condition in which Figs 1 and 2 for high and low registers, 11 intensity differences as in the previ- the noise maskedthe tone, since the in- respectively.Note that when the noise is ous series, but here they were per- tensitylevel of the noise remainedcon- weakcompared to the level of the tone, formed for each of five noise band- stant at a level difference found to be low ratingsare given (tone heard sepa- widths varying from 50 to 200 Hz in within the fusion response area for a rately) and when the noise is intense, 5SHz increments. The noise bands were bandwidthof 100 Hz in the first experi- high ratings are given (tone not heard created with a second-order bandpass mental series. Thus, for this series, re- separately).The curves for both regis- filter. Each condition was repeated three sponse category 6 (Tone NOT heard ters suggesta gradualprogression of the times for each listener. These responses separately:I am sure) was taken to indi- percept from separationthrough fusion were then averaged across listeners and cate the greatest degree of fusion, and to masking. repetitions. For this series, the same response category 5 (Tone NOT heard Presumingthat high ratings (5-6) in- types of mean response curves resulted, separately:I am fairlysure) was consid- dicate that the tone may be masked by with the same difference in registers, as ered the fusion threshold.In this series, the noise and that low ratings(1-2) indi- were seen for the first series with a noise in other words,judgments of non-sepa- cate that the tone is heard separately bandwidth of 100 Hz. In order to com- ration were taken to indicate fusion from the noise, we takeintermediate rat- pare the intensity differences between rather than masking, given the signal ings (3 4) as indicativeof the conditions noise and tone in the fusion region (re- parametersused. under which the tone and noise are be- sponses 3-4) across the different band- Mean responses across listeners and ginning to fuse. Figures 1 and 2 show widths, the intensity differences corre- repetitions as a function of the fre- that the degree of noise-tonerelation in sponding to mean responses of 3 and 4 quency difference are shown in Figs 5

Fales and McAdams,The Fusion and Layering of Noise and Tone 71 2010- 179it - - | - | - r-rit _li -I; | 0 ; . t . 1\. _ 1. . 1\ .. 1 .. xl S . l . ||tks v | |

de and 6 for both registers. These curves demonstrate that fusion is affected by

l l the way the auditory system Ellters the incoming sound into frequency bands 50 , ------. ... _ in a manner similar to that shown for I i

40 - t ------. . . . masking by a vast body of psychoacoustic research. The peripheral auditory sys- 30 t ------tem can be considered to be a bank of overlapping bandpass filters (called "au- ditory filters'') whose center cover the audible frequency range and whose bandwidths (often called "critical

°O 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 bandwidths") vary systematically with the center frequency, being smaller at low frequencies and larger at high fre- Fig. 7. Narrowband of elements of flute tone of fundamental fre- quencies. Thus, as the center frequency quency 1,032 Hz. The horizontal axis corresponds to frequency (in Hz), and the vertiical of the noise moves away from the tone axis to amplitude (in dB). The is approximately 1,032 Hz. frequency and its energy starts to fall outside the auditory filter center on the tone, the degree of fusion decreases.

de This effect gives rise to the bell-shaped curves in Figs 5 and 6.

- - - Discussion of the Perceptual Experiments --+ From an ethnomusicological point of view, a primary interest of the results ob- - - tained so far resides in the relatively small range of the acoustic parameters that determine fusion. While we have not yet tested the three parameters as they might operate for fusion with com- plex tones, we can hypothesize that if a musician were to construct an ideal Hz noise device intended to fuse with a con- tinuum of pure tones that made up the Fig. 8. Narrowband spectrum of noise elements of flute tone of fundamental frequen 1,032 Hz. The horizontal axis corresponds to frequency (in Hz), and fhe vertical axis CYor pitch inventory (an altogether unrealis- responds to amplitude (in dB). Note fhe similarity in envelope (outline of peaks) of Jdhe tic scenario), the device would produce noise spectrum to fhe harmonic spectrum above, indicating fhat noisebands are rouglhly noise having the following characteris- centered on primary . tics: (1) a bandwidth as large or larger than that of an auditory filter centered on the highest pitch of the instrument; de (2) a center frequency as close as pos- sible to the frequency of the tone with which it is to fuse, but with a frequency offset small enough to still allow the re- quired noise power to enter an auditory filter centered on the lowest frequency produced by the instrument; and (3) an intensity level that allows the required noise power to enter the auditory filter centered on the highest frequency pro- duced by the instrument, given the cho- sen bandwidth of the noise. Informal testing of laboratory stimuli has demon- strated that the fusion of noise with a Hz more realistic complex tone is more Fig. 9. Narrowband spectrum of flute tone (complete) of fundamental frequency 1,032! Hz complicated than fusion with a pure The horizontal axis corresponds to frequency (in Hz), and the vertical axis to amplitude tone. It is not a simple matter of sur- (in dB). Note presence of a second set of Zharmonics," which are marked by arrows. NoteI rounding each harmonic with noise or also that the width of noisebands sulsounding fundamental and second two harmonic sare adding a single block of noise with roughly as large as the bandwidfhs of the auditory filters centered on these componen 8 bandwidth equal to the bandwidth of the complex tone. However, though confirmationwill only be possible with

72 Eales and McAdams,The Fusion and Layering of Noise and Tone continued experimentation,we suggest tone and whose spectrum is also rela- lated harmonic portion, though at a that to the extent that the noise phe- tivelycomplicated, noise is produced by loweramplitude. nomena of fusion and masking are re- metal isclackers'attached to the gourd lated, then our resultson the fusion of a resonator,producing a metallicjingling Flute Analyses pure tone might be relevantto complex sound superimposedover the sound of For the flute, the width of the noise tones as well. Since there are indeed in- the struckcord. band surroundingthe fundamentaland struments (such as the sanza of South- Ironically,the majorproblem encoun- the second two harmonicsis at least as ern, CentralSand West Africa, an ex- tered in noise analysis of real instru- large as the bandwidthsof the auditory ample of whichwe will discussbelow) in ments is exactly the result of the phe- filters centered on these components which a unitarynoise device contributes nomenon that is the object of our (for the fundamentalfrequency at 1,032 a cross-frequencyi'buzz" to the timbre,it research:both acousticallyand percep- Hz, the noise bandwidth must be ap- is probable that the invention of such tually,noise is intertwinedwith other el- proximately136 Hz; for the second [4] devices is neither haphazard nor ements of the music with tremendous harmonic at 2,016 HzSthe bandwidth epiphenomenal. It does not appear to complexity.Whether the noise is layered must be approximately241 Hz, for the be the case that the use of fused noiseS or fused, the resultingacoustic signal, of third harmonic at 3,105 HzSthe band- at anz7rate, is simplyanother example of course, includes both noise and tonal width must be 349 Hz). In addition,the the Africanpredilection for layeredtex- components and the distinction be- noise in the frequencyregion of the fun- tures. Rather, as demonstrated below, tween them is difficult to detect. The damental and third harmonic has the sllch devlcesmust be carefullychosen or noise-separationsoftware that we used highest level, correspondingto the fact constructed in order to achieve a de- operates on the assumption that the that these are the strongestharmonics. sired effect. constituentnoise elements are lower in Therefore,for the perceptionof the en- amplitude within a given frequencyre- tire complex conglomerateSthe audi- gion than the harmonicelements. The tory filters in the region of the funda- AGOUSTIGANALYSES results of our perceptual experiments mental and the third harmonicare the The second part of our study aims to show that it is possible to achieve per- most stimulatedby the noise. In the case complement the perceptual research ceptual fusion of noise and tone even of the higher harmonics,it is difficultto with acousticanalyses of noise devicesin when the noise is at a higher level than determine where the noise bandwidth Africanmusical instruments. The analy- the tone. Indeed the last instrumentwe beginsSsince the noise level flattensout ses whose results are presented here will discuss-the musical bow- shows considerablyaboure about 55000Hz. were done using two kinds of digitalsig- noise elements that are more intense We can make several other observa- nal processing software developed at than periodic elements in some fre- tions based on the spectra,as well as on IRCAM.The firstof these (whichwe will quencyregions. In the case of the musi- the sound of the signalwe are analyzing. call Znoise-separation")separates the cal bow then, the noise-separationsoft- As mentioned above, the noise element non-periodic part of the signal (the warewas unable to distinguishperiodic in the timbre of this flute is slight, as noise) from the periodic part (har- from aperiodic elements, and we used might be predictedfrom the level of the monic tone). The second is a filtering filtering softwareto isolate noise from noise. If the fusion of noise with a com- program that allows very fine pass- or the harmonicelements. Even in the case plex tone has some relation to noise- stopband filtering by literally "drawing of the flute and sanzaewhere the noise pure tone fusion}then, accordingto our ins'the desired filter on a spectrum.We separation program was 1lsed the experiments, the difference in noise present the results from three instru- sampleshad to be resubmittedfor pro- and tone level here ought to make this ments, a traditional bamboo flute, a cessing several times, in order to sepa- combination difficult to fuse And yet, sanza [2] with a single level of metal rate noisy from periodic elements as they arefusing, as is evidenced by the mellae (tonglles) and a musical bow much as possible. sound of the entire signal as well as the [3]. All three intruments are from Wewill look firstat the instrumentsin sound of the separate noise and tone Bllrundi and were chosen for analysis which noise and tone fuse. Figures7-12 signals (both of which are noticeablyal- because the first two demonstratefused below show spectra of the flute and tered by their isolation from one an- noise and tone} and the last demon- sanza.In the Elrstgroup of three Fig. 7 other). Fromthis example,we can arrive strateslayered noise and tone. shows the harmonic elements only of at one of two conclusions.It maybe that Regardingthe fused instrumentssthe the flute tone with a fundamental fre- noise fusion in the case of a complex flute has a relativelysimple spectrum quency of 1,032 Hz; Fig. 8 shows the tone differsmarkedly from noise fusion while the sanza is more complicated. aperiodiccontent of the same tone; Fig. with a pure tone. However,insofar as the Perceptually,the flute timbre is charac- 9 shows the entire conglomerate The noise phenomena of fusion and mask- terized by breathiness or air noise, second three figures show, respectively, ing are related,Moore's observation [5] which- although weaker in regard to the harmonic structure (Fig. 10) the that the maskingof a complex tone by fusion than the noise in our synthesized aperiodic content (Fig. 11) and the en- noise is predictablefrom the detectabil- examplesemonstrates severalparam- tire sanza tone (Fig 12) with a funda- ity of the most prominent harmonics eters of concern in our fusion experi- mentalfrequency of 285 Hz. The figures may be equally applicable to fusion. ments. The sanza noise is produced by for both instrumentsshow that primary Therefore,our resultson the fusion of a beer bottle capsthat havebeen attached harmonics are surrounded by noise. pure tone oughttobe relevant to com- to the resonator of the instrument. Both noise spectra,in factyappear to fol- plex tones as well. Noise-tone fusion in the case of this in- low the harmonic spectra of their re- An alternativeexplanation of the fail- strumentis perceptuallymuch stronger spective instruments,so that the enve- ure of the flute example to conform than for the flute. For the musicalbow, lope of just the noise portion of each more exactlyto our experimentalresults which demonstratesunfused noise and signal resemblesthe envelope of the re- is that these resultsmay need to be rein-

FaZesand MsAdams,The Fusion and Layering of Noise and Tone 73 30__A201 o #ki1|| _ .....+__ 0 >1lr__ _ 1 _ 1_11 t t. +_ ---| q , 1 1C- 1 .. wil--'t------1l1 of1[-, 1111+. M1 0q . ]51 Isic,i i .U, t 1l-- --lr stonly( Gregory aln I thelightlypoint timbre Sandellofinfluential view-i.e.of the distinguishedother.inin theterms perceptualThus; of thewe

terpretedto accountfor degreesof fusion. dB In order to do this, we need to establish a definition of fusion that accounts for i _ the varyingstrengths of listeners'sensa- t tions.Does "weakfusion" occur when the l J noise accompanyinga periodic tone is

i I composition of the timbre? And does

20 J __ | l_ F -|--[ L | ^ y thatmean thatthe n) se it elfmustbe at 4 11 } t , : } 1 1 low level?Or can tl oi e be relatively I i t ll w5 J 11 (, i tense, but with its c ter equencyoff- 10 0-- 1I t by such a degree from the tone that - - t- l - t 9 r I 1! I I 2 sonly a partof it is contributingto the sen-

°O ' 1000 1 20001 3000 4000 5000 5000 7i0 8000 9000 10000 sation offusion? If the second arrange- ment of noise and tone is allowed as a

ig. Fi10. Narrowband spectrum of harmo?zi& elements of sanza tone of fundamental fre- kind of fusionwith only partof the noise uency 285 Hz. The horizontal ax}s corresponds to frequency (in Hz), and the vertical axis fusing, then can noise both fuse and >amplitude (in dB). Note the relative of partials, as indicated by their un- layer with the tone it accompanies?In ven distribution across the frequency axis * our experiments,we have consideredall stimuliprovoking responses 3 or 4 to be dB fused, but perhapsresponse 2 ought to be consideredto reflect a less strongde- I | gree of fusion thus provoking a less

j | 01 ||\ 1 F | y r 8 10 peAttI *t ^ nlerencc ol tl e Et ro- } | | I I be ween "emergent"timbre, produced II q by the perceptual fusion of the entire | spectraof two separateinstruments, and i ' "augmented"timbre, produced by the fu- °O 1000 2000 30iOo 4000 6000 60'00 7000 8000 9000 10000 sion of several harmonicsfrom one in- strumentwith the entire spectrum of a Hz second instrument, thus changing the Fi ig. 11. Narrowband spectrum of M notseelements of sanza tone of fundamental fre- timbre of the second while leaving the

uency 285 Hz. The horizontal axis corresponds to frequency (in Hz), and the vertical axis

) amplitude (in dB). Notice the similarity in envelope (outline of peaks) of the noise spec- pe ceptual d stlnCtve e s o t e st um to the harmonic spectrum above, indicating that noisebands are roughly centered on lntact [7] . Somethlng of the same dis- pl rimaryharmonics. tinction might well be relevant here, where the flute timbre is augmentedby the presenceof noise, althoughthe noise ds itself is not sufElcientlyintense to pro- duce the kind of emergenttimbre tested E I * s I for in our experimentation. With the 1 1 I flute as an example, it appears that fu- 4D lil ,1 1 1 1 11 | j _ 1 5 on ought to be defined from a percep-

2 A 18 4 }t111 1 l i V 21 { l{"2| s \r N 0 I lutual influence of the noise and tone

,1 1 r | i L 102 it 1ll ,1dl^M conditions are met: (1) the overall tim- rotf T - | --I - - -- i -_ T r 1l| 1 r bre of the sound must demonstratethe I I I ! t 111 11 11 i It|ll I 11 mutualinfluenceofthetwocomponents o I I ; I | I W 1F 1 (so that in the case of two of the instru- 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 ments representedhere, we have a noisy HZ flute and a noisy sanza, as opposed to a

'ig. 12. Narrowband spectrum of sanza tone (complete) of fundamental frequency 285 Eiz. flute and sanzain the presenceof noise), T Nhe horizontal axis corresponds to frequency (in Hz), and the vertical axis to amplitude and (2) when the soundis analyticallydi- in dB). Here noise bandwidths begin to approximate the size of respective auditory filters vided into its two components,both the Inly starting at the fifth harmonic. noise alone and the tone strippedof the noise must sound discernibly different in the absence of the other. The second observation we can make from the flute spectra is that while the

74 FaZesand McAdams,The Fusion and Layering of Noise and Tone 1 1, 1 11111, ,1 .,1

dB noise envelopedoes followthe harmonic envelope,beginning with the second har- monic, it is actuallyshifted slightly,such ___. ... - t--- 40 : -- - that noise peaksare higher in frequency than harmonicpeaks. Furthermore, Fig.

9 whichincludes spectra of both the har- 30 + - - = a R monics and the noise, shows what ap- pears to be a second set of harmonics, 20 h't - & beginningbelow the fundamentalvwhose t!ltlll l I 1ll-11'''11 '/1lpll x peaks are in fact as regular as the real harmonics. Our noise-separation pro- gram assignsmost of these peaks to the i ' , i ! noise component. However,their regu- 3000 4000 5000 6000 7X0 8000 SO00 10000 11000 120X 13000 ) 18000 lariqrmade us suspiciousof the accuracy Hz of this assignmentuniil we matchedthe Fig. 13. Narrowband spectrum of musical bow tone (comptete) of fundamental frequenc pitch of the tone and found it in fact to 302 Hz. The horizontal axis corresponds to frequency (in Hz), and khe vertical axis correu correspondto a frequencyof about1,030 sponds to amplitude (in dB). Notice the presence of two kinds of noise (see text): noise Hz. These regularpeaks, then, become khat surrounds primary harmonics, as in sanza and flute tones, and high frequency noise the peaks of the noise, which are them- (above about 6,000 Hz) that appears to produce the layering audible in its timbre. selves quite regular harmonically.The shift of the noise center frequency is dB greater than that predicted by our ex- periments on the coincidence of tone frequencywith the center frequencyof i ; the noise which may contribute to the weaker influence of the noise on the flute timbre.At the same time, the real partials of this traditional flute are re- markablyinharmonic for a flute [8] al- though not so much in comparison to the sanzadiscussed below. If the partials were ideally harmonic, their values would lie somewherebetween their ac- jl--- a °o 1000 2000 3000 4000I 5000 q##ts-6000 7000 8000 gO00 1000011000 12000 13000 14000 1500016000 17000 18000 tual peaks and the harmonicallyregular peaks of the noise that surroundsthem. Hz Moore [9] and others have shown that especiallywithin the first six harmonics Fig. 14. Narrowband spectrum of high frequency noise filtered from spectrum. The hori- zontal axis corresponds to frequency (in Hz), and the vertical axis to amplitude (in dB). of a complex tone inharmonicpartials Notice the relative formlessness of fhe noise in comparison with the noise in the remaining tend to pull the sense of pitch awayfrom signal (shown in Fig. 16). that indicated by the frequency of the fundamental in the direction of the inharmonicity.If this is true, then a pos- dB sible effect of the regularnoise peakslo- catedabove the true harmonicsmight be the undoing of the tendency of the flute's (inharmonic)partials to pull the sense of pitch downward,by addingwhat might almostbe considereda second set of partials,each paired with one of the real harmonics,but offset in a direction opposite to the real harmonics' in- harmonicity.Indeed, in comparisonwith the complete tone (noise and harmonic frequencies) the harmonic-onlysignal (withoutnoise) soundslower. Even stron- ger confirmationof the role of noise in Hz undoing the pitch effects of Fig. 15. Narrowbandspectrum of portion of signal remaining(after high frequencynoise inharmonicityis heard in a comparison was filtered from spectrum)with noise removedby noise4eparationprocess. The hoxizon- of the harmonic-only signal with the tal axis correspondsto frequency (m Hz), and {he verticalaxis correspondsto amplitude noise-and-harmonicsignal when both (in dB). have been filtered to exclude all but the first three harmonics.Informal listening to these soundsproduced general agree- ment among listeners that not only was

Fales and McAdams,The Fusion and Layering of Noise and Tone 75 dB were unable to hear any periodic ele- ments whatsoever;it was evident from 30 this result that the bow was an example of an instrument on which our noise- separation process is ineffective, since

20 the noise elements in part of the spec- trum appear to be louder than the har- monic elements in the same regions. .lll Thus the software'schoice of the high- 10 est intensitypeaks as harmonicswas in- appropriatein the case of the musical bow. However, having filtered out the frequency region above 6,000 Hz, we os ) 7000 8000 9000 10000 11000 12000 13000 14000 15000 16000 17000 18000 could then apply the noise-separation process to the sample that remained. Fig. 16. Narrowbandspectrum of portion of signal remaining(after high frequencynoise Figures 13-16 are spectra of the entire was ffltered from spectrum)with noise only, isolated by noise-separationprocess. The bow tone (Fig. 13), the high-intensity horizontalaxis correspondsto frequency(in Hz), and the verticalaxis correspondsto am- noise region removed by filtering (Fig. plitude (in dB). Note, for this portion of the signal, the similarityin envelope (outline of 14), the remainingsignal with noise ex- peaks) of the noise spectrumto the harmonicspectrum above, indicatingthat noisebands tracted (Fig. 15), and the noise only of are roughlycentered on primaryharmonics. the remainingsignal (Fig. 16). If we look firstat the spectraof the sig- nal remainingafter filtering out all fre- the pitch higher for the reduced noisy noise-separationprocess, of course,can- quencies above 6,000 Hz, we see some- complex,but the "roughness"or "graini- not distinguishbetween the two sources. thing of the same pattern of noise-tone ness" of the noiseless complex was re- This is alwaysa problemwhen there is a relationas we sawin the flute and sanza. duced by the presenceof noise. supplementary noise device acting on The noise followsthe harmoniccontent an instrument.For the analysisof overall of the tone, forming noise bands sur- Sanza Analyses noise content, it probablymakes little rounding each peak. Again, The spectra for the sanza are remark- differencewhere the noise originates,as when we furtherfiltered this part of the ably complex when comparedwith the long as it is discerniblyfusing or layer- signal to leave only the first four har- flute spectra. The partialsof the sanza ing. However,when discussingthe reac- monics, those harmonics were inhar- have so inconsistenta relationship(the tion of supplementarynoise devices on monic to the extent that theywere heard distancebetween them varyingfrom 212 the "natural"timbre of the instrument, to be so weaklyfused that, ratherthan a to 320 Hz, with a quasi-fundamentalof it is importantto distinguishnoise pro- single, unified sound with a discernible 285 Hz) that it is difficult to decide duced in the absenceof the device from pitch, almostall listenersheard a collec- which peaksare legitimateharmonics- noise producedby the device.Thus, hav- tion of three or four pitches, none of even though the sample was run ing suggested a possible "function"of which appearedto be the same pitch as through the noise-separation process the sanza noise, we also point out that the entire tone. When we combined three times to eliminate as much noise propositions as to noise devices that these four harmonics with the highfre- as possible. Here again, the noise enve- "compensate"for inharmonicity,for ex- quencynoise (Fig. 13), the resultingper- lope follows the spectralenvelope. Un- ample, must remain tentativeas long as cept was of the separate pitches of the like the flute, however,the noise band- there is no sample of the same harmonicswith noise superimposedor widths surrounding the harmonics instrument's sound with its inherent layeredabove. Evidently the noise above begin to approximatethe size of the re- noise and withoutthe external noise de- 6,000 Hz contributedneither to the fu- spective auditory filters only with the vice- that is, as long as there is no pure sion of the harmonicsnor to the sense of fifth harmonic.Further, with the excep- sample of what it is that the noise com- pitch but, rather,added an additional, tion of the fundamental, the intensity pensatesfor. layeredsound. When we mixed back in level of the noise relative to the har- the noise below 6,000 Hz (Fig. 17), how- monic it surroundsis high enough that Musical Bow Analyses ever, listeners reported that the same the total noise power entering each fil- As mentioned earlier, the musical bow four harmonicsfused into a slightlynoisy ter appears to exceed the noise-to-har- example differs from the other instru- tone and yielded a sense of pitch match- monic rationecessary for fusion. ments describedhere in that it demon- ing that of the entire tone. Again, it ap- A problemin the noise analysisof the strates layered rather than fused- pears that only the noise surrounding sanza (whichdid not arisein the case of tone and noise. When we tried our harmonicsof the bow could aid in their the flute) is that, in all likelihood, the noise-separationprocess on samples of fusion and the resulting pitch percep- instrumentused for analysishere actu- the bow,we found that the softwareleft tion, and further,that this noise wasalso ally contains more than one noise so much noise in the frequencyregion adding a slight noisy quality compa- source. We have mentioned alreadythe above 6,000 Hz that a spectrumof what rable in intensityto the noisy qualityof use of bottle caps attached to the reso- was meant to be harmonic-only ele- the flute. When we then mixed back in nator of the instrument. At the same ments looked quite similarto the spec- the high frequency noise (above 6,000 time, there is probablysome degree of trumfor the entire tone. When we next Hz), the resulting percept was almost noise producedby the instrumentitself, bandpass-filteredthis extractedsignal to identicalto the entire tone as it occursin as it is traditionally performed. The leave only the region above6,000 Hz, we performance, with a unified pitch and

76 Fales and McAdams,The Fusion and Layering of Noise and Tone timbre-senseand an additionalnoise lay- bottle-cap trick appeared to work. It is in hferences and Notes ered over this complex. stories such as this one that the analyti- 1. See, for example, J*H. Kwabena Nketia, Muszcof Thusnlt appearsthat, in fact, the musi- cal and experimental parts of our noise Afrzvs(New York:W.W. Norton, 1973). cal bow recorded for this sample pro- fusion study come together. Anyone who 2. The sanza, also called mbtruin parts of Africa, is a duces two kinds of noise: one of lower is familiar with the noisy timbre of a small, handheld lamellophone with a rectangular frequency and intensity that fuses with bottle-cap-laden sanza such as the one shallow box resonator and a clavier of wooden or metal lamellae that are "pluckedt with the thumb- the tones it surroundsand that contriS described above will recognize that the nails. In some regions of AfricaXthe sanza may be utes to their fusion and pitch sense and level of the noise could not have been put inside a larger calabasse for extra resonance. In the United States, the instrument is colloquially another of higher frequencyand inten- nearly sufflcient to maskthe mismatched called a zithumbpiano.t sity that produces the layeringof noise intonation of the two sanzas. If there is any validity to our earlier suggestion that 3. The musical bow is a struck string instrument re- audibleabove the tone. As in the case of sembling the bow o£ a bow-and-arrow with a gourd the sanza,there is no wayof knowingif the noise described above counteracts resonator attached to the wooden bow part of the both kinds of noise arisefrom the noise the inharmonicity of the instruments instrument. This gourd is held against the musician's chest cavitwfor further resonance while device attached to the instrumentor if that produce it-and if we consider the he or she strikes the string with a short baton. the fusingnoise is inherentto the instru- comments of our experimental subjects 4. For musicians5 whose system begins counting ment,while the layerednoise is the result concerning the "spreading out of pitch harmonics a£ter the fundamental frequency7 this of the "metal clackers."Whatever the as an effect of noise-then it is possible will be the first harmonic or . sourcesof the noise, their configurations that a similar phenomenon is working 5. B.C.J. Moore, An Int;roductionto the Psychologyof as revealed by the spectra above are for the Central Mrican sanza musicians Hearzng(London: Academic Press, 1989). again in agreement with the results of to compensate for the imperfect intona- 6. I)egree of £usion was an issue of concern in factS our experiments.The fusing noise cen- tion between the two instruments by in the original design of the experiment, but the ters roughly on prominent harmonics blending one intonation into the other. intial difficulty encountered in proposing the no- tion of noise fusion to subjects made the proposi- while the layering noise is well set off tion of degreesof fusion unrealistic. from audibleperiodic elements and is of CONCLUSION 7. Gregory Sandell, 'iAnalysis of Concurrent Tim- sufficientintensity that it masksany peri- bres with an Auditory Model,s' paper presented at odic elements that lie in the same fre- Verification of hypotheses concerning the Third International Conference for Music Per- ception and Cognition (ICMPC), hosted by the Eu- quencyregion. the perceptual effects of noise in regard ropean Society for the Gognitive Sciences of WIusic One of the intriguing aspects.of the to faulty tuning, inharmonicity or am- (ESCONI), Liege Belgium, 23-27July 1994. studydescribed in this paper-although biguous pitch sensation will require fur- 8. Personal communication, Marc Pierre Merge (in- it is one whose implicationscan only be ther, extensive laboratory research. Veri- strument acoustician, IRCAh4) concer ning inhar- speculated occurred in informal re- fying that such perceptual effects are monicity of flute-like instruments, March 1994. sponses from listeners describing the intentional on the part of musicians will 9. Moore [5]. require additional extensive field re- fused noise-tone percept. Four com- 10. Personal communicatiesnx see also Vincent ments that were repeated frequentlyby search. Similarly an investigation of the Dehoux, Les Chnts a Penser(Paris: SELAF,Place de subjectswere that the noise-tonefusion intended effect of noise fusion and layer- la Sorbonne, 1992) . (1) seemed to '4spreadout" the pitch ing-of whether one was more desirable (2) gave the sensationof more than one or productive of the illusion than the Bbliography pitch, (3) left listeners unsure of the other-will also require inventive field re- search The use of other noise phenom- A.S. Bregman and S. Pinker 'Auditory Streaming pitch7 and (4) caused the illusion of and the Building of Timbret Cvnadian Journst of asyrlchronybetween the noise and tone. ena, such as auditory restoration effects P$ychobe32 (1978) pp. 19-31. Vincent Oehoux of the Departmentof might also contribute to the intended ef- B.R Glasberg and B CJ. Moore "Derivation of Au- Ethnomusicology in the laboratory of fects of noise in African music. From the ditory Filter Shapes from Notched Noise the Langues et CivilisationsTradition- perspective of ethnomusicology, in order Data,BHearznghsesrch47 (1978) ppw10>138. nelles Oralesof the Centre National de to combine the controlled results of labo- B.C.J. Moore "The Perception vf Inharmonic Recherche Scierltifique ( LACITO- ratory research with the richness of un- Complex Tones, in W. Yost and C. Watson, eds.} AuditoryProcessing of ComplencSounds (Hillsdale, : CNRS)in Paristells the storyof being in controlled musical behavior experimen- Lawrence Erlbaum Associates, 1987). the presence of two sanza musicians tation of the kind described here must B.C.J. Moore, B. Glasberg and R. Peters, 'Relative from the CentralAfrican Republic who consist of equal parts of laboratory re- Dolninance of Individual Partials in Determining had been workingfor some time to tune search acoustic analyses of the instru- the Pitch of Complex Tones,l' Journal of the Acousti- their instruments together for a duet ments and field research. cat Soczet of America77, (1985) pp. 1853-1866. they were to perform. After a finalsun- B+C.J. Moore, R. Peters and R.W. Peters, Thresh- successfulattempt they gave up the ef- olds for the Detection of Inharmonicity in Com- Acknowledgments plex TonesnJournat of theAcxs6cat icst of Amsa fort, deciding instead simply to attach 77 (1985) pp. 1861-1867. still more bottle caps to the resonators This research was funded in part by National Sci- ence Foundation and Fyssen Foundation grants to of their instruments.And, said Dehous the first author and by a grant from the French Min- [10], whatever the mechanismS the istry of Research and Space to the second author.

Fales and M%cAda7rJ The Fusion and Layering of Noise and Tone 77