arXiv:chao-dyn/9907002v1 26 Jun 1999 where r otiilsltoso h equation the of solutions nontrivial are and necmiaintnsi h olnaiiso h a.A frequencies ear. with the sources two of by differ nonlinearities fed of the nonlinearity generation in passive the tones to combination it ence attributed who [3], Helmholtz perception. residue to is equal this and fundamental; unchanged missing remains the tone the timbre of few the pitch first Although the the 1b). changes, (Fig. of removed some are note. and harmonics musical fundamental higher same the distin the that to playing suppose violin one Now a allow from which tim- trumpet a sound, give guish the amplitudes a number to relative of The characteristics their that bral be and fundamental. to the harmonics perceived of higher is frequency of pitch the Its at 1a tone ear. Fig. pure in the shown to that as presented such stimu- tone is the periodic of a that component Suppose frequency lus. any to mapped be cannot perception rbe fthe of problem system. auditory pro- the extensive of for levels do need higher as at the well cessing removes as that least and at models, sounds current complex of results perception experimental pitch describes percep- the that pitch tones of complex theory for nonlinear tion a processi develop peripheral we and Here [9 neural cortex [10,11]. integrate auditory models primary pro- the latest this before The that out is carried evidence is necessar experimental cessing is the been However, processing has system it [6–8]. nervous recently central peripher more that [1–5]; a thought system at auditory arise the first to in The level pitch several considered understood. with well theories tones not perceptual complex still of is pitch components the frequency of Pythag origin to in- back the scientific dating but The history ras, the long pitch. a by same has given the pitch is of of vestigation terms sound tone pure arbitrary in a an pitch of of of frequency pitch definition the operational en- frequency: an and of adopt monotonic to is us relation sin- ables a the with component, tone this frequency pure a in gle For stimulus, frequencies. the component of its qu attributes mainly subjective physical a to is pitch related sensations, tity all Like scale. musical h rtpyia hoyfrtersdei u ovon to due is residue the for theory physical first The e hnmnni ic ecpini nw sthe as known is perception pitch in phenomenon key A a on lie to it perceive we where is sound a of pitch The ω 2 p eeae obnto oe ffrequency of tones combination generates and 1] n ossso h ecpino ic that pitch a of perception the of consists and [12], ASnmes 54.a 36.y 87.19.La 43.66.+y, 05.45.-a, numbers: PACS kno sounds the especially intelligibility. sounds, speech and of pitch the of perception q isn fundamental missing r nees o amnccmlxtone complex harmonic a For integers. are eapyrslsfo olna yaist nodpolmin problem old an to dynamics nonlinear from results apply We 3 ntttMdtraidEtdsAacas MDA(CSIC–U Avanc¸ats, IMEDEA d’Estudis Mediterrani Institut 1 olna yaiso h ecie ic fCmlxSounds Complex of Pitch Perceived the of Dynamics Nonlinear nttt nau eCeca el ira AT(CSIC-UGR) IACT Tierra, la de Ciencias de Andaluz Instituto uynH .Cartwright E. H. Julyan , ita pitch virtual pω Pyia eiwLetters, Review (Physical 1 2 + siuoLml N,I419Blga Italy Bologna, I-40129 CNR, Lamel, Istituto qω 2 or , + ω C 1 ω which , , residue C ∗ ig .Gonz´alez L. Diego , 0 = case an- ω on ng o- al ]. y 1 - - , 1 82 edsrbdb ifrnecmiaintn:i esital amount shift same we the if by tone: partials cann combination the difference residue the a of by behavior described the be Schout that demonstrated residue. [4] the al. et of theories nonlinear challenged ously ( hsb reo h eiu.Isedi sfudta h per- the that on found dependence is linear it a Instead showing shifts, also residue. should pitch the ceived same of the true and be thus unchanged, remains tone combination oe oee,frsmall for However, num- harmonic in tone. the lines of the inverse of the slopes on ber the be- roughly that the depend qualitatively shows 1d model shift Fig. pitch to the attempt of first havior A accurac an [13]). attain 0.2% can of shift pitch pitch on experiment first experiments many the (psychoacoustic in measured as accurately known been has is and effect, phenomenon This 1d). (Fig. (i.e., h rqec ftemsigfundamental missing the of frequency the Fg b,tedfeec obnto tone combination difference the 1b), (Fig. ual tbebhvo fti ls fdnmclsse.W system. dynamical of str class this with of data behavior stable experimental turally identify nonlinear pro generic and central a oscillator, as extensive con- forced system to auditory be the resort can model We to tones cessing. need complex the for without perception structed pitch of terms in theory such described a systems, dynamical accurately nonlinear be of attractors can generic al. of et Schouten of ment neur [10,11]. integrated models centra to peripheral of and thence result [6–8]; a processing tha be to system theories tones nervous of complex development of and pitch the [2,4] the to considered (periodicity and ears peripheral theories both [1,3]) of or place one abandonment exciting the stimulus to the led of th (all monoti of just diotically rest not the and and and ear) ear controlateral, dicho other, one the elicited exciting stimulus stimulus is the residue of the (part it cally that effect, experiments demonstrated shift Pitch-shift that pitch [4]. others it second within behavior the included anomalous with usually this is correlated As be pitch. to residue seems produce the frequency in central decrease the a pa fixed between maintaining spacing while the of tials enlargement an Also, effect. shift lgtybtcnitnl agrta hs u mle fw if smaller but this, place than larger consistently but slightly k 3959,1999) 5389–5392, , na ope oe htaeipratfrmusic for important are that tones complex as wn edmntaeblwta h rca ic-hf experi- pitch-shift crucial the that below demonstrate We 1) + k p 1 + k B,E001Plad alra Spain Mallorca, de Palma E-07071 IB), ω 1 + 1 = 0 cutclpyis h ehns fthe of mechanism the physics: acoustical 2 ftecnrlprilo he-opnn complex three-component a of partial central the of − , † by , n rsePiro Oreste and , kω q -87 rnd,Spain. Granada, E-18071 , k = 0 hsbhvo skona h eodpitch second the as known is behavior This . = − ω 1 0 ewe w ucsiepril has partials successive two between ) oee,acuileprmn seri- experiment crucial a However, . k 3 tlat h hnei lp is slope in change the least, at , ‡ ∆ ω Fg c,tedifference the 1c), (Fig. ω ω 0 C htis that , = ω 2 ω − and , cally C re- e ∆ that uc- ω en ti- = ot al ω r- y 1 e e s s ) - l l t emphasize that since we are interested in universal behavior, Since complex tones can be decomposed as a series of the results we obtain are not dependent on the construction partials — a sum of purely sinusoidal components — and of a particular model, but rather represent the behavior of any residue perception is elicited with at least two of these, we dynamical system of this type. search for suitable attractors in the class of two-frequency quasiperiodically forced oscillators. Quasiperiodically forced oscillators show a great variety of qualitative behavior that falls into the three categories of periodic attractors, quasiperi- odic attractors, and strange attractors (both chaotic and non- Temporal Fourier Pitch Waveform Spectrum chaotic). We propose that the residue behavior we seek to 1 explain is a resonant response to forcing the auditory system a 2 quasiperiodically. From stability arguments [14–17] we sin- 3 4 5 P=ω gle out a particulartype of two-frequencyquasiperiodicattrac- 6 7 0 8 tor, which we term a three-frequencyresonance, as the natural

ω0 candidate for modelling the residue. Three-frequency reso- 4 b 5 nances are given by the nontrivial solutions of the equation 6 7 pω1+qω2+rω =0, where p, q, and r are integers, ω1 and ω2 8 P=ω R 0 are the forcing frequencies, and ωR is the resonant response, and can be written compactly in the form (p,q,r). Notice that ω 0 combination tones are three-frequency resonances of the re- c stricted class (p, q, 1). This is the only type of response possi- P=ω +∆P ble from a passive nonlinearity, whereas a dynamical system 0 such as a forced oscillator is an active nonlinearity with at ω least one intrinsic frequency, and can exhibit the full panoply 0 ∆ω of three-frequency resonances, which include subharmonics d k=6 k=7 k=8 P of combination tones. The investigation of three-frequency systems is a young area of research, and there is not yet any ∆ ∆P ∝∆ω P consistency in the nomenclatureof these resonancesin the sci- ω0 ∆ω entific literature: as well as three-frequency resonances, they are also called weak resonances or partial mode lockings; see ω ω ω f 70 8 0 9 0 Baesens et al. [18] and references therein. It is known that FIG. 1. Fourier spectra and pitch of complex tones. Whereas three-frequency resonances form an extensive web in the pa- pure tones have a sinusoidal waveform corresponding to a single fre- rameter space of a dynamical system. In particular, between quency, almost all musical sounds are complex tones that consist of any two parent resonances (p1, q1, r1) and (p2, q2, r2) lies the a lowest frequency component, or fundamental, together with higher daughter resonance (p1 + p2, q1 + q2, r1 + r2) on the straight frequency overtones. The fundamental plus overtones are together line in parameter space connecting the parents. collectively called partials. a A harmonic complex tone. The over- For pitch-shift experiments, the vicinity of the external fre- tones are successive integer multiples k = 2, 3, 4 . . . of the funda- quencies to successive multiples of some missing fundamen- mental ω0 that determines the pitch. The partials of a harmonic com- tal ensures that (k + 1)/k is a good rational approximation to plex tone are termed harmonics. b Another harmonic complex tone. their frequency ratio. Hence we concentrate on a small inter- The fundamental and the first few higher harmonics have been re- val around the missing fundamental between the frequencies moved. The pitch remains the same and equal to the missing fun- damental. This pitch is known as virtual or residue pitch. c An ω1/k and ω2/(k + 1), which correspond to the resonances − − anharmonic complex tone. The partials, which are no longer har- (0, 1, k) and ( 1, 0, k + 1). We suppose that the residue monics, are obtained by a uniform shift ∆ω of the previous har- should be associated with the largest resonance in this inter- monic case (shown dashed). Although the difference combination val. In numerical simulations and experiments with electronic tones between successive partials remain unchanged and equal to the oscillators [14–17], we have confirmed that for small nonlin- missing fundamental, the pitch shifts by a quantity ∆P that depends earity the daughter of these resonances, (−1, −1, 2k + 1), is linearly on ∆ω. d Pitch shift. Pitch as a function of the central fre- the resonance of greatest width between its parents. Hence we quency f = (k + 1)ω0 +∆ω of a three-component complex tone propose that the three-frequency resonance formed between {kω0 +∆ω, (k+1)ω0+∆ω, (k+2)ω0 +∆ω}. The pitch-shift effect the two lower-frequency components of the complex tone and is shown here for k = 6, 7, and 8. Three-component complex tones the response frequency P = (ω1 + ω2)/(2k + 1) gives rise are often used in pitch experiments because they elicit a clear residue to the perceived residue pitch P . Since in pitch-shift exper- sensation and can easily be obtained by amplitude modulation of a iments the external frequencies are ω1 = kω0 + ∆ω and pure tone of frequency f with another pure tone of frequency ω0. When ω0 and f are rationally related, ∆ω = 0, and the three fre- ω2 = (k+1)ω0 +∆ω, the shift of the response frequencywith quencies are successive multiples of some missing fundamental. At respect to the missing fundamental is ∆P = 2∆ω/(2k + 1). this point ∆P = 0, and the pitch is ω0, coincident with the frequency This equation gives a linear dependence of the shift on the of the missing fundamental. detuning ∆ω, in agreement with the first pitch shift effect.

2 nonlinear terms without fitting parameters, and can overcome the objections against nonlinear theories raised by pitch-shift experiments. Dichotic perception has been another argument against nonlinear theories, because it has been thought to imply cen- tral nervous system processing [21]. However, subcortical nervous paths, for example through the superior olivary com- plex, allow the exchange of information between both — left and right — peripheral auditory systems. Moreover, fre- quency information up to 5 kHz is preserved until at least this point in the auditory system [22,23]. These two factors al- low frequency components of the stimulus that arrive at dif- ferent ears to interact at a subcortical level. This implies that three-frequencyresonances can be generated, not only monot- ically but also dichotically, by a mechanism such as that which we are proposing. Experimental evidence for the existence of FIG. 2. Experimental data with our predictions superimposed multifrequency resonant responses is given by electrophysio- show pitch as a function of the central frequency f =(k+1)ω0+∆ω logical records in single units of the auditory midbrain nucleus for a three-component complex tone {f − g,f,f + g}, for k = 6, of the guinea fowl [24]. Our mechanism is also consistent 7, 8, 9, 10, and 11. The component spacing is g = ω0 = 200 Hz. Circles, triangles and dots represent experimental data for three dif- with neuromagnetic measurements performed on human sub- ferent subjects (from Schouten et al. [4]). The perceived pitch shifts jects showing that pitch processing of complex tones is carried linearly with the detuning ∆ω. The pitch shift effect we predict is out before the primary auditory cortex [9]. shown superimposed on the data as solid lines satisfying the equa- Thanks to the residue we can appreciate music in a small tion P = g + 2(f − (k + 1)g)/(2k + 1). The lines describe the radio with negligible response at low frequencies; but it is behavior of the response frequency in a three-frequency resonance not just an acoustical curiosity. The residue seems to play an formed with the two lowest-frequency components of the stimulus. important rˆole in music perception and speech intelligibility. The lines agree with the psychophysical data without any fitting pa- The importance of the residue for the perception of musical rameters. sounds has long since been recognized. It has been proposed [8] that residue perception is at the heart of the fundamental In Fig. 2 we have superimposed the behavior of the cor- bass of Rameau [25]. As the fundamental bass and its more responding three-frequency resonances on data for the pitch modern counterparts form a key element for the melodic and shift for three different subjects. There is good agreement, harmonic structuration of musical sounds, a physical basis for which explains both the first pitch shift effect and the first the residue may contribute to the construction of an objec- aspect of the second pitch shift effect, because the predicted tively grounded theory of music [14]. As for speech intelligi- slope is 1/(k +1/2), between 1/k and 1/(k + 1). The second bility, hearing aids that furnish fundamental frequency infor- aspect of the second pitch shift effect can be interpreted as mation produce better scores in profoundly hearing impaired follows: the term 2∆ω in the equation for ∆P arises from the subjects than simple amplification [26]. It is clear that a better two equal contributions ∆ω obtained by a uniform shift in the knowledge of the basic mechanisms involved in pitch percep- two forcing frequencies. If, while maintaining ω2 fixed, we tion will allow a similar improvement in its applications to increase the interval to ω1 — we enlarge the spacing between technology and medicine. successive partials — the first contribution remains constant and equal to ∆ω while the second diminishes, to give a de- The principle of economy, widespread in nature, suggests crease in the response frequency of the resonance and thus in that by means of our proposed mechanism, complex tones the residue. may be preprocessed at a subcortical level, to feed optimized pitch candidates to more central zones of the nervous system, For small harmonic numbers k our model data fall within experimental error bars for most of the pitch-shift experiments freeing them of an enormous quantity of calculation. In this way, the nonlinear behavior of the auditory periphery may reported [4,19,20]. For larger k there are systematic devia- tions of the residue slopes which become shallower than theo- greatly contribute to explain the astonishing real-time analyt- retical predictions of both our theory, and the central theories. ical capabilities of the auditory system [27]. An accepted explanation for this is that difference combina- JHEC and OP acknowledge the financial support of the tion tones generated as a consequence of passive nonlinear Spanish Direcci´on General de Investigaci´on Cient´ıfica y mixing in the auditory periphery can play the rˆole of the low- T´ecnica, contracts PB94-1167 and PB94-1172. We are est frequency components of the stimulus. The same explana- pleased to thank Egbert de Boer, Manfred Schroeder, and tion is also valid for our approach. Our goal here, however, Gene Stanley for helpful conversations. is simply to demonstrate that physical frequencies other than combination tones can accurately describe residue behavior in

3 ∗ [email protected], WWW http://formentor.uib.es/∼julyan. [14] D. L. Gonz´alez, Ph.D. thesis, Universidad Nacional de la Plata, † [email protected]. Argentina, 1987. ‡ [email protected], WWW http://www.imedea.uib.es/∼piro. [15] O. Calvo et al., IEEE Trans. Circuits and Systems, to appear [1] G. S. Ohm, Ann. Phys. Chem. 59, 513 (1843). (1999). [2] A. Seebeck, Ann. Phys. Chem. 60, 449 (1843). [16] J. H. E. Cartwright, D. L. Gonz´alez, and O. Piro, Phys. Rev. E [3] H. L. F. von Helmholtz, Die Lehre von dem Tonempfindun- 59, 2902 (1999). gen als physiologische Grundlage f¨ur die Theorie der Musik [17] O. Calvo et al., Int. J. Bifurcation & Chaos, to appear (1999). (Braunschweig, 1863). [18] C. Baesens, J. Guckenheimer, S. Kim, and R. S. MacKay, Phys- [4] J. F. Schouten, R. J. Ritsma, and B. L. Cardozo, J. Acoust. Soc. ica D 49, 387 (1991). Am. 34, 1418 (1962). [19] R. D. Patterson, J. Acoust. Soc. Am. 53, 1565 (1973). [5] R. Plomp, J. Acoust. Soc. Am. 41, 1526 (1967). [20] A. Gerson and J. L. Goldstein, J. Acoust. Soc. Am. 63, 498 [6] J. L. Goldstein, J. Acoust. Soc. Am. 54, 1496 (1973). (1978). [7] F. L. Wightman, J. Acoust. Soc. Am. 54, 407 (1973). [21] A. J. M. Houtsma and J. L. Goldstein, J. Acoust. Soc. Am. 51, [8] E. Terhardt, J. Acoust. Soc. Am. 55, 1061 (1974). 520 (1972). [9] C. Pantev, M. Hoke, B. L¨utkenh¨oner, and K. Lehnertz, Science [22] N. Y. S. Kiang, Discharge patterns of single fibers in the cat’s 246, 486 (1989). auditory nerve (MIT, 1965). [10] R. Meddis and M. J. Hewitt, J. Acoust. Soc. Am. 89, 2866 [23] D. Johnson, J. Acoust. Soc. Am. 68, 1115 (1980). (1991). [24] G. Langner, Exp. Brain Res. 52, 333 (1983). [11] M. A. Cohen, S. Grossberg, and L. L. Wyse, J. Acoust. Soc. [25] J. P. Rameau, Trait´ede l’harmonie (Paris, 1722). Am. 98, 862 (1995). [26] A. Faulkner et al., J. Acoust. Soc. Am. 91, 2136 (1991). [12] E. de Boer, in Handbook of Sensory Physiology. Auditory Sys- [27] Sample sound files to demonstrate the phe- tem, edited by W. D. Keidel and W. D. Neff (Springer, 1976), nomena we describe in this Letter may be found at the URL Vol. V, pp. 479–584. http://www.imedea.uib.es/∼piro/PitchPage/index.html. [13] W. M. Hartmann, J. Acoust. Soc. Am. 100, 3491 (1996).

4