European Journal of Neuroscience, Vol. 18, pp. 432–440, 2003 ß Federation of European Neuroscience Societies

Tonotopic representation of missing fundamental complex in the human auditory cortex

Takako Fujioka,1,2,3 Bernhard Ross,1 Hidehiko Okamoto,1,2,4 Yasuyuki Takeshima,2 Ryusuke Kakigi2 and Christo Pantev1 1Rotman Research Institute, Baycrest Centre for Geriatric Care, University of Toronto, Toronto, Canada 2Department of Integrative Physiology, National Institute for Physiological Sciences, Myodaiji, Okazaki, Japan 3Department of Physiological Sciences, School of Life Science, The Graduate University for Advanced Studies, Myodaiji, Okazaki, Japan 4Department of Otolaryngology and Sensory Organ Surgery, Osaka University Graduate School of Medicine, Osaka, Japan

Keywords: auditory cortex, , magnetoencephalography, tonotopic organization,

Abstract The N1m component of the auditory evoked magnetic field in response to tones and complex sounds was examined in order to clarify whether the tonotopic representation in the human secondary auditory cortex is based on perceived pitch or the physical spectrum of the sound. The investigated stimulus parameters were the fundamental (F0 ¼ 250, 500 and 1000 Hz), the spectral composition of the higher of the missing fundamental sounds (2nd to 5th, 6th to 9th and 10th to 13th harmonic) and the frequencies of pure tones corresponding to F0 and to the lowest component of each complex sound. Tonotopic gradients showed that high frequencies were more medially located than low frequencies for the pure tones and for the centre frequency of the complex tones. Furthermore, in the superior–inferior direction, the tonotopic gradients were different between pure tones and complex sounds. The results were interpreted as reflecting different processing in the auditory cortex for pure tones and complex sounds. This hypothesis was supported by the result of evoked responses to complex sounds having longer latencies. A more pronounced tonotopic representation in the right hemisphere gave evidence for right hemispheric dominance in spectral processing.

Introduction Listening to a harmonic complex sound, a unified pitch is perceived these models fail to explain fully pitch perception over a wide rather than separated frequency components. The pitch of the complex frequency range with either temporal or spectral encoding mechanism. sound is mainly determined by its , F0, which is The periodicity of the sound is conveyed along the auditory pathways the repetition rate of temporal periodicity in the sound. Although the as phase-locked activity (Rose et al., 1967). However, more centrally, ‘missing fundamental’ (MF) complex sound contains no spectral the upper frequency for phase-locked activity degrades from about energy at F0, it produces a pitch that can be matched by F0 (Schouten, 700 Hz in the central nucleus of the inferior colliculus (Langner & 1940). This phenomenon of ‘virtual pitch’ (Terhardt, 1974) is even Schreiner, 1988) to about 100 Hz in the auditory cortex of the cat recognized when the harmonic components are separately presented to (Schreiner & Urbas, 1988). Therefore, it seems unlikely that the each ear (Houtsma & Goldstein, 1972). Additional noise does not mask periodicity is the sole coding strategy for pitch information of complex the virtual pitch (Patterson, 1969). Thus, both observations indicate sounds. that pitch processing based on temporal sound properties is more By contrast, the tonotopic organization that is established in the centrally processed in the auditory system. By contrast, the spectral cochlea and preserved along the auditory pathway has been demon- structure of a complex sound is perceived as its and it is an open strated in the human auditory cortex using auditory evoked potentials question as to how temporal and spectral encoding are integrated in the (AEPs) (Bertrand et al., 1991) and auditory evoked fields (AEFs) auditory system causing the senses of pitch and timbre. recorded using magnetoencephalography (MEG) (Elberling et al., Several psycho-acoustical studies model pitch perception by means 1982; Romani et al., 1982; Pantev et al., 1988). Furthermore, the of temporal encoding or spectral pattern analysis (Goldstein, 1973; recent studies combining MEG with intracerebral evoked potentials Wightman, 1973; Terhardt, 1974). However, both models fail to have demonstrated that auditory responses of middle to late latency explain more general phenomena such as the pitch of non-harmonic range have different source locations that can be separated with complex sounds (Schouten, 1940; Schouten et al., 1962). In addition, excellent spatial and temporal resolution. For example, Godey et al. (2001) revealed that N1m responses (most pronounced late latency response, peaking around 100 ms after stimulus onset) have different

Correspondence: Dr Takako Fujioka, at present address below. sources in the intermediate and lateral parts of the Heschl’s gyrus and E-mail: [email protected] in the planum temporale. They also showed that middle latency components Pam and P1m (30–50 ms) are generated in Heschl’s gyrus Present address: The Rotman Research Institute, 3560 Bathurst St., Toronto, Ontario, (Godey et al., 2001). Although other brain imaging techniques such as M6A 2E1, Canada positron emission tomography (PET) (Lauter et al., 1985; Lockwood Received 3 February 2003, revised 21 April 2003, accepted 12 May 2003 et al., 1999) and functional magnetic resonance imaging (fMRI) doi:10.1046/j.1460-9568.2003.02769.x Cortical representation of missing fundamental sounds 433

(Wessinger et al., 1997; Bilecen et al., 1998; Talavage et al., 2000) have also demonstrated the tonotopic representation of the auditory cortex, the accumulated literature clearly confirms that magnetoence- phalographic techniques are appropriate for investigating fine spatial– temporal representation in the auditory cortices in terms of the functional differences to the various sounds. The tonotopic organiza- tion reflects, in general, processing of the sound spectrum across the whole auditory pathway. However, the spectral resolution in the periphery itself is not high enough to account for the fine frequency discrimination. Therefore, it is reasonable to assume a hybrid system in the afferent auditory pathway gradually integrating both the temporal and the spectral processing. In an MEG study, the N1m source of AEFs elicited by an MF complex sound was found to be located near to the F0 as compared with the locations of individual spectral components of that sound (Pantev et al., 1989), even for dichotic stimulation (Pantev et al., 1996). Therefore, the N1m reflects the perceived pitch rather than the single spectral components of the sound. An additional ‘periodotopical’ cortical map, with orientation different from the tonotopic representa- tion of pure tones, was demonstrated by Langner et al. (1997). This result was interpreted to represent the independent perception of pitch (periodicity) and timbre (spectrum) as seen in an orthogonal source configuration in the inferior colliculus of cat (Langner & Schreiner, 1988; Schreiner & Langner, 1988). In addition to the spatial repre- sentation, the latency of the N1 response is thought to reflect the encoding process of MF sounds (Crottaz-Herbette & Ragot, 2000). Fig. 1. Schematic representation of the spectral components of the complex However, it is not possible simply to combine the results of all these stimuli. Each row represents seven stimuli corresponding to a common funda- mental frequency (F ¼ 250, 500 and 1000 Hz). These are three MF sounds in studies because of their limited sound parameter variations. For 0 three harmonic order conditions and the F0 and LC pure tones. The numbers example, it is still unclear whether the perceived pitch or the stimulus above the symbols indicate the harmonic order for each component of the MF periodicity is the most relevant parameter when a single MF sound is sounds and the LC tones. investigated. Moreover, psychophysically pitch and timbre appear to be partly dependent, but related to periodicity and spectrum, respectively. Stimuli Pitch and timbre depend strongly on combinations of the F0 The stimuli used in this study consisted of three groups of seven stimuli and its corresponding spectral components (Singh, 1987; Singh & (Fig. 1). Each group was characterized by the periodicity F0 (250, 500 Hirsh, 1992). Previous studies have shown that the timbre of stimuli or 1000 Hz). One stimulus was a at the F0 frequency. Three can change substantially by varying the F0, even though the fre- other stimuli were missing fundamental complex sounds consisting of quency bands of the sounds are kept constant (Langner et al., 1997; the 2nd, 3rd, 4th and 5th harmonics in the low harmonic order Crottaz-Herbette & Ragot, 2000). This is because the stimuli contain condition (MF-Low), the 6th to 9th harmonics in the middle harmonic a higher number of components for the lower F0.Inorderto order condition (MF-Mid), or the 10th to 13th harmonics in the high differentiate the effect of periodicity and spectral composition of harmonic order condition (MF-High). All spectral components of a complex sounds on the perception of pitch and timbre, the spectrum complex sound were of equal amplitude. This is in contrast to natural of higher harmonics should be considered in close correspondence sounds such as in speech or music, which show characteristic ampli- with the F0 instead of assuming independence of spectral condition tude patterns. In this study the effects of specific processing of natural and periodicity. sounds were avoided. The remaining three stimuli were pure tones The present study aims to explore whether the tonotopic gradient of with frequencies equal to the lowest components (LC) of the three MF the N1m response to MF sounds is more likely to reflect the periodicity sounds corresponding to 2F0 (LC-Low), 6F0 (LC-Mid) and 10F0 of the sound or its spectral contents. Furthermore, it investigates how (LC-High). the N1m sources are spatially mapped in relation to the periodicity (F0) The sounds were presented as tone-bursts of 70 ms duration includ- of the sound and the structure of its harmonic components. Results ing 10 ms rise and decay time at interstimulus intervals (ISIs) rando- from the current study also help us to understand better how the mized between 800 and 1000 ms. All tone-bursts were prepared as cortical maps in humans are related to the perception of pitch and sound files at a sampling rate of 44 100 Hz and were presented using a timbre of complex sounds. non-magnetic loudspeaker (Panasonic WM501, Yokohama, Japan) located at a distance of 135 cm in front of the subject. The speaker Materials and methods had a flat transfer characteristic in the frequency range from 50 to 10 000 Hz. The intensity of the sounds was set up at 60 dB SPL (sound Subjects pressure level) as measured at the subject’s external ear. Three Twelve right-handed volunteers (two females) between 27 and successive experimental sessions, each with a different F0, were 42 years of age participated in the study. None had any history of carried out. Therefore, in each session the seven different sound otological or neurological disorders. Informed consent was obtained stimuli for one periodic level were presented in random order. Each from the participants prior to the study, which was approved by the stimulus appeared 200 times, which resulted in a duration of about Ethical Committee in accordance with the Declaration of Helsinki. 23 min for each session.

ß 2003 Federation of European Neuroscience Societies, European Journal of Neuroscience, 18, 432–440 434 T. Fujioka et al.

MEG recordings Magnetic resonance imaging (MRI) overlay The MEG recordings were performed in a magnetically shielded MRI scans (Shimadzu 150 XT 1.5 T, Kyoto, Japan) were obtained for room using a dual 37-channel neuro-magnetometer (Magnes, 4D- all subjects. T1-weighted coronal, axial and sagittal images with Neuroimaging Inc., San Diego, CA, USA). The detection coils of the continuous 1.5-mm slice thickness were used for overlays with instrument were arranged as a uniformly distributed array in con- ECD sources detected by the MEG. The same fiducial points (nasion centric circles over a spherically concave surface 144 mm in dia- and pre-auricular points) that defined the MEG coordinate system were meter. The coils, with diameters of 20 mm and with a distance of visualized in the MRI images by affixing to these points high-contrast 22 mm between the centres of each, were configured as first-order cod liver oil capsules (3 mm diameter), whose short relaxation time gradiometers and connected to super-conducting quantum interfer- provides a high-intensity signal in T1-weighted images. The common ence devices (SQUIDs). The magnetic field signals were band-pass fiducial points allowed easy transformation of the MEG coordinates filtered between 0.1 and 100 Hz and digitized at a sampling rate of into the MRI device. 520.8 Hz. The two sensor arrays were centred above the C3 and C4 positions (International 10–20 system) as close as possible to the subject’s head. Thus, the left and right auditory cortices were Results optimally covered. Clear auditory evoked fields were obtained from all subjects for the F0, three MF sounds and three corresponding LC tones at each periodicity Data analysis (250, 500 and 1000 Hz). A single subject’s auditory evoked magnetic Stimulus-related signal epochs of 500 ms duration were selectively field waveforms for each sensor are depicted in Fig. 2 for the seven averaged according to the various types of stimuli. The DC offset has stimuli having an F0 of 250 Hz. The AEF components (P1m, N1m and been corrected with respect to the 100 ms prestimulus interval. Mag- P2m) were clearly identified for all stimulus types in this example. The netic source localization was performed in a head-based Cartesian most pronounced N1m component, with a latency of about 80–120 ms, coordinate system with an origin in the midpoint between the pre- was consistently observed in all subjects, whereas the other compo- auricular points. The x-axis connects the origin with the nasion, nents, P1m (40–70 ms) and P2m (120–350 ms), were more variable representing the anterior–posterior direction, with positive to the among subjects. nasion. The y-axis connecting the pre-auricular points represents the lateral–medial direction with positive values in the left hemisphere. N1m latency and amplitude The z-axis (superior–inferior direction) is defined as perpendicular to The group averages of the N1m peak latencies are shown in Fig. 3A for the x–y plane, and is positive to the vertex. all stimulus conditions and both hemispheres. For the ANOVA, latency The localization of the equivalent cortical source was based on a and amplitude values were available from all subjects in all stimulus moving single equivalent current dipole (ECD) model in a sphere. This conditions. The ANOVA results confirmed that all stimulus parameters was applied independently for both hemispheres to the data-points affected the N1m latency [periodicity: F2,22 ¼ 10.36, P ¼ 0.0007; around the N1m maximum of the global field power (root mean square harmonic order level: F2,22 ¼ 15.79, P < 0.0001; stimulus type value, RMS). The best sphere was locally fitted to the left and to the (MF/LC): F1,11 ¼ 11.748, P ¼ 0.0056]. However, there was no sig- right hemisphere of the digitized head-shape of each subject. The ECD nificant effect of the hemispheres. The ANOVA showed also an inter- parameters location (x, y and z coordinates), orientation and magnitude action between periodicity and stimulus type (F2,22 ¼ 9.52, P ¼ were determined as the best fit that maximally accounts for the 0.0011). Thus, the latency was not an independent function of a single measured magnetic field distribution. The source analysis was per- stimulus parameter. Tukey’s multiple comparisons revealed very small formed only on those magnetic field datasets showing iso-contour but still significant differences in latency. The N1m latency was 3.5 ms maps, which indicated a single dipolar source in each supratemporal shorter at F0 ¼ 1000 Hz compared with F0 ¼ 250 Hz (95% confidence plane. Dipole solutions were accepted for further analysis if the interval: 1.8 ms) and 2.47 1.5 ms shorter at 1000 Hz compared with correlation between the measured and the predicted fields was larger F0 ¼ 500 Hz with no significant difference between 250 and 500 Hz. than 90%. A further criterion for acceptance of the estimated source The latency was prolonged in the highest (4.2 1.6 ms) and mid was the stability of the dipole location, which should not vary by more harmonic order condition (3.4 1.4 ms) as compared with the low than 5 mm for the data-points around the N1m peak of the global field harmonic order condition, but no differences were found between the power. The latency and amplitude (RMS value in fT) of the N1m mid and high harmonic order condition. The N1m latency was response was measured at the maximum of the global field power. The 3.9 1.3 ms longer for the MF sounds compared with the LC tones. effects of the stimulus parameters and the hemisphere on the amplitude Figure 3B shows the group averages of the global field amplitude at and latency of the N1m response to the LC-tones and MF-sounds were the N1m peaks. The ANOVA revealed significant effects of the factors statistically analysed by a repeated-measure analysis of variance hemisphere and all stimulus parameters [periodicity: F2,22 ¼ 39.02, (ANOVA) with four within-subject factors: hemisphere (left and P < 0.0001; harmonic order level: F2,22 ¼ 46.14, P < 0.0001; stimulus right) stimulus type (MF, pure tone) periodicity (250, 500, type (MF/LC): F1,11 ¼ 15.65, P ¼ 0.0023; hemisphere: F1,11 ¼ 10.46, 1000 Hz) harmonic order (low, mid, high). Tukey’s multiple com- P ¼ 0.0079]. The ANOVA showed interactions between periodicity parisons of means were applied as a post-hoc test. Significance was level and all other factors [periodicity harmonic order level accepted at the level of 0.05 in all statistical analyses. (F4,44 ¼ 5.33, P ¼ 0.0014), periodicity stimulus type (F2,22 ¼ 8.55. In order to test the tonotopic organization of the N1m source P ¼ 0.0018) and periodicity hemisphere (F2,22 ¼ 3.58, P ¼ 0.0449)]. coordinates, a linear model was approximated to the characteristics There are also interactions between harmonic order level and of the source coordinates vs. the logarithm of the frequencies. An other factors [harmonic order level stimulus type (F2,22 ¼ 15.35, ANOVA was applied to verify that the slope of the regression lines were P < 0.0001), harmonic order level hemisphere (F2,22 ¼ 4.978, significantly different from a horizontal line. For this regression P ¼ 0.0165) and harmonic order level periodicity stimulus type analysis, the complex MF sounds were represented by the geometric (F4,44 ¼ 7.098, P ¼ 0.0002)]. The N1m amplitudes were significantly mean of the frequency components. smaller at F0 ¼ 1000 Hz (46.2 5.4 fT) compared with 250 Hz, at

ß 2003 Federation of European Neuroscience Societies, European Journal of Neuroscience, 18, 432–440 ß 03Fdrto fErpa ersineSocieties, Neuroscience European of Federation 2003 uoenJunlo Neuroscience of Journal European otclrpeetto fmsigfnaetlsud 435 sounds fundamental missing of representation Cortical , 18 432 , – 440

Fig. 2. Superimposition of the 37-channel auditory evoked magnetic field waveforms obtained above the left hemisphere of an individual subject. 436 T. Fujioka et al.

Tonotopic organization of the N1m source locations

The overlay of the results of the magnetic source locations onto MR images is shown in Fig. 4 for one subject (right hemisphere, source coordinates obtained for the seven stimuli related to F0 ¼ 250 Hz). Figure 4 demonstrates that the tonotopic organization can be observed in a single subject. It is also clear that the N1m sources are located within a small volume. In order to obtain the differences in location related to the various stimulus parameters, the regression lines were analysed as results of the group statistic. However, in some subjects it was not possible for all stimulus types to explain the magnetic field of N1m with a single current dipole model because of an insufficient signal-to-noise ratio. Therefore, for further statistical analysis we used the dipole parameters of those N1m responses for which a reliable dipole solution was obtained. Table 1 shows the number of subjects of the 12 in each condition for whom dipole estimation was successful. Figures 5 and 6 show the dependency of the N1m source locations for pure tones and MF sounds on the stimulus frequency. In Fig. 5 all individual source coordinates in anterior–posterior (x), medial–lateral (y) and inferior–posterior (z) direction are plotted vs. the pure tone frequencies or the lowest component of the MF sounds. Furthermore, the mean and its 95% confidence interval are shown for each frequency as well as regression lines approximated to all individual data. The Fig. 3. Group averages of (A) the latency and (B) the magnetic field amplitude slopes of these regression lines are measures of tonotopic organization. (RMS) of N1m responses of F0 tones (black), MF sounds (dark grey) and LC- tones (pale grey). The horizontal axis represents the three periodicities (250, An enlarged view of the trace of the tonotopic organization is given for 500, 1000 Hz), and the different harmonic order conditions (F0, Low, Mid, the y–x and y–z planes for both hemispheres in Fig. 6. Significant High) for both hemispheres. The error bars denote the standard errors of the tonotopic organizations were found for both pure tones and MF sounds means. in the medial–lateral direction (y-axis) in both hemispheres. The tonotopic gradients for this direction were 3.49 mm/decade (per 10-fold frequency increase) (F1,89 ¼ 10.58, P ¼ 0.0016) for the MF F0 ¼ 1000 Hz (31.5 6.2 fT) compared with 500 Hz, and smaller sounds and 1.76 mm/decade (F1,121 ¼ 5.08, P ¼ 0.026) for pure tones (14.7 5.4 fT) at F0 ¼ 500 Hz than at 250 Hz. The N1m amplitudes in the left hemisphere. The corresponding tonotopic gradients in the decreased with increasing harmonic order. The decrements were all right hemisphere were 2.68 mm/decade (F1,88 ¼ 8.25, P ¼ 0.005) for significant between order levels (20.1 5.8 fT from low to mid, MF sounds and 2.10 mm/decade (F1,120 ¼ 10.69, P ¼ 0.0014) for 29.2 6.2 fT from low to high, and 9.13 3.2 fT from mid to high). pure tones. The minus sign of the tonotopic gradients indicates more The N1m amplitudes were significantly larger in the right hemisphere medial source locations with increasing frequency, an effect that is (27.4 4.8 fT) compared with the left hemisphere, and also common for all four conditions. In addition, a systematic shift to more (10.1 4.2 fT) larger for the MF sounds than LC tones. superior source location with increasing frequency was found to be

Fig. 4. Example of an overlay of the N1m source coordinates on the individual MRI in the right hemisphere for the seven stimulus types having an F0 of 250 Hz. The F0 tone is indicated with a circle (* number 0). MF sounds are shown in the upper row with circles ( 1, 2 and 3) corresponding to their harmonics order (1: Low, 2: Mid, 3: High). LC tones are represented by square symbols (&1, 2 and 3) in the lower row.

ß 2003 Federation of European Neuroscience Societies, European Journal of Neuroscience, 18, 432–440 Cortical representation of missing fundamental sounds 437

Table 1. Numbers of successfully estimated dipoles under the different stimulus conditions

Left hemisphere Right hemisphere

MF sounds LC tones MF sound LC tones

F0 Low Mid High Low Mid HighF0 Low Mid High Low Mid High

250 12 12 12 11 12 9 11 12 12 12 9 12 10 10 500 12 11 11 11 12 11 10 12 12 11 11 12 12 11 1000 12 11 9 10 10 8 9 12 10 10 11 8 11 10

2.03 mm/decade (F1,108 ¼ 9.50, P ¼ 0.0026) for pure tones in the left tonotopic gradients for the MF sounds. The N1m sources for MF hemisphere. A tendency in the same direction was found for pure tones sounds with F0 ¼ 250 Hz are located in the immediate vicinity of the in the right hemisphere with a shift of 1.25 mm/decade (F1,120 ¼ 3.54, source found for the 250-Hz pure tone in the x–y plane. P ¼ 0.06). Only the right hemisphere showed a strong tonotopic gradient directing more posteriorly with increasing frequency for Discussion MF sounds [7.19 mm/decade (F1,92 ¼ 37.4, P < 0.00001)] and for pure tones [3.24 mm/decade (F1,125 ¼ 18.95, P ¼ 0.00003)]. In the y–z plane In this study of 12 normally hearing adults, reliable estimates of the [(medial–lateral)–(inferior–superior)] a similar representative schema amplitudes, latencies and source coordinates of the N1m wave of AEFs is observed for both hemispheres, with higher frequencies located in response to pure tones and ‘missing fundamental’ sounds under more medially. In both hemispheres MF sounds are located more various spectral and periodicity conditions were obtained. The N1m superiorly with increasing frequency, whereas pure tones tend to show sources were found closely spaced in a volume of less than 1 cm3, an organization in the opposite direction. In the x–y plane the MF which corresponds well with the small volume of the auditory cortex. sounds and pure tones show organization in the same direction. The discrimination between sources within such a small volume However, the tonotopic organization for the MF sounds extends over represents the limits of spatial resolution of the MEG. Nevertheless, a larger distance. Furthermore, Fig. 6 shows the mean N1m source the group statistics allowed observations of systematic changes in location for MF sounds with common F0. These source locations, location and spatial distribution related to the different stimulus marked with filled symbols in Fig. 6, are in general aligned along the conditions.

Fig. 5. Individual source coordinates vs. the logarithm of the stimulus frequency for pure tones and MF sounds in both hemispheres and regression lines fitted to the individual data.

ß 2003 Federation of European Neuroscience Societies, European Journal of Neuroscience, 18, 432–440 438 T. Fujioka et al.

spectral ranges, all spectral components of the stimuli were above 5000 Hz. In these cases, the activation of the auditory nerve is based mainly on the spectral pattern, in which each of the higher harmonic components is resolved in the filter bank at the peripheral level. Therefore, it is likely that the pitch perception in this case is strongly based on spectral encoding. Several studies using pure-tone stimuli showed the existence of tonotopic organization of the N1m source (Pantev et al., 1988, 1995; Yamamoto et al., 1992; Tiitinen et al., 1993; Verkindt et al., 1995). Results from these studies showed for stimuli in the frequency range from 250 to 4000 Hz a tendency for a more medial or posterior location within the secondary auditory cortex in both hemispheres. A recent study demonstrated that this tonotopic gradient seems to be preserved for frequencies above 4 kHz up to 14 kHz (Fujioka et al., 2002). If the tonotopic gradient of N1m source locations reflects merely the spectral stimulus patterns, the N1m sources for LC tones are expected more anteriorly or laterally located as compared with those for MF sounds. This is because each MF sound has a higher spectrum than the corresponding LC tone. Additionally, the location for LC tones might be closer to the F0 than that for MF sounds according to the tonotopic mapping, despite any periodic or spectral conditions. However, such an interpretation might be too simple according to the results described above, which showed that different tonotopic organizations were found for pure tones and complex sounds. Instead of assuming the same tonotopic representation for pure tones and complex sounds, it seems more reasonable to consider different representations of the sounds in the auditory cortex. Despite the different tonotopic representations for pure tones and complex sounds, both stimuli exhibit a common tonotopic organiza- tion in the medial–lateral direction with the sources related to higher Fig. 6. Projection of the tonotopic gradients of N1m source locations into the y–z plane (inferior–superior vs. medial–lateral direction) and the y–x plane frequencies located more medially. This corroborates the tonotopy (anterior–posterior vs. medial–lateral direction). The filled symbols represent found by Pantev et al. (1989). In their pioneering work, the source of the mean source location corresponding to the various F0. the N1m evoked by an MF sound (3rd to 6th harmonic of F0 ¼ 250 Hz) was located closer to the source of the N1m evoked by an F0-tone, in the medial–lateral direction, than to the source related to the mean Tonotopic organization of N1m source for MF sound spectral component of the MF sound. Because at that time only a single Similar to the results for pure tones, a tonotopic representation related channel magnetometer was available, the extensive measurement time to the corresponding mean frequency of the MF sound was preserved. did not allow for the investigation of multiple MF sounds. The N1m A tonotopy showing sources located more medially with increasing source organizations, in the present study, showed that the bilateral frequency was found consistently for both pure tones and complex MF sources of an MF sound with spectral components between 1000 and sounds. By contrast, the tonotopic organization in the inferior–superior 1750 Hz, have similar location in the medial–lateral direction as a 250- and anterior–posterior directions was significantly different between Hz pure tone. This is in complete agreement with the results of Pantev the stimulus types. These results are interpreted as evidence for et al. (1989). different representation of pure tones and complex sounds in terms Different spatial organizations of N1m sources related to the sound’s of tonotopic organization and indicate the existence of different spectrum or its periodicity have been reported by Langner et al. (1997), perceptual mechanisms for the pitch of MF sounds reflecting N1m who suggested a periodotopic organization orthogonal to the tonotopic responses. Additionally, for the MF sounds a significant tonotopic gradient. A common result of the present study is the different cortical gradient ordered by their F0 was found in the anterior–posterior representation of pure tones and complex sounds. However, it seems direction in the right hemisphere. This direction was parallel to the problematic to assume independent effects of periodicity and spectral tonotopic gradient for the pure tones and for the mean spectrum of the contents. As discussed above, the complexity of the results provides MF sound, suggesting that the tonotopic gradient found in the anterior– evidence to support not just one principle of cortical coding of the pitch posterior direction more probably reflects the spectral effect, which is of complex sounds, but also different principles, suggesting the overlaid on the periodicity effect rather than a real ‘periodotopic’ existence of multiple spatial representations of complex sounds. organization. There is an agreement in the results of various animal studies that In the 250-Hz mid and high harmonic order ranges, the spectral temporal encoding in the auditory pathway deteriorates in the central components are closely spaced in the frequency domain and single part of the auditory system (Creutzfeldt et al., 1980; Steinschneider components are not resolved in the auditory periphery. It seems most et al., 1980; Rees & Møller, 1983; Rees & Palmer, 1989; Langner, likely that in this case pitch processing is based on the temporal 1992; Bieser & Mu¨ller-Preuss, 1996). The limited frequency range of information of the 250-Hz rhythm. Correspondingly, the source of the phase-locked responses to the periodicity of sound signals does not N1m response was located closer to the source of the F0 tone than to the seem to be sufficient to explain pitch perception in general (Langner & tone corresponding to the spectrum of the MF sound. By contrast, in Schreiner, 1988; Langner, 1992). For example, the maximum modula- the 500-Hz high spectral range and in the 1000-Hz mid and high tion frequency eliciting synchronized responses in cats was found to be

ß 2003 Federation of European Neuroscience Societies, European Journal of Neuroscience, 18, 432–440 Cortical representation of missing fundamental sounds 439 around 800 Hz in the cochlear nucleus (Glattke, 1969), 700 Hz in the spatial distribution of MF sounds, suggesting a preference for proces- central nucleus of the inferior colliculus (Langner & Schreiner, 1988), sing of spectral information in the right hemisphere and of the temporal 300 Hz in the medial geniculate body (Rouiller et al., 1981) and 100 Hz information in the left hemisphere. in the auditory cortex (Schreiner & Urbas, 1988). Therefore, it is likely that stimulus periodicity is transformed into a place-coding schema in Acknowledgements the auditory cortex in conjunction with a synchrony code for lower temporal modulations. The source coordinates of the N1m responses We are grateful to Mr O. Nagata for technical help and Dr A. Herdman for obtained in our study provide evidence for such a place-code scheme, critical remarks on an earlier version of the manuscript. This study was supported by the Grant-in-Aid for Scientific Research (07458215 and suggesting a theory of multiple strategies of periodicity coding within 09558102), Grant-in-Aid for Scientific Research on Priority Areas the central auditory system. In other words, our results suggest that the (08279244) and Grant-in-Aid for Exploratory Research (08878160) from the sources of N1m in the secondary or associated auditory cortical areas Ministry of Education, Japan, the Joint Research Programmes from The reflect the process of extracting both the spectral and the periodic Graduate University for Advanced Studies, Japan, the International Foundation information. for Music Research and the Canadian Institutes of Health Research. More pronounced tonotopic representation was found in the right hemisphere. This was expressed by the larger tonotopic gradient in the Abbreviations anterior–posterior direction that was seen in the left hemisphere. This AEF, auditory evoked magnetic field; AEP, auditory evoked potential; ANOVA, finding may correlate with the right hemispheric preference for pitch analysis of variance; ECD, equivalent current dipole; F0, fundamental fre- processing of MF sounds, which has been reported in previous studies quency of periodic sounds; fT, femto Tesla, 1015 T; ISI, interstimulus interval; (Zatorre, 1988; Zatorre et al., 1992, 1994; Laguitton et al., 1998). LC, the lowest component of MF; MF, missing fundamental of complex sounds; MEG, magnetoencephalography; MRI, magnetic resonance imaging; N1, AEP N1m latency and amplitude at a latency of about 100 ms; N1m, AEF corresponding to N1 of AEP; nAm, nano Ampere metre, 109 Am; PET, positron emission tomography; RMS, root It is assumed that the mechanisms of pitch and timbre processing for mean square; SPL, sound pressure level; SQUID, super-conducting quantum MF sounds are based on both temporal and spectral information. interference device. Processing the timbre of a complex sound seems to be a more elaborate and more time-consuming task than the processing of pure tones in the References same spectral range. This is a possible explanation for the longer latencies obtained for MF sounds as compared with the corresponding Bertrand, O., Perrin, F. & Pernier, J. (1991) Evidence for a tonotopic organiza- LC tones. A similar latency prolongation is demonstrated in previous tion of the auditory cortex observed with auditory evoked potentials. Acta Otolaryngol. Suppl., 491, 116–122. studies for vowels and musical sounds compared with pure tones Bieser, A. & Mu¨ller-Preuss, P. (1996) Auditory responsive cortex in the squirrel (Zouridakis et al., 1998) and for synthetic vowels compared with the monkey: neural responses to amplitude-modulated sounds. Exp. Brain Res., tone of the lowest vowel component (Diesch & Luce, 1997). Our 108, 273–284. results demonstrate that the N1m latency for F and MF sounds Bilecen, D., Scheffler, K., Schmid, N., Tschopp, K. & Seelig, J. (1998) 0 Tonotopic organization of the human auditory cortex as detected by become shorter as the periodicity is increased. This is consistent BOLD-FMRI. Hear. Res., 126, 19–27. with previous results obtained with pure tones (Verkindt et al., Creutzfeldt, O., Hellweg, F.C. & Schreiner, C. (1980) Thalamocortical transform- 1995; Stufflebeam et al., 1998) and complex sounds (Ragot & ation of responses to complex auditory stimuli. Exp. Brain Res., 39, 87–104. Lepaul-Ercole, 1996; Langner et al., 1997; Tiitinen et al., 1999; Crottaz-Herbette, S. & Ragot, R. (2000) Perception of complex sounds: N1 Crottaz-Herbette & Ragot, 2000). The N1m latency in our study also latency codes pitch and topography codes spectra. Clin. Neurophysiol., 111, 1759–1766. shows a tendency to increase with higher order level of the harmonics, Diesch, E. & Luce, T. (1997) Magnetic fields elicited by tones and vowel at which level pitch recognition becomes more difficult (Ritsma, reveal tonotopy and nonlinear summation of cortical activation. 1967). This result is compatible with the interpretation that a delayed Psychophysiology, 34, 501–510. N1m reflects cognitive processing of the complexity of the sound Elberling, C., Bak, C., Kofoed, B., Lebech, J. & Særmark, K. (1982) Auditory magnetic fields from the human cerebral cortex: location and strength of an (Roberts et al., 1998; Simos et al., 1998). equivalent current dipole. Acta Neurol. Scand., 65, 553–569. However, our results are different from those examining the elec- Fujioka, T., Kakigi, R., Gunji, A. & Takeshima, Y. (2002) The auditory evoked trical N1 component in AEPs (Ragot & Lepaul-Ercole, 1996; Crottaz- magnetic fields to very high frequency tones. Neuroscience, 112, 367–381. Herbette & Ragot, 2000). These studies suggest that the N1 latency Glattke, T.J. (1969) Unit responses of the cat cochlear nucleus to amplitude- does not vary with the harmonic order level, but that it does vary with modulated stimuli. J. Acoust. Soc. Am., 45, 419–425. Godey, B., Schwartz, D., de Graaf, J.B., Chauvel, P. & Lie´geois-Chauvel, C. the difference in the periodicity. Another study using pure tones (2001) Neuromagnetic source localization of auditory evoked fields and (Roberts & Poeppel, 1996) showed that the N1m latency is shortest intracerebral evoked potentials: a comparison of data in the same patients. between 1000 and 2000 Hz and is elongated in the low and high range Clin. Neurophysiol., 112, 1850–1859. of audible frequencies. Their conclusion is that this behaviour resem- Goldstein, J.L. (1973) An optimum processor theory for the central formation of the pitch of complex tones. J. Acoust. Soc. Am., 54, 1496–1516. bles the curve of hearing sensation. We used MF sounds with compo- Houtsma, A.J.M. & Goldstein, J.L. (1972) The central origin of the pitch of nents mostly above 1000 Hz. Considering the significant decrease in complex tones: evidence from musical interval recognition. J. Acoust. Soc. N1m amplitude with increasing harmonic order level in our results, we Am., 51, 520–529. assume that this delay is mainly the consequence of decreased hearing Laguitton, V., Demany, L., Semal, C. & Lie´geois-Chauvel, C. (1998) Pitch sensation level at higher frequencies. From this point of view, we perception: a difference between right- and left-handed listeners. Neurop- sychologia, 36, 201–207. cannot confirm the suggestion of Ragot & Lepaul-Ercole (1996) and Langner, G. (1992) Periodicity coding in the auditory system. Hear. Res., 60, Crottaz-Herbette & Ragot (2000) that N1m latency reflects the per- 115–142. iodicity instead of the harmonic order level. Langner, G., Sams, M., Heil, P. & Schulze, H. (1997) Frequency and periodicity In summary, we propose that the tonotopic gradient of N1m source are represented in orthogonal maps in the human auditory cortex: evidence from magnetoencephalography. J. Comp. Physiol. [A], 181, 665–676. locations represents the perception of pitch and timbre information Langner, G. & Schreiner, C.E. (1988) Periodicity coding in the inferior rather than the spectral pattern or the periodicity of the complex sound. colliculus of the cat. I. Neuronal mechanisms. J. Neurophysiol., 60, Our results are also consistent with the hemispheric asymmetry in the 1799–1822.

ß 2003 Federation of European Neuroscience Societies, European Journal of Neuroscience, 18, 432–440 440 T. Fujioka et al.

Lauter, J.L., Herscovitch, P., Formby, C. & Raichle, M.E. (1985) Tonotopic Schreiner, C.E. & Urbas, J.V. (1988) Representation of amplitude modulation in organization in human auditory cortex revealed by positron emission tomo- the auditory cortex of the cat. II. Comparison between cortical fields. Hear. graphy. Hear. Res., 20, 199–205. Res., 32, 49–63. Lockwood, A.H., Salvi, R.J., Lou Coad, M., Arnold, S.A., Wack, D.S., Murphy, Simos, P.G., Breier, J.I., Zouridakis, G. & Papanicolaou, A.C. (1998) MEG B.W. & Burkhard, R.F. (1999) The functional anatomy of the normal human correlates of categorical-like temporal cue perception in humans. Neurore- auditory system: response to 0.5 and 4.0 kHZ tones at varied intensities. port, 9, 2475–2479. Cerbral Cortex, 9, 65–76. Singh, P.G. (1987) Perceptual organization of complex-tone sequences: a Pantev, C., Bertrand, O., Eulitz, C., Verkindt, C., Hampson, S., Schuierer, G. & tradeoff between pitch and timbre? J. Acoust. Soc. Am., 82, 886–899. Elbert, T. (1995) Specific tonotopic organizations of different areas of the Singh, P.G. & Hirsh, I.J. (1992) Influence of spectral locus and F0 changes human auditory cortex revealed by simultaneous magnetic and electric on the pitch and timbre of complex tones. J. Acoust. Soc. Am., 92, recordings. Electroencephalogr. Clin. Neurophysiol., 94, 26–40. 2650–2661. Pantev, C., Elbert, T., Ross, B., Eulitz, C. & Terhardt, E. (1996) Binaural fusion Steinschneider, M., Arezzo, J. & Vaughan, H.G. Jr (1980) Phase-locked cortical and the representation of virtual pitch in the human auditory cortex. Hear. responses to a human speech sound and low- frequency tones in the monkey. Res., 100, 164–170. Brain Res., 198, 75–84. Pantev, C., Hoke, M., Lehnertz, K., Lu¨tkenho¨ner, B., Anogianakis, G. & Stufflebeam, S.M., Poeppel, D., Rowley, H.A. & Roberts, T.P. (1998) Peri- Wittkowski, W. (1988) Tonotopic organization of the human auditory cortex threshold encoding of stimulus frequency and intensity in the M100 latency. revealed by transient auditory evoked magnetic fields. Electroencephalogr. Neuroreport, 9, 91–94. Clin. Neurophysiol., 69, 160–170. Talavage, T.M., Ledden, P.J., Benson, R.R., Rosen, B.R. & Melcher, J.R. (2000) Pantev, C., Hoke, M., Lu¨tkenho¨ner, B. & Lehnertz, K. (1989) Tonotopic Frequency-dependent responses exhibited by multiple regions in human organization of the auditory cortex: pitch versus frequency representation. auditory cortex. Hear. Res., 150, 225–244. Science, 246, 486–488. Terhardt, E. (1974) Pitch, consonance, and harmony. J. Acoust. Soc. Am., 55, Patterson, R.D. (1969) Noise masking of a change in residue pitch. J. Acoust. 1061–1069. Soc. Am., 45, 1520–1524. Tiitinen, H., Alho, K., Huotilainen, M., Ilmoniemi, R.J., Simola, J. & Na¨a¨ta¨nen, Ragot, R. & Lepaul-Ercole, R. (1996) Brain potentials as objective indexes of R. (1993) Tonotopic auditory cortex and the magnetoencephalographic auditory pitch extraction from harmonics. Neuroreport, 7, 905–909. (MEG) equivalent of the mismatch negativity. Psychophysiology, 30, Rees, A. & Møller, A.R. (1983) Responses of neurons in the inferior colliculus 537–540. of the rat to AM and FM tones. Hear. Res., 10, 301–330. Tiitinen, H., Sivonen, P., Alku, P., Virtanen, J. & Na¨a¨ta¨nen, R. (1999) Electro- Rees, A. & Palmer, A.R. (1989) Neuronal responses to amplitude-modulated magnetic recordings reveal latency differences in speech and tone processing and pure-tone stimuli in the guinea pig inferior colliculus, and their mod- in humans. Brain Res. Cogn. Brain Res., 8, 355–363. ification by broadband noise. J. Acoust. Soc. Am., 85, 1978–1994. Verkindt, C., Bertrand, O., Perrin, F., Echallier, J.F. & Pernier, J. (1995) Ritsma, R.J. (1967) Frequencies dominant in the perception of the pitch of Tonotopic organization of the human auditory cortex: N100 topography complex sounds. J. Acoust. Soc. Am., 42, 191–198. and multiple dipole model analysis. Electroencephalogr. Clin. Neurophysiol., Roberts, T.P., Ferrari, P. & Poeppel, D. (1998) Latency of evoked neuromag- 96, 143–156. netic M100 reflects perceptual and acoustic stimulus attributes. Neuroreport, Wessinger, C.M., Buonosore, M.H., Kussmaul, C.L. & Mangun, G.R. (1997) 9, 3265–3269. Tonotopy in human auditory cortex examined with functional magnetic Roberts, T.P. & Poeppel, D. (1996) Latency of auditory evoked M100 as a resonance imaging. Human Brain Mapping, 5, 18–25. function of tone frequency. Neuroreport, 7, 1138–1140. Wightman, F.L. (1973) The pattern-transformation model of pitch. J. Acoust. Romani, G.L., Williamson, S.J. & Kaufman, L. (1982) Tonotopic organization Soc. Am., 54, 407–416. of the human auditory cortex. Science, 216, 1339–1340. Yamamoto, T., Uemura, T. & Llina´s, R. (1992) Tonotopic organization of Rose, J.E., Brugge, J.F., Anderson, D.J. & Hind, J.E. (1967) Phase-locked human auditory cortex revealed by multi-channel SQUID system. Acta response to low-frequency tones in single auditory nerve fibers of the squirrel Otolaryngol., 112, 201–204. monkey. J. Neurophysiol., 30, 769–793. Zatorre, R.J. (1988) Pitch perception of complex tones and human temporal- Rouiller, E., de Ribaupierre, Y., Toros-Morel, A. & de Ribaupierre, F. (1981) lobe function. J. Acoust. Soc. Am., 84, 566–572. Neural coding of repetitive clicks in the medial geniculate body of cat. Hear. Zatorre, R.J., Evans, A.C. & Meyer, E. (1994) Neural mechanisms under- Res., 5, 81–100. lying melodic perception and memory for pitch. J. Neurosci., 14, Schouten, J.F. (1940) The residue, a new component in subjective sound 1908–1919. analysis. Proc. Kon. Ned. Akad. Wetensch., 43, 356–365. Zatorre, R.J., Evans, A.C., Meyer, E. & Gjedde, A. (1992) Lateralization Schouten, J.F., Ritsma, R.J. & Cardozo, B.L. (1962) Pitch of the residue. of phonetic and pitch discrimination in speech processing. Science, 256, J. Acoust. Soc. Am., 34, 1418–1424. 846–849. Schreiner, C.E. & Langner, G. (1988) Periodicity coding in the inferior Zouridakis, G., Simos, P.G. & Papanicolaou, A.C. (1998) Multiple bilaterally colliculus of the cat. II. Topographical organization. J. Neurophysiol., 60, asymmetric cortical sources account for the auditory N1m component. 1823–1840. Brain Topogr., 10, 183–189.

ß 2003 Federation of European Neuroscience Societies, European Journal of Neuroscience, 18, 432–440