Perception & Psychophysics 2003, 65 (4), 602-612 Auditory imagery from musical notation in expert musicians WARREN BRODSKY and AVISHAI HENIK Ben-Gurion University of the Negev, Beer-Sheva, Israel BAT-SHEVA RUBINSTEIN Tel Aviv University, Tel Aviv, Israel and MOSHE ZORMAN Levinsky College of Education, Tel Aviv, Israel Anecdotal evidence has suggested that musical notation can trigger auditory images. Expert musi- cians silently read scores containing well-known themes embedded into the notation of an embellished phrase and judged if a tune heard aloud thereafter was the original theme (i.e., melodic target) or not (i.e., melodic lure). Three experiments were conducted employing four score-reading conditions: nor- mal nondistracted reading, concurrent rhythmic distraction, phonatory interference, and obstruction by auditory stimuli. The findings demonstrate that phonatory interferenceimpaired recognition of orig- inal themes more than did the other conditions. We propose that notational audiation is the silent reading of musical notation resulting in auditory imagery. The research suggests that it also elicits kinesthetic-like phonatory processes. The focus of empirical investigation of imagery has that there exists a single seat or locus of auditory-based been on the visual modality. However, several recent ini- imagery. tiatives have concerned auditory imagery (for extensive A special case of auditory imagery is musical imagery. reviews, see Godoy & Jorgensen, 2001; Reisberg, 1992). In a host of experiments, Hubbard and Stoeckig (1988, In the form of inner speech,auditory images serve an im- 1992), Halpern (1988, 1989, 1992, 2001), Halpern and portant function of human cognitionin tasks such as ver- Zatorre (1996), and Zatorre (1994) had subjectscreate an bal memory (rehearsal), speech perception, silent read- image of raising a tone or chord a full step, set metronome ing, and thought processes. This type of imagery, also beats to imagined music, tap out the tempo of imagined referred to as subvocalization or inner ear, is the experi- songs, imagine a song’s starting tone, judge the similar- ence of an inner voice without vocal output or environ- ity between tones heard externally and those heard inter- mental input (Intons-Peterson, 1992; Smith, Reisberg, & nally, and mentally scan a melody for pitch and duration Wilson, 1992). The seemingly auditory quality of inner characteristics of heard stimuli. Collectively, these re- speech is not necessarily auditory in origin, and it should searchers found that musical images are generated in real not be attributed exclusively to events or systems that are time, encode fairly precise information about tempo and strictly auditory in nature. For example, Baddeley and pitch, and contain information concerning melodic and Logie (1992) found that echoic memory (i.e., temporary harmonic relationships.Hence, musical images possess a storage of auditory material) operates only in the pres- sensory quality that is similar to the experience of per- ence of auditory stimuli and, hence, cannot be the seat of ceiving.Moreover, EEG studies (Beisteiner, Altenmuller, auditory imagery. Additionally, Mackay (1992) found Lang, Lindinger, & Decke, 1994), PET studies (Halpern that inner speech is linked to the phonological system. & Zatorre, 1999; Zatorre & Halpern, 1993; Zatorre, Such findings may, then, negate the common assumption Halpern, Perry, Meyer, & Evans, 1996), and event-related potential(ERP) studies (Janata, 2001a,2001b) have shown that imaginal and perceptual processes for melodies share The first author wishes to acknowledge the support of the Council for neural structures and topographies. But, for the majority Higher Education (VaTat Committee), which granted the postdoctoral research fellowship under whose auspices this research took place. This of musically naive (untrained) people, music may still be research was supported in part by the Israel Science Foundation, a predominantly external experience. For example, mu- founded by the Israel Academy of Sciences and Humanities. The au- sically naive individuals have been found to be signifi- thors are indebted to Bruno Repp and two anonymousreviewers for their cantly worse than highly trained individuals on all audi- comments on two earlier versions of the manuscript. Correspondence should be addressed to W. Brodsky, Music Science Research, Depart- tory imagery tasks (involving both music and everyday ment of the Arts, Ben-Gurion University of the Negev, P. O. Box 653, sounds), but not on visual imagery tasks (Aleman, Beer-Sheva 84105, Israel (e-mail: [email protected]). Nieuwenstein, Bocker, & de Haan, 2000). The ability of Copyright 2003 Psychonomic Society, Inc. 602 AUDITORY IMAGERY FROM MUSICAL NOTATION 603 musicians to experience musical images, then, may be of peoplewho claim to be able to read a score in complete the outstanding mark of a musical mind. Musical images silence, without mediation of instrument or voice. How- have been viewed as a necessary condition for learning, ever, I choose not to make anything of this example here. retention,recall, recognition,and the anticipation of mu- There are several reasons for this. One is that there is no sical events (Seashore, 1938). obvious way of assessing the claim. Another is that The research literature is unequivocal in pointing to among the population of fluent music readers those who can read silently are statistically rare, and so third, they the underlying cognitive system, which accounts for the may be employingprocessesof silent reading that are not maintenance and processing of auditory representations normally employed in reading for performance(Sloboda, as expressed in musical images (for a review, see 1984, p. 224). Kalakoski, 2001). One question often raised is the fol- lowing: To what extent do musical imagery and the But musicians have always given credence to the pres- phonological working memory overlap? This is espe- ence of imagery from musical notation. Long ago, the cially interesting because vocal music has many acous- Romantic composer Schumann (1848/1967) declared tic features in common with vocal and subvocal speech. that his students must get to the point where they could Given that interference of imagery has been found to be hear the music from the page and be able to picture the most effective when both the image and the distracting piece at first hearing as though the score were in front of stimuli originatefrom the same sensory modality(Intons- them. Peterson, 1992), and given that auditory imagery has To date, there is virtually no empirical evidence to been blocked most effectively by auditory input (Smith support the notion of notational audiation. Most unfor- et al., 1992), what, then, is to be expected in the case of tunately, the two major research compilations on audi- musical imagery? Will musical imagery be affected most tory imagery (Reisberg, 1992) and musical imagery by the presentationof irrelevant musical, nonmusicalau- (Godoy & Jorgensen, 2001) do not include so much as a ditory, or verbal material? The initial studies linked both research note on the subject.There is one music cognition auditory and musical imagery to the crisscrossing of study (Experiment 6 in Waters, Townsend, & Underwood, aural and oral channels (Reisberg, 1992), and the trig- 1998) that shows that skilled musicians can successfully gering of musical images was also found to be linked to match one bar of musical notationto a subsequentlypre- motor imagery (Mikumo, 1994; Petsche, von Stein, & sented auditory sequence in a same–different matching Filz, 1996). More recently, PET studies (Halpern, 2001; task. However, although the task is referred to as con- Halpern & Zatorre, 1999) have found that the supple- verting visual representations of music to auditory rep- mentary motor area (SMA) is activated during musical resentations,the study does little to demonstrate that task imagery, and most specifically during covert mental re- performance is based on evoked musical imagery (rather hearsal (Langheim, Callicott, Mattay, Duyn, & Wein- than on structural harmonic analysis or visual surface berger, 2002). Accordingly, the SMA may mediate re- cues found in the notation). In addition, an fMRI study hearsal that involves motor processes such as humming. (Nakada, Fujii, Suzuki, & Kwee, 1998) and a magneto- Furthermore, the role of the SMA during imagery of fa- encephalography(MEG) study (Schurmann, Raij, Fujiki, miliar melodies has been found to include both auditory & Hari, 2002) clearly show audiocortical activity while components (of hearing the actual song) and carrier musical notation is read. However, these studies do little components (such as an image of subvocalizing, fingers to reveal the cognitive process underlying this phenom- moving on a keyboard, or someone else performing; enon. Therefore, as far as cognitivescience is concerned, Schneider & Godoy, 2001; Zatorre et al., 1996). it appears that auditory imagery as a response to musical For some individuals, musical imagery can also be notation continues to remain essentially conjectural in triggered by reading the graphic representation of music nature. (i.e., musical notation).Following Gordon (1975, 1993), The Present Study this phenomenon will be referred to here as notational The purpose of the present investigation was to de- audiation. Yet there are musicians who possess high- velop an experimentalparadigm to investigatenotational level skills without knowledge of how to read or write