From the to the Perpetual Simple Melody Auditory

Pedro Patrício∗

Portuguese Catholic University - School of Arts (EA-UCP) / Research Center for Science and Technology in the Arts (CITAR), Rua Diogo Botelho 1327, 4169-005 Porto-Portugal [email protected] http://pp2007.pt.googlepages.com

Abstract. This paper discusses the use of Shepard Tones (ST) as digital sound sources in digital music composition. These tones have two musical interests. First, they underline the difference between the tone height and the tone chroma, opening new possibilities in sound generation. And second, considering the fact that they are (in a paradoxical way) locally directional while still globally stable and circumscribed allows us to look differently at the instrument's range and the phrasing in musical composition. Thus, this paper proposes a method of generating ST relying upon an alternative spectral envelope based on the 40 Phon equal-loudness curve of Fletcher & Munson (1933), which as far as we know, has never been used before for the reproduction of the Shepard Scale Illusion (SSI). Using the proposed digital sound source, it was possible to successfully reproduce the SSI in the scope of several music exercises, which include a simple melody. Moreover, we composed a digital musical composition, “Perpetual Simple Melody – contrasting moments”, using the digital sound source as a sound generator and the simple melody as musical content.

Keywords: Shepard Scale Illusion, Shepard Tones, Spectral envelope, 40 Phon equal-loudness curve, A-Weighting curve, Digital music composition.

1 Introduction

Shepard [1] built an auditory illusion widely known as the Shepard Scale Illusion (SSI) through digital sound synthesis. The SSI could generate the auditory illusion that rises perpetually along a spiral as is the case with the oblique lines of a Barber Pole in motion [2], [3] or with the visual illusion of Escher’s “staircase to heaven” [1], [4].

∗ This work is developed within the framework of a doctorate grant provided by Fundação para a Ciência e Tecnologia/Ministério da Ciência, Tecnologia e Ensino Superior (Portugal). From the Shepard Tone to the Perpetual Simple Melody Auditory Illusion

In general terms, Shepard conceived the auditory illusion by annulling the vertical or rectilinear relationship of the sounds through the creation of the Shepard Tone [3]. The Shepard Tone (ST) is a complex sound consisting by ten components that are separated by octave intervals and in which a spectral envelope for the amplitudes with the shape of a bell or Gaussian Curve was applied. For complex sounds that are separated by octave intervals, through digital sound synthesis techniques by computer, it is possible to isolate perception attributes related to the height of the sound (i.e. pitch), which are commonly correlated. These attributes are the tone height and the tone chroma [3], [5]. In relation to pitch, the tone height is its rectilinear dimension while the tone chroma is its circular dimension [1], [5], [6], [7], [8]. Thus, the tone height of the sounds is eliminated through the ST and the position of the sound within the octave (the tone chroma) is simultaneously preserved [3]. It will probably be easy to identify the musical note of the sound (the tone chroma) but it will be very difficult to identify the octave that the sound belongs to (the tone height), [4]. In the SSI, each complex sound has a fixed position and always represents the same sound along any octave, i.e. a C represents all Cs that are in a guitar. Namely, the tone chroma is always the same for all C, D, E, etc. [5]. The circular dimention of the ST allows representing them visually as equally spaced points around a circle [1], (see Fig. 1).

Fig. 1. The figure represents the tone chroma of the ST in the SSI. The original SSI is a chromatic scale of D# divided into 12 parts.

In the course of our process for the composition of digital music, two questions have emerged. Firstly, how could one use, both the ST as a digital sound source and the SSI as musical content for composing digital music? And secondly, what kind of an alternative spectral envelope could one use to reproduce the SSI successfully, as far as we know, has never been used before? From the Shepard Tone to the Perpetual Simple Melody Auditory Illusion

We decided to design a digital sound source based on the ST and compose a digital music using the SSI reproduced through a simple melody as musical content so as to respond to the aforementioned questions. In the sections which follow, related work with spectral envelopes used in works that revisited the SSI or the ST and a detailed section about the construction of the digital sound source will be presented, followed by the sections of the evaluation and conclusion of this proposal.

2 Related Work

To better structure this section, two types of spectral envelopes sets have been created according to the following criteria: one set for the authors who used Fixed Spectral Envelopes (FSE) and another set for the authors who used Adjustable Spectral Envelopes (ASE). In the FSE, the individual amplitudes of the components are always the same for all complex sounds, while in the ASE, the individual amplitudes vary along the complex sounds. However, when a cycle of the scale divided into 12 parts is completed, the individual amplitudes always return to their original values [8].

2.1 Fixed Spectral Envelopes (FSE)

The following authors used FSE: Pollack [9] used a triangular spectral envelope with fixed amplitudes, logarithmically weighted. The individual amplitudes are always the same for the central components (five and six), while the amplitudes of the components either below or above the central components decrease logarithmically and progressively in both directions (see Fig. 2).

Fig. 2. The figure depicts Pollack’s spectral envelope for the individual amplitudes. The vertical lines symbolize the ten components of the complex sound separated by octave intervals, while the oblique lines represent the spectral envelope. The image was extracted and adapted from Pollack [9].

Burns [6] presented a triangular spectral envelope very similar to the shape of the spectral envelope used by Pollack [9], in which the individual amplitudes are From the Shepard Tone to the Perpetual Simple Melody Auditory Illusion weighted in the following manner: minus 6 dB per octave from the central component (five) in both directions, (see Fig. 3).

Fig. 3. The figure depicts Burns' spectral envelope for the individual amplitudes of a complex sound. The dotted line represents the amplitude envelope. The image was extracted and adapted from Burns [6].

In Nakajima et al. [10] the four central components have the same amplitude, while toward both ends of the spectrum, the sound pressure levels are attenuated by leaps of minus 20 dB per octave (see Fig. 4).

Fig. 4. The figure depicts Nakajima’s spectral envelope for the individual amplitudes of a complex sound. The dashed lines represent the amplitude envelope. The image was extracted from Nakajima et al. [10].

2.2 Adjustable Spectral Envelopes (ASE) Authors such as Shepard [1], Risset [5], Ueda & Ohgushi [8], (see Fig. 5) or Deustch [11], (see Fig. 6) used a spectral bell-shaped envelope (Gaussian Curve). This kind of shape of spectral envelope allows a smooth transition among the components and simultaneously highlights the central components while making the components of the extremities become inaudible [9]. From the Shepard Tone to the Perpetual Simple Melody Auditory Illusion

Fig. 5. The figure depicts Shepard’s spectral envelope for the individual amplitudes. The vertical lines represent the ten components of the complex sound separated by octave intervals. The image was extracted and adapted from Shepard [1].

Fig. 6. The figure depicts Deutsch’s spectral envelope for the individual amplitudes. The vertical lines represents six components of the complex sound separated by octave intervals. The image was extracted and adapted from Deutsch [11].

Of all previously mentioned authors, Risset (as composer) used the original SSI in the second movement (Fall) of his digital music called “Computer Suite for Little Boy” (1968). The SSI could be heard between minute 1.15 and 2.20. One of the purposes of this article is not to use the original SSI in digital music compositions, but to go a step further and compose with the SSI through the digital sound source that is being proposed. Furthermore, to successfully reproduce the SSI, an alternative spectral envelope for the individual amplitudes that, as far as we know, has never been used before, is also being proposed in this work. The construction of the digital sound source will be discussed in detail in the following sections. From the Shepard Tone to the Perpetual Simple Melody Auditory Illusion

3 Digital Sound Source

The digital sound source proposed in this work will be called “Our Shepard Sound” (OSS) as a tribute to (1964). In this work, only digital sound synthesis was used to construct OSS.1 OSS consists of nine components separated by octave intervals, in which an alternative spectral envelope for the amplitudes was applied. In this section, the SSI will be reconstructed and reproduced through OSS, with the purpose of verifying whether the SSI will be successfully reproduced. Primarily, the individual frequencies of the OSS will be automatically determined followed by the corresponding individual amplitudes.

3.1 Automatic Determination of the OSS individual frequencies

The first Shepard’s equation [1] was implemented in Pure Data (PD) so as to automatically determine the heights of the OSS.

f (t, c) = fmin * 2^[(c – 1) * tmax + t – 1] / tmax . (1)

Where:

• fmin = 32.72-Hz, represents the frequency of the first component (fundamental frequency) of the SSI’s first complex sound;2 • tmax = 12, represents the number of the scale’s complex sounds; • cmax = 9, represents the number of each complex sound’s components.3

At the end of this procedure, the chromatic scale of C, divided into 12 parts, was obtained.4 The process of automatic determination of the individual amplitudes of the OSS will be described in detail in the following section.

1 A MacBook Intel Core 2 Duo of 2-GHz, an Operating System Mac OS X (version 10.4.11) and the Pure Data (PD) software (version 0.39.3 – extended) were utilised in the construction of OSS. More information about PD at http://puredata.info/ 2 In this work, the original value of fmin (4.863-Hz) of the Shepard’s equation [1] was replaced by 32.72-Hz with the purpose of moving the spectrum to the right and thus, obtain a more brilliant timbre and musically more interesting than the original ST. 3 Only nine components were used in this work, because after the ninth component and from the fourth complex sound, the individual frequencies overcame the relative maximum limit of the human capacity of a young person, which is around 20-kHz [2], [12], [13]. 4 The table that contains the individual frequencies of the OSS is in the Appendix section. From the Shepard Tone to the Perpetual Simple Melody Auditory Illusion

3.2 Automatic Determination of OSS Amplitudes

A filter used in the measurement of the intensity of noise, called A-Weighting curve or Filter A, was extracted from the inversion of 40 Phon equal-loudness curve of Fletcher & Munson (1933), [2], (see Fig. 7).

Fig. 7. The figure depicts the A-Weighting curve or Filter A, based on the inversion of the 40 Phon equal-loudness curve of Fletcher & Munson (1933). The image was extracted and adapted from Wong [14].

The A-Weighting curve simulates the reaction of the human ear throughout the equal-loudness curve of 40 Phon of Fletcher & Munson (1933), [14].

Wong [14] presents a mathematic equation for determining the amplitudes along the A-Weighting curve corresponding to the frequency introduced in the equation 2. Namely, different frequencies whose amplitudes lie along the A-Weighting curve sound equally loud.

Wong’s equation is:

WA (f) = 20 log [f4^2 * f^4 / (f^2 + f1^2) (f^2 + f2^2)^1/2 * (f^2 + (2) f3^2)^1/2 (f^2 + f4^2)] – WA1000 .

Where: • WA1000 = -2 dB, represents the constant of normalisation; • f = represents the variable in Hz; • f1 = 20.6-Hz; • f2 = 107.7-Hz; • f3 = 737.9-Hz; • f4 = 12194-Hz.

From the Shepard Tone to the Perpetual Simple Melody Auditory Illusion

To automatically extract the individual amplitudes of OSS, the equation 2 was implemented in PD.5 Figure 8 depicts the image of the first alternative spectral envelope obtained for the first complex sound of OSS.

Fig. 8. For musical purposes, the initial values obtained, which ranged between zero and two were converted in Root mean squares (rms), namely values that range between zero and one.

From now on, this spectral envelope will be called Alternative Self-Generative Spectral Envelope (ASGSE). It is self-generative, in the that allows the preservation of equal-loudness perception for any frequency entered in OSS in real- time.

The following section will be presented so as to test if OSS is able to successfully reproduce the SSI, even when applied to musical exercises.

4 Musical Exercises Applied to OSS

OSS was applied to the chromatic scale of C divided into 12 equal parts. The following have been chosen from the total amount of exercises: • Melodic progression of a descendant major third interval by ascendant chromatic half-tone (E-C, F-C#, F#-D and so on until D#-B); • Melodic progression of a descendant major triad by descendant chromatic half- tone (G-E-C, F#-D#-B, F-D-A# and so on until G#-F-C#); • Melodic progression of a descendant arpeggio of a major chord with major seventh by descendant chromatic half-tone (B-G-E-C, A#-F#-D#-B, A-F-D-A# and so on until C-G#-F-C#); • Harmonic progression of a major chord with major seventh by ascendant chromatic half-tone (C E G B, C# F G# C, D F# A C# and so on until B D# F# A#).

5 The table obtained from the individual amplitudes is annexed in the Appendix section From the Shepard Tone to the Perpetual Simple Melody Auditory Illusion

• A simple melody. For a better view of the simple melody, its pentagram is presented in the following figure.

Fig. 9. The simple melody is constituted by ten musical notes spread along two measures that are repeated 12 times. The melody rises one semitone every two measures.

An interface for listening to the reproduction of the SSI through OSS and for listening to the musical exercises applied was designed in PD.6

5 Digital Music Composition

The digital music composition was called “Perpetual Simple Melody - contrasting moments”.7 Apart from the aesthetic aspects, the musical work also had the function of testing the musicality of OSS. It was integrally composed and sequenced in PD. Processes of digital sound synthesis, OSS as sound source and the simple melody as musical content were used exclusively in the composition. On the one hand, to further explore the timbre of the musical work and, on the other hand, to produce sonic segregation between the musical voices, we used vibrato produced through Frequency Modulation (FM), [15]. Namely, we modulated an OSS with another OSS. As musical gestures, transpositions applied to the perception of the height of sound (pitch), envelopes applied to the perception of the intensity of sound (loudness), envelopes of tempo and envelopes of amplitude applied specifically to the attacks and decays were mainly used. Additionally, envelopes applied to the main FM

6 The material programmed in PD can be downloaded at https://sites.google.com/site/pp2007pt/uk/auditory-illusion 7 The PD patch for playing the “Perpetual Simple Melody - contrasting moments” can be downloaded at https://sites.google.com/site/pp2007pt/uk/auditory-illusion

From the Shepard Tone to the Perpetual Simple Melody Auditory Illusion parameters as the carrier frequency, the modulator frequency and the modulation index were also used.

6 Conclusions

The initial challenges of applying an alternative spectral envelope to the ST which, as far as we know, had never been used before for reproducing the SSI and compose digital music using exclusively ST (i.e. OSS) as digital sound source and the SSI as musical content were widely overcome. OSS has successfully reproduced the SSI, even when applied to several musical exercises, including the simple melody. The SSI was always perceived in all exercises. The simple melody will be called "Perpetual Simple Melody Illusion" because when it is heard it can create the auditory illusion that it never ends, as is the case with the SSI. Through the digital composed music “Perpetual Simple Melody - contrasting moments”, OSS has proven that it could be a digital sound source with an interesting timbre and be musically flexible: it works either with slow or fast rhythms, as well as with short or long attacks in amplitude envelopes for high, middle and low sound registers. For these reasons, OSS designed and presented in this work could be a fruitfully digital sound source for composing digital music. Furthermore, OSS has the particularity, as does the ST, of being ambiguous in height [3]. One could probably say what OSS’ musical note is, but it would be very difficult to say which octave it belongs to. Due to the ASGSE used in this approach, one could play or introduce any fundamental frequency in OSS, and theoretically an equal-loudness perception would be perceived. It was mentioned theoretically, in the sense that any equal-loudness curve must be considered as a general indicator and not as a prescription of individual capability of one person’s hearing. Thus, the feature mentioned in the ASGSE has an advantage in relation to the previous set of spectral envelopes due to the fact that it could be used during the composition processes and performance of digital music in real-time. If the 40 Phon equal-loudness curve of Fletcher & Munson (1933) was inverted, and if compared to either of the A-Weighting curve (Filter A) or with the ASGSE, one could verify that all three envelopes are similar in terms of shape. Applying OSS to other digital sound synthesis techniques, such as Ring Modulation, with the purpose of finding new timbres and gathering a new sound bank for composing new digital music will be the future direction of this work. From the Shepard Tone to the Perpetual Simple Melody Auditory Illusion

Acknowledgments

We would like to thank Eric Oña, António Ferreira, Helena Mena Matos and especially to Luis Gustavo Martins and António de Sousa Dias for their suggestions and critical comments.

References

1. Shepard, R. N.: Circularity in Judgments of . JASA 36 (12), 2346–2353 (1964) 2. Loy, G.: Musimathics 1: The Mathematical Foundations of Music. Cambridge University Press, Cambridge, UK (2006) 3. Shepard, R.: Pitch Perception and Measurement. Cognition, and Computerized Sound: An Introduction to . In: P. R., Cook (eds.), pp. 261–275. Cambridge University Press, Cambridge, UK (1999) 4. Fugiel, B.: Waveform Circularity From Added Sawtooth And Square Wave Acoustical Signals. 28 (4), 415-423 (2011) 5. Risset, J. C.: Paradoxes de Hauteur. Communication au Symposium IRCAM sur la psychoacoustique musicale. University Lyon II, France, 1–6 (1977) 6. Burns, M. E.: Circularity in relative pitch judgments for inharmonic complex tones: The Shepard demonstration revisited, again. Perception & Psychophysics 30 (5), 467–472 (1981) 7. Deutsch, D., Dooley, K., Henthorn, T.: from tones comprising full harmonic series. JASA 124 (1), 589-597 (2008) 8. Ueda. K., & Ohgushi, K.: Perceptual components of pitch: Spacial representation using a multimensional scaling technique. JASA 82, 1193-1200 (1987) 9. Pollack, I.: Decoupling of Auditory Pitch and Stimulus Frequency: The Shepard Demonstration Revisited. JASA 63, 202-206 (1978) 10. Nakajima, Y., Tsumura, T., Matsura, S. Minami, H., & Teranishi, R.: Dynamic Pitch Perception for Complex Tones Derived from Major Triads. Music Perception 6 (1), 1-20 (1988) 11. Deutsch, D., Moore F. R., Dolson, M.: The perceived height of octave-related complexes. JASA 80 (50), 1346-1353 (1986) 12. Benson, D.: Music: A Mathematical Offering. Web version (2008). Retrieved 2 November 2010 from http://www.maths.abdn.ac.uk/~bensondj/html/music.pdf 13. Mathews, M.: What is Loudness?. Cognition, and Computerized Sound: An Introduction to Psychoacoustics. In: P. R., Cook (eds.), pp. 261–275. Cambridge University Press, Cambridge, UK (1999) 14. Wong, G.: Sound Level Meters. The Handbook of Noise and Vibration Control. In: M. J., Crocker (eds.) Chapter 38, pp. 455-464. New York (2007) 15. Chowning, J.: Perceptual Fusion and Auditory Perspective. Cognition, and Computerized Sound: An Introduction to Psychoacoustics. In: P. R., Cook (eds.), pp. 261–275. Cambridge University Press, Cambridge, UK (1999)

From the Shepard Tone to the Perpetual Simple Melody Auditory Illusion

Appendix

Table 1. Values of the individual frequencies obtained through the equation 1. These values correspond to the scale of C divided into equal 12 parts.

Table 2. Values of the individual amplitudes obtained through the equation 2. For all complex sounds, the amplitudes increase gradually from the first to the seventh component and decrease from the seventh to the ninth component.