array ICMC | SMC 2014 Keynote Speech John Chowning 2015/2016 in any other medium. Along the way, Instrument [3], because the sampling that were exclusively electroacoustic. the discovery of FM Synthesis provided theorem is the foundation on which Some of the music, composed for four ICMC | SMC 2014 Keynote a means of creating diverse spectra Mathews based much of his early work. channels was, quite literally, head turning. Address that, coupled with a ratio from Euclid’s His research included speech, hearing and From my youth I had a fascination with Mathews’ Diagram and Elements, produced an unusual and where the loudspeaker cavernous spaces and echoes, their Euclid’s Line: Fifty Years Ago productive connection between spectral is the ultimate sound source. Mathew’s disorienting effect on otherwise familiar space and pitch space. diagrammatic representation of the sounds and the spatial aspect of this by John Chowning sampling theorem opened the door to music provoked a desire to compose for my understanding of what was otherwise loudspeakers—to put imagined sounds in 1. Introduction Onassis Cultural Centre, 16 September incomprehensible because of my own imagined spaces. 2014 Claude Shannon’s 1948 paper, A non-scientific background. Mathematical Theory of Communication Euclid’s line, to which I refer in the title, is Abstract [1] is the hard-edged theory that underlies its division into extreme and mean ratios, the flow of information in today’s Making the science and technology now commonly known as the golden complex digital world of computers, large of computer music comprehensible ratio. This ratio became of interest to me and small, tablets, mobile phones, pads to musicians and composers who had after composing Turenas (1972), in which I and pods—capable of ‘sensing’ sound, little or no background therein was made extensive use of both harmonic and image, touch, location—all complex a part of Mathews’ genius. In inharmonic spectra. I looked for other machines, the complete understanding of this presentation I will show how a irrational numbers to produce inharmonic which is beyond the capacity to know of simple diagram led to the essential spectra and found that the Golden ratio any single human being. It is a summation understanding of Claude Shannon’s had particularly interesting properties in sampling theorem, which in turn of Shannon’s own work and that of his this application. Figure 1. This is Mathews’ schematic diagram opened up a conceptual path to colleagues and predecessors. The timing was propitious as the first stored-program of the sampling process from 1963 [3], at which composing music for loudspeakers 2. Mathews’ diagram time electro-acoustic music was exclusively in the computers were just being developed. The that had nothing to do with wires, analog domain. cables and electronic devices, but paper includes the first use of the word My interest in composing music for 1 However, I was well aware that the led to learning how to program a ‘bits’. And theorem 13, the sampling loudspeakers stemmed from a few musical stringent technical requirements, computer—to write code. The change theorem, is critical to the connection experiences that had a profound effect on knowledge and means to create music from device-determined output (analog) between continuous and discrete signals. the way I thought about composing. From for loudspeakers in the 1960s were to program-determined output (digital) In his article, The Origins of the 1959 until 1962 I studied in where inaccessible to all but a few composers. was a major change in paradigm that Sampling Theorem, H.D. Luke traces a contemporary music was notably present. led to my realization of an integral rich history of the sampling theorem that Some concerts included electroacoustic In 1964, because of a bit of serendipity, I sound spatialization system that would extends back to 1848 [2]. music— the Domaine Musicale concerts was given Mathew’s article. It was the first at the Théâtre de l’Odéon and the have been impossible for me to achieve Shannon’s paper is the first reference diagram, which caught my attention (see Groupe de recherches musicales (GRM) in ’ famous 1963 article Figure 1). It presented a comprehensible presented concerts at the French Radio The Digital Computer as a Musical face of the sampling theorem that, for

1313 14 array ICMC | SMC 2014 Keynote Speech John Chowning 2015/2016 me and perhaps others, was suggestive seemed to be undaunted by learning to two very different but complementary and inspiring. It carved out a path to program a computer to do the same. processes — joining the structure of the electroacoustic music that bypassed (what Having read Mathew’s article early in sound itself to the structure of musical for me was) technological clutter; a path 1964 and the comprehensive article by form. that would allow the composition of ‘any James Tenney, Sound Generation by • I realized that those having motivation perceivable sound’ [3] bringing musical Means of a Digital Computer [4], in and perseverance, but no special creation to the edge of my imagination. April, I took a new course offered at competence in building electronic devices, called, Computer were presented with a means to engage 2.1 Sampling’s Simplicity Programming for Non-Engineers. With in a medium, and at a high level of the confidence that I could program a Immediately striking in the diagram is abstraction, that was a defining musical computer, I set about to learn acoustics that there are but three devices and a advance in the 20th century—music and psychoacoustics, the latter highlighted computer, none of which have changed composed for loudspeakers. in Mathew’s article as an area of special over time in their functional relationship, importance to music perception. The discovery of FM Synthesis in 1967 but all of which have changed over time Figure 2. Finding a graphic solution: the distance, was the result of searching for lively in their cost, quality and precision—for Tutored by the undergraduate math azimuth and velocity cues of a moving sound are captured by plotting points along the trajectory at sounds that had some internal dynamism the better! major, tuba player, and incipient hacker, that made them easy to localize. Armed David Poole (my angel!), by September a constant interval of time. Doppler shift is derived Now, dear reader, imagine a 29 year from the radial velocity. I used the Cartesian with insights3 derived from Jean-Claude 1964 (just 50 years ago!) we had generated old graduate student composer, fifteen quadrants for naming the channels. Risset’s analysis-synthesis of trumpet our first sounds using Mathew’s Music IV years from his last math class, never tones, [5] I spend the next few years 2 Completing the quad spatial system was program. developing FM synthesis. having seen a computer, but with a very important moment in my personal The Artificial Intelligence Laboratory vivid imaginings, however vague and history, and in the direction that the After seven years of development and provided me off-hour computer time and inchoate, of composing music in Computer Music Project—and eventually study, I had acquired sufficient knowledge a population of skilled researchers in fields space. Imagine further, the conceptual CCRMA—would take. There are several to build sophisticated enough tools to ranging from linguistics to philosophy, breakthrough when with images in mind reasons for this: be able to realize two compositions— of electroacoustic music studios—filled speech, physics and, of course, computer Sabelithe (1971) and Turenas (1972). • While computers were not yet powerful with electronic equipment, cables, science and electrical engineering - any An extensive account of this early work, enough to synthesize and process sound wires, multiple microphones, spinning one of whom could answer the many Turenas: The Realization of a Dream, was in real-time—hands-on and favoring loudspeakers and austere-looking questions that I posed as I developed presented at the Journées d’Informatique immediate response—they would be some engineers in white coats—I understood a sound spatialization program. After Musicale in 2011 [6]. the implications of Mathew’s Figure 1. cajoling an electrical engineering student day (as we know very well with today’s to build me a 4-channel DAC, I realized a technology). 3. Euclid’s Line 2.2 The Soft Complexity Behind the quad system in 1968 (Figure 2). • Computer synthesis provided the Euclid defines what is now known as Samples composer direct control of the material the Golden ratio in Elements, Book VI, of music, as a painter has with paint and Already familiar with complex symbols Definition 3 [8]. It reads: as representation of sound, musicians canvas, allowing the accomplishment of

15 16 array ICMC | SMC 2014 Keynote Speech John Chowning 2015/2016 A straight line is said to have been cut ratio. When the carrier and modulating [7], with an equal tempered division of if it is a sine wave (this is demonstrated in extreme and mean ratio when, as the frequencies are both different powers of the pseudo-octave into nine scale steps. I in his work Dreamsong, 1978), I set about whole line is to the greater segment, so φ, four of the resulting partials are also call this the ‘Stria scale’ (StrScl), after the to synthesize the singing voice with FM is the greater to the less. powers of φ (see Table 1). composition in which it was first used. synthesis. Taking advantage of McNabb’s important insight and Johan Sundberg’s In three of my compositions I exploited vast knowledge of the science of the AB : AC = AC : CB (1) this division of the pitch space and the singing voice, I profited greatly from his complementary inharmonic spectra based presence at IRCAM and was able to 1+ 5 on the φ and FM synthesis (φFM) shown j = synthesize a number of sung vowel tones. or 2 (2) in Table 1. By setting the frequency at the j =1.618033... • Stria (1977) used φFM spectra [9] phonation frequency (pitch frequency), The ratio in its algebraic form (equation • Phoné (1981) used harmonic spectra of and the carrier frequencies at the closest synthesized singing voice mixed w/φFM 2) is one of the most studied of numbers, Table 1. Shaded cells show the four low-order harmonics to a given vowel’s formant with many claims being made over partial frequencies that are powers of φ when both spectra and synthesized singing voice. frequencies, I successfully modeled the the centuries as regards its presence in the carrier and modulating frequencies are powers • Voices (2005, v.3 2011) used harmonic target spectrum, as shown in Figure 3. The nature, art and music (many are probably of φ (but not equal). spectra of soprano’s voice mixed with relationship of the spectral model to the extravagant claims). The ratio is implicit This unique attribute caught my φFM spectra and synthesized singing signal generation can be seen in Equation in the formation of the pentagram and attention, as this is not the case with √2 voice. 4. With an appropriate mix of a piecewise perhaps known to the Pythagoreans linear random function, and a periodic or any other irrational number that I am Together with a longstanding interest in almost three centuries earlier. However, sinusoidal function to approximate the aware of. aspects of Greek mythology and history, my interest in this ratio came from micro-modulation of pitch (phonation especially the Pythia and her origins, the frequency) through time, the simulations another point of view. 3.2 The Golden ratio and the Pitch golden ratio and the Oracle of Delphi were convincing. This work is described in Space came together in Voices for soprano and 3.1 The Golden ratio and FM Spectra my paper, Synthesis of the Singing Voice I then ‘discovered’4 that powers of φ were interactive computer. Along the way, I by Means of Frequency Modulation [10]. In FM synthesis the distribution of related in the same way as Fibonacci became fascinated with the singing voice. the spectral (side-band) components numbers, as seen in Equation 3. are determined by the relationship 3.3 The Singing Voice: Phoné and e =A sin(2p f t + I sin 2p f t) 1 c1 1 m (4) between the carrier and the modulating n+1 n n-1 j =j +j Voices +A sin(2p f t + I sin 2p f t) frequencies. For inharmonic spectra in (3) 2 c2 2 m n =1, 2,3... In 1978 Jean-Claude Risset invited me Turenas, I used a carrier-to-modulating to spend a year at IRCAM. Based on frequency ratio of 1:√2. Looking for Expanding out powers of φ in log Michael McNabb’s demonstration that other irrational numbers that satisfied the frequency results in an equal intervallic capturing the fundamental frequency constraint that their fractional part not division of pitch, as is the case with (phonation frequency) of a sung female be small (as is, for example, π) I explored powers of 2. I have referred to the interval vowel tone through time is to capture the sound and attributes of the golden based on this division as a pseudo-octave the signature of the singing voice, even 17 18 array ICMC | SMC 2014 Keynote Speech John Chowning 2015/2016 ordered but not in the harmonic series. 0. Rather, they rise and are joined by the So, too, in Jean-Claude Risset’s Mutations other three components of the Equation Figure 4. (1969), where a set of pitches is heard, 4, A2, I1 and I2, as the micro-modulation Figure 5.1 first as melody, then as harmony, and is faded into the mix—a smooth finally folded into timbre [7]. It is the transformation to multiple singing voices. last stage which, again, is composed of precisely tuned partials from the set of pitches. It gives it an inharmonic, gong- like sound an ineffable quality of sounding ‘imprinted’ pitches. It was Mutations that inspired me to extend Risset’s powerful idea to another level of Figure 4. A collection sinusoids with frequencies control based on my research with the from the pitch space sound like a bell at the Figure 5.2 singing voice and perceptual fusion [12]. onset. Continuing, they each become a harmonic Phoné was premiered at IRCAM in 1981. in singing voice tones, where the change in hue \ represents the additional harmonics. Figure 3. Spectral modeling of the singing voice Over several years I developed the SAIL5 (or any sound having prominent resonances) can code around the idea of continuous Voices makes use of synthesized sounds be realized by setting the carrier frequencies, transformations of sounds through only and the amplified and processed fc1 and fc2 at the harmonic frequencies, 2f and sound of a soprano. The sounds and 7f, closest to the resonance peaks. The target detailed control of the partials and the spectrum in red, was captured by sndpeek. Band- conditions in which they cohere, or pitches are based upon φFM spectra and widths of the resonances (blue curved lines) are fuse, to be perceived as a single source the StrScl (and its pseudo-octave). The determined by the indices I1 and I2, here ≈ 1.0. rather than individual partials. As noted question at the outset was whether or not above, Risset demonstrated in Mutations a well-trained singer could comfortably Figure 5.3 One might ask: why synthesize a singing that sinusoids that begin together with sing in an unfamiliar spectral complex voice when one can sample and then amplitude envelopes that are exponential and in an artificial tuning system? [7] process a real voice? One answer lies in shape, and then fall off in duration in the kind of control one has over the (Details of how the piece was composed with increased pitch height, sound ‘gong’ details of the sound material. With have been previously described [7].) or ‘bell’-like, yet they are imbued with synthesis, sound can be formed in ways harmony. The onset of such a tone is that are not possible in transformations of shown in Figure 4. sampled sounds. Extending this process to another level Figure 5.4 John Pierce’s Eight-tone Canon (1966) [11] of complexity in Phoné, each of these The answer seems to be yes and I have could only have been realized by synthesis sinusoids is the f of a two carrier FM found independent confirming evidence as because the timbres are composed of c1 process as shown in Equation 4. The to why this may be so. precisely arranged partials that are amplitude envelopes A1 do not decay to

19 20 array ICMC | SMC 2014 Keynote Speech John Chowning 2015/2016 4. Partials and Tuning • In Figure 5.3 the base of the spectral 5. Conclusions founded in 1974. equation is increased by 10% to 2.1. Hiding (from me, at least) in the ever Understanding the implications of 4. This was a ‘discovery’ in that in 1974, increasing corpus in the hearing • In Figure 5.4 the base of both in Mathews’ diagram freed musical ideas I knew that the ratio between consecutive sciences, is a demonstration CD that is increased by 10% that led me into a field of study, research numbers of the Fibonacci sequence an astonishing and relevant example The 1st corresponding sound example and creation that I could not have were an approximation of φ, but I had that shows the importance of the sounds as expected - simple and boring. anticipated. The golden ratio fell into no knowledge of the same relationship complementary relationship between The 2nd and 3rd sound examples sound my ‘ear lap’ simply because it was ‘in the between the pow-ers of φ. spectral space and pitch space. It is out-of-tune - again, as expected. But the air’— in the culture of the 1970s with 5. Stanford Artificial Intelligence astonishing partly because the example 4th example, where both tuning and M.C. Escher t-shirts, computer graphics, Language is a procedural language is not cast in the context of new music, partials are stretched was not as expected. and D. Hofstadter’s Gödel, Escher, Bach: An developed at Stanford in the 1960s-70s. where it is often difficult to make critical, I had expected it to sound out-of tune, but Eternal Golden Braid. The Phoné code was derived from the objective judgments because both material in a different way than the previous two. Much of my inspiration is close to the bits Stria code in the same language. and context are unfamiliar. This example In, fact it sounded good, surprisingly— and bytes of sound, the spectral-temporal is a synthesized Bach chorale [13] without more interesting that the 1st sound detail. But also to the programming 7. REFERENCES artifice, where the tones are composed of example! language itself, abstract and cool in its partials produced by individual oscillators, [1] C. Shannon, “A mathematical When I formed the theoretical generality, but often provocative and theory of communica-tion.” ACM the amplitudes of which are similar to animating when engaged. those of a sawtooth wave. However, it is underpinnings for Stria and began the SIGMOBILE Mobile Computing and not a sawtooth wave and could not be! time-consuming sound realization, I had Communications Review 5, no. 1, pp. wondered if its lissome sound surface 6. Notes 3-55, 2001. The chorale is presented four times was unique because of its φFM spectra. 1. ‘If the base 2 is used the resulting units [2] H. D. Lüke, “The origins where each iteration sounds a different And so with Phoné. Engaing a soprano may be called binary digits, or more of the sampling theorem,” IEEE relationship of tones and tuning. The in Voices was a special challenge, because briefly bits, a word suggested by J. W. Communications Magazine 37, no. 4, pp. spectral/tuning renderings of the chorale I was unsure how the digital precision Tukey’ [1]. 106-108, 1999. are represented in Figure5.1-4 by a tone of synthesis would interact with the 2. The program was run on an IBM 7094, having a pitch frequency of 110Hz, where suppleness of a real singing voice. But [3] M.V. Mathews, “The Digital a 1301 disk drive, which was shared with the red colored equation and division again, the piece is built on the same Computer as a Musical Instrument,” a Digital Equipment Co. PDP-1, whose along the x axis stand for the pitch space ‘plinth’ as Stria and Phoné. Finding the Science, Vol. 142, No. 3592, pp. 553-557, graphics display’s x, y ladders provided scale and the gray equation and grey Tones and Tuning with Stretched Partials 1963. DACs. partial components their frequency [13] example pointed toward, and gave relation to the pitch space. [4] J. Tenney, “Sound-generation by weight to, a generalization: building sound 3. Joined by , then in 1968 means of a digital computer,” Journal of • In Figure 5.1 the base of both equations structures where pitch space and spectral by J. “Andy” Moorer and then later by Music Theory, pp. 24-70, 1963. is 2.0. space are complementary may open to an John Grey and Loren Rush, the research entirely new soundscape. at the Computer Music Project flourished. [5] J-C Risset, and M.V Mathews, • In Figure 5.2 the base of the pitch The Center for Computer Research in “Analysis of Musical-Instrument Tones,” equation is increased by 10% to 2.1. Music and Acoustics (CCRMA) was Physics Today, vol. 22, no. 2, 23-30, 1969.

21 22 array ICMC | SMC 2014 Keynote Speech John Chowning 2015/2016 Computer Music Currents 13, Schott Wergo, 1995. [12] J. Chowning, “Perceptual Fusion and Auditory Perspective,” in [6] J. Chowning, “Turenas: the P. Cook (ed.), Music, Cognition, and realization of a dream.” Proc. of the 17es Computerized Sound: An Introduction to Journées d’Informatique Musicale, Saint- Psychoacoustics, Cambridge, MA: MIT Etienne, France, 2011. Press, 1999, pp. 261-275. [7] J. Chowning, “Fifty Years of [13] J. Bach, “Als der gütige Gott,” Computer Music: Ideas of the Past in Tones and Tuning with Stretched Speak to the Future.” Computer Music Partials, Auditory CD, Acoustical Society Modeling and Retrieval. Sense of Sounds. of America, 1987, No 31. (http://asa.aip. Springer Berlin Heidelberg, pp. 1-10, org/discs.html) 2008. [8] Euclid, and D. Joyce. Euclid’s elements. Clark Uni-versity, Department of Mathematics and Computer Science, 1998. [9] O. Baudouin, D. Dahan, M. Meneghini and L. Zat-tra, The Reconstruction of Stria, Computer Music Journal, Vol. 31, Num. 3. MIT Press, Cambridge, 2007. [10] J. Chowning, “Synthesis of the Singing Voice by Means of Frequency Modulation,” in Sound Generation in Winds, Strings, Computers. Royal Swedish Academy of Music, 1980, No. 29. Reprinted in Current Directions in Computer Music Research, Edited by M. Mathews and J. Pierce, MIT Press, 1989. [11] J. Goebel, (producer). The Historical CD of Digital Sound Synthesis. Computer Music Currents 13, Schott Wergo, 1995.

2323 24