Proceedings of the International Computer Music Conference (ICMC 2009), Montreal, Canada August 16-21, 2009

SYNAESTHETIC ANALYSIS, EXPLORING MUSIC STRUCTURES BY MULTIMEDIA REPRESENTATION. PROMETHEUSZ12, A NOVEL 3D VISUALIZATION TOOL.

Paolo Tagliolato Universita` degli Studi di Milano Dipartimento di Informatica e Comunicazione - Laboratorio di Informatica Musicale, Milan, Italy [email protected]

ABSTRACT

Visual representations have always been fundamental tools to communicate about music. While common music nota- tion was for centuries the standard way to encode western music information, different visual representations were de- veloped and exploited by music analysis. The importance a. b. of such representations lies in their great explanatory power. The real-time exhibition of visual models of music features joins together aesthetic and analytical perspectives, favor- Figure 1. a. Light instrument used by Scriabin (from [15]) ing the comprehension of analytic systems. In this article b. Scriabin’s map of sounds to colors and emotions, on the I present PrometheusZ12, a novel 3D visual representation circle of fifths (from [16]) tool for music, based on the concept of product GIS intro- duced by D. Lewin. As suggested by its name, a tribute to the synesthaetic explorations of A. Scriabin, the software in- great explanatory power. It could be said that it is a case in which analysis and aesthetic are joined together. Tools volves Z12, the well known mathematical model of the space, providing the real-time geometrical rendering of which let this type of visualization can favor the compre- hension of analytic systems. Moreover, they could help their the product extensions of Z12 with some rhythmic models. learning. To learn the association between sound aggregates and 1. INTRODUCTION. MULTIMEDIA REAL-TIME forms is one of the basic aspects in instrumental apprentice- SYNAESTHETIC SYSTEMS: ART AND ANALYSIS ship, forms being in that case mental images which musi- Multimedia real-time association of visual aspects and sounds cians must develop, like the positions (or configurations) of can be exploited from an aesthetic and artistic point of view. hands on their instrument. The automation of the move- An early attempt in this direction is Scriabin’s Prometheus ments to translate those mental images into instrumental (Symphony No.5, Op.60), in which colors and sounds are gestures are the basis for learning an instrumental technique. associated. More than a simple sort of choreography, the re- Also music notation, in alternative to the CMN abstrac- search of Prometheus originated from Scriabin’s hypothesis tion, historically made use of the configuration of hands about the existence of a synaesthetic correspondence among on musical instruments: “intavolature per liuto” of Renais- sounds, colors and emotions. Figure 1 b. presents this as- sance, guitar tablatures, organ tablatures used by the Ger- sociation on the circle of fifths. It is interesting to observe man Baroque organ school (see e.g.[17]) are some exam- how this map makes use of enharmonic equivalence. [15] ples. The resorting to the description of physical configu- presents a picture (fig. 1 a.) of a light instrument used by ration of the control interface of an instrument was also the Scriabin, which is precisely a 12 tone wheel whose pitch practice adopted for notation by many electroacoustic com- classes are colored light bulbs. posers. On the other hand, the real-time exhibition of visual Coming back to analysis, music analysts like Allen Forte models of music features can be linked not only to aesthetic or David Lewin start from their listening experience, edu- but it can also be regarded in an analytical perspective. E. cated by theory and trained on geometric analogies. It is Isaacson, in [6], outlined the importance of visual repre- my opinion that real-time models could favor this analogi- sentations of music founded on analytical models, for their cal and synaesthetic training.

97 Proceedings of the International Computer Music Conference (ICMC 2009), Montreal, Canada August 16-21, 2009

2. RELATED WORKS This kind of visualization exposes very clearly what we can call the interval structure of a pcs. Moreover, this simple Musical set-theory, developed for the analysis of atonal mu- geometrical representation is very useful, when one com- sic, focuses on classes of pitches under octave and enhar- pares different pcs. monic equivalence. The quotient set, isomorphic to Z12, is In the context of , Thomas Noll explored affine trans- PC that of the twelve pitch classes, whose well-known picto- formations in Z12 and widely made use of 2D computer an- rial two dimensional representation is also called the twelve imations of the twelve tone circle [12]. Three-dimensional tone circle or wheel.Chords can be regarded as elements of representations of music structure have been developed in the power set of Z12, called pitch class sets (pcs). Set-theory theories of tonal space (see [7] for an introduction, [5] for founds its analyses on pcs, their classifications (cf. Forte [4] recent works in the field). Noteworthy is E. Chew’s Spiral and Solomon [14]), their intervallic structures and proper- Array 3D model [3], exploited for key-finding, which led to ties. In the mentioned geometric representation, which well a software tool for real-time representation of midi perfor- exposes the cyclic nature of Z12,apcs becomes a polygon mances. inscribed in the circle. In the framework of transformational analysis, the set- theoretical model can be regarded as an example of a more 3. PROMETHEUSZ12 complex mathematical structure, namely a Generalized In- terval System ( ) [9], which interprets musical intervals Pitch class sets have their balance and symmetries on which GIS some analyses are founded. The geometrical representation as elements of a group acting simply transitively on Z12. In this context the pitch class space ( ) is then a pitch of as points of a circumference is useful to visualize PC PC related . The same theoretical framework can model these properties. GIS also rhythmical features of music. For example it can be I developed PrometheusZ12, a novel 3D visual represen- shown that the space of onsets ( ns), whose “intervals” are tation tool for music, with the intent of seeing through the O analysts’ eyes the geometrical content of -related mu- time-spans between events, shares the same mathematical PC structure and can be regarded as a rhythmic . More- sic analyses evolving in time. Differently from what can be GIS over, different features can be combined, considering that done with the existing systems, I wanted a system which the product of two is also a . could maintain the geometrical peculiarity of the rep- GIS GIS PC The visualization of , as said, has been widely used resentation but which could let also the “navigation” of the PC in . Moreover we can find it in several software resulting representation of music, in order to better analyze it. It should appear clear how the concept of product tools. IRCAM’s OpenMusic [2] and Laurson’s PWGL [8], GIS two Computer Assisted Composition systems, offer their were well suited to these aims. users modules for that kind of visualization, which can be PrometheusZ12 shows a three dimensional model of a product hythmic, which extends the 2D rep- exploited both for analysis and composition (figure 2 a., GISPC×R resentation of with rhythmic information. The resulting from [1]). Malinowski’s Music Animation Machine [10] - PC geometry is that of a cylinder made up of pc circles disposed along a time axis centered in the center of circles and per- pendicular to circles’ planes. Each harmonic change triggers the addition of a new pcs to the tube, whose distance from the preceding depends upon the choice of the rhythmic model 1. Pcs are depicted as polygons inscribed in the circumferences, and optionally their semi-transparent areas are visualized, whose colors are a. b. combination of those of their vertices. Starting as said with Scriabin’s association, I tried also different combination of colors2. Pcs can be presented, like in MAM’s dyad view, Figure 2. Set theoretical visualization in other software along with their intervallic structure. While the areas of pcs tools. a. Open Music. b. Malinowski’s MAM. with the same transpositional prime-form have different col- ors, depending on their transposition, intervals’ color are in- a tool for the visualization of MIDI performances - among variant: one interval, one color. several other representations, with the “interval dyad” view presents music by means of the geometric model for . 1 Two models are available: ns or cc - this second being a trivial PC O O While the software plays music, the current pcs is shown model in which subsequent events are equally spaced along the time axis. For the characterization of this second simple rhythmic space as a , and all the intervals are depicted as the chords of the arcs GIS see [13] between each “dyad”, that is between each couple of pitch 2Following some ideas from Malinowski, see “Harmonic Coloring classes (figure 2 b.). based on the Perfect Fifth”, in [10])

98 Proceedings of the International Computer Music Conference (ICMC 2009), Montreal, Canada August 16-21, 2009

4. GEOMETRIES OF MUSIC THROUGH PROMETHEUSZ12

Let me show some examples of what we can see through PrometheusZ12. Figure 3 compares two segments of “Lasciate i monti” for Mixed Choir and Continuo from Monteverdi’s Orfeo. Here, following Malinowski, adjacent pitch classes in the circle of fifths are mapped to adjacent colors in a color wheel. 1. 2. Moreover, coloring of pcs’ area is enabled: this way, in this almost triadic piece, tonality changes are somehow evi- denced: the change of accidentals for B, E, F acts so that the overall colorings of the two segments are well distinguish- able. Left screenshot shows note names, in the right picture this feature is off, making the view less confusing.

3. 4.

5. 6. Figure 3. Monteverdi - Orfeo “Lasciate i monti” for Mixed Choir and Continuo. Figure 4. J.S.Bach Toccata, Adagio e Fuga in C major Figure 4 shows the PrometheusZ12 3D rendering of the BWV 564. Two different sections are well distinguishable. Adagio from J.S.Bach’s “Toccata, Adagio e Fuga in C ma- jor” BWV 564. It is easy to distinguish two sections of the piece, the first (in 1. and 2.) with a simple chordal accom- paniment, the second (4.-6.), which is marked grave, char- for . It is not difficult to observe how, effectively, acterized by chromatic progressions and dissonances. The many of the chords used in the Poem of Fire are related double descending scale (3.) functions as a separator of the to the Prometheus chord. In 1. we find a half-diminished two sections. Here each interval is associated to a specific seventh (prime form 0258, classified by Forte with its in- color. This way the interval structure of pcs is more evident. verted form 0368, a more familiar dominant-seventh, as 4- 1. Succession of consonant triads which are characterized 27). This chord is also known as since its by green, blue, red intervals (resp. major third, minor third, use - moving towards - by Wagner in “Tristan und perfect fifth). 2. A major consonant triad following two Isolde”. Tristan is a subset of Prometheus, as we can eas- non-consonant triads. 3. The melodic line is well outlined (a ily see comparing the two figures. It is interesting also to nice spiral): PrometheusZ12 was set up in this case to show observe in this screenshot, that Tristan is here the resolu- hythmic intervals only in presence of melodies. 4. tion of a french- (prime form 0268, Forte’s 4-25) PC×R The interval structure of a more complex , towards similarly to its first appearances in the Wagnerian Prelude. the end of the piece. 5. Concluding harmonies of the piece, Moreover, the which maps the french-sixth to in Scriabin’s colors. 6. Same as 5 but in Malinowski’s col- Tristan chord (F into E) is a movement within the same ro- ors, this time showing all the hythmic intervals, that tation of Prometheus. 2. The superposition of some pcs is PC×R is all the intervals between successive pcs. shown. Being extremely near in time, we could associate Figure 5 shows Alexander Scriabin’s Poem of Fire, op.72. them to a single harmony properly containing Prometheus. Like the other late compositions of Scriabin, the Poem of In 3. a chord is shown, obtained by the same voice lead- Fire is an atonal work. Several authors indicated the Prome- ing described in 1. but in which all the notes are main- theus or (figure 5 a.) as central in Scriabin’s tained, so it is the union of the two chords Tristan and the compositional technique ([11] pg.313, serving as the basis french-sixth chord, and it is a subset of Prometheus. The

99 Proceedings of the International Computer Music Conference (ICMC 2009), Montreal, Canada August 16-21, 2009

6. REFERENCES

[1] M. Andreatta, “Methodes´ algebriques´ en musique et musicologie du xxe siecle,”` Ph.D. dissertation, EHESS, 2003. [2] G. Assayag, C. Agon, J. Fineberg, and P. Hanappe, “An object oriented visual environment for musical composition,” in ICMC, 1997. a. 1. [3] E. Chew, “Towards a mathematical model of tonality,” Ph.D. dissertation, MIT, 2000. [4] A. Forte, The Structure of Atonal Music. Yale Uni- versity Press, 1973. [5] W. B. Hewlett, E. Selfridge-Field, and E. Correia, Eds., Tonal Theory for the Digital Age, ser. Comput- ing in Musicology. CCARH, 2008, no. 15. 2. 3. [6] E. J. Isaacson, “What you see is what you get: on vi- sualizing music,” in ISMIR, 2005, pp. 389–395. [7] C. L. Krumhansl, “The geometry of musical structure: A brief introduction and history,” ACM Computers in Entertainments, vol. 3, no. 4, 2005. [8] M. Laurson and M. Kuuskankare, “PWGL: a novel visual language based on Common Lisp, CLOS and 4. 5. OpenGL,” in ICMC, 2002, pp. 142– 145. [9] D. Lewin, Generalized Musical Intervals and Trans- Figure 5. a. The mystic or Prometheus chord. 1.-5. A. formations. Yale University Press, 1987. Scriabin - “Poem of Fire”, op.72. [10] S. Malinowski. (2005) Music animation machine. [Online]. Available: http://www.musanim.com/ [11] G. Monro and J. Pressing, “Sound visualization using second-last chord in 4. and the chord in 5. are other two embedding,” CMJ, vol. 22, no. 2, pp. 20–34, 1998. Prometheus subsets. In 4. the last chord, a Minor-Major (prime form 01348), is obtained by symmetriz- [12] T. Noll, “Geometry of chords,” Electronic Bulletin of ing the second-last one. the Sociedad Matematica Mexicana, vol. 1, 2001. [13] A. Pinto and P. Tagliolato, “A generalized graph- spectral approach to melodic modeling and retrieval,” in ACM MIR 2008, pp. 89–96. 5. CONCLUSIONS [14] L. Solomon, “The list of chords, their properties, and uses,” Interface, JNMR, vol. 11, 1982. In this article I introduced PrometheusZ12, a novel visual- ization tool intended for music analysis popularization and [15] I. Vanechkina and B. Galeyev. “Prometheus” (Scriabin presentation. I proposed a 3D geometrical interpretation of and Kandinsky) - Prometheus Institute, RU. [Online]. a Generalized Interval System hythmic. In this Available: http://prometheus.kai.ru/ckkande.htm PC×R way the classic geometrical model of pitch class space is ex- [16] ——. Was scriabin a synaesthete? [Online]. Available: tended with rhythmic informations, letting the navigation of http://prometheus.kai.ru/skriab e.htm this peculiar analytic view. I showed, with PrometheusZ12, how does music appear through centuries, following the evo- [17] P. Wollny and M. Maul, “The Weimar organ tab- lution of pcs complexity, which exploded in 19th century, lature: Bach’s earliest autographs,” Understanding when this kind of geometrical schematizations were firstly Bach, vol. 3, pp. 67–74, 2008. conceived.

100