REGARDS ON TWO REGARDS BY MESSIAEN: AUTOMATIC SEGMENTATION USING THE SPIRAL ARRAY Elaine Chew University of Southern California Viterbi School of Engineering Epstein Department of Industrial and Systems Engineering Integrated Media Systems Center [email protected] ABSTRACT the same space and uses spatial points in the model’s interior to summarize and represent segments of music. Segmentation by pitch context is a fundamental process The array of pitch representations in the Spiral Array is in music cognition, and applies to both tonal and atonal akin to Longuet-Higgins’ Harmonic Network [7] and the music. This paper introduces a real-time, O(n), tonnetz of neo-Riemannian music theory [4]. The key algorithm for segmenting music automatically by pitch spirals, generated by mathematical aggregation, collection using the Spiral Array model. The correspond in structure to Krumhansl’s network of key segmentation algorithm is applied to Olivier Messiaen’s relations [5] and key representations in Lerdahl’s tonal Regard de la Vierge (Regard IV) and Regard des pitch space [6], each modeled using entirely different prophètes, des bergers et des Mages (Regard XVI) from approaches. his Vingt Regards sur l’Enfant Jésus. The algorithm The present segmentation algorithm, named Argus, is uses backward and forward windows at each point in the latest in a series of computational analysis techniques time to capture local pitch context in the recent past and utilizing the Spiral Array. The Spiral Array model has not-too-distant future. The content of each window is been used in the design of algorithms for key-finding [2] mapped to a spatial point, called the center of effect and off-line determining of key boundaries [3], among (c.e.), in the interior of the Spiral Array. The distance in other things. The previous model for determining key the Spiral Array space between the c.e.’s of each pair of boundaries is the one most relevant to this paper. This forward and backward windows measures the difference earlier algorithm requires knowledge of the entire piece in pitch context between the future and past segments at and the total number of segments, and does not compute each point in time. Segmentation boundaries then in real-time. So far, the Spiral Array has only been used correspond to peaks in these distance values. This paper in the analysis of tonal music. This paper extends the explores and analyzes the algorithm’s segmentation of Spiral Array model’s applications to real-time post-tonal music, namely, Messiaen’s two Regards, segmentation and to the analysis of post-tonal music. using various window sizes. The computational results The Argus algorithm for automatic segmentation uses are compared to manual segmentations of the pieces. only the outermost pitch spiral in the Spiral Array model Taking into account the entire piece, the best case and the interior space to compute a distance between the computed boundaries are, on average, within 0.94% (for local context (captured by a pair of sliding windows) Regard IV) and 0.11% (for Regard XVI) of their targets. immediately before and after each point in time. The distance measure peaks at a segmentation boundary and 1. INTRODUCTION the peaks can be used to identify such boundaries in real- Segmentation by context is a necessary part of music time. Since the segmentation algorithm detects processing both by humans and by machines. Efficient boundaries between sections employing distinct pitch and accurate algorithms for performing this task are collections, the procedure does not depend on key critical to computer analysis of music, to the analysis context and can be applied in general to both tonal and and rendering of musical performances, and to the atonal music. The method is highly efficient, indexing and retrieval of music using largescale computing in O(n) time, and requires only one left-to- datasets. Computational modeling of the segmentation right scan of the piece. The algorithm is tested on process can also lead to insights on human cognition of Messiaen’s Regards IV and XVI and the results presented music. This paper will focus on the problem of for various window sizes. The computational results are determining boundaries that segment a piece of music compared to manual segmentations of the piece. into contextually similar sections according to pitch Related work on finding local tonal context include content. In particular, the algorithm will be applied to Temperley’s dynamic programming approach to Olivier Messiaen’s (1908-1992) Regard de la Vierge determining local key context [10], Shmulevich & Yli- and Regard des prophètes, des bergers et des Mages, Harja’s median filter approach to local key-finding [9] the fourth and sixteenth pieces in his Vingt Regards sur and Toiviainen & Krumhansl’s self-organizing map l’Enfant Jésus (1944). approach to determining and visualizing varying key The O(n) segmentation method uses the Spiral Array strengths over time [11]. The focus of these methods are model [1], a mathematical model that arranges musical on determining the local key context rather than finding objects in three-dimensional space so that inter-object the segmentation boundaries. The methods center distances mirror their perceived closeness. The Spiral around key-finding, which applies only to tonal music. Array represents tonal objects at all hierarchical levels in (c) major key (b) major triad representations (a) pitch representations representations Figure 1. The Spiral Array model. The remainder of the paper presents a concise overview indexed by its number of Perfect Fifths from a reference of the Spiral Array model followed by a description of pitch. The model assumes octave equivalence so that the segmentation algorithm. Then, in Section 3, a all pitches with the same letter name map to the same descriptive analysis of Regard IV is followed by a spatial point. detailed analysis of the computational results and Pitches that define a triad form compact clusters. comparisons between the computed and manually They also form the vertices of a triangle. Each triad is assigned boundaries. A similar treatment of Regard represented by the convex combination of its component XVI is presented in Section 4, followed by discussion pitches, that is to say, a point in the interior of the and conclusions in Section 5. triangle. For example, the major triad is defined as: C (k) = w P(k) + w P(k+1) + w P(k+4), (2) 2. THE SPIRAL ARRAY MODEL M 1 2 3 3 where w1 ≥ w2 ≥ w3 > 0 and w = 1. Âi=1 i This section provides an overview of the structure of, and underlying concept (namely, the center of effect) The sequence of major triad representations also forms behind, the Spiral Array model [1]. The description of a spiral, as shown by the inner spiral in Figure 1(b). Three adjacent major triads uniquely define the pitch the proposed segmentation algorithm follows the † introduction to the Spiral Array. collection for a major key. They also form the IV, I and V chords of the key. Hence, major keys are defined as: 2.1. The Model’s Structure TM(k) = w1C(k) + w2C(k+1) + w3C(k–1), (3) The Spiral Array model represents pitches on a spiral so 3 where w1 ≥ w 2 ≥ w 3 > 0 and w = 1. that spatially close pitch representations form familiar  i=1 i higher-level tonal structures such as triads and keys. Again, the sequence of major key representations form a The model represents each higher-level object as the spiral. This major key spiral is shown as the innermost convex combinations of its lower-level components. spiral in the illustration in Figure 1(c). Corresponding These weighted sums of representations of the definitions exist for† the minor triad and key components result in spatial points in the interior of the representations. pitch spiral. For example, Figure 1 shows the The Spiral Array model is calibrated using hierarchical construction of major key representations, mathematical constraints to reflect perceived closeness from pitches to triads to keys. among the different entities. For example, in the Figure 1(a) shows the pitch spiral – adjacent pitches definition of the major triad, the weights are constrained along the spiral are related by intervals of a Perfect so that the weight on the root is no less than the weight Fifth; each turn of the spiral contains four pitch on the fifth, which is no less than the weight on the representations, as a result, vertical neighbors are a third. Since the segmentation algorithm uses only the Major Third apart. Pitch representations can be pitch representations, we shall concern ourselves only generated by the following equation: with parameter selection for the pitch spiral, namely, the choice of r and h. È r sin(kp 2)˘ The pitch spiral is uniquely defined by its aspect ratio Í ˙ h/r. The goal is to constrain the parameters so that the P(k) = Í r cos(kp 2)˙ (1) Í ˙ distance between any two pitch representations kh ÎÍ ˚˙ correspond to their perceived closeness. Suppose the desired rank order of the interval distances is as follows: where r is the radius of the spiral and h is the vertical {(P5/P4), (M3/m6), (m3/M6), (M2/m7), (m2/M7), ascent per quarter turn. Each pitch representation is (d5/A4)}, where P denotes a perfect interval, M a major † interval, m a minor interval, d a diminished interval and b n j 1 p i, j A an augmented interval. Algebraic manipulation shows c a,b =  (4) that the mathematical constraints on the aspect ratio, b - a +1 j=a i=1 n j √(2/15) £ h/r £ √(2/7), produce the desired ranking of interval relations.