Lawton Hall Music Composition and Technology LEANDER: NAVIGATING University of Wisconsin–Milwaukee 2400 E Kenwood Blvd. MUSICAL POSSIBILITY SPACE Milwaukee, Wisconsin 53211 United States through COLOR DATA SONIFICATION [email protected] Abstract Introduction fixed sounds or musical parameters, the color values are used as probability vectors Leander is an experimental film that sonifies Even in the dreariest early spring months, Lake representing the likelihood that particular color data to generate its musical soundtrack. Michigan seems to change its colors almost sound-events with particular characteristics The colors of Lake Michigan, captured in daily, reflecting the most minute changes will occur at each moment in the film. time lapse video, constitute ever-changing in the sky, tides, temperature, and other Leaving the moment-by-moment specifics probability vectors that govern the behavior atmospheric conditions in the appearance to weighted random chance means that of musical sound-events over time. This of its waves. To document this chameleonic no two realization of Leander are exactly stochastic, or probabilistic approach to data behavior, I took time-lapse videos of the lake alike, while maintaining an overall musical sonification imagines the musical experience from the same point on a cliff in Milwaukee structure and audio-visual synchronicity. as movement through a virtual possibility County, Wisconsin over a series of days in space, rather than the end result of a causal March 2020. Each video began thirty minutes Named after the young man of Greek myth process. This pictorial describes how color before sunset and ended thirty minutes after who swam nightly across the Dardanelles data guides the various musical parameters sunset. I also made audio recordings of the toward his lover Hero’s lighthouse, at play in Leander through weighted chance. surrounding environment. After gathering Leander is a guide for navigating a musical and assembling the footage, I extracted the landscape on a journey of patience and Authors Keywords quantifiable color data of the video using solitude. The lake and its colors provide a Processing into a table of timestamped color compass and bearing, but the specific path stochastic music, data sonification, values which can be read like a musical score taken is a product of chance and algorithms, information art, parameter space, time into the audio programming environment discovered only by experiencing the piece. lapse, SuperCollider, electronic music, SuperCollider, sonifying (or perhaps musifying) video art, modern composition the color data in sync with the video. The Leander can be viewed at film, in a sense, writes its own soundtrack. www.lawtonhall.com/leander. Rather than mapping the data directly to VISAP’20, Pictorials and annotated portfolios. VISAP’20, Pictorials Flowchart of the process of generating music from video through color Chance Music data sonification in Leander. Delegating compositional decisions to random, famously) I Ching divination to determine or aleatoric, chance is not a new technique musical parameters. Cage mapped these Saturation Brightness and the use of mechanized automation to chance techniques to musical parameters assist with musical chance procedures long (pitch, rhythm, etc.) much like the parametric predates the digital age. Musical games mapping in data visualization and sonification.5 (musikalisches würfelspiele) in which players COLOR build pieces of music by rolling dice to choose The sonic environment generates the musical Hue DATA Red from precomposed fragments were popular experience in some aleatoric works, much from video pixels in eighteenth-century Europe. A notable 1792 like how Leander generates music from its würfelspiel attributed to Mozart is capable of visual landscape. In Cage’s 1952 piece 4’33”, producing over 45 trillion different waltzes the solo performer is instructed to tacet (not from its possible combinations of measures.1 play) for the entire duration of the piece, Blue Green allowing the listener to focus exclusively on In 1821, Dutch-German inventor Diederich ambient sounds in the concert hall.6 Composer Nikolaus Winkel mechanized the earlier Alvin Lucier’s I Am Sitting in a Room from determines dice games with an instrument called the 1969 features a recording of his spoken voice Componium, a clockwork pipe organ that being played into an empty room and re- WEIGHTD PROBABILIT plays combinatorially-generated music recorded. This process is repeated numerous from material pre-programmed onto times,with the acoustic qualities of the pinned wooden cylinders like a music box.2 room becoming increasingly apparent with determines The Componium uses a roulette wheel-like each iteration until all that remains are the device to choose fragments randomly and resonant frequencies of the room itself.7 can produce over 53 trillion different pieces 3 Pitch Rhythm without the use of dice or human intervention. In the würfelspiele and more recent aleatoric works, the piece of music is not fixed entity, This proto-chance music failed to catch on but rather a possibility space (or parameter as music composition became increasingly space) that encompasses a large number of MSICAL complex and deterministic in the late possible outcomes that each occupy a discrete 4 8 Bells PARAMTRS Timbre nineteenth and twentieth centuries. In point within such a space. Every dynamic reaction to these modernist tendencies, musical parameter (i.e. parameters whose American composer John Cage championed values may change between realizations) chance procedures beginning in the 1950s. represents a dimension in possibility space Many of his works from this era, such as and the location of one particular outcome is Pads Soundscape Music of Changes, are almost entirely aleatoric, determined by the value of each parameter relying on coin flips, graphical scores, and (most at each moment as a piece is realized. VISAP’20, Pictorials and annotated portfolios. VISAP’20, Pictorials Random chance, then, drops the listener, may be lost when exploratory methods are medieval isorhythms and the probabilistic performer, or even composer into an strictly causal, but interactive algorithms methods of stochastic music. It is both a unexpected location within a possibility allow the composer, guided by her tastes and generative musical possibility space defined space without the personal biases that values, to take a hands-on role in finding by a small number of dynamic parameters, often blind us to out-of-the-box creative “good” trajectories through parameter space.12 and a wide, winding path through this space thinking. Each realization of an aleatoric In his 1963 book Formalized Music, Greek defined by the fixed-media visuals. The colors work provides a different view of a musical composer Iannis Xenakis detailed a synthesis of the time lapse video determine those regions landscape even when the full breadth of a of aleatoric and causal techniques by using in this musical landscape where listeners may possibility space is incomprehensibly vast. probability to narrow the range of possible find themselves at every moment in the piece. The results are often exciting and surprising outcomes in an aleatoric context and to and allow for the creation of works that make some outcomes more likely than are greater than the sum of their parts, others through statistical distribution. especially when the process is automated. Xenakis called this composition method stochastic music and demonstrated its use Of course, random chance is not the only way in Pithoprakta and many later works.13 to explore a possibility space. Composers of isorhythmic motets in the fourteenth century Complex natural phenomena, such as the used fixed, repeating cycles of pitches and summer songs of cicadas, can be imitated rhythms to explore the uncharted frontier musically by using statistical probability to of polyphonic music.9 In the twentieth determine the morphology and characteristics century, serialism and other modernist of large numbers of discrete sound-events. composition methods used rigid algorithms Each sound-event behaves independently of to avoid areas of musical possibility space the others but all are governed by the same that carried historical or extramusical probability distributions, allowing a composer baggage.10 Even Mozart’s würfelspiel could, to assert creative agency without the rigidity in theory, be explored systematically, e.g. by of causal determinism. Most importantly, choosing the fifth option every time instead the probabilities used in stochastic music are of rolling dice or by using an integer series. not fixed, but rather the primary dynamic variables that a composer uses to shape Such causal, deterministic algorithms pieces of music over time. The same vast are a means of locating those regions in a probability space exists for the duration of a possibility space that meet certain aesthetic work, but sound-events can be constrained criteria.11 Algorithms have the advantage of to specific subspaces as the music requires. tweakability; if one fails to produce pleasing results, it can simply be adjusted and repeated Leander hybridizes the combinatorial while avoiding having to manipulate musical indeterminacy of the würfelspiele, the parameters directly. The element of surprise deterministic clarity of data sonification and Camera placement for Leander VISAP’20, Pictorials and annotated portfolios. VISAP’20, Pictorials The Leander Possibility Space SIA PAAS Only three