“As unified, bi-sensorially, as the sound film can be” 1: Composed interdependence of abstract moving image and synthesized sound in John and James Whitney’s Five Film Exercises

Volker Straebel, Audio Communication Group, Technical University Berlin Paper at Music and the Moving Image , New York University, Steinhardt School of Culture, Education, and Human Development, May 30, 2009

Introduction

When John and James Whitney started working in their studio in Los Angeles in 1943 on what was to become the Five Film Exercises , the brothers were ambitious young men, 26 and 22 years of age. After they had completed their abstract 8mm silent Twenty-Four Variations in 1940, they were about to no less than create a truly audio-visual abstract sound film that was meant to overcome the structural discrepancy of earlier work in this field. In an unpublished essay from 1943 or 44, they declared: “We extended our approach to organization within the new cinema medium, into its audio as well as visual elements. [... O]ur objective is organization on strictly new lines which we believe to be inherent within the medium itself.” 2 This approach implied on the technical level the development of an improved animation technique utilizing stencils that were directed by means of pantographs, and the creation of a technique to design the optical soundtrack directly on film, without recording audible sound. Both elements, visuals and audio, were precisely composed and notated in an unifying score, that to the scholar seems to pose more questions than answers. In this paper, I’m trying to answer some of these questions. Drawing from mostly unpublished material from the estate of John and James Whitney, to which John Whitney Jr. kindly and generously provided me access, I’m going to reconstruct the artistic background of the Whitney brothers at the time of the creation of the Five Film Exercises , describe the technique of animation and sound creation in use, and present my understanding of the score. Further research will be necessary to extract a structural analysis from the score to gain a better understanding of the interdependence of image and sound.

Artistic Background

John Whitney spent the year of 1938/39 on his grand tour in Europe. On a Rotterdam-bound freighter he shot non-narrative sequences of geometrical elements and abstract forms with his 8mm camera. It was not before his return to California that he learned about Walter Ruttmann’s Berlin – Sinfonie der Großstadt of 1927, that not only had extended the montage of observations of everyday life to a feature-length film, but that also carried the musical genre of the symphony in its title, thereby emphasizing the non-narrative, abstract nature of the film, John Whitney had been interested in. 3

1 John Whitney and James Whitney, "Audio visual music", in Art in Cinema. A Symposium on the avantagrde film , ed. Frank Stauffacher (San Franciso: Art in Cinema Society; San Francisco Museum of Art, 1947): 31-34, 32. 2 James Whitney and James Whitney, [Both of us began our film work four years ago] , John and James Whitney Estate, 1943, 3. 3 John Whitney, Digital Harmony. On the Complementarity of Music and Visual Art (Peterborough, New Hampshire: Byte Books, 1980), 21-22. –2–

Of higher significance was Whitney’s stay in Paris. There he was a neighbor of René Leibowitz, the conductor and composer, who at that time had just begun to absorb Arnold Schoenberg’s 12-tone technique. The two young men – Leibowitz was Whitney’s senior by just four years – met two or three times a week over a period of several months, during which Whitney gained a broad knowledge of serial composition techniques. 4 Back in California, he would buy a copy of Ernst Krenek’s Studies in counterpoint. Based on the Twelve-Tone Technique right after it’s publication by Schirmer, New York in 1940. 5 During his stay in Paris, John Whitney also started experimenting with abstract animation. Again, he wasn’t aware of earlier work done in this field and didn’t know of Oskar Fischinger e.g., at that time a refugee living in Los Angeles. But while Fischinger’s abstract animation visualized the progression of melodies and motives mostly of classical music, Whitney was interested in applying the concept of serial music composition to the changing film images themselves. The first outcome of this artistic enterprise were the Twenty-Four Variations , a 8mm color silent of five minutes duration, which he created together with his younger brother James after his return to Los Angeles an 1939/40. Here, basic geometric shapes move in simple ways, thus creating visual- kinetic motives. As the title suggests, they are subjected to a circle of variations, in which’s course speed and direction are changed, shapes and/or movements are mirrored, inverted and superimposed. An array of forms, from which the variations were organized, suggests the nearness of the Whitneys approach to form in time to the serial thinking of 12-tone technique. It is not as surprising as it might seem at first glance, that an essay John Whitney had written on his early work, was first published in German translation in the journal Die Reihe , the famous organ of the serial school of post-war Cologne, edited by Herbert Eimert and Karlheinz Stockhausen. 6 Another artistic framework of importance to the Whitney brothers was the work of Paul Klee, most notably Klee’s thoughts on line, shape, time and space in drawing, as expressed in his Pädagisches Skizzenbuch of 1925. 7 In his instructional graphics, Klee explained how opaque shapes such as squares and circles can be understood as derived from moving or rotating lines . This concept of so-called “passive lines” is of double interest here. First, it might have served as an inspiration to not only move shapes in time, enlarge and diminish them, but to change the shapes themselves in a disciplined and structurally clear manner. Secondly, the concept of understanding shapes as derived from movements of lines might have inspired the use of pantographs for creating the abstract animation. Klee also spoke of visual structure in terms of rhythm. Hereby, he implied a succession of elements in time, even though the elements of a drawing are of course visible to the observer simultaneously. It is to the observer to structure his or her experience in time. In what they called a “visual time art” 8, the Whitney brothers were able to subject what Adorno called the “Zeitkern”, the time core of the art work, 9 to their artistic organization.

Animation Technique

For creating the abstract animation of their Five Film Exercises , the Whitney brothers basically made use of the technique they had developed for the Twenty-Four Variations. “Two or more shapes cut

4 Ibid., 174. 5 Ernst Krenek, Studies in counterpoint. Based on the Twelve-Tone Technique (New York: Schirmer, 1940). 6 John Whitney, "Bewegungsbilder und elektronische Musik", Die Reihe , no. 7 (1960): 62-72. 7 Paul Klee, Pädagogisches Skizzenbuch , Bauhausbücher 2 (München: Langen, 1925). 8 James Whitney and John Whitney, "Audio-Visual Music", Arts and Architecture 61, no. 12 (Dec. 1944): 28, 42, 28. 9 Theodor W. Adorno, Aesthetic Theory , ed. Gretel Adorno and Rolf Tiedemann, trans. Robert Hullot-Kentor (London - New York: Continuum, 2004), 174-198. –3– out of opaque papers are manipulated one over the other with a diffused area of light underneath. [...] The only image recorded by the camera is that part of the light area contained within the cut-out shapes of the opaque papers: i.e. where the cut-out portions overlie each other permitting the light to pass through [...].” 10 The stencils were moved by means of two pantographs. The use of pantographs allowed to precisely transfer a movement path, laid out beforehand and followed by a tracer, to a stencil. Which stencils where to use and what movements they had to follow, was defined in a score. Thus, “thematic raw material” 11 was produced on black and white 16mm film. To build the desired sequences, this material was transferred to color film by means of an optical printer. Thereby, speed, size and orientation could be changed, color added and several images could be superimposed. The complexity of this process made it difficult for the artists to score their work, which makes it difficult for us to decode the animation sheets. The surviving score material of Five Film Exercises consists of some 400, mostly single-sided folio of letter size paper. Image and music are usually scored on different pages, and when too many actions are superimposed, more than one page per track is used. Image and music are linked together by the frame count, however, the music often starts before the image. The offset of image and sound on the final print is sometimes indicated (e.g. fol. A-155), but usually taken for granted. Fol. B-109v shows the beginning of the third section of Film Exercise #1. On the left, time is indicated by frames and feet, thereby representing 20 seconds. We see five visual movements superimposed, with numbers indicating the stencils in use, while figures that follow the letter R refer to a movements path – they stand for “radical action”. Rhomb shaped lines, finally, symbolize the change of size and/or focus in the optical printer.

Sound

Before I focus on the means of sound production in the Five Film Exercises , I would like to play the beginning of Exercise #4 (film) . The main undertaking of the Five Film Exercises was the integration of image and sound in a truly audio-visual art form. As the Whitney brothers stated in the catalog of the first Art in Cinema festival at the San Francisco Museum of Art in 1947: “It is a commonplace to note that film and sound today have become a permanent unity. We are attracted by the prospects of an idiom as unified, bi- sensorially, as the sound film can be.” 12 And in working notes from 1943 they declared: “Since our objective is organization on strictly new lines which we believe to be inherent within the medium itself, we find it necessary to reject all existing music. [...] We believed [sic] that ultimately a completely interrelated audio-visual relationship would be possible only when both elements are under control of one composer.“ 13 – Incidentally, the design of abstract animation was hereby integrated into the composer’s field. “Thereafter, little thought was given to any other consideration than to search for a method of creating our own sound by some means near as possible to the image animation process, technically and in spirit.” 14 As László Moholy-Nagy, Oskar Fischinger, Norman McLaren and Rudolph Pfenninger (no completeness intended) had done before, John and James Whitney set about to create sound by directly designing the optical soundtrack. While Moholy-Nagy did not further develop his more conceptual approach of reading images such as letters and fingerprints as music and the other

10 James Whitney and James Whitney, A Method of Producing Animated Abstract Images , John and James Whitney Estate, 1943, 1. 11 Ibid., 3. 12 Whitney and Whitney, "Audio Visual", 32. 13 Whitney and Whitney, [Both of us began our film work four years ago] , 3. 14 Whitney and Whitney, "Audio Visual", 32 –4– advocates of animated sound after initial intermodal experiments soon focused on the creation of tonal music by photographing frequency templates on the optical soundtrack, the Whitney brothers stood out with their approach to create music “inherent within the medium itself.” 15 They consulted Carlos Chavez’s book Toward a New Music of 1937, that with its chapter on sound transmission and electronic sound production was also influential to John Cage, and were “stimulated [...] to seek a music source within the film itself.” 16 For that, between 1940 and 1943, John and James Whitney designed and built a device, with which the movements of swinging pendulums were recorded on film. The pendulums, varying in size from 4 feet to about 6 inch long, are linked by a steel wire that is attached to a triangular wedge. The opaque wedge is oscillating directly above a light slight, so that its movement creates a variable area sound track. The soundtrack is recorded at very low speed. Thereby, the subsonic sound of the slow swinging pendulums gets transposed into audible range when the film is played at regular speed The pendulums can be tuned by adjusting of sliding weights. Altogether, the resulting sound depends on the pendulums operated, their tuning or scale, and the speed of the film on which their movements are recorded. A frame counter (in the image on the upper right) becomes an essential device for measuring the resulting time. Being influenced by the concept of 12-tone technique, John Whitney tuned the 12 pendulums according to the diatonic scale (fig: fol. 1r - Tuning 1). For different sections different tunings were established, e.g. Tuning 0 for the beginning of Film Exercise #4 (fig: fol. 217r - Tuning 0). The table indicates the intervals – 5:6 for minor thirds and 9:16 for minor sevenths – and the positions of the adjustable weights, measured from the bottom of the pendulums. Unfortunately measured from the bottom, since this makes the calculation of resulting intervals quite difficult (fig: fol. 218r – Tunings). Let’s have a look at the beginning of the sound score of Film Exercise #4 (fig: sd IV-1-t2 fol.180). Every 10 frames, a new pendulum comes in, swinging a minor third faster for 40 frames, the sixth for 50, the seventh for 60, and so on. At frame 180, three pendulums swing, and their transposition is changed abruptly to 9. From frame 270 on, we see short glissandi. The colon on the very right stands for dynamics, that is the width of the variable area sound track. This part is superimposed with a second recording (fig: sd IV-1-t1 fol.181), that starts at frame 180. Here, the same three pendulums 9 to 11 swing, but while the recording speed was 9 before, this time we have a vibrato oscillating between 9 and 7, that later leads into a slow glissando downwards to 4. A spectrogram illustrates these observations (film: Whitney sonagram.swf).

Conclusion

The research that I presented in this paper is just a starting point for further analytical observation of the inter-medial approach to abstract film John and James Whitney established in their Five Film Exercises . Since the surviving score material contains some sections in differing versions, we might be able to gain some understanding of the genesis of this outstanding work of art, that its authors considered the foundation of a new art form. Especially the problematic scoring of the moving image makes it difficult to reveal the precise interdependence of sound and image. That structural units in sound and image often do not match, is, according to the score, no coincidence. Often the sound anticipates structural turning points in the image, thereby avoiding the simple doubling of both media, that had been criticized by avant-garde filmmakers like Moholy-Nagy or Hans Richter from early on. That this separation, or relative free relation, of sound and image also follows serial calculations, seems to be suggested by certain parts of the score. If this is a consistent underlying principle of the Five Film Exercises remains to be seen.

15 Whitney and Whitney, [Both of us began our film work four years ago] , 3. 16 Ibid., 4.