Visual Music: Searching for an Aesthetic
Total Page：16
File Type：pdf, Size：1020Kb
Visual Music: Searching for an Aesthetic Tom DeWitt Abstract-If the nonverbal and emotionally evocative aesthetic of music is to be extended to visual art, we must examine the psychology of sight and the intuitive uses of this psychology by visual artists. The structure of the eye, the nerve connections between the eye and the brain, and the visual cortex can give us clues as to how human perception is uniquely sensitive. These sensitivities, amplified through the self-expression of artists, can induce emotional responses that are more related to the form of the expression than to the content of the imagery. Works by the author and by other artists who influenced him are used to illustrate these points. I. INTRODUCTION apparatus simplifies some image processing when the modulating frequency is at integral During the nineteenth century, Helmholtz [1] problems, particularly geometric rotation. Their multiples of the sine wave frequency. The studied the relationship between musical model of visual perception and cognition seems pattern in the petal leaves is determined by the harmony and the human perceptual apparatus. to be related to an aesthetic that I developed timbral qualities (wave shape) of the His discoveries can guide an investigation into independently in my art. modulating wave form. An image made by this the relationship between music and visual art, In 1974 I produced a work called technique is shown in Fig. 1. Since the process not because he uncovered a direct correlation, Philharmonia [4], which was based on the was realized with an audio synthesizer, I was but because his method of analysis can be used logarithmic spiral. It was realized using an able to record a sound track simultaneously. to find one. In a key finding, Helmholtz audio synthesizer to draw pictures. If a sine and My endeavors in this area were by no means concluded that we appreciate the geometric its cosine wave form are displayed on an the first. Probably the most impressive formal progression in sound frequencies because our oscilloscope as the vertical and horizontal body of work of this type was realized by the ears seem to produce these overtones even in inputs, a circle or oval will be produced. If Whitney brothers, James and John Sr. I had the absence of their physical presence. In other these wave forms are amplitude modulated by been influenced by these artists at the words, we enjoy the art born from the subtle wave forms at frequencies higher than the sine beginning of my filmmaking career, and in manipulation of our aural psychology. wave frequency, the circle is pinched into a 1965 I approached John seeking an If we apply this methodology to sight, it is shape like a flower or a rose window. The apprenticeship. He declined my request but did reasonable to suggest that the eye has intrinsic petals move dynamically but are stable physical properties that point toward an aesthetic. The centric structure of the retina with its logarithmic density of sensor cells radiating from the fovia, the cone and rod retinal cells that are sensitive respectively to color and monochrome, the crossing of left/right eye neurons in the optic nerve chiasma, and the processing of the visual cortex itself are all unique psychological phenomena that must be examined. II. VISUAL HARMONY Interesting work has been pursued by Chaikin and Schwartz at the New York University Brain Research Lab, showing how visual processing might work [2, 3]. Noting that cells in the eye and brain are distributed in a polar coordinate system, as opposed to the Cartesian coordinates that usually are used in computational geometry, they postulated that our visual Tom DeWitt (multi-media artist), Image Processing Laboratory, Rensselaer Polytechnic Institute, Troy, NY 12180, U.S.A. Received 7 July 1986. © 1987 ISAST Fig. 1. Frame from Studies for Philharmonia, videotape, 1974 showing a pentatonic petal made using Pergamon Journals Ltd. the algorithms of digital harmony. The lobes of the flower correspond to the harmony of a Printed in Great Britain. simultaneously created musical composition. 0024-094X/87 $3.00+0.00 LEONARDO, Vol. 20, No. 2, pp. 15-122, 1987 share his teaching with the entire world by piano. If chordal structures are desired during the high-contrast, black-and-white film stock publishing a treatise on the subject, Digital a piano performance, the physical size of two used to record the image. Harmony [5], in which he described his hands is imposed on the composition. I The superimposition of a color pattern simple and effective algorithms for producing propose that we construct cluster keyboards over a black-and-white pattern places the a pleasing time variant image by modulating based on that more recently evolved digital color pattern in sharper relief. In Koan [11], spirals. These can be described as digital interface, the typewriter. With a richly saturated flesh tones in the foreground computer programs, providing a two-dimensional keyboard, virtually any 10 image are juxtaposed over muted much-needed notation for the emerging notes could be played simultaneously. backgrounds (Color Plate No. 1). The eye visual music art form. When computers are used to make visual sees the muted background as distant and In my efforts to notate, I resorted to or aural music, they can be programmed to remote. Yet the background motif was conventional musical practices, a linear strip create a polyphony from a single command. actually derived by electronically tracking the of notes on paper. I used graph paper and a Consider the implementation of a graphic movement of the foreground subject, a form of notation explored in this century by mouse by Laurie Spiegel [7]. In her couple clapping hands. Technically speaking, such artists as Villa-Lobos and Gershwin— computer-based instrument, the the background in this image is an integral skyline notation. The choice was premised on two-dimensional position of the graphic part of the foreground image, but the use of a an automated score reader I was developing. screen cursor, as controlled by a mouse or muted color scheme for the background The notated graph paper was placed on trackball, triggers the production of chordal resulted in a clear sense of separation registration pins, much like those used in cell structures. I have proposed a similar between the hands in the foreground and the animation, and a video camera converted it to performance controller based on my tracking sky-like background their movement created. an electronic signal. An electronic circuit system, Pantomation. In this system, the Perhaps because the monochrome retinal reduced the video image to a set of timed three-dimensional movement of a conductor's neurons, called rods, are more sensitive to pulses which corresponded to the position of baton is interpreted by a music program as an light than the color-sensitive cones are, the each note on each line, and this could be read interactive control for changes in the music transition from black and white to color can by a small computer. [8]. produce the sensation of awakening. As our My idea was to liberate the artist from the level of visual perception grows in increased electronic music studio by allowing notation amplitudes of light, visual perception at the piano or simply from memory, just as III. COLOR changes from black and white to color. My composers score for orchestras. I started to film, The Leap [12], pivots on this develop the machine to read scores in 1976, Universal agreement on a color aesthetic monochrome to color transition, in that the when electronic music equipment was only has not yet been reached, nor is there a set of moment of denouement—a leap from slightly more accessible to composers than rules correlating music tones and visual confusion to enlightenment—is accompanied professional orchestras. By the time the work colors. This contrasts with Western music by the introduction of color into the motif. was published in 1981 [6], the modest price theory, which long ago settled on a set of The reverse transition from color to black and of home computer systems had provided aural frequencies, suggestively named the white leaves a feeling of release into twilight. composers with accessible tools for chromatic scale, upon which generations of I suggested this to Dean Winkler when he composition. Digital music synthesizers with composers have built aesthetic architecture. was producing First Experiment with Bryon's piano keyboards have since become John Whitney reflects on this difficult Solids [13], and the effect can be observed in ubiquitous. problem of achieving a formalism for color his tape's finale: intensely colored geometric My notation for visual harmony presumed when he writes: objects transform to black and white before that a composition was to be constructed they fade out completely. from a fixed vocabulary of visual notes, much I confess my own puzzlement Another interesting psychological effect of like the diatonic scale. John Whitney affirms despite nearly forty years of color black and white seems to be its association this assumption of mine. In a manner filmmaking . One propounds with the factual as opposed to the emotional. remarkably similar to aural harmony, theories for the use and effect of Our emotions are often described as colors, logarithmic spirals have stable structures only color; I make a new plan with each e.g. green with envy, blue with heartbreak, at fixed intervals of frequency. If a piano film. Rarely have the best ideas red with anger. Newspapers and journals, by keyboard is used for a musical performance, lived up to expectations [9]. way of contrast, fulfill their primary roles it can be used simultaneously to produce without the use of color.