Sonic Images and Visual Poetics
Total Page:16
File Type:pdf, Size:1020Kb
Sonic Images and Visual Poetics Exploring a methodology for sound and image relationships by Martha W. Swetzoff BA, Social Anthropology Harvard College, Cambridge, Massachusetts 1984 Submitted to the Media Arts and Sciences Section in partial fulfillment of the degree of Master of Science in Visual Studies at the Massachusetts Institute of Technology June 1989 Copyright © 1989 Martha W. Swetzoff. The author grants MIT permission to reproduce this thesis in whole or in part. Signature of Author Martha W. Swetzoff Media Arts and Sciences Section February, 14, 1989 Certified by Glorianna Davenport Assistant Professor of Media Technology Thesis Supervisor Accepted by Stephen A Benton Chairman Departmental Committee on Graduate Students MASSACHUSETTS INSTITUTE OF TECHnt OGY OCT 2 3 1989 LIBRARIES Rawc Room 14-0551 77 Massachusetts Avenue Cambridge, MA 02139 Ph: 617.253.2800 MIT Libraries Email: [email protected] Document Services http://libraries.mit.edu/docs DISCLAIMER NOTICE The accompanying media item for this thesis is available in the MIT Libraries or Institute Archives. Thank you. Sonic Images and Visual Poetics by Martha W. Swetzoff Submitted to the Media Arts and Sciences Section on February 14, 1989 in partial fulfillment of the requirements for the degree of Master of Science in Visual Studies. Abstract This thesis emerged in parallel from specific ideas within musical and cinematic traditions. In the course of developing a methodology for working with sounds and images I completed a half-hour video and sound piece, entitled "Tales of Love and Glory" (TLG), included as part of this thesis. This project brought up artistic dilemmas for which the existing models of sound production in the traditions of film and video failed to present satisfactory aesthetic and technical solutions. In the search for a new paradigm I explored the growing field of computer music. There, research and methodology encompass a level of scientific description of sonic materials, ideas about higher-level cognitive processes of auditory perception, and musical aesthetics. This approach offers a powerful set of theoretical and technical tools for artists, composers and sound designers engaged in creating a language of sound and picture relationships for motion pictures. Included in this thesis is a discussion of two digital audio workstations: the AudioFrame, designed by the WaveFrame corporation, and the Synclavier, designed by the New England Digital corporation. Thesis Supervisor: Glorianna Davenport Title: Assistant Professor of Media Technology ACKNOWLEDGEMENTS The author would like to thank the following individuals and corporations: Steve Cunningham Steve Krampf Andrew Currie The WaveFrame Corporation Ted Pine The New England Digital Corporation Curtis Roads Adrian Freed Tod Machover Glorianna Davenport Alan Lasky Sabrina Birner Bruce Middleton-Baird Alan Ruttenberg Rainer Boesch Jean-Claude Risset Gwen Gordon David Heeger Contents PARALLEL STREAM S ..................................................................... 6 Introduction ............................................................................. 6 A SHORT ESSAY ON TIM E...............................................................7 "Tales of Love and Glory"................. ............................... ... 11 The sources of artistic dilemma..................................12 The Fragmentary Behavior of Memory ...... .......................... 15 Musical streams.......................................... 16 Traditional film sound technique ............................ 17 M usique Concrete ................ ........... .................................... 17 Musique Concrete in the cinema.............................19 Wishart and Aural Landscape.............................22 Sound images as metaphors.....................................25 Moving from the tape recorder to the computer..................27 How do sounds carry meaning? . ....... ...... ...... ...... 31 MEANING AND INVENTIVENESS IN SOUNDS AND PICTURES....... 34 The importance of the photographic image.................................. 35 The inventiveness of sight .................................. 37 THE PROCESS OF CREATING A UNITY OF SOUND AND IMAGE......40 Montage and synchronization.................................. 40 Synchronization as an extension of cinematic language ............ 41 Sound and Image as formal relations..........................................44 Towards a cinema of the mind: poetics ..................................... 47 "Tales of Love and Glory"...... .. ................................... 51 SEARCHING FOR THE RIGHT TOOL...............................53 Constraints ........ ..................................... 53 The nature of physical objects helps clarify needs...................57 Developments that push the technology..................... ........ 59 History of the technology............... ... ..................................... 60 Soundroid..... ....... .... ........ .......................................... 61 Current developm ents ............................................................ 65 SPECIFICS ABOUT THE SYSTEMS..............................66 Conditions of the Evaluation. ..... ................................... 66 The A udioFram e......................................................................67 Analog-to-digital m odule......................... ..................... .......... 68 Digital-to-analog module...................................68 Sampling synthesis module........................................................ 68 14-Megabyte memory expansion module ...................................... 68 The Digital Audio Bus......................o ........................ 6 Software................................................ .70 SoundProcessor..................................... ............ 70 EventProcessor .......................................... 73 M ID Icad ..................................................................................... 74 Soundstore .............................................................................. 74 A metaphor for the overall software design ..................................... 75 Overall Evaluation.....................................................................76 Mental Model............................................................................77 DSP pros and cons...................................................................... 78 SYNCLAVIER ............................................................................... 78 The Synclavier as a tool.................................................................81 Evaluation.............................................. 83 Storage media...........................................................................88 Marketplace Considerations.......................................................88 Personal musings ........................................................................ 90 CONCLUSION ............................................................................... 90 ILLUSTRATIONS .......................................... 98 BIBLIOGRAPHY .. .. o-........ .......-.......................... o......... 113 PARALLEL STREAMS Introduction This thesis emerged in parallel from specific ideas within musical and cinematic traditions. In the course of developing a methodology for working with sounds and images I completed a half-hour video and sound piece, entitled "Tales of Love and Glory" (TLG). This project brought up artistic dilemmas for which the existing models of sound production in the traditions of film and video failed to present satisfactory aesthetic and technical solutions. In the search for a new paradigm I explored the growing field of computer music. There, research and methodology encompass a level of scientific description of sonic materials, ideas about higher-level cognitive processes of auditory perception, and musical aesthetics. This approach offers a powerful set of theoretical and technical tools for artists, composers and sound designers engaged in creating a language of sound and picture relationships for motion pictures. I became interested in the language of sonic images in contemporary music. This direction of compositional thought embraces timbres, or sound colors, and sound-objects, as opposed to traditional musical notation, as the building blocks of composition. This branch of musical thinking has been greatly affected by the introduction of the computer as a means to analyze, order, synthesize or modify sonic materials. Works such as Sud, by Jean-Claude Risset, and Mortuos Plango, Vivos Voco, by Jonathan Harvey, excited me because they contained extremely supple transformations of naturally- occurring sounds. I became interested in the way, due to the precise control offered by the computer, perceptual illusions might evolve from such treatments. I became particularly intrigued with combining this approach to sound and music composition with moving pictures. The first half of this thesis begins with a short essay on time. I then discuss the artistic conditions which provoked this search, the background of traditional sound use in motion pictures, and the work of the composers Trevor Wishart, Jonathan Harvey, and Jean-Claude Risset as examples of a new methodology. The first section ends with a discussion of visual concepts that have informed this thesis work. These include, but are not restricted to, certain areas of photography and cinema. "Tales of Love and Glory" incorporates a diverse set of materials,