INFORMATION TO USERS
The most advanced technology has been used to photo graph and reproduce this manuscript from the microfilm master. UMI films the original text directly from the copy submitted. Thus, some dissertation copies are in typewriter face, while others may be from a computer printer.
In the unlikely event that the author did not send UMI a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyrighted material had to be removed, a note will indicate the deletion.
Oversize materials (e.g., maps, drawings, charts) are re produced by sectioning the original, beginning at the upper left-hand corner and continuing from left to right in equal sections with small overlaps. Each oversize page is available as one exposure on a standard 35 mm slide or as a 17" x 23" black and white photographic print for an additional charge.
Photographs included in the original manuscript have been reproduced xerographically in this copy. 35 mm slides or 6" x 9" black and white photographic prints are available for any photographs or illustrations appearing in this copy for an additional charge. Contact UMI directly to order.
Accessing the World'sUMI Information since 1938
300 North Zeeb Road, Ann Arbor, Ml 48106-1346 USA Order Number 8824447
Part I. Using a digital soundfile mixing program as a compositional tool. Part II. "Extensions” for concert band and tape. [Original composition]
Yocom, Neal Wesley, D.M.A.
The Ohio State University, 1988
Copyright ©1988 by Yocom, Neal Wesley. All rights reserved.
UMI 300 N. Zeeb Rd. Ann Arbor, MI 48106 PLEASE NOTE:
In all cases this material has been filmed in the best possible way from the available copy. Problems encountered with this document have been identified here with a check mark V .
1. Glossy photographs or pages ______
2. Colored illustrations, paper or print ______
3. Photographs with dark background _____
4. Illustrations are poor copy ______
5. Pages with black marks, not original copy ______
6. Print shows through as there is text on both sides of page ______
7. Indistinct, broken or small print on several pages ^
8. Print exceeds margin requirements ______
9. Tightly bound copy with print lost in spine ______
10. Computer printout pages with indistinct print ______
11. P age(s)______lacking when material received, and not available from school or author.
12. P age(s)______seem to be missing in numbering only as text follows.
13. Two pages num bered . Text follows.
14. Curling and wrinkled p ag es ______
15. Dissertation contains pages with print at a slant, filmed as received ______
16. O t h e r ______
UMI Part I: USING A DIGITAL SOUNDFILE MIXING PROGRAM AS A COMPOSITIONAL TOOL
P art II: EXTENSIONS for Concert Band and Tape
D.M.A. Document
Presented in Partial Fulfillment of the Requirements
for the Degree Doctor of Musical Arts
in the Graduate School of The Ohio State University
By
Neal Wesley Yocom, B.M.E, M.A.
oo oooooooooooooooooo
The Ohio State University
1988
Dissertation Committee:
Dr. David Butler Approved by
Dr. Gregory Proctor Dr. Thomas Wells, Adviser Dr. Thomas Wells School of Music Copyright by Neal Wesley Yocom 1988 for my wife
ii ACKNOWLEDGMENTS
Thanks are due to many colleagues and friends who have supported my work, in particular Rocky and Judy Reuter and Chris Chater. Special thanks are certainly due to two persons at IRCAM who helped me greatly, David Wessel and Marco Stroppa.
Without the assistance of Marc Battier, Jan Vandenheede, Denis Lorrain, Thierry
Lancino, and the others at IRCAM who assisted me with my research, the results of this investigation would not have been nearly as well grounded in practice.
I would like to thank the members of my dissertation committee, Dr. David
Butler and Dr. Gregory Proctor, for their personal interest and assistance, and for their exemplary instruction over the past years. My adviser, Dr. Thomas Wells, has given vital encouragement to my work, insight into the craft of musical composition, and a new appreciation of the music of the masters. I am glad to have had the opportunity to study and work with a musician and composer of his stature.
My wife, Judy, has provided continuous positive support during the research and preparation of this document, without which its completion would not have been imaginable. VITA
Oct. 3, 1946 ...... Bom: Bryan, Ohio
1968 ...... B.M.E., Music Education: Baldwin-Wallace College
1968-72 ...... Teacher, Instrumental Music: Mulligan Jr. High School, Central Islip, New York
1973-7 4 Director of Bands: Hauppauge High School, Hauppauge, New York
1974 ...... M.A., Music: C.W. Post College, Long Island University Greenvale, New York
1974-7 9 ...... Instrumental, Choral, Classroom Music: The Frankfurt International School, e. V., Oberursel, West Germany
1979-82 ...... Instrumental Music Director: American School of Paris, 92210 St. Cloud, France
1982-85...... Graduate Teaching/Research Associate: School of Music, The Ohio State University, Columbus, Ohio
1985-198 8 ...... Research in Computer Music: Institut de Recherche et Coordination Acoustique/Musique (IRCAM), Paris, France
1986-198 8 ...... Instrumental Music Director: American School of Paris, 92210 St. Cloud, France TABLE OF CONTENTS
Page
VITA ...... iv
LIST OF TABLES...... viii
LIST OF FIGURES...... ix
ABSTRACT...... x
PART I: USING A DIGITAL SOUNDFILE MIXING PROGRAM AS A COMPOSITIONAL DEVICE
CHAPTER
I. Introduction to Mix-Oriented Composition ...... 1
Basic Assumptions ...... 3
Definition of Term s ...... 4
Limits of the S tudy ...... 6
II. The Digital Audio Environment ...... 7
Introduction ...... 7
Random-Access Editing of Digital Audio ...... 9
The IRCAM Setting ...... 16
Sum m ary...... 20
v CHAPTER
III. The Mix-Oriented Compositional Environment ...... 21
Introduction ...... 21
Physical Components of the Environment ...... 21
Software Components of the Environment ...... 26
The IRCAM Mix Program ...... 33
U sing M ix ...... 34
Inside Mix: Design Features of the Program ...... 37
Using Mix as a Soundfile Manipulation T ool ...... 39
Command File Generation ...... 41
Creating and Altering Mixscores via Awk ...... 41
LISP and C Programs ...... 44
M acM ix ...... 45
Unusual Applications of M ix ...... 45
Some Technical Considerations ...... 46
Summary...... 49
IV. Using Mix as a Compositional Tool: Small-Scale Techniques... 50
Introduction ...... 50
Composition via M ixing ...... 50
Small-Scale Compositional Mixing Techniques: ...... 51
Splicing ...... 51
Crossfading ...... 52
L ooping ...... 61 CHAPTER IV (continued)
Layering ...... 64
Fragmentation ...... 73
Spatial Considerations ...... 73
Sum m ary...... 75
V. Using Mix as a Compositional Tool: Large-Scale Techniques ..77
Introduction: Prolongation and Extension ...... 77
Streaming and Musical Continuity ...... 77
Grouping Mechanisms and Musical Continuity ...... 79
Extension via Pitch Matching ...... 81
Extension via Timbre/Texture Matching ...... 83
Extension via Rhythm M atching ...... 86
Examples of Mix-Oriented Compositional Techniques in Extensions ...... 87
Summary...... 92
PART II: Extensions for Concert Band and Computer-Generated Tape
Musical Score ...... 93
APPENDICES
A. List of Mix Flags and Keywords ...... 136
B. Programs Used To Generate Mixscores ...... 138
BIBLIOGRAPHY ...... 144 LIST OF TABLES
Table Page
1. Disk Storage Devices, Filesystems on Erik ...... 24
, 2. Comparison of Relative Speeds of Mix and Mix4 ...... 40
3. Comparison of Execution Times: Integer vs. Floating Point M ixes ...... 48
4. Frequency Cancellation and Reinforcement Resulting from Delays 66
viii LIST OF FIGURES
Figure Page
1. Erik-VAX 780 System ...... 23
2. Conceptual Model of Computer Music Compositional Environment ...... 27
3. Physical Configuration of Computer Music Compositional Environment 28
4. IRCAM Soundfile Design ...... 32
5. Mix Control Flow ...... 38
6. Overflow Bit Arrangements ...... 47
7. Information Revealing Nothing About Loudness ...... 53
8. Results of Crossfading Using Various Slope Functions ...... 55-59
9. Looping Techniques ...... 63
10. Frequency Cancellation and Reinforcement Resulting from Delays 67-68
11. Amplitude Differences Due to Phase Cancellation and Reinforcement 69
12. Spectral Plots Showing Comb Filtering Effects ...... 70-72
13. Gestalt Grouping Principles ...... 80 ABSTRACT
Often, the final step in the creation of a computer music composition is the mixing of a number of carefully prepared digital soundfiles into a definitive performance. At this stage, the composer is working with a familiar repertory of sounds, creating a finished composition by arranging these selected sounds in time and space. A digital soundfile mixing program can be used to arrange the sounds precisely as specified by the composer. This treatise proposes a mix-oriented compositional environment, where the focus of compositional activity is on the manipulation of sounds and the creation of whole pieces, rather than on the synthesis of individual sound events.
Rather than being an exhaustive investigation of many compositional mixing techniques, this study focuses on a few techniques at various levels of the compositional process, describing some useful procedures for both small- and large-scale applications.
This treatise is intended to provide an introduction to the compositional possibilities of mixing and a model for developing yet other techniques and tools for mix-oriented composition.
The second part of this document is a musical composition, Extensions , for
Concert Band and Tape.
x CHAPTER I
INTRODUCTION
Often, the final step in the creation of a computer music composition is the mixdown of a number of carefully prepared digital soundfiles into a definitive performance. At this stage, the composer is working with a familiar repertory of sounds, creating a finished composition by arranging these selected sounds in time and space. A digital soundfile mixing program can be used to arrange the sounds precisely as specified by the composer. If further sound conditioning is required, signal processing programs can be used to quickly carry out the desired alterations.
The composer working with synthesis programs to generate sounds often spends a great deal of time experimenting to find usable parameter settings for those programs.
The computation : time ratio is often great and the results are often not predictable.
Although it is theoretically possible for a single software synthesis program to be used to create an entire composition, in reality this is seldom the case. A more typical approach to using software synthesis programs is to discover useful parameter values by making numerous test runs, saving successful test soundfiles for eventual inclusion in a composition. The output of a given synthesis program usually requires some further processing (i.e., reverberation or altered spatial placement) to satisfy the composer’s needs. At some point, a mixing program must be used to assemble the chosen sounds into a finished composition.
1 The composer who works with a digital soundfile mixing program has several
significant advantages over another working with analog tape recorders. In addition to
the advantage of not having the physical problems involved with managing tapes, by
working exclusively with digital signals there is no signal loss problem due to signal
degradation during transfers; a composer can make unlimited successive copies of a
digitally stored signal with no signal loss. Since digital soundfiles can be processed in
random-access memory, the composer may select precisely any part of any soundfile for
mixing, listening, or viewing. New soundfiles can be created by editing and mixing
existing soundfiles without having to resort to elaborate synchronization techniques, as
is the case with analog tapes. Yet trial-and-error experimentation and spontaneity is
facilitated; the composer is able to keep successful mixes and discard others.
A mix-oriented approach to composition puts the focus of the compositional act
on the arranging of sounds into a composition, allowing for intuitive, direct control of
the sound material. If the source sounds are well known to the composer, compositional decisions can be made away from the computer. The composer may choose to work at any level, from the whole piece to the single sample, either conceptually or practically.
High-level programs that implement compositional algorithms can be used to create text files containing mix commands—mixscores—to direct the work of the mixing program.
The mixing program itself may be used as a signal processor capable of reshaping the amplitude envelope of a soundfile, altering the spatial placement of sounds, creating chorus and delay effects, transposing and making other transformations. By mixing together sounds with their transformations the composer can produce a great number of useful, related sounds. A mixing program may also be used to overcome some of the limitations of software synthesis programs by making it possible to select and combine usable portions of soundfiles that may not be usable in their entirety. Thus, a mixing 3 program, which allows for the quick editing, playback and transformation of soundfiles, is an essential tool in a computer music composition environment.
At the Institut de Recherche et Coordination AcoustiquelMusique (IRCAM) in
Paris, a powerful, yet easy to use program for mixing digital soundfiles, Mix, has enabled composers such as Stephen Beck and Trevor Wishart to create finished compositions within the computer system. Using Mix and the signal processing programs that are available, composers have been able to transform and extend a limited amount of original sonic material into a large body of related sounds. An important feature of this working environment is the possibility for the composer to select freely among the various soundfile manipulation and signal processing programs, using one program at a time or creating complex sequences of operations via UNIX shell scripts.
In a mix-oriented compositional environment, the com-position of sounds into a finished piece is both the immediate activity and the long term goal. This study will focus on two principal topics:
1. What are the components of a mix-oriented compositional environment?
2. What are some useful compositional strategies for such an environment?
Basic Assumptions
This study is concerned with compositional activity in a time-sharing computer music environment, as can be found at IRCAM and at a number of American universities. The essential features of this mix-oriented compositional environment include:
• A minicomputer system, such as the VAX 750 or 780. Workstations or
other systems could be used if sufficient soundfile storage and
other components of the environment are available. • Sixteen-bit analog-to-digital (ADC) and digital-to-analog (DAC)
conversion; arbitrary sample rates up to 48kHz stereo should be
supported by play and record programs.
• A soundfile storage and manipulation system, built around random-
access rotating disk storage media.
• A program to mix soundfiles, such as the Mix program at IRCAM.
• Software synthesis and signal processing programs, such as those
found in the University of California at San Diego CARL/cmusic
system.
Definition of Terms
A soundfile is a sound, stored in a computer system as a sequence of discrete
values, or samples, representing the sound pressure wave at each sample time. A
soundfile may be the output of a software synthesis program or the result of sampling a
natural sound to convert it to digital data. “Sound” will be used interchangeably with
“soundfile” in this study, in order to emphasize the importance of keeping sound manipulation as the primary concern. The difference will be made explicit in places where the context does not provide sufficient clarity.
The number of samples per second, the sample rate, is expressed in Hertz, or
“Hz”. The most common sample rate used for soundfiles at IRCAM, for example, is
32000 Hz, or 32 kHz. This study supports the convention that the small letter “k” means “times 1000,” and the large letter “K” means “times 1024”. Although this difference is not always observed in the literature, the difference is significant and will be adhered to here. Furthermore, the letter “K” (or, sometimes, “Kb”) is often used to indicate a number of 8-bit bytes when referring to computer memory or storage sizes. A tool is a device used to accomplish a specific task. The digital soundfile mixing program, Mix, is a tool used to combine soundfiles. If the normal usage of a tool is known, then an unusual implementation of the tool, perhaps in combination with other tools, to accomplish complex or unusual tasks can be conceived. However, if a tool’s appearance or its implementation is complicated, then unusual or extended implementations may not be as easily conceived. This study investigates both “normal” and unusual compositional applications of a soundfile mixing program.
The user interface, the appearance of the program to the user, may be thought of as a tool, because it affects how the user forms conceptual models for the programs
(tools) used. The user interface has a great impact on how tools are designed and used.
Mix is an example of a command-driven program. This sort of design allows the user to decide what is to be done and then to specify the commands, chosen from a limited menu, that will accomplish the task. This design also allows a composer to use programs that create text files containing lists of commands, eliminating the need to specify every single command when creating large or long mixes. In this manner, larger-scale composition is encouraged.
A computer music compositional environment is a physical environment consisting of physical devices—computers, programs, operating systems, digital tape and disk storage media, digital or analog tape recorders—used by a composer for sound synthesis, transformation, analysis and composition. It is also a conceptual environment consisting of tools to produce, transform and combine sounds. A mix-oriented compositional environment is a computer music environment in which the composer’s attention is focused on using a digital soundfile mixing program to arrange sounds into a finished composition. Limits of the Study
Rather than being an exhaustive investigation of many compositional techniques, this study focuses on selected small- and large-scale techniques for using a soundfile mixing program to assemble a number of existing soundfiles into a composition. Small- scale techniques are concerned with short time spans, usually less than one second.
Large-scale techniques are concerned with the combination of individual sound events.
It will be assumed that a number of soundfiles have already been selected for use in a composition and that the sounds are well known to the composer. This study is intended to provide an introduction to, and a model for, developing additional mix- oriented compositional techniques.
This study is intended to be relevant to the kind of computer music installation described above, rather than exceptional facilities, such as the Lucasfilm Audio Signal
Processor. Systems controlled by means of a Musical Instrument Digital Interface
(MIDI) are, generally speaking, not dealt with in this study, because, at present, the signals produced by most MIDI-controlled devices are analog signals. Several strategies for storage, playback and editing of digitally recorded signals that have been described in the literature are examined in Chapter Two. Chapter Three describes the hardware and software components at IRCAM as of May 1988 and how they contribute to the mix- oriented compositional environment being studied. During the course of the present research, the author revised the soundfile mixing program at IRCAM to include several features and capabilities required for mix-oriented composition. A description of this program is included in Chapter Three. Chapter Four presents a detailed investigation of problems that are commonly encountered when employing small-scale mixing techniques. Chapter Five deals with larger-scale mixing and compositional techniques based on writings about psychoacoustics and composition. CHAPTER II
THE DIGITAL AUDIO ENVIRONMENT
Introduction
Since 1958, when Max Mathews and his colleagues at Beil Labs first produced computer-synthesized sounds, the kind of equipment available has been one of the factors that has determined the type of music computers have made. Early computer music synthesis programs generally simulated analog techniques familiar to the authors of the programs. Musique concrete techniques, on the other hand, were not widely used for computer synthesis because they require great amounts of memory and storage which were not generally available to computer musicians. In addition, the storage and playback of computer-generated sounds was sometimes done from half-inch digital storage tape, making the many storage and retrieval operations necessary to use musique concrete techniques difficult, if not impossible. Musique concrete could be done with greater ease and better sound quality using analog equipment. There was simply no advantage to making it digitally.
Computer sound synthesis has historically been a time-consuming process often complicated by problems such as insufficient computer memory, restricted access to sufficiently powerful computing and conversion equipment, lack of programs or equipment to accomplish the desired tasks, unpredictable hardware devices, and, not least, relatively limited results. Often, a software synthesis program would require hours of computation time in order to produce a few minutes, or even seconds, of sound. The final composition would be assembled by first recording these short sounds
onto analog tapes, and then arranging, mixing and remixing them.
With the advent of large capacity random-access storage devices, it has become
possible to assemble and mix all the sounds for a finished composition within the
computer system, without having to resort to analog mixing. One important benefit of
this progress is that computer music compositions need not be limited to the raw sounds
produced by software synthesis programs. Since the output of these synthesis programs
is stored as digital data, it can be transformed, edited, duplicated or rearranged as
desired. Analog-to-digital converters for recording “natural” sounds into the computer’s
memory are now generally available, making virtually any acoustic sound available to
the composer. With the availability of sufficient amounts of random-access storage for
high quality audio production, computer music studios with “tapeless” editing and
mixing facilities have expanded the range of compositional techniques available to the
computer music composer.
The most apparent advantage of mixing digital signals, rather than analog tapes,
is the elimination of the physical problems involved with analog tapes—mounting and
unmounting reels, keeping track of where sounds are located, cueing up tape recorders,
winding and rewinding, resetting switches, making numerous trial runs, and so on.
Another important advantage of working exclusively with digital signals is that there is
no signal loss caused by transfers between tapes, as is the case with analog tape signals
which can lose from six to ten decibels (dB) of signal-to-noise ratio with each successive
generation of copying. Digitally-stored signals suffer no such loss because of the nature of digital storage, which is also much less susceptible to signal degradation due to such magnetic media problems as printthrough, improper head alignment and head wear, and the mechanical problems of tape transports. A composer can make unlimited successive
copies of a digitally stored signal with no noise buildup.
The fact that digital soundfiles are stored on random-access memory devices allows the composer to select quickly and precisely any part of any soundfile for mixing or auditioning. The computer keeps track of the actual physical locations of the digital soundfiles, freeing the composer from the burden of having to remember exactly where any particular sound is located once the sounds have been stored on a random-access storage device. New soundfiles can be created by editing or mixing existing soundfiles without the need for elaborate synchronization techniques, as with analog tapes. Thus, trial-and-error experimentation is encouraged; the composer is working directly with sound, and is able to concentrate on compositional considerations.
Random-access editing and mixing of digital soundfiles has been developed for commercial uses at Soundstream, Lucasfilms and other locations. These facilities have been documented in periodicals such as the Computer Music Journal and the Journal o f the Audio Engineering Society. Although the programs and equipment used in these
“tapeless” studios may vary widely, they do have some common equipment requirements, standards, techniques and design considerations. Descriptions of several tapeless mixing/editing facilities reveal some of the common requirements and some significant individual features as well.
Random-Access Editing of Digital Audio
The first commercial all-digital recording studio was the Soundstream studio in
Salt Lake City, started in 1976. It first began using random-access editing in 1976 with experimental digital recordings and the first commercial application took place in 1978. 10
(Ingbretsen and Stockham 1984) Ingbretsen and Stockham cite some of the advantages
of digital audio, including,
freedom from generation loss during copying, the ability to play repeatedly without degradation, archiving without gradual deterioration over time, the ability to perform complex digital processing (such as deconvolution and mixing), and the fact that the time base of the recording is “frozen,” thus allowing rapid and flexible operations to be implemented. (Ingbretsen and Stockham 1984,114)
One such “rapid and flexible operation” is random-access editing, a process used
at the Soundstream studio between early 1978 and 1982 to edit over 200 recordings.
Editing begins by playing back the source material, the recorded sound, and making
selections. These selections may be transformed and replicated if desired, and then
arranged into a sequence. This process can be repeated as often as needed to produce
the desired result.
A random-access digital editing session at Soundstream would begin by copying
source material into an area of random-access memory. From here, editing proceeded
by listening, selecting passages, creating splice transitions (crossfades), and copying the
material in final order to the destination. The continuous random-access editing system
(CRAES) that Ingbretsen and Stockham describe employs a time-base smoothing buffer
that allows the final edited material to be played from random-access memory, rather
than having to copy out the finished sequence before playback. The resulting savings in
the amount of disk space required is one remarkable and desirable feature of this system.
The principal elements of this system, as described by Ingbretsen and Stockham,
are summarized here to provide the reader with a working model of a random-access
editing environment. The first principal element of the Soundstream system is a large
random-access memory consisting of two 300 Megabyte removable media disk drives.
These provide a combined maximum storage capacity of 256 million audio samples, or just over 42 minutes of stereo data at a sampling rate of 50,000 samples per second. At 11 the beginning of an editing session, source selections, or “takes,” are transferred from storage tape into random-access memory. As editing proceeds, reference locations such as the beginning of a particular take can be identified by assigned names, by time, and by sample number.
The second principal element is a small area of volatile memory that holds the bridges used to produce the splice point transitions. Each of these bridges is a properly contoured crossfade (or “smear”) between the lead-out of one take and the lead-in of another. These transitions, or crossfades, are typically in the range of 20-100 milliseconds, but may be as long as a second or more, depending on the material. As splices are created, heard, and accepted, these crossfades are saved in the volatile memory area, awaiting final playback.
The third element—the key to the operation of this system—is the time-base smoothing buffer, the basic component of which is a high-speed electronic memory
(FIFO) controlled by an external clock. The clock provides the time base for the system and thus controls the passage of samples out of the system. At the same time as samples are being clocked out of the buffer, new samples are being stored in the next available locations in memory. The smoothing buffer described by Ingbretsen and Stockham contains up to a megabyte of memory and is controlled by a programmable clock, allowing all commonly used sample rates, including 50kHz, 48kHz, and 44.1kHz. The buffer can be operated in two-, four-, or eight-track mode, which would require a throughput of up to 400,000 16-bit samples (6.4 million bits, 0.8 Megabytes) per second.
The fourth element, a splice table constructed during the editing process, directs the playback of the recorded material. This table consists of instructions that describe what portions of what selections are to be used, the order in which they are to be used, 12 and the parameters of the crossfades at splice points or fades to be performed on particular sections. All or any part of the edited material may be heard at any time during the editing process. A digital controller uses instructions from the splice table to get the appropriate material and put it into the smoothing buffer, in order to assure continuous playback.
The essence of this system reduces to a simple concept: creating the splice table that will control the playback of source material and crossfades. By creating several such splice tables, several proposed splices or even several complete jobs can be heard instantly and the best selected. Since all source data can be accessed at random, there are no editing delays imposed by rewinding or fast-forwarding time, and as previously mentioned, the material can be played back in any order at any time.
Ingbretsen and Stockham cite the following criteria among the important considerations for designing the system:
The system had to be fast and highly interactive. It was (and is) our firm belief that the rapid ability to conceive of an idea, try it, and evaluate it does much to enhance creativity in editing (or, for that matter, in any other aspect of recording). (Ingbretsen and Stockham 1984, 117)
Other studios that are capable of random-access editing require a similar equipment configuration; large-capacity disk storage devices provide sufficient random- access memory to hold the sounds (soundfiles) to be edited, and a general purpose computer running specially designed software programs to carry out the sound editing and mixing operations. The various implementations will be determined by the work being done at each individual site.
The digital sound editing system at the New York Institute of Technology was developed to meet the needs of musicians, sound engineers, video artists and research scientists (Kowalski and Glassner 1982). According to Kowalski and Glassner, the system was designed to be a highly adaptable and easy-to-use system for displaying, 13
editing, and playing back audio sample data. They describe the system as a bare-bones
editor, intended to be further developed as part of a production-oriented system. They
make an interesting observation concerning the nature of the user interface, “We are of
the opinion that any computer system, whether intended for research or production, is
useful in inverse proportion to its complexity—or, more precisely, to its perceived
complexity.” (Kowalski and Glassner 1982, 66)
The N. Y.I.T. system is similar to the Soundstream system in fundamental
operation. The user has a disk pack, mounted at the beginning of the editing session.
Crossfades are calculated and written onto the user’s disk. Playback is guided by an
elaborate internal pointer scheme, so the original sound data is unchanged. Graphic
displays of soundfiles allow great control in selecting specific time points, which may be
labeled with alphanumeric names or graphic symbols. In the N.Y.I.T. system these
time points can be located by dragging a cursor with a light pen. Editing proceeds by
first setting pointers to define a segment, and then specifying what action is to be taken:
play, copy, move, insert, or delete. The visual display allows for multi-channel display
and playback, but the individual channels of stereo files cannot be edited or spliced.
Gcomp (Graphic Control of Mixing and Processing) is part of a system for
digital musique concrete composition developed at Queen’s University, Kingston,
Ontario, Canada. Gcomp is “an interactive graphics program that permits the user to define and edit time-varying control functions for processing individual sound files and mixing together several sound files stored on disk.” (Banger and Pennycook 1983,33)
The central operation in Gcomp is the specification and adjustment of time- varying functions to be used to control the operation of the mixing program. In this manner, Gcomp simulates an automated mixing console, although it does not operate in real time. Gcomp provides control over five basic functions found on audio mixing 14
consoles: signal input level, four parametric equalizers, reverb send/return mix, output
stereo placement, output signal level. Gcomp allows each parameter to be controlled by
a user-designed time-varying curve. A feature called the “trigger track” enables the
composer or engineer to mark and label selected moments in any soundfile (although the
procedure for choosing such moments is not described). The trigger track is a file that
stores these marked times for synchronizing events in any soundfile with events in other
soundfiles. Thus, automatic interchannel timing coordination can be achieved. The
output of Gcomp is a file of linked lists containing the time-varying functions and
pointers to the soundfiles declared during a Gcomp session. This program is intended to
produce functions that change values at a control rate rather than at the signal rate. The
default time increment is set at 1/10 second, which could give rise to audible distortion
depending on the source sound material and the functions being used, but this default
value can be overridden.
The Lucasfilm Audio.Signal Processor (ASP) is a special-purpose, dedicated machine, rather than a general-puipose computer, as in the previously discussed studios.
The primary purpose for constructing the Lucasfilm ASP was to facilitate the mixing of the many different sounds used for movie soundtracks. This mixing task requires great control over time relationships, and signal processing may be needed to provide filtering, reverberation, and so on, for each sound individually. The equipment needed to accomplish this pioneering achievement was assembled by James A. Moorer, and is described in his article, “The Lucasfilm Audio Signal Processor” in the Fall, 1982 issue of the Computer Music Journal.
The Lucasfilm ASP, as described in Moorer’s article, uses several 300 Megabyte disk packs to furnish sufficient random-access memory to carry out the simultaneous mixing of up to 64 channels of sound. When formatted, one such pack, can hold 42 15
minutes of monophonic sound (16-bit integers) at 50Khz, or about seven minutes of six-
track movie sound. The ASP can control up to eight digital signal processors (DSP’s),
which do the actual signal processing. Each DSP is capable of a computation rate of
about 18 million 24-bit integer multiply-adds per second, simultaneous with a sustained
disk transferrate of 6.4 million bits per second (800 kbytes per second) and a sustained
conversion transfer rate of 6.4 million bits per second. Each DSP is capable of handling
up to eight channels of audio at 50kHz sampling rate, providing a grand total of 64
channels in a fully loaded ASP.
In an accompanying article, John Snell describes the user interface
considerations for a real-time audio signal processing computer to control the Lucasfilm
ASP, which would allow users to create, analyze, modify, and edit sound, utilizing a
wide variety of techniques. Snell makes the analogy between the computer console and a
musical instrument:
At Lucasfilm, we are developing several consoles with a moderate number of controls—the minimum that one would need in any given session. Musicians are able to add more expression to their music if they know their instruments well. The more controls they have available, the less familiar they will be with the location of each control. One human being can have fine control of only a small number of input devices. (Snell 1982,37)
A disk storage method described by Curtis Abbott allows for more efficient use of disk storage in a computer music situation. (Abbott 1984) Abbott’s goal is to take advantage of the computer’s random-access storage capabilities, as opposed to the storage strategy used by Ingbretsen and Stockham where the computer is used as a serial device analogous to a digital tape recorder, storing the nth samples of each channel in adjacent locations. (This interleaved storage scheme is the one most widely used for multichannel soundfile storage in computer music centers.) Abbott proposes storing each channel separately, allowing the blocks of samples of a soundfile to be “scattered in 16
a deliberately random way across the disk.” This system is intended to allow greatly
increased flexibility in adjusting timing relationships between channels. Abbott’s
method, which is intended to use disk space more efficiently, appears to ignore the need
for high transfer bandwidth, the critical factor in disk-based digital sound recording
systems. The solution to this apparent problem is provided by 700K bytes of buffering
available in the audio signal processor. Scheduling is done based on a list of soundfiles
and their start times and durations. This schedule can be thought of as a “virtual
multitrack tape recorder.” Abbott states that this may not be the solution for everyone;
indeed, it was designed to allow very precise time placements of sound for film editing
purposes.
The IRCAM Setting
The computer music composition environment at IRCAM—the setting for this
study—includes programs for recording, playing, editing, mixing, transforming and copying of soundfiles. A brief summary of the facilities at IRCAM that make up the environment under study is offered here. A more detailed description of this sub environment is provided in Chapter Three.
At IRCAM, sounds can be recorded into the system at any sample rate up to
48000 Hz, mono or stereo. The “play” program can play back mono or stereo soundfiles. Soundfile editing programs provide a great range of features for displaying, analyzing and editing soundfiles. A composer can use these editors to precisely locate some point in a soundfile by manipulating the graphic display pointers. A section, thus selected, can be played, plotted on the user’s screen or on a printer, submitted to Fourier analysis, saved to disk, and so on. Editing is often carried out using the interactive mode of the “play” program which allows the user to specify new beginning and ending 17 points. When a section has been selected, either by interactive play or one of the editing programs, it can be saved to disk as a new soundfile. The disadvantage with this process is that each new soundfile consumes valuable disk space.
Sounds can be adjusted or transformed using programs that reside online.
Standalone programs that provide reverberation, delays, filtering, normalization, amplitude envelope, panning and other modifications are available. These “black box” programs are very, popular and much used. The phase vocoder program, a program that requires a great amount of computation time, has been implemented on the FPS-100
Array Processor, greatly speeding up its work. Extensions to the phase vocoder program have been written to use this device to provide sample rate conversion, transposition, filtering, time-base variation and cross synthesis. These programs have been used extensively by composers to create large amounts of source material for compositions.
Sounds can be combined via Mix, a command driven program that reads commands from either a terminal or a text file. Mix allows for precise combination of soundfiles, control of their amplitude envelopes, channel assignments, looping, panning, transposition and other possibilities, all of which are covered in greater detail in
Chapter Three. An alternative user interface to Mix, the MacMix program, takes advantage of the menu and mouse user interface provided by the Macintosh microcomputer. Among the composers who have used early versions of MacMix, some have found the program to be a fast, efficent way to accomplish the work they had to do, but others prefer to use the Mix program because of the greater precision and flexibility possible by using text files to specify the mixer’s work.
The composer who is ready to assemble a piece of music from some carefully prepared soundfiles may wish to adjust or condition these soundfiles further before 18
combining them. For instance, the composer may wish to add reverberation to a sound
in order to give it a certain ambience. This decision is often made after hearing the
results of a mix. One advantage of this environment is that a mix can be made, heard,
and, if needed, redone conveniently and quickly by altering the mix command file. The
source soundfiles can also be altered via soundfile utility and transformation programs
before redoing the mix.
Composers such as Stephen Beck and Trevor Wishart have used the soundfile
mixing program to create finished compositions. Other composers, including York
Holler, Michael Obst, Kaija Saariaho, David Evan Jones and Yoji Yuasa, have used Mix
to assemble sections of compositions that they eventually completed using multitrack
tape recorders. Because Mix can read commands from a text file, compositional
algorithms to produce such command files can be implemented using a programming
language such as Lisp, or C, or the UNIX tool, AWK. Pierre-Franijois Baisnee, Jan
Vandenheede and other assistants at IRCAM have written programs that implement
compositional algorithms to create text files containing lists of Mix commands.
Examples of this powerful technique are investigated in Chapter Three.
Given the advantages of using a digital soundfile mixing program to combine
sounds, it is important to note that the practice of transferring the sounds to be used in a
composition to multitrack tape for final mixdown persists. The use of multitrack tapes
may be necessary if a composer is working in two or more studios with incompatible
hardware or software. Also, composers have become accustomed to “fine tuning” their
sounds and adjusting equalization and effects in the final mixdown, a situation that has become conceptually analogous to a performance. But it takes concentration and patience to carefully synchronize all the sounds into their precise positions on a multitrack tape.
Making subsequent changes is a laborious and time-consuming task. It would seem that 19 a more efficient procedure would be to assemble the sounds within the computer system, adding equalization and other effects when transferring the finished piece to the final playback medium.
The mode d’emploi of new tools must be learned. Whether these tools are computer programs or manual tools, the familiarity gained through limited practical experience with a tool is seldom sufficient to provide the understanding needed to use that tool effectively. Moreover, acquiring faulty or incomplete knowledge about a tool may cause misunderstanding that can lead to problems or limit the user to the most rudimentary applications. Stanley Haynes writes, “Despite improvements in computer music facilities which are making them more accessible to musicians without specialized training, it would be unwise to predict that a composer new to the medium is likely to produce a piece which uses it completely effectively.” (Haynes 1980, 164) He also notes, “All too often composers settle for trivial results because an interesting solution is too complicated for them to realize.” (Haynes 1980,159) The effort spent to study documentation of how a tool works and what its capabilities are, will ideally result in more productive work techniques, and may reveal unusual implementations for accomplishing special tasks. Such an effort may at least save the user some time and frustration by revealing that this particular tool is indeed not the best choice for the job.
The importance of good documentation in a computer environment is noted by various authors, but seldom with the emphasis it deserves.
Haynes calls for sound synthesis systems with more intelligence built in. “There is a great need for increased intelligence in sound synthesis systems so that composers can be protected from at least some of the numerical complexities and so that instructions with a format akin to human languages can be incorporated.” (Haynes 1980,147) Many of the sound synthesis and soundfile manipulation programs at IRCAM supply default 20
values for most or all optional parameters, shielding users from the burden of having to
supply large numbers of parameter values each time the program is run. These default
values represent the experience of these programs’ authors, and supply built-in
intelligence. As computer music software continues to evolve, it will, ideally, continue
to become more intelligent, with each generation incorporating the best new knowledge
available into its design. Likewise, as better techniques for using computer music
software are discovered, they should be made known by means of new or updated
documentation and, ultimately, by incorporation into subsequent versions of the
programs involved.
Summary
The nature and the availability of computer equipment has had a significant effect
on the kinds of computer music synthesis and composition techniques that have been
used. With the arrival of random-access digital audio editing and mixing systems, it has
become possible to create a composition totally within the computer system. Equipment
requirements and techniques for random-access mixing/editing of digital soundfiles from
several sites where such facilities have been developed reveal some of the ways that
computer music composition has been done. A good user interface includes good
program design and good documentation, but the user must make an effort to learn how
to make effective use of the tools chosen. The compositional environment for this study
exists as a subset of programs residing on the VAX 780 (“erik”) at IRCAM. Using the programs in this environment, a composer can play, record, edit, analyze, transform and mix sounds. The soundfile mixing program at IRCAM, Mix, is an efficient,
straightforward program that has been used by composers to assemble some, if not all, of the sections of their compositions. CHAPTER III
THE MIX-ORIENTED COMPOSITIONAL ENVIRONMENT
Introduction
“What are the components of a mix-oriented compositional environment?”
“What are some useful compositional strategies for such an environment?”
These two questions are addressed directly in this chapter. The environment for this study is a subset of the programs and facilities available at IRCAM, where much of the research for this document took place. A description of the hardware components and software programs that make up this environment is given first. The IRCAM Mix program, written by Robert Gross and revised by Neal Yocom, is described in some detail, including the design and implementation of Mix. An introduction to using Mix demonstrates how the program works, and how it appears to the user. Techniques that have been used for creating Mix command files, and techniques for using Mix in combination with other programs are examined. Some examples of how Mix can be used to accomplish various compositional tasks are cited.
Physical Components of the Environment
IRCAM supports a number of areas of musical work, including research in acoustics, music perception research, musical analysis, development of the 4x real-time synthesizer, the development of microcomputer-based personal systems, and the
21 22 production and performance of new music. This study is concerned with the programs, equipment and methods used for the production of musical compositions us.tng the computer facilities at IRCAM. The compositional environment under study is built around a VAX 11/780 general purpose computer known to IRCAM system users as
“erik.” As of May, 1988, “erik” is configured with four megabytes of internal memory, uses the UNIX 4.2bsd operating system, has the standard peripherals—printers, terminals, disk storage, tape drive—and also has analog-to-digital and digital-to-analog conversion equipment for recording and playing back sound. This equipment configuration has much in common with that found at other centers for computer music research and composition. “Erik,” the computer environment for this study, is shown in figure 1. 23 ERIK Digital Eq Corp VAX 780
4 Megabytes
FPA
hpO hpl
disk drives Tape TU77 Drive RM05 256 Mb hp4 Eagle Lines 450 to/from DSC 200 Mb Studios dacs/adcs hp5 FPS 100 Printronix S.l. 9766 Array Line Printer 256 Mb Processor Removable Disk Packs
3 Transpac lines
2 modem lines Tektronix 4662 60 terminals Plotter
Ethernet
Kinetics □ Interface Macintosh Plus
AppleTalk Network LaserWriter
Figure 1 Erik - VAX 780 System 24
Computer music composition requires great amounts of disk storage for sound
production purposes. For example, a ten-minute stereo composition with a sample rate
of 48kHz will require about 115 megabytes of disk storage. Storage devices at IRCAM
and their capacities are listed in table 1. A TU-77 tape drive is available for storing
soundfiles offline onto digital tape.
Table 1. Disk Storage Devices, Filesystems on Erik
Physical Device Drive/Partition Kbytes Filesystem
DECRM05 hpOa 7429 / 256 MB hpOg 74691 /u sr hpOh 137616 /u
DECRM05 hpla 7429 /tm p 256 MB hp1g 74691 /space hplh 137616 /u 1
Fujitsu hp4d 2687 /sound Eagle hp4e 229120 /snd 450 MB hp4f 155200 /sndl
S.l. 9766 hp5f 234880 /snd2 256 MB Removable Disk Packs
Sixteen-bit linear digital-to-analog and analog-to-digital converters provide conversion between digitally stored samples inside the computer system and the analog waveforms to be played or recorded. The converters presently in use at IRCAM are
Digital Sound Corporation DSC-200 driven by “erik,” capable of handling arbitrary sample rates up to 48kHz stereo.
Finished compositions may be transferred to analog or digital tapes for playback.
If a piece has been composed and stored in the computer, this operation can be a simple 25
matter of recording the finished composition onto analog or digital tape as it is played
back through the DACs. The “original” or “master” in this case is a digital soundfile in
the computer system. This “master” soundfile can be stored permanently in digital
format on tape by means of a tape drive such as the DEC TU-77. A more commonly
used method for producing a composition is to transfer the source soundfiles to a 24-
track digital tape recorder for a final mixdown. In this case, the soundfiles must be
carefully synchronized as they are being recorded to the 24-track tape. SMPTE time
code synchronization is used for this step. A computer-controlled mixing board is then
used to mix the various tracks down to a digital quad or stereo master, which is then
copied to the playback medium. Signal processing equipment, for adding reverberation,
filtering, or equalization, and so on, can be used when mixing down or transferring
sounds.
The question arises, “Why would a composer want to transfer and synchronize
all the soundfiles on a 24-track machine when all the mixing could be done inside the computer system much more conveniently?” When this question was posed to several composers and composers’ assistants on the staff at IRCAM, most of the replies indicated that the final mixdown was a sort of performance where the composer could make on-the-spot decisions, based on how the mix sounded. The possibility of adjusting the various parameters of reverberation to achieve the desired ambience was the one feature mentioned most often in favor of the 24-track method. In this situation, the composers are willing to set the sounds into their final temporal order so that they can have the freedom to make adjustments to the amplitudes, processing, equalization, and spatial placement in real time. 26
Software Components of the Environment
The computer music compositional environment at IRCAM includes programs
for sound synthesis, including CHANT/FORMES, CARL/cmusic and others. There are
also programs for the recording, playing, editing, mixing, transforming, copying, etc.,
of soundfiles. These programs can be used one at a time, or they can be chained
together into sequences of actions by means of UNIX shell scripts. The creation of -shell
scripts combining several operations into one user command is an important tool for the
computer music composer. In this environment, if no single program does what the
composer wants, it is likely that there is some sequence of programs that can be
constructed to achieve the desired goal.
The “classical” way of working in this environment involves experimentation
with the various sound synthesis programs, or combinations of programs, using the trial
and error method. Once a composer has discovered a particular synthesis algorithm that
produces a range of sounds that seem interesting and useful, further experimentation is
needed to find the full range of effectiveness for that method. A conceptual model of the
computer music composition environment is shown in figure 2. The actual physical
configuration is shown in figure 3. These two figures illustrate how a computer music
compositional environment needs to provide for testing the results of each stage of the compositional process, and they illustrate that the physical manipulation and storage of
soundfiles is an essential ingredient in the mix-oriented compositional environment. SynthesizeRecord
Process Mix
Yes, but... Play: Delete Keep?
Yes
Soundfile Storage
Figure 2. Conceptual Model of Computer Music Compositional Environment 28
Record Synthesize
Process Soundfile Mix Storage
Delete
Play: Yes, but... No Keep'?
Yes
Figure 3. Physical Configuration of Computer Music Compositional Environment
Several important features of the IRCAM soundfile storage system and the implementation of soundfile manipulation programs are closely bound to the UNIX operating system. At the lowest level, the soundfiles themselves are stored as regular
UNIX files. This is a remarkable departure from previous systems for storing soundfiles which sought a way to obtain the necessary throughput for audio playback in a multi-user environment. Other systems have been based on storing soundfiles on adjacent tracks, or cylinders, so as to minimize delays due to head seek times. The key to the IRCAM soundfile storage system is an alteration to the UNIX kernel to allow an 29 allocation size of 16,384 bytes for soundfiles. This means that the smallest possible soundfile, consisting of just one sample would take up 16,384 bytes of disk storage space. Larger soundfiles grow in chunks of 16,384 (16K) bytes. These chunks are scattered across the disk in a typical fashion, but their large size provides sufficient throughput on read or write operations to support playback and recording at rates up to
48,000 Hz stereo. A disklocking option has been added to permit uninterrupted recording and playback in real time if the system is being used heavily.
The soundfile storage system at IRCAM is maintained on three UNIX file systems named /snd, /sndl, and /snd2 (see table 1). The default storage location for sound synthesis, transformation, recording, and playback is /snd. The soundfiles on
/snd are temporary and will be deleted if they have not been touched for two days. The soundfiles on /sndl are permanently available, frequently used soundfiles; each user is allocated a certain amount of space and is expected to remain reasonably near this limit.
File system /snd2 is set up on the removable disk packs, shared by groups of users, providing much larger work and storage areas. Composers who are working on pieces are generally allotted parts of one or more disk packs, in order to provide them with sufficient work space.
Because soundfiles require tremendous amounts of storage, they are grouped in separate file systems on separate devices, but they are still accessible from all terminals in the same way as regular files. At IRCAM the soundfiles are stored as regular UNIX files, so UNIX commands, tools, and system calls operate on them as well, with some exceptions. A set of utility commands has been created to manipulate soundfiles in a manner analogous to normal UNIX file operations. This system makes it possible to carry out quickly and easily not only common file operations such as copying and 30 moving, but also soundfile manipulations such as transposing, normalizing, panning, and so on. For example, the UNIX command “mv” is used to move regular Files from directory to directory; “mvsf ’ is used for moving soundfiles from directory to directory.
The following two command lines both effect the same task:
% mvsf pizz pizza
% mv /snd/usemame/pizz /snd/usemame/pizza
“Cpsf ’ works analogously to “cp,” “tarsf ’ replaces “tar,” and so on. There are, under the IRCAM Soundfile System, two kinds of programs, or activities, each with their own characteristic nomenclature. The soundfile utility programs for moving, copying, and so on, end with “sf,” as just shown. The soundfile treatment programs begin with
“snd,” for instance, “sndnorm,” the program to normalize the amplitude of a soundfile so that the maximum sample is set to the sixteen-bit signed-integer limit of 32767.
“Sndreverse” creates a reversed version of a soundfile, for playing the sound backwards.
The format, handling, and even the existence of soundfile headers is an issue with no universal agreement. At IRCAM, the soundfile treatment programs use the information from these headers to help them carry out their work. The IRCAM soundfile design is shown in figure 4. Soundfiles are made up of two parts: a header, which is always the first 1024 bytes, and the rest of the file, the sound samples, which may be any number from zero to the storage device’s limit. The soundfile header, in turn, has two parts. The first four items are fixed: a “magic” number indicating “is-a- soundfile,” the sample rate, the number of channels, and the number of bytes per sample. The last item in a soundfile’s header, “sf_code,” is used to indicate if any further coded information is contained in the header, such as the maximum amplitude per channel, or a comment. Full documentation about the use of soundfile headers is available in “The IRCAM Soundfile System,” a set of online manual pages documenting the use and operation of the soundfile manipulation programs. 32 Soundfile Configuration
header samples 1024 bytes
Soundfile Header
sf_magic sf_srate sf_class sf_packmode sf_code ...zeroes .7 7 samples...
------1024 bytes ------
fixed size items variable size items
sf_magic sf srate sf class sf_packmode sf code
bytes bytes bytes bytes
code bsize sf_maxamp code ...zeroes .7 » code = 1 (sf_maxamp) code = 0 bsize = 40 bytes (includes code, bsize and data) nothing else until samples Figure 4. IRCAM Soundfile Design 33 The IRCAM Mix Program Sounds can be combined via Mix, a command-driven program that will accept mix commands from either the standard input or a text file. Mix allows for precise specification (sample resolution) of starting times and durations in both the input and output soundfiles, precise amplitude envelope control for each soundfile, output channel assignment (mono, stereo, or quad), looping, panning, transposition and more. Mix commands consist of keywords followed by values. If the mixer is reading the commands from a terminal, a control-d signals the end of the command list and causes the mixer to begin execution. If a text file consisting of mix commands is being read, the end-of-file marker has the same effect. Mix will handle either integer or floating point soundfiles or both. It produces a short integer (playable) soundfile by default. A floating point mix and output file can be specified, using one of several optional command line flags. Mix sets the default sample rate from the first file opened. A warning is given if the sample rates of the input and output files are not the same. Files with different sample rates will be read at the output sample rate, resulting in transposition. The default number of channels is the greatest number found among the input files. Channel assignments remain unchanged unless a new assignment is given, except in the case of mono soundfiles, which are sent to all output channels by default. Panning and soundfile transposition values can be specified, either as single values or as a list of value/time breakpoints. Envelope values and output destinations for any or all channels of an input soundfile can be specified. 34 Using Mix The capabilities of Mix can be illustrated by demonstrating how to carry out some typical operations. The command to launch a mix is to type “mix”. If there is a text file named “fred,” containing a number of mix commands, the command would be, “mix fred”. Several optional command line flags may be used to specify how Mix is to perform. For instance, the command “mix - f ’ requests a floating point mix and a floating point output file (which must be converted to an integer file to be played back). Another command line flag is the “-F” flag, which causes Mix to do a floating point mix, and then to normalize the output file and convert it to an integer (playable) soundfile. A list of all command line flags and mix commands is included in Appendix A. If a mix command file has been specified on the command line, Mix reads its commands from that file and executes the mix, reporting warnings and errors that may be found. If, however, the program has been launched by typing only “mix,” the mixer will expect mix commands to be entered from the terminal. The simplest possible use of Mix, to read in a file named “pizz,” and copy it out as another soundfile, would require mix to be launched by typing “mix,” followed by the command, “input pizz,” and control-d to signify the end of input. The default output soundfile will be named, by default, “mixtest”. The following list of mix commands would read in “pizz,” fade it out in . 1 second, read in “vibe,” skipping over the attack portion of the sound, and name the output soundfile “pvibe”: 35 input pizz All text after the “<” is ignored by Mix. Starting times for all files are, by default, zero. Amplitude envelopes are specified by sets of value/time pairs. All amplitudes values are multipliers, usually within the range 0-1. The word “end” will be interpreted by Mix as the duration of “vibe”. Since only the first two letters of a mix command are significant, and since mix commands can be separated by a semicolon, the above lines could be condensed, for example, as follows: in pizz; du .1; en 1 0 0 1; in vibe; sk .3; en 0 0 1 .1 0 end; ou pvibe Mix can change output channel assignments and relative amplitudes quite simply. If channel one of a stereo input file named “duet” is to be sent to channels three and four of a quad mix, and the amplitude envelope is to be altered, the mix commands would be: 4 input duet send 113 < send all of channel 1 to channel 3 send .7 14 < send .7 of channel 1 to channel 4 env 0 0 1 .5 0 end ochans 4 < number of channels in output soundfile output rearchans Exponential envelopes can be used by specifying “xenv,” which requires three values for each breakpoint, specifying value, time, and slope, respectively. The slope value determines the shape of the transition to the next breakpoint. Positive slope values create exponential curves, which change slowly at first and faster toward the end, while negative slope values create inverse exponential curves, which change quickly at the 36 beginning and slower toward the end. This feature allows for more “natural” envelopes to be imposed on sounds more easily than by meticulously specifying a great number of “env” breakpoints. A mono soundfile can be panned between two channels by specifying a series of angle/time pairs. The value of the angle must be between 0 and 90, corresponding to channel one and channel two, respectively: input zoom pan 0 0 90 .5 0 1 90 2 0 4 < angle/time pairs If a single angle is specified, then the sound will be “located” at that angle by adjusting the amplitude of that sound sent to channels one and two: input zoom pan 20 The pitch of a soundfile can be altered by specifying transposition values. The value given is taken to be a multiplier for setting the new sample rate used for “playing back” the current soundfile. As with panning, the amount of transposition can be varied according to a series of value/time pairs. Note that the duration of the soundfile will be affected by this manipulation. This change is handled automatically by Mix, but the user who wants to specify an envelope for the new sound should be aware of this situation. Some typical uses of transpose could be achieved by the following sets of mix commands: input toot transpose 1.059 < transposition factor in toot trl.0 0 1.059 2 1.12 4 < transposition/time pairs 37 Inside Mix: The Design of the Mix Program The Mix program incorporates several notable features that are designed to its execution time and to provide an uncomplicated user interface. The commands that a user specifies in a file of mix commands, or at the terminal, are interpreted by the program segment named “setup”. Here the commands and their arguments are read in, checked for validity, the appropriate flags are set, and values are assigned to the appropriate variables. When an error is encountered, a warning will be sent to the screen, and, if the error was fatal, the program will quit. If, for example, a non-existent or invalid type of file was specified as the “input” file, Mix will report the error, then quit. Such errors have been designated as fatal because the user probably does want a specific soundfile to be used, so it is better to quit and allow the user to correct the command file than to continue to mix and produce something that is not desired. Unrecognized commands (and typing errors) that are encountered will provoke a warning, but are not fatal. A chart showing the program structure is given in figure 5. 38 Mix Control Flow m ixer Main loop: scan command line, if no command file, read commands from terminal setup Interpret mix commands, set up lists of values for envelopes, etc., for each input file, then install build a run queue according to start times. sec2unit Convert all timings to output sample numbers, sort envelopes into time-ordered lists. If global skip or global duration specified, adjustq adjust the run queue accordingly. dsetup Set up a queue in order of ascending end times. check Final check before mixing begins; check durations, channel assignments, envelope values. If the print flag is set, create a "mix.print" file qprint containing a synopsis of the run queue for this mix. Allocate input and output buffers, open output file, m ixctl drive the mixer through the run queue, opening and closing files as needed. qmixem If all files have the same class (float or short) and there are no envelopes or altered channel assignments, then qmixem does the mixing. setmode If, however, there are mixed classes, envelopes, or altered channel assignments, cmixem is cmixem required to carry out the work. Figure 5. Mix Control Flow 39 In order to improve a program’s speed of execution it is essential to maximize the efficiency of disk read and write operations. Mix carefully checks file statistics before the mixing begins, so that once it starts running, no more checking will be necessary, and the program can run at top speed. In addition, reading and writing is done in blocks of 16K bytes whenever possible, reducing the number of input and output operations required. If many envelope segments are used, or if there are a large number of soundfiles starting or stopping at different times, the number of read/write operations is greatly increased and the speed of execution is reduced. Using Mix as a Soundfile Manipulation Tool One of Mix’s best qualities is its relatively fast speed of execution. One benefit of this speed is that Mix can be used to carry out tasks other than mixing soundfiles. For example, Mix can be used to select a portion of a soundfile and to write it out as a new soundfile. If the beginning and end of the new soundfile are too abrupt, Mix can quickly taper the ends, using the “env” or “fin” and “fout” commands. In tests conducted at IRCAM, using Mix to carry out this function has been shown to be two to four times faster than the same function implemented via the CARL “Janus” program. Mix is probably most often used in its role as a soundfile manipulation tool to alter the amplitude envelope of existing soundfiles, using either linear or exponential transitions between breakpoints specified by the user. The most often cited advantage of Mix over its predecessor, Mix4, is its speed. Some comparisons of the speeds of the two programs are offered in table 2. The total execution time is the actual time elapsed from the moment the command was given to the completion of the program. The load average indicates how heavily the system is being 40 used. It varies with the number of users and with the types of programs that are being run. Table 2. Comparison of Relative Speeds of Mix and Mix4 Times given in seconds. All soundfile durations 1.0 seconds. Test 1: Copy a mono soundfile into new soundfile Mean results from four trials, mean load average: 0.68 Program User System Total Execution Mix4 14.0 2.25 21.5 Mix .2 .60 1.0 Test 2: Mix two mono soundfiles; output mono soundfile Mean results from four trials, mean load average: 0.91 Program User System Total Execution Mix4 22.80 2.9 41.00 Mix .35 .7 1.25 Test 3: Mix two stereo soundfiles, use 4 segment envelopes for each, output stereo soundfile Mean results from four trials, mean load average: 2.32 Program User System Total Execution Mix4 22.3 5.9 81.0 Mix 2.2 1.2 10.0 UNIX shell scripts and aliases can be used to gather a series of program calls under one command. A number of shell scripts that reside online at IRCAM have been developed by tutors and by visiting composers and researchers. Denis Lorrain has created a shell script, “etir,” that makes it possible for users to time-stretch stereo or quad soundfiles using the phase vocoder program, which can only process mono files. Lorrain’s script uses Mix to reassemble the channels into a transformed version of the original soundfile after processing each channel separately. Trevor Wishart has written a shell script that he uses to eliminate clicks from source soundfiles. His script calls on Mix to splice out the short segment containing the click. When transferring soundfiles to 41 the 24 track digital recorder, it is usually necessary to add a certain amount of silence before the beginning of each soundfile, because the DACs produce audible “thumps” at the start of playback. This task is most often accomplished via Mix. Command File Generation Mix, being a command-driven program, requires that the user specify exactly what is to be done. Mix can read these commands from a text file, allowing the composer to specify a set of mix commands, run the mix, and then make adjustments to the command file if the mix does not yield the desired results. Mix command files, also called “mixscores,” can also be created by programs that create text files, thus saving the composer from having to type every line to be executed by Mix. These programs typically implement some algorithm, acting on given soundfile names according to some specifiable parameter values. Compositional algorithms for creating mixscores can be implemented by programs written in programming languages such as C or Lisp. Such programs have been developed to meet certain composers’ needs, as will be shown below. Creating and Altering Mixscores via Awk The UNIX pattern scanning and processing language, Awk, is useful for mixscore creation and transformation. Awk scans a set of input lines, usually from a text file, searching for specified patterns. When a pattern is matched, a specified action is taken on the line or a particular field in the line. Awk commands can be entered on the command line or can be read from a file. The following command line could be used to remove all lines using the mix command “ampfac” from the mixscore “mixscore,” and 42 write out the new mixscore, “mixscore2”: awk ' $1 !~ /am/ {print $0}' mixscore > mixscore2 The following set of Awk commands, written into a file named “spreadout,” would add a fixed amount of time to every start time found in a mixscore (text following the “#” sign is treated as a comment): BEGIN { incr = .5 } $1 !~ /st/ {print $0} # if the line does not begin with “st,” copy it out as is. $ 1 ~ /st/ { # if the line begins with “st,” it is a start time, adjust it. if($2 > 0) { # if the start time is greater than zero, adjust it $2 += incr; # increment the start time print "st ",$2 # copy out the line with new time } else {print "st 0.0"} # for files starting at time zero } This program probably would not produce what is wanted, however. By merely adding the same amount to every start time, the time relationships between all soundfiles except the first will be unchanged. One would probably want to add a certain amount of time between every starting time. What is needed for this is a variable to accumulate the additional time needed before each soundfile starts. This can be done by adding one line of code to the Awk command file, just after the second print statement: incr += incr # update the amount to be added 43 Awk can be used to create a mixscore. The following shell script creates a dummy file for Awk to work on, then calls Awk, which writes out the mixscore. This Awk routine writes a mixscore that overlaps the files repeatedly, fading in and out to create a continuous sbund. cat "dummy" » dummyfile awk ' BEGIN { totaldur = 8 numfiles = 2 file[0] = "/snd/usemame/original" dur = .4 overlap = . 1 fin = fout = overlap*.45 } END { while(time < totaldur) for(i=0;i Although Awk is somewhat cumbersome to learn and use, the advantages it offers in manipulating large mixscores are valuable enough to recommend it to a composer who will be working with mixscores. Once an Awk program for manipulating mixscores has been developed and put into a shell script, it can be called 44 and used with great ease. The user can easily alter existing Awk programs to accomplish new tasks as needed. LISP and C Programs Composers at IRCAM have used programs written in C and in LISP to create mixscores. Developing programs in these or any other programming languages requires some time, but, as with Awk, the benefits may warrant the time involved. The great benefit of such programs is that if a mixscore that has been generated by one of these programs does not produce what the user wants, it is a simple matter to go back and specify a different set of parameters, and then to generate a new mixscore. This is a more efficient way of working than by typing in every line of a mixscore, and then editing it “by hand” if it does not produce the desired results. In a community situation, such as at IRCAM, once such a program has been created, it can be made available to other users, allowing each user to build up a personal “toolbox” of programs. Typically, programs written in languages such as C or LISP generate mixscores that implement some algorithm or compositional procedure. The LISP program, “mixer.ll,” written by Pierre-Fran^ois Baisnde for use by Kaija Saariaho, writes a mixscore that restarts a given soundfile according to the durations it reads in from a text file. The C program, “makenums,” writes out a text file containing a list of starting times to be used by another C program, “startem,” which reads in the list of starting times and writes out a mixscore using the durations to set start times. The C program, “detune.c,” creates a mixscore that slightly transposes and delays several copies of a soundfile and mixes them together. These programs are included in Appendix B. Another C program, “renv,” detects the amplitude envelope of one soundfile, writes out 45 a mixscore containing this series of normalized amplitude values, or “ampfacs,” and then calls Mix to impose the envelope on another soundfile. MacMix A Macintosh interface for Mix, “MacMix,” provides the user with the familiar menu-and-mouse interface for specifying the operation of the mixing program. MacMix, which was written in Aztec C by Adrian Freed, runs on a Macintosh acting as an intelligent terminal connected to the host VAX 780. “Mixd,” a program that runs on the VAX 780, acts as a host daemon for the Macintosh, interpreting commands it receives, and then calling Mix, “play,” or other programs to carry out the work to be performed. Although not all the features of Mix are available in MacMix, some of the graphic display features and some of the extra mixing procedures it offers make MacMix interesting for certain applications. MacMix has been used by George Benjamin, Stephen Beck, and other composers for assembling sounds and sections of compositions. It should be noted that a much more recent version of MacMix, “MouseMix”, is being distributed along with the Dyaxis digital soundfile conversion equipment for the Macintosh. Unusual Applications of Mix In addition to carrying out standard mixing techniques, there are some unusual ways of using Mix to create and transform sounds. For example, it is possible to “erase” part of a mix, if the mixscore has been preserved. If a sound that starts at 5.6 seconds into a mix is to be erased, the mix commands relevant to that file should be put into a new mixscore, with the amplitude multiplying factor, or “ampfac,” set to -1, or the 46 inverse of the original ampfac. Thus, whatever the original sound contributed to the mix will be cancelled. Delay techniques can be accomplished quite easily and effectively via Mix. If a soundfile is to be mixed with a reverberated version of itself, it is important that the reverberated version be delayed slightly, and panned to the opposite channel if possible, to achieve a more natural effect. Phase cancellations and reinforcements occur if a soundfile is mixed with a delayed version of itself, causing unexpected and sometimes dramatic alterations of both timbre and amplitude. This topic is investigated in Chapter Four. Some Technical Considerations Soundfiles at IRCAM, as at many such centers, are stored as 16-bit integers, because the playback system uses 16-bit digital-to-analog converters. If a number of soundfiles are to be mixed together, there is a chance that the the instantaneous sum for a given output sample will exceed the 16-bit signed integer maximum, 32767. If the sum of all the individual samples being mixed together at any instantaneous point exceeds 32767, overflow occurs, and the result will not merely be clipped, but will be interpreted as whatever signed integer the 16 bits indicate. An input value of 32768, for example, will be interpreted as -32768, usually resulting in an audible click because of the great discontinuity between adjacent samples that is the likely result. Figure 6 illustrates the problem. 47 Bit arrangements, two’s-complement encoding MSB LSB Decimal Value 0111 1111 1111 1111 32767 0000 0000 0000 0001 + 1 1000 0000 0000 0000 -32768 11111111 11111111 - 1 Figure 6. Overflow: Bit Arrangements Overflow can be avoided by carrying out all the numerical operations in floating point format, and then normalizing the output file so that the maximum value in the soundfile is 32767 or less. This feature is implemented in Mix by specifying the command line flag, “- F \ This is recommended for crucial mixes, and would be a good idea for all mixes, except that it causes the mixer to run more slowly. A comparison of integer and floating point runs of the same mixscore is shown in table 3. It can be seen that when the same file format is used for both input and output, Mix runs faster than when conversion to one or the other format is required. It is interesting that when the F” flag is specified, the extra time required to convert and normalize the floating point output is relatively small, suggesting that this should perhaps be the default for floating point mixes. 48 Table 3. Comparison of Execution Times for Integer vs. Floating Point Mixes Times given in seconds. All soundfile durations 4.0 seconds. Test 1: Mix two mono integer-format soundfiles Mean results from four trials, mean load average: 1.83 Output File Format User System Total Execution Integer 1.35 1.40 7.25 Floatingpoint 5.3 2.15 23.00 Test 2: Mix two mono floating-point format files Mean results from three trials, mean load average: 3.1 Output File Format User System Total Execution Integer 5.2 2.2 27.0 Floatingpoint 2.1 2.3 15.3 Test 3: Mix two mono files: one integer and one floating-point format Mean results from four trials, mean load average: 1.95 Output File Format User System Total Execution Integer 4.4 1.9 11.3 Floatingpoint 4.5 2.2 12.0 FI. Pt. /Norm.->Int. 9.4 4.9 22.0 Test 4: Mix two mono floating point format files: use four-segment amplitude envelope Mean results from two trials, mean load average: 1.3 Output File Format User System Total Execution Integer 9.9 2.65 27.5 Floatingpoint 7.9 3.15 30.5 FI. Pt. /Norm.->Int. 12.8 6.30 42.0 Test 5: Mix two mono files, one int., one fit. pt.: use four-segment amplitude envelope Mean results from two trials, mean load average: 2.8 Output File Format User System Total Execution Integer 9.15 2.55 23.5 (load avg: 2.2) Floatingpoint 9.0 3.3 55.0 (load avg: 3.2) FI. Pt. /Norm.->Int. 13.75 7.4 60.0 (load avg:3.0) 49 Summary The setting for this study is a subset of the programs and facilities at IRCAM that are used for the production of musical compositions, but the focus of compositional activity is oriented around the soundfile mixing program, “Mix”. The IRCAM soundfile storage system, central to all activities involving sound, and the soundfile manipulation programs are all part of a unified UNIX environment, which has several benefits, including the possibility to create shell scripts that gather sequences of operations into one command. The digital soundfile mixing program, Mix, is a powerful, yet easy-to- understand tool. Its design, flexibility and speed allow users to carry out some soundfile manipulations other than mixing quickly and easily. Since Mix can read its commands from a text file, or “mixscore,” programs written in higher level languages, such as C or LISP, can be used to create mixscores. The UNIX pattern scanning tool, Awk, can be used to create or make adjustments to mixscores. Such programs as these can be gathered into a composer’s “toolbox” for composing larger, more complex sound events or sequences of sound events. CHAPTER IV USING MIX AS A COMPOSITIONAL TOOL: SMALL-SCALE APPLICATIONS Introduction This chapter focuses on some fundamental mixing techniques, pointing out problems that often accompany their implementation, and presents the results of tests carried out to find the causes of, and solutions to, these problems. Some useful “small- scale” mixing techniques, including crossfading, looping, layering, fragmentation, and other procedures such as spatial location techniques will be examined. Composition via Mixing Once a composer has selected a repertoire of sounds to be used in a computer music composition, the task remaining is to arrange these sounds in time and space. The composer’s attention at this point, therefore, is on the shaping of a composition from these sounds, using a soundfile mixing program. Although it is possible to return to the synthesis environment, and it is likewise possible to adjust or “fine tune” a sound using sound manipulation programs, the central activity is composition via mixing. The focus of compositional activity may change freely from the composition of individual sound events, to small segments, to larger sections, according to the composer’s momentary concerns. Otto Laske writes, “Compositional activity is difficult to analyze because it is characterized by instantaneous changes in compositional strategy from top-down to bottom-up approaches to the material.” (Laske 1978, 558) Mix-oriented composition 50 51 provides a working environment that accomodates such instantaneous changes, allowing the composer to freely change the focus from any level of compositional activity to any other. Small-Scale Compositional Mixing Techniques Splicing Splicing is a fundamental technique from analog tape recording which has been adapted to compositional uses by many composers of electroacoustic music, most notably by John Cage. Splicing provided these composers with the capability to join sounds end-to-end, with or without overlapping at the join point. The digital equivalent of these analog splicing techniques can be carried out quickly and more precisely using a soundfile mixing program. The simplest method of joining two sounds is simply to start a sound after the previous one finishes, the equivalent of a “butt joint” in analog tape terms. The sounds may be faded on the ends to avoid clicks or thumps resulting from sudden changes of signal amplitude. The composer may also wish to separate the sounds by inserting some silence between them. For example, a composer who wanted to use Mix to set up an ostinato rhythm repeating a single sound at various time intervals would indicate the name of the soundfile to be used and the starting times needed to produce the desired rhythm. Pitch variations could also be created by using the Mix transposition feature. The resulting ostinato could then be duplicated by looping to fill out whatever duration was wanted. A number of complementary ostinatos could then be mixed together in various combinations to create a complex, rhythmically interesting result, composed entirely by mixing. Programs to carry out such compositional sequences have been created for use by composers at IRCAM, as discussed in Chapter Three. 52 Crossfading A valuable technique for joining two soundfiles sequentially is to overlap them slightly at the joint. In order to achieve a smooth transition, one soundfile will often be “faded out” while the other is “faded in,” a standard mixing technique known as “crossfading”. Crossfading requires that a number of choices be made concerning the joining of two sounds. The duration of each side of the crossfade, the shape of the envelope function to be used, and the amplitude of the crossover point are all dependent on the timbres, amplitudes and pitches of the sounds being crossfaded. There is, unfortunately, no formula that can be used.to guarantee an indetectable or smooth transition from one sound to another, so trial-and-error methods must be used to find the best values for any given crossfade. Research in the field of psychoacoustics has produced some useful information that can be used to help find useful values for crossfade parameters. In order to achieve a smooth transition, the relative loudness of the two sounds must be adjusted so that the overall amplitude behavior at the crossfade is natural. This is not a simple operation. The perceived loudness is dependent on a number of factors including not only amplitude and envelope, but also pitch and timbre. In their study of the perceived loudness of tones, Fletcher and Munson showed that the relationship between sound pressure level and the perceived loudness level is pitch-dependent. (Fletcher and Munson 1933) The composer can use Fletcher and Munson’s findings to help predict, but only in a general way, the loudness behavior of sounds being used. It should be noted that Fletcher and Munson were using sine waves as the sound source. Subsequent studies have shown less dramatic differences when sounds with greater spectral content are used. (Rossing 1982, 84; Roederer 1975, 85) Even if the pitches of the two sounds being crossfaded are the same, their timbres must be carefully considered. A sound having more spectral components across 53 a greater part of the audible spectrum will be perceived as louder than a sound with fewer spectral components, distributed across a smaller part of the audible spectrum, even though their amplitudes may be equal. Figure 7 shows how graphic or numeric information concerning amplitude may be misleading, and thus not be useful for predicting a sound’s apparent loudness. The judgments of the apparent loudness of the two sounds are the result of an informal study carried out in the course of this investigation. Peak rms Level Waveform Frequency Amplitude power Meter Sine 400 Hz. .4 .282828 0 dB Ramp 400 Hz. .4 .233780 -2.5 dB The ramp wave sounds at least twice as loud as the sine wave. (similar results obtained for tones at 1000 Hz and 4000 Hz) Figure 7. Information Revealing Nothing About Loudness If two soundfiles having the same pitch and timbre are being joined by crossfading, the shape of the attack and decay envelope functions being used must be selected carefully in order to avoid audible dips in loudness. Values must be determined for the crossover point (level), the crossfade envelope functions and their starting times and durations (Ingbretsen and Stockham 1982). The fade out of the first sound and fade in of the second one will not necessarily begin simultaneously. By maintaining the first sound at its full level while the second one begins to fade in, there may or may not be a rise in overall sound pressure level, due to the effects of phase cancellation and reinforcement, but the result can be a more satisfactory crossfade. Figure 8 illustrates the results of experiments using different envelope functions to crossfade from a sound 54 into a copy of itself. As shown above, one cannot rely on graphic or other representations of signals in creating crossfades. The ear will always be the final judge; any composer working with mixing must keep this in mind. The experiment summarized in figure 8 began by selecting a .5 second segment of the “steady state” portion of a clarinet tone. Figure 8b shows plots of the soundfiles created by imposing three basic fade-in and fade-out slope shapes: linear, and concave, and convex curves of varying sharpness, produced by an algorithm similar to the CARL/cmusic GEN4 function generator. Figure 8c shows plots of the soundfiles resulting from crossfading the linear-slope soundfile with itself, (1) starting the fade-in and fade-out together, and (2) starting the fade in .02 seconds before the fade-out begins. Figure 8d shows the results of mixing the convex fade-out with the concave fade-in. Figure 8e shows the results of mixing the concave fade-out with the convex fade-in. None of the combinations tested (including many not shown here) produced a perfectly smooth crossfade. The best result was produced by the last combination in figure 8e, using a concave fade-out with a GEN4 slope factor of 4, with at convex fade- in produced by a GEN4 slope factor of -4. Crossfade Functions Crossfade Result / \/ i/\ i i i * i i '0.1 sec \ / , \/ 1: sec i / X \ / v \ i 1 start together i ( convex y \ concave V f : : •0.1 lec ------convex concave J 1 0.1 sec Figure 8a. Results of Using Various Slope Functions for Crossfades. *rv[ irtVFO^w '00000 50000 -50000 • -'0000.0 -'5000.0 TMC »AVtrOfrV »n(M '5000.0 - 100000 * 5000.0 -5000.0 «- -1 0 0 0 0 0 -'5000.0 rwt wvcrotw '50000 10000.0 * 5000.0 -50000 1 0 0 0 0 . 0 • -15000.0 TWC Figure 8b. Results of Using Various Slope Functions for Crossfades. TIME W A v r r c ^ V Str‘.str‘. 10000.0 r 5000.0 - A M P L C e -5000.0 ■10000.0 TIME WAVEFORM strtx.3B 10000.0 r "xiftyif'iiiPiiHi**, 5000.0 - A M P L i T 0.0 U D E -5000.0 - - 10000.0 0.176 0.352 0.528 0.704 0.88 Figure 8c. Results of Using Various Slope Functions for Crossfades. time wAvrroRv /s''c/yoco~'/‘es'.s/v*cvl A W P L i T 0 0 U D r -5000.C -10000.0 0.35 0.39 0.43 0.47 0.51 0.55 T'WE T |V l w a v e p o r m v x c v 4 10000.0 5000.0 A M P L I T U c E -5C0C.0 6991^11924 - 10000.0 0.55 Figure 8d. Results of Using Various Functions for Crossfades. 59 TIME WAVEFORM /snd/yocon/Tesls/cwx" 10000.0 5000.0 A M P L I T 0.0 L D r -5000.0 -10000.0 0.39 0.43 0.51 0.550.35 TIME TIME WAVEFORM Cvvx4 10000.0 5000.0 A M P I T 0.0 U D E -5000.0 - 10000.0 0.35 0.39 0.43 0.47 0.51 0.55 TIME Figure 8e. Results of Using Various Functions for Crossfades. 60 One well-known compositional mixing technique is to use the attack portion of one sound, such as a pizzicato violin tone, followed by the continuation portion of another sound having the same pitch—a vibraphone, for instance. (The usefulness of this technique is evidenced by its appearance in commercial synthesizers, such as the Keytek CTS-2000, the RolandD-50, the EMAX SE.) Setting the parameters to crossfade from the pizzicato attack into the vibraphone continuation involves a number of steps. The vibraphone attack has to be removed, but finding just how much to remove involves some experimentation. The attack transients, especially those of a percussive sound, persist for a certain time before the continuation timbre establishes itself. This is also a consideration in deciding how much of the pizzicato attack to use, as well. The effect depends on the listener’s persistence in “hearing” the first timbre longer than it is actually there. The second timbre must be perceived as emerging from within the first. The duration of the crossfade must not be too long, or two timbres will be heard, cancelling the “fused” effect, and it must not be so short as to cause a noticeable click as timbres change between the two sounds. Trevor Wishart, in his composition Vox 5, uses spectral interpolation to crossfade from one sound gradually into another. The spectral interpolations are done by manipulating the data file from a phase vocoder analysis, and then using the altered data file for resynthesis. This technique was developed by Wishart to extend the range of sounds that could be crossfaded. His previous work had been done by editing, splicing and mixing analog tape recordings. At one point he crossfades from crows cackling into the sounds of a children’s playground. Another time, he crossfades from a vocal voiced “zzz” into the sound of bees buzzing. There is rich ground for musical composition here, especially in manipulating the ambiguous in-between timbres that can be produced. 61 Looping In an analog tape studio, a composer wanted to extend a sound or to repeat a portion of a sound several times could create a tape loop by splicing together the two ends of a segment of magnetic tape containing the sound to be repeated. The composer using a computer mixing program to do the same operation specifies the start and end points in a source soundfile, and then the duration for which this segment is to be repeated. Looping is a quick and easy technique for extending the duration of a sound event, but there are a number of problems that the composer must cope with in order to use looping successfully. The ideal loop would have endpoints with equal amplitudes and perfectly matching waveforms so that the end and the beginning join indetectably. This situation is rarely found, but there are methods for overcoming such obstacles and creating smooth loop points. In choosing a segment to be looped, there are several considerations that will affect the success of the operation. First, the timbres at the beginning and end of the segment should match, so that no sudden change is apparent at the loop point. This is generally not a problem when dealing with very short segments, but if a fairly long segment is being looped, the timbral change can cause a click or pop at the loop point. Likewise, the amplitudes at both ends of the segment must match or the amplitude discontinuity at the periodic return of the loop point will be obvious to the listener. Also, if the amplitude contour of the segment has any characteristic features, the looping will make them even more noticeable by repeating them at a regular interval. There are some techniques that can help overcome these problems and create smoothly joined loops. Finding beginning and ending points near zero crossings (points where the samples are changing from negative to positive values or vice-versa) can help to avoid discontinuities at the loop point. If the last few samples at the end of the segment are 62 negative, but approaching zero, and the first few samples at the beginning of the segment are increasing from zero, the transition has a much greater chance of success than if a sudden change of direction occurs. Ideally, the end and the beginning of the waveform could be joined. This method works well if the segment’s waveform is simple and unvarying. If the segment’s waveform is complex, having several zero crossings within one period, for example, or if the timbre or amplitude at the end is different from that at the beginning, it could be difficult or even impossible to find two suitable points to join smoothly using this method. In back-and-forth looping, once the end of the segment is reached, the retrograde inversion, as it were, of the segment is appended, assuring a loop point which will have (ideally) no amplitude or timbral discontinuity. A problem with this technique is that part of the sound is now being played backwards, so this method may not work as well for long segments with identifiable features. Overlapping the endpoints and crossfading is one technique that can often be used to create smooth loops. The unpredictable phase cancellation and reinforcement that occurs during the overlapping can, unfortunately, cause a noticeable timbral change. This particular aspect of looping problems has not been documented well in the literature. The results of phase cancellation and reinforcement are investigated in the section below on “Layering”. In order to loop a sound with large differences in amplitude, nearly the entire length of the loop may need to be crossfaded, which may seriously alter the timbres involved. Another solution is to overlap several copies of a segment. This will assure a smoother transition, but the timbre will, again, be altered to some extent. It is a good idea, as well, to vary the distance between restarting times to avoid noticeable periodicity effects. Some looping techniques are illustrated in figure 9. Original Waveform v v Join Copies End-to-End Join Segment with Reversed Copy vv Join Segment with Reversed and Inverted Copy v v Join Two Segments By Crossfading at the Joint Figure 9. Looping techniques. 64 Layering Layering is the building up of new sounds by overlaying existing sounds or selected parts of existing sounds in a sort of additive synthesis. Layering is a valuable technique for extending a limited repertoire of source sounds. Although the process would seem to be straightforward, there are some problems that can cause unexpected and undesirable results. For instance, a composer might wish to create the illusion of a number of players playing at once by mixing together several copies of a pizzicato violin sound with slight transpositions and offsets in the starting times in order to simulate the “chorus effect”. The resulting sound, however, may have any number of unexpected problems such as clicks due to overflow, or dropouts resulting from phase cancellation. This latter phenomenon may be responsible for several common problems in mixing digital signals that have not been thoroughly investigated. Experiments carried out in the course of this study have shown how phase cancellation and reinforcement affects amplitudes and timbres of sounds being mixed in a digital system. Furthermore, these effects are predictable, and can be used for compositional purposes. When two copies of a sound are mixed together with starting times offset by a few milliseconds, phase cancellation and reinforcement also may lead to such unforeseen effects. Comb filtering always results when a periodic sound is mixed with itself, offset by a delay of a few milliseconds. The frequencies affected by the resulting phase cancellation or reinforcement are dependent on the delay interval. The frequencies of cancellations (nulls) occurring in this situation will be found at odd integer multiples of 1/2 * delay time. Reinforcements will occur at integer multiples of the frequency of the delay time. (Wells 1981,164) Experiments carried out using Mix to mix together a delayed signal with the original produced dramatic timbral alterations. In one case, a 65 clarinet tone was transformed into a timbre more like a muted-trumpet tone, and a trumpet tone into a clarinet-like tone. Table 4 shows frequency cancellations and reinforcements that result from mixing a soundfile together with a delayed version of itself. Figure 10a and 10b illustrate those frequencies that are affected when mixing a soundfile with a delayed version of itself. For clarity, all spectral components in figure 10a and 10b have the same amplitude, to show the effects of mixing using very short delay times, as is done in creating artificial reverberation, for example. The time vs. amplitude plots in Figure 11 illustrate the dramatic amplitude differences that result from mixing a soundfile with a delayed version of itself. The time waveform plot at the top of figure 11 is the original sound, a short clarinet tone at 311 Hz (its period is ca. 0032 seconds). The middle plot of figure 11 shows the cancellation that results from mixing the original tone with a copy delayed by a half period (.0016 seconds). The bottom plot of figure 11 shows the reinforcement resulting from using a delay of one full period (.0032 seconds). Figure 12a shows the spectral plot of a clarinet tone (above) and the result of mixing that tone with a copy delayed by a half period (below). Figure 12b shows the difference between the spectrum of a trumpet tone (above) and the result of mixing that tone with a copy delayed by a half period (below). The result sounds much like a trumpet tone an octave higher than the original. Figure 12c shows the clarinet spectrum (above) and the result of mixing a phase-inverted trumpet tone with a copy of the original delayed by one half period (below). The two spectra both have fundamentals at about 311 Hz with stronger odd-numbered partials, especially below the tenth harmonic. The trumpet mixture shown here also sounds quite a bit like a clarinet tone. 6 6 Table 4. Frequency Cancellations and Reinforcements Resulting from Delays If a delayed signal is mixed with the original: -Cancellations will be found at odd-integer multiples of (.5*frequency of delay time) -Reinforcements will be found at integer multiples of (frequency of delay time) F = Frequency of fundamental tone P = Period of fundamental tone Delay Cancelled Reinforced Frequency Frequencies Frequencies .5 F (2P) .25F, .75F, 1.25F, ... .5F, IF, 1.5F, ... 1 F (IP) .5F, 1.5F, 2,5F, ... IF, 2F, 3F, ... 1.5 F (.75P) .75F, 2.25F, 3.75F... 1.5F, 3F, 4.5F, ... 2 F (.5P) IF, 3F, 5F, ... 2F, 4F, 6F, ... 3 F (.33P) 1.5F, 4.5F, 7.5F, ... 3F, 6F, 9F, ... 4 F (.25P) 2F, 6F, 10F, ... 4F, 8F, 12F, ... 5 F (.2P) 2.5F, 7.5F, 12.5F... 5F, 10F, 15F, ... If the inverted original signal is mixed with the delayed signal: -Cancellations will occur at integer multiples of (frequency of delay) -Reinforcements at odd-integer multiples of (.5*frequency of delay) Delay Cancelled Reinforced Frequency Frequencies Frequencies .5 F (2P) .5F, IF, 1.5F, ... .25F, .75F, 1.25F, ... 1 F (IP) IF, 2F, 3F, ... .5F, 1.5F, 2,5F, ... 1.5 F (.75P) 1.5F, 3F, 4.5F, ... .75F, 2.25F, 3.75F... 2 F (.5P) 2F, 4F, 6F, ... IF, 3F, 5F, ... 3 F (.33P) 3F, 6F, 9F, ... 1.5F, 4.5F, 7.5F, ... 4 F (.25P) 4F, 8F, 12F, ... 2F, 6F, 10F, ... 67 Frequency Cancellations and Reinforcements Resulting from Delays 0) T3 3 Original Spectrum E < P a rtia l: 2 3 4 5 6789 Original + Delay (IF) a) 3 P< E < P a rtia l: 123456789 Inverted Original + Delay (IF) o T3 3 P. E < P a rtia .•1 2 34 5 6 7 8 9 Original + Delay (2F) 0) X3 3 a, £ < P a r ia l: 1 2 3 4 5 6 7 8 9 Figure 10a. Frequency Cancellations and Reinforcements Resulting from Delays 68 Frequency Cancellations and Reinforcements Resulting from Delays Oh b < Partial: 2345 6789 Original + Delay (2F) O h B < JGL. JEL. P artia : 1 2345 6789 O h E < JL JL JLJL Partial: 1 234567 89 Original + Delay (3F) o ■o p O h E < P a r tia l:! 2345 6789 Figure 10b. Frequency Cancellations and Reinforcements Resulting from Delays 69 fiv[ WAvTr0®M toot 0.168 0.252 0.336 0.*2 loot 0 0 '6 AU P I T 0.0 0U z 35000.0 0 0.4216 35000.0 r '!l 'II I.''1!1! i - 17S00.0 0.06*6* 0.16M 0J5M 0 .U 6 0.«2S2 Figure 11. Amplitude Differences Due to Phase Cancellation and Reinforcement 70 POWER SPECTRUM tort -20 -3 0 -4 0 - -51 ~60 H -70 0 1200 3500 4800 6000 POWER SPEC1RUW toot.0016 -10 -20 -3 0 -4 0 -6 0 - -70 0 1200 2400 3600 4800 6000 FREQUENCY Figure 12a. Spectral Plots Showing Comb Filtering Effects 71 pOWtR SPECTRUM irpi A M P I- I T C Q E 1200 2400 3600 4800 6000 FREQUENCY POWER SPECTRUM trpt.cb - 10 - A M P L I T U D - 4 0 - E -6 0 - -7 0 1 UUu 1200 2400 3600 4800 6000 FREQUENCY Figure 12b. Spectral Plots Showing Comb Filtering Effects 72 POWER spectrum tael - ‘0- -20 A M P L -30 I T U D -40 E -50 -60 -70 'JL 1 J L l JLJL 1200 2400 3600 4800 6000 POWER SPECTRUM tpeb.i -10 -20 A M P L -30 I T U D -4 0 E -50 -6 0 -70 iyii.il j_U—LL 1200 2400 3600 4800 6000 Figure 12c. Spectral Plots Showing Comb Filtering Effects 73 Fragmentation The fragmentation of sounds can be used to generate useful sound material from existing sounds. Although granular synthesis, as described by Curtis Roads (Roads 1985, 145), could be implemented using a soundfile mixing program such as Mix, it would require the development of one or more sophisticated programs to calculate all the variables, and then to write out the appropriate mixscores. Even then, the sheer amount of control data could require more memory than is available in most computers capable of running Mix. Nevertheless, soundfile fragmentation techniques can be used to produce useful sound transformations. Roger Reynolds has used a technique he calls “microsurgery” to select a portion of a soundfile, the attack, for example, to “chop” it into hundreds of very short segments (.01-.1 second), and then to write these out in a different order. These fragments may overlap or may be separated by silence. Each fragment may have a slightly different amplitude envelope. Such fragmentation and subsequent reassembly of very short segments of a sound will alter the timbre, duration, and also the pitch of a sound. A computer program, “frag,” generates a mixscore that makes this process easy to accomplish. When a clarinet tone was fragmented and reassembled as part of this study, the resulting sound ranged from one like a fluttertongued clarinet tone to a complex, rich sound, having the same envelope shape of the original clarinet tone. Spatial Considerations Every sound is accompanied by a set of cues that indicate its setting in physical space. This setting, always a complex mixture of component cues, may consist of a general ambience, or it may incorporate an elaborate scheme of spatial movement involving reverberation, delay, filtering, and Doppler shift. For example, as we listen to the sound of a truck passing at a busy city street comer, many sounds that are unrelated 74 to the truck are also present—heels clicking, children’s voices, the roar of small and large gasoline and diesel engines, construction noises, shuffling packages, distant music, vehicle horn sounds echoing off buildings, approaching traffic— but are neglected if we focus our attention on the sounds of the truck. If these other phenomena were to be reduced or eliminated, as usually happens at different times of the day, or if we were to move further away, the sound of the passing truck would likely take on a different significance, and our perception of the total physical event would be greatly different. If we were to compare recordings of the sound of a truck at a busy intersection and the sound of a truck passing on a highway, the difference in sound would be immediately recognizable and our perception also altered. If the two sounds were to be incorporated into a musical composition, the composer would have to carefully consider the differences in the two settings and their acoustic and perceptual effects. The composer should be concerned with creating the proper ambience and spatial location cues for sounds. It is not possible to create a realistic illusion of a sound moving into the distance by merely reducing the amplitude of that sound. It is possible to add interest to sounds by changing the relative amplitudes of the signals between two stereo loudspeakers, or “panning.” But we have fine-tuned acoustical systems that are so well accustomed to more elaborate spatial movement cues that sounds lacking such cues may fail to create the desired illusion. Our primary cues to the distance of sounds are the amount of reverberation and some loss of high frequency components, so a composer wishing to make a sound seem to move to a distant point would have to vary the reverberation, low-pass filtering and Doppler shift. If a composer imagined a sound’s spatial setting as being distant, and the sound had not been given the proper spatial setting prior to mixing, the composer could create 75 the desired spatial effect at that time. The techniques used in spatial location of sounds involve the mixing of slightly varied copies of the original sound. For example, if a composer wishes to give a sound a more natural-sounding ambience, some sort of reverberation has to be created, often by using a special program to produce a reverberated copy of the original soundfile. Then, rather than mixing the “dry” and reverberated signals monophonically, a more realistic spatial effect can be achieved by locating the “dry” sound more toward one channel, and a delayed and reverberated copy of the sound toward the other. Nearly all stereo effects depend on the listener being in a relatively small area in front of two loudspeakers. If the listener is closer to one side than the other, subtle stereo effects may be lost. It is possible that the listener who is outside of the central stereo area will only be able to hear one channel even though both are playing simultaneously. For this reason, a composer should not count too heavily on subtle stereo effects. For better localization of sounds, the best solution is to use more loudspeakers, along with more channels of appropriately equalized, delayed, reverberated and filtered sound. Experimentation with different settings for these effects is needed to find a satisfying mixture. Even after a definitive performance version of a piece has been recorded on tape, the playback situation must be adjusted to provide an appropriate presentation of the composition. S um m ary When using a soundfile mixing program to carry out small-scale mixing techniques, the composer must be aware of the effects of phase cancellation and reinforcement, which can produce dramatic changes of timbre and amplitude. Because of this phenomenon, it is impossible to predict which envelope functions will produce 76 the most successful crossfades. The frequencies that are reinforced or cancelled when mixing together two copies of a soundfile, offset by a short delay, are predictable, and this information can be used to alter timbres. Looping is a small-scale mixing technique which is difficult to use successfully for many reasons, most of which are concerned with the identifiably periodic return of one or more characteristic features of a sound event, especially the starting point. A useful solution is to overlap several copies of the looped segment, starting each copy at a different offset starting time. Using segments that are taken from slightly different places in the original soundfile also helps avoid the identifiable periodic return of the loop point. The fragmentation and subsequent recombination of soundfiles can be used to produce useful timbral transformations. The composer using a computer to assemble a piece can control, to some extent, the spatial placement of the sounds being used. This resource can be manipulated to enhance the presentation of sounds played through loudspeakers, and can become an integral part of the musical ideas. CHAPTER V USING MIX AS A COMPOSITIONAL TOOL: LARGE-SCALE APPLICATIONS Introduction: Prolongation and Extension The composition of a piece of computer music may involve using various techniques to extend, prolong, or draw out aspects of the source material chosen by the composer. To Schenker, dealing with tonal music, prolongation of the underlying musical idea, the “Ursatz,” was the goal of composition. Musical ideas (i.e., compositional ideas) often arrive in a very condensed or incomplete form. The composer who wants to use one or several such ideas (or sounds) in a composition must find a way of extending (prolonging) and arranging these ideas (sounds) into a finished piece. Some compositional mixing techniques for extending (prolonging) musical sounds, or sound events, based on principles derived from writings on psychoacoustics and composition are described in the first part of this chapter. Instances of these and other mix-oriented compositional techniques used in the composition of Extensions are described in the latter part of this chapter. Streaming and Musical Continuity In their article, “Hearing Musical Streams,” McAdams and Bregman write: The perceptual effects of a sound depend on the musical context in which that sound is imbedded. That is, a given sound’s perceived pitch, timbre, and loudness are influenced by the sounds that precede it, coincide with it, and even follow it in time. This context influences the way a listener will associate the sound with various melodic, rhythmic, dynamic, harmonic, 77 78 and timbral structures within the musical sequence. (McAdams and Bregman 1987, 658) McAdams and Bregman demonstrate how a repeating six-tone sequence could be perceived as several different groupings, depending upon the tempo and frequency separation, the intensity, timbre, and spatial location. They discuss how the psychological organizing strategy used by the listener can be guided by manipulating these factors. McAdams and Bregman continue: The point being made is that in the framework of music, where all these complex interactions are of great importance, the context that is created may be the essential determinant of the musical result of a given sound. One sound can be perceived in a great number of ways, depending on its context. This leads us to believe that the fundamental perceptual element in music may be the “melody” rather than the isolated tone, or (in the terminology of auditory perception) that the fundamental structure is the auditory stream. (McAdams and Bregman 1987, 689) Some of the principles described by McAdams and Bregman can be applied to composition. By manipulating the context in which a sound is presented, a composer can associate various musical meanings with that sound, and can combine that sound with others to create various musical contexts. Consider, for example, the difference in effect resulting from placing a bass drum fortissimo in the middle of a quiet musical moment, as compared to placing it at the end of a passage which steadily builds up rhythmic and dynamic intensity. A composer can use a soundfile mixing program to control the order and rate at which sound events are presented, thus directly controlling the musical context and the resulting musical effect. Rhythmically complex and texturally dense sounds can be created by using a soundfile mixing program to overlay several periodically repeating sounds. If we create several soundfiles, each using the same source sound, but each with a different period of repetition (.2, .3, .5 seconds), a number of combinations is possible, some with surprising results. If we mix the soundfile that repeats every .2 seconds with the one 79 that repeats every .3 seconds, we hear the familiar two-against-three rhythm, with an accent resulting from the coincidence of note attacks every three beats, or .6 seconds. We could, however, invert the phase of one of the soundfiles, so that at every coincidence of note attacks, total cancellation would occur, producing silence, a rest. Another method of combining these repetitive sounds is to mix a repetitive soundfile with a slightly delayed copy of itself to create a feeling of dotted rhythms. It is possible to create complex rhythmic and timbral combinations, but with dense mixes there is an accompanying loss of rhythmic clarity, and, as a result, a loss of the complexity that was originally the goal. However, as this point of dense texture is reached, it becomes possible to treat the sound as a background, and to move elements of the texture toward the foreground, then to let them disappear as other elements are made prominent. Spatial movement is a valuable resource in this sort of compositional context. Grouping Mechanisms and Musical Continuity Deutsch states that the primary task of our auditory system, when confronted with a complex stimulus such as music, is to “inteipret this spectrum in terms of the behavior of external objects.” (Deutsch 1982,99) The auditory system interprets the attributes of these stimuli-frequencies, amplitudes, spatial location, timbres-and invents groupings of stimuli according to principles that it selects on a “best fit” basis. Deutsch cites the Gestalt principles that govern our grouping of stimuli: proximity, similarity, good continuation and common fate. The principle of Proximity groups elements on the basis of nearness, grouping elements which are closer in space. The principle of Similarity groups like elements together. Good Continuation groups elements that follow each other in a given direction. The principle of Common Fate groups elements that seem to move in the same direction. 80 According to Gestalt theory, our perception of the immediate environment is made according to such principles. Things that are close together are more likely to be interpreted as a group than things that are farther apart. Similar sounds are attributed to a single source, unless conflicting information (e.g. spatial information) forces another interpretation. When multiple, or rapidly changing, or complex sounds are encountered, these grouping principles may provide percepts that are ambiguous or false. The manipulation of such ambiguity is a rich technique which can be exploited by a composer to produce musically interesting results. The Gestalt principles of Proximity, Similarity, and Good Continuation are illustrated in figure 13. Proximity o # o o o O o o Similarity o # o o Good Continuation • • • • Figure 13. Gestalt Grouping Principles (after Deutsch) The grouping principles cited by Deutsch may be used by a composer to construct musical groupings with some degree of assurance that the listener will have some basis for following the evolution of the music. Such consideration is not as 81 important when composing music in a more traditional style, where a large body of listeners bring to the music listening strategies based on great numbers of previous experiences, and the composer can (and should) take advantage of this knowledge in constructing his music. But if a composer is working with a style unfamiliar to the listener, computer music, for example, consideration of what listening strategies the listener is likely to use to follow the music can help him make compositional decisions that will be effective. Keeping in mind the principles described by Deutsch and by McAdams and Bregman, it is possible to develop compositional techniques for arranging sounds in time and space which take into account the listener’s perceptual strategies and limits. The next sections propose ways of arranging sounds based on findings discussed above. The goal of this activity is to discover compositional mixing techniques that can be used to extend, or prolong, musical ideas by shaping the contexts in which they are presented. Extension via Pitch Matching Pitch is a strong organizing factor in many different kinds of music, and for good reason; the human auditory system is very sensitive to pitch changes. Given a sound with a discernible pitch, a second sound with the same pitch can be presented in such a way as to be taken as a continuation of the first. An example of this technique easily accomplished by mixing is the crossfading from the attack of one instrumental sound to the continuation of another playing the same pitch. The second sound does not necessarily have to overlap the first in order to be heard as a continuation. The repetition of a single pitch by different timbres can be done in such a way that the repetitions are accepted as an extension (prolongation) of one long event, rather than as several 82 different musical events, because they are judged to be “similar,” or “good continuations.” Electronic tape compositions often use pitch matching to extend sound material and provide smooth transitions. Jonathan Harvey's Mortuous Plango, Vivos Voco, for example, contains several moments where a pitch present in one sound is taken over by the next sound. Such smooth transitions give the listener a path to follow as the piece unfolds. Harvey's composition provides us with excellent examples of pitch matching because the pitch classes used in the piece are limited to a collection derived from the partials of the great bell of Winchester Cathedral. Given a readily recognizable pattern of pitches or motif, it is relatively easy to create a subsequent passage that is perceived to be derived from, or closely relatedto, that original pattern. Much of our music depends upon the ability of the listener to recognize such relationships. Repeating the pitch behavior of a motif at the same, or at a different pitch level, is a technique well known by the great masters. If a composer wishes to extend a musical event that has a clear pitch behavior, continuation of the same or recognizably similar pitch behavior is essential for extending that musical event. Good examples of this technique are those passages in the orchestral literature where a scalar pitch pattern is heard, even though it is achieved by having several different instruments play parts of the scale in quick succession. Proximity (of time, pitch, timbre, space, intensity) is also essential if the repeated pitch or pitch behavior is to be taken as an extension of previous activity, although a certain amount of variety is necessary to maintain musical interest. Extension via Timbre/Texture Matching Our sensitivity to pitch changes applies also to the spectral changes of an evolving sound— its timbre. A number of musicians and writers have speculated about systems of musical organization based on timbral organization. The notion of klangfarbenmelodie as an organizing principle has been discussed, but there is little, if any, agreement as to how it would be organized. It is difficult to be specific when writing about a subject that is so broadly defined. If we speak of a clarinet-like timbre, which of the many timbres that the clarinet is capable of producing do we mean? Timbre is a multidimensional concept that will be defined here as the spectral evolution of a sound. The spectral content is the primary component of timbre, but it is important to keep in mind that a sound’s spectrum changes through time. Robert Erickson, in Sound Structure in Music” describes a number of ways of organizing timbres in composition. Erickson devotes the fourth chapter of his book to a discussion of drones— sounds that are sustained for long periods of time. He notes that our musical tradition is so pitch-oriented that, except for bagpipe music, where “drones are suffered as a noisy adjunct to a loud national music,” we generally have no background to help us understanding how drones function. Drone music can, however, be a sophisticated sort of polyphonic texture where the drone interacts subtly with the other (usually upper) parts. A drone can also be used to provide an ambience, or environment, which changes little if any during the course of a piece of music, providing a “timeless” sort of background. He continues, “The conception of drone music as an ambient or an environment is at the root of much contemporary music in this genre and is, I think, only at the beginning of a long and fruitful development.” (Erickson 1975, 84 Some of Erickson’s ideas about timbre-oriented composition can be implemented by compositional mixing techniques. The idea of creating an ambience, whether it stands alone or serves as a background for a piece, is one of great interest to the composer of electroacoustic music. All sounds need a context. Much electronic music has been produced without any regard for creating a “sound landscape,” a setting, an ambience for the unfolding of the sound piece itself. If a piece is to be performed, even partially, by loudspeakers, the composer should consider creating an ambience or some sense of spatiality. Every natural sound is attributable to some physical source, or, going one step further, to some physical action or gesture that causes that sound. If we imagine every musical gesture as if it were the result of some physical gesture, we could develop a new approach to understanding the expressive quality of music. A composer using a soundfile mixing program is able, to some extent, to control the gestural quality of individual sounds by reshaping their amplitude envelopes. Large-scale musical streams and musical gestures can be created by shaping and arranging timbral events into groupings that have recognizable direction. Individual sounds have no meaning without context. It is only when they are placed in relationships with other sounds that we become attracted enough to follow where they are going. The focus of mix-oriented composition, as described here, is on the combination and juxtaposition of “whole” sounds into large-scale musical streams and musical gestures. Erickson notes that “during the past half-century more music has been written in which texture itself is primary,” citing works such as Penderecki’s Threnody to the Victims of Hiroshima, Ligeti’s Atmospheres, even Terry Riley’s In C, where “contrapuntal and tonal construction is made to yield a predominantly textural experience.” (Erickson 1975,139) In compositions by Varese and others, textures, 85 especially dense textures, have been composed for structural purposes, and are not merely “a by-product of contrapuntal, harmonic, melodic, or rhythmic processes.” (Erickson 1975, 139) Several different types of thick textures are described by Erickson—sound masses, layers, clouds, clusters. Stratification—the building up of layers of sound— may result in a texture which can be heard as a continuous mass of sound, or as a sound texture, or as individual layers of sounds. Steve Reich’s Music fo r 18 Musicians and gamelan music are examples of such constructions. The composition of many simultaneous, contrapuntal voices can create a dense web of sound, a sound mass or sound block. Penderecki, Ligeti, and Lidholm have used multivoiced canon (micropolyphony) to produce thick textures. In these micropolyphonic textures, speed, pitch repetitions, pitch distributions, articulations, and dynamics are more important than melodic/contrapuntal formations. If players were merely repeating notes, the effect would be quite different. In John Cage’s HPSCHD, 51 channels of computer-generated sound are played back while several harpsichordists play fragments taken from Mozart and others, creating a dense texture in which the listener may occasionally recognize a familiar pattern. Composers such as Varfcse, Xenakis and Ligeti have composed transformations within massed sound formations from layered texture to homogeneous timbre, from clouds made up of grains to solid blocks of continuous tone. To the composer working with a soundfile mixing program, the creation and manipulation of thick textures, or drones, or ambient settings is a fairly straightforward task. A mixing program makes it possible to use any soundfile as source material, to reshape its amplitude envelope (its evolution), and to use spatial placement, delay, and transposition as part of the same operation. 86 The concept of klangfarbenmelodie, a way of composing sequences of timbres analogous to melody, was proposed by Schoenberg in 1911, but, as Erickson points out, “Nothing is said about the nature of relationships between timbres that might be equivalent to the inherent logic of pitched melodies.” (Erickson 1975,106) Erickson questions “whether there is an ‘internal logic,’ meaning musical sense, to be found in such timbre sequences.” (Erickson 1975,107) He reviews how timbre is most often used in conjunction with pitch to organize sequences of sounds, and observes that klangfarbenmelodie is most likely to be useful as an organizing technique when pitch is negligible, vague or clouded. In sum, klangfarbenmelodie seems to be something left over after other organizing principles are removed, and does not provide a strong primary technique for organizing musical events. But Erickson does not entirely dismiss the possibility of having a linear organization principle based on timbre. He notes with interest the research on channeling (streaming) by Bregman and others, stating: It seems certain that a klangfarbenmelodie having any musical interest cannot be a mere linear succession of contrasting timbres. There must be internal organization on several levels, and channeling may be significant on one or more of those levels. (Erickson 1975, 118) Extension via Rhythm Matching Rhythm, along with pitch, is a strong organizing factor in music. Sound events become related through memory of, and association with, past events. If the space between the onset of sounds forms a recognizable pattern, incoming information is evaluated against that pattern. If it fits, the pattern is strengthened, if not, an alternative pattern is developed to accommodate the new information. Once a rhythm or metrical pattern is developed, incoming events can have different timbres, or intensities, or pitch behaviors, and still be accepted as part of the same rhythmic stream. Conflicting grouping interpretations give rise to musical 87 ambiguities that can be exploited in order to extend a passage without a surfeit of repetition. An example of such a passage would be a three note repeating stream in which two timbres alternate, giving rise to two different grouping interpretations of the same passage. Brahms and other composers have used the compositional technique of hemiola to exploit such ambiguity. More recently, composers such as Aaron Copland have used accents to accomplish similar rhythmic multiplicity of interpretation. Short rhythmic motifs are perhaps the easiest sort of sounds to prolong, because they are so easily memorized, and can be recognized even when other parameters— timbre, pitch, intensity—are greatly changed, and even when the original motif is altered. The composer’s challenge when handling such readily recognizable material is to extend the rhythmic motif by altering it—adding to the beginnning or end, altering the durations, subtracting, and so on—without overexposing it. Recalling a rhythmic motif used in an earlier section of a piece is an effective technique that reinforces unity within a piece, but can also extend the range of sound events that can be used simultaneously. Examples of Mix-Oriented Compositional Techniques in E x te n sio n s My original idea was to make a piece in which the instrumental and the tape parts each act as extensions of the other, each responding to and complementing the other's material, using techniques described above. I wanted to create a piece of music with unambiguous correspondences. I also wanted to incorporate tape material generated by small-scale mixing techniques described in Chapter Four. I had been able to generate a wide range of timbres using a single clarinet tone as the source sound in various experiments related to this treatise, and decided to use these as my primary source material. All but the fourth tape segment in Extensions contain examples of these clarinet tone transformations; these are described below. The instrumentation and the 88 technical demands of the instrumental players were to be kept within the range of a good high school concert band because there is not yet much music of this sort that such groups can play. The percussion sounds at the beginning of the first tape segment are an answer to and extension of the percussion event at measure 40 (rhythm-matching). The dotted- note rhythm heard in both the tape and the instrumental parts here is also used in the first measures of the piece, and will be heard throughout the composition in various transformations. There is an extension of the instruments playing B-natural in measure 41 via pitch- and timbre-matching in the tape part in the next measures, being taken over by the instruments, extending through the clarinet solo at measure 53. The drum strokes were all derived from one recorded bass drum sound. The rhythmic pattern, the different intensities, and slightly different attack times, and the spatial alternations were all created with Mix. The gong sound that emerges in the background, originally a major third higher, was transposed down to B-natural, the first several seconds removed, and the amplitude envelope adjusted, all by Mix. The filtered-noise sound that emerges was synthesized by the CHANT program, specifying five band-pass filters with frequencies and amplitudes based on the first five harmonics of the clarinet tone. The last sound to emerge, an undulating stereo sound, was created by feeding the clarinet tone into the CARL “lprev” reverberation program, specifying a 20 second decay time. The resulting reverberated sound has a pulsing amplitude variation that was minimized by sending the original sound to channel one and a slightly delayed version to channel two, resulting in a general reduction of the undulation and a slight side-to-side movement of the sound. The second tape segment takes over and extends the clarinet whole-tone scale activity around measure 60, providing a transition to the metallic percussion activity in 89 the short section that follows. The first sounds heard in the tape part are recorded clarinet tones moving upwards through the same whole tone scale as the live clarinets. The original tones were too short, so it was necessary to extend their duration by crossfading at a point where the timbre is relatively steady. Between 12 and 22 seconds into the segment, synthesized triangle wave tones playing the same pitches fade in as the tape clarinet tones fade out. The final sound in this segment, which has a triangle-like timbre, was created by mixing together (layering) two triangle waves, a G6 and an F7 (USA Standard). All the sounds in the first 30 seconds of the third tape segment were derived from the same source—a clarinet tone. This tone was analyzed using the phase vocoder program, then the partials were “stretched”—transposed upwards by a common multiplier—and resynthesized. Ten different tones were produced using this process, each tone having a different “stretch” factor. These inharmonic tones were mixed together in rapid succession and these mixes were then restarted in close succession for four generations. The resulting soundfile, “tinmix,” was transposed to one-fourth its original pitch, segments were extracted, reenveloped to be more percussive, and then added to the mix. The first two long, high, “grainy” sounds heard in tape segment three were obtained by band-pass filtering “tinmix.” This tape segment begins as an extension of the the high metallic sounds (triangles and muted sleighbells) being played by the percussion . The character of the music changes suddenly at approximately twenty seconds into the segment from a quiet, ethereal passage to a loud, percussive, “dramatic” section. The live percussion instruments answer the tape's bass drum strokes at approximately 30 seconds into the segment, which recalls the earlier moment in segment one where the tape echoes the live percussion. The dotted-note and other irregular rhythms used in both of these places 90 have been used throughout the piece as the percussion motif. The clarinet-like pipe sounds at about fifty seconds into the tape segment were produced by filtering the gong sound with the CHANT filters discussed above in segment one. The low instruments enter after about 48 seconds, taking over the Eb of the gong in the tape part, and building up a repetitive bass line. Over this bass line, the other instruments develop a complementary repetitive part that culminates in measure 80. Tape segment four is an active section that serves as an answer to the instrumental section which precedes it. The sounds in this section were all created by resonant filter synthesis, sending a pulse to ring CHANT band-pass filters set to frequencies related to or derived from the clarinet tone. The resulting percussive sounds were mixed with slightly detuned versions of themselves to add some depth, and then these sounds were mixed with each other, shortened, re-enveloped, and then set into a number of repetitive patterns. These patterns were mixed together in various combinations, sometimes being offset and sent to the opposite channel to produce a sound that “bounces” back and forth between channels. The gestural image I had in mind for this segment was a decaying echo, which can be heard at the beginning and the end. The last sound in this segment was made by clipping the attack off some of the percussive, marimba-like sounds from the section, giving them a .1 second rise time, and keeping the decay as it was. These tones create a marvelous effect, continuing but softening the forward motion and providing a haunting, disappearing gesture. Just after the return of the initial percussive sounds at about 1'23" into segment four, the band enters on pitches that are derived from the tape sounds (Eb, F, Bb). The fifth and last tape segment draws upon sounds previously heard in the other tape segments to serve as a coda. The instrumental part that leads into this segment, and 91 continues through it to the end is no more or less important than the tape sound here. Both are intended to be slowing down and coming to an inevitable ending. The structural divisions at measures 11, (26), 41, 63,84, 105, 121 are all set off by changes of pitch material. The pitch structure of the piece is oriented around the pitch of the original clarinet tone—Eb. The opening section introduces one of the two main pitch structures of the piece, the C-D-B figure in the trumpet in measure two, and the C- Eb-B in measure 6, and their elaboration in the measures leading up to measure 26, where the second set of pitch material appears—the Eb, Bb, F above the ongoing C-D ostinato. The section from measures 26 to 41 expands the first pitch material. The sudden change of mood at measure 41 is accompanied by the appearance of new pitch material derived from an Eb whole tone scale. The band entrance at measure 63 returns to the first pitch material of the piece, moving toward the second pitch material which has arrived by measure 82, and remains until around measure 105. After a momentary return to the first pitch material, the band moves on to the second to set up the similarly- pitched sounds in tape segment 4. After returning with these same pitches at the end of tape segment 4, the band moves on to pitch material related to the second set for the ending. I discovered early on that the tape part would have to be completed before the band part could be completed. This procedure seemed necessaiy because I was often unable to predict the behavior of sounds that would be created by experimenting with different unusual ways of combining soundfiles, especially soundfiles with inharmonic spectral content. 92 S um m ary The composition of a piece of computer music may involve using various techniques to extend, prolong, or draw out aspects of the source material chosen by the composer *Some large-scale compositional mixing techniques for extending (prolonging) musical sounds, or sound events, based on principles derived from writings on psychoacoustics and composition, proposed in the earlier part of this chapter, have been used in the compositon of Extensions . Instances of how both large- scale and small-scale compositional mixing techniques have been used in Extensions are described in the latter part of this chapter. Extensions for Concert Band and Tape by Neal Yocom (1988) 93 Extensions Neal Yocom (1988) Concert-Pitched Score Instrumentation: Flutes/Piccolo Oboe I & II Clarinet I in Clarinet II in B^ Bass Clarinet Bassoon Alto Saxophone I & II Tenor Saxophone Baritone Saxophone Horn I & HI in F Horn II & IV in F Trumpet I in B^ Trumpet II in B^ Trombone I Trombone II Euphonium Tuba Percussion: Timpani, Piano, Bass Drum, Tam-Tam,Tom-Toms, Marimba (sub. vibraphone), 4 Triangles, Sleighbells, Crash Cymbals Tape: Loudspeakers to be located behind the band, to the left and right. If the tape machine does not have a silent start, without any audible clicks, it should be located outside of the performance room. Performance Notes The repetitive figure that appears in several places in this piece should be performed with more emphasis on the first note than on the second. The second note should be shortened, giving the figure a bit of a lilt. This figure grows out of the chaos in measures 9-10, then carries the piece forward. The notation of the tape part is minimal, but should provide enough "landmarks" to permit accurate entrances. There are two seconds of silence at the beginning of each segment, allowing the volume to be brought up gradually before the sounds begin. In measures 43-53, all instrumental parts should fade in and out smoothly. In measures 44-51, the piano trill should be played by never letting the key come all the way up, resulting in a light, fast, uneven trill. The four triangles, ms. 72-75, should be placed around the back of the band as follows: triangle 1—extreme left; triangle 2— left center, triangle 3—right center, triangle 4— extreme right. The sleighbells should be placed near the left speaker. The slight muffling of the sleighbells is done to match the first sounds of tape segment 3. In tape segment three, a clear tempo is established by the bass drum rhythm at ca. 38". The rests leading up to the band entrance are based on this tempo (J =120). The section beginning at measure 77 reaches a climax at measure 95. The conductor will have to rely on musical intuition to "shape" this section, as it would be impossible to supply all the necessary performance indications. The section starting at 99 should be played with great energy, but not too fast. At measure 128, there is an abrupt change of character from the marcato section into a series of smooth, undulating figures that echo after the percussive chords at 128 and 132. The band entrance at measure 159 comes only one beat after the tape cue arrives. The notation of the tape solo is approximate, and it may take several attempts before the musicians are able to make their entrance with confidence. The last tape segment should start (on cue) while the clarinet and hom soloists are holding the last beat of measure 177. The tempo is taken from the vibrato of the Eb in the tape part. Starting the band exacdy with the tape is practically impossible. The beat in the tape part is quite clear on the third counts of measures 183 and 184, where the band and tape must be in synchronization. 96 Extensions Neil Yocom 1988 Fluta/Piccolo Oboe Clarinet X i Clarinet XX Bass Clarinet Bassoon Alto Saxophone & - - Tenor Saxophone > = = = Bari. Saxophone B om 1/111 Horn l l / T V Trumpet X Trumpet XX Trombone X Trombone XX Euphonium Percussion X Percussion XX Tape neoorder © 1988 NEAL WESLEY YOCOM 97 4 5 6 F its . Oboe Cl.l Cl.2 BsCl. B s s n . r x r p p A.Sx. ST ^ ...... T.Sx. ST U " y ------_------B.Sx. *•' U ’ 7' *------H. 1/3 i f f T T -----■" ■ * ; n 1 l------r s’- i .... --J f l j - - ^ 4 T ^ h .J' i* J i H.2/4 ----- EB Jr n n i ------r "' XT= • >^ > p p ------*.J-^ w tutti: tolo: — m u b s s u - — = Trp.l ------WWY j J ~ M i H aw * iHiiJd ^ >> > >> > i t Trp.2 ..—r i" l — ------~ W W |"| 1------** > > >J'------ Trb.l t _ = Trb.2 . Q - r ------■ * -.* ------jcr m --- £------Euph. sr m Tuba : Prc.l ^!I i 1 a Prc.2 j n h ------JiJ #i------* XT > Tape 98 F i t s . ; ^ ...... - ■■ ~= Oboe !r ====== Cl.l Lfc . p—^ —_ & J — J — 3 — ■ i -J- o r jj- i>—*flj -j J ---- l^ p i ^ p _ . ----1------p M L - J Cl.2 - v — j : ~ P p p ♦ " v p fct: - j |— ■■ f — f 1 f t fT r f r r rT BsCl. r f""7^ ^Tr f Bssn. 3 3 p $ ______, J~"'J--—-J JJi*' ^-tJ ^ ^ ^ V »11 - - H.2/4 Trp.l Trp.2 Trb.l Trb.2 Euph Tuba Prc.l p ~ - '~= j , , - .;=.-1 |-|— Prc.2 1 > I i H 3 — 3=3 Tape 99 A llp/iretto G r a a i d S O 11 4 12 1'ineuo tempo J-IOC F its . Oboe C l.l Cl.2 BsCl. Bssn. A.Sx. T.Sx. B.Sx. H. 1/3 H. 2/4 Trp.l Trp.2 Trb.l Trb.2 Euph. Tuba PUno A Mtriaba cia. Prc.l — 4 = m m p w w w TV*1- *, M M M Prc.2 ^ 4------Tape 100 13 14 15 16 Fits. > ' Cboe > ' > ' Cl.2 BsCl. Bssn. A.Sx. T.Sx. B.Sx. H.l/3 H.2/4 Trp.l Trp.2 Trb.l Trb.2 Euph. Tuba Prc.l Prc.2 Tape 101 17 18 19 20 F its . Oboe C l.l Cl.2 BsCl. Bssn. A.Sx. T.Sx. B.Sx. H.l/3 H.2/4 Trp.l Trp.2 Trb.l Trb.2 Euph. Tuba Prc.l Prc.2 Tape i 102 21 22 23 F its . •%rfr r- fn «:--r- > > LJ * ^ — Oboe —*--#-F^-r jt] = 4 - J* ^ ^> * a >■ > f > r ?— C l.l >w» Cl.2 ------fW -.d' A 9 J «l JJ \J. J > > > > > > > # « > y1 - BsCl. ----- ' / Bssn. A.Sx. tJ v> *♦ "W- * f Trb.l Trb.2 Prc.l 9 JJJJJJJJ . - '. . . r r r . ' Prc.2 ------4----- Mt— ‘ / Tape 103 26 2S > > > 27 Fits. crcic. dim . > > > > Oboe C l.l l T * , J Cl.2 ± J '3 5 - j j 3 3 BsCl. .r. r..-X -dua. Bssn. iisum * A.Sx. >■ ...— T.Sx. OWC ~~~~—. di'n_____ ■— —> B.Sx. -----1 ‘ /—-dim- H.l/3 H. 2/4 cnf- t - L T - J r- Trp.l dim. Trp.2 £■** r r r- T> ? f f f J§ Trb.l Trb.2 Euph. Tuba Prc.l -n S > > » > V Prc.2 m e . hw. A Mar.: Pno. 104 20 Fits. Oboe C l.l Cl.2 S< w * BsCl. = = 'lW:-= ■ ■■■■ Bssn. »Irm TrfTr rrf fTFflrntm fTr rr= M £i A.Sx. T.Sx. -O- H.l/3 B. 2/4 Trp. 1 Trp.2 Trb.l Trb.2 Euph. Tuba Prc.l Prc.2 Pno. 105 32 33 34 35 Fits. dim. ■asr C l.l Cl. 2 BsCl. Bssn. rrrrrrrrirrrrr rrrirrrrrrrrirrrrrrrr A.Sx. dun. > T.Sx. «■ B.Sx. H.l/3 H.2/4 Trp.l Trp.2 Trb.l Trb.2 Euph. Tuba Prc.l Prc.2 Pno. 106 Fits. > > > .>> > > #i«. > ■ 0 M .0, O .... Oboe f =f=f=t- = f - > #inn. ft .._> > ** T > ■ » j p——— - — -4^ C l.l r r ^ t i Cl.2 V J" J—J-JW- «l JJ- BsCl. Si** ff r f -r. £■ ~T f Bssn. / 1 --I—1 f-l—■ 1 ■— LI id- d_i j-.yf• I--- fi -1 - d-l f f r T J — + - A.Sx. ------E ^ $ > ^ *> > > >5 ► > > * > > > > :► > > U = : . = - T.Sx. | ------W J-J -+ — it1 " ^ > > > > B.Sx. H.l/3 - ” r ' H.2/4 I .jj -4-J-4 J-- ■■ J j-j -J -J J--IJ j. > > — > I ^.=^5|^j=)=-gq Trp.l e tc i - i-i!- 1 = * — — — -EJ r- 1 1 p ■m * J - Trp.2 r ~ii j ±=h=?= aim. > L ? > > > *<*•■ * ------.— : p - f : y------N ffff=#=f=fH Trb.l ------= ± = b = — / > > > > > Trb.2 J : ± : ■:-----1------z — / Euph. Tuba : Prc.l , > » * ► 1 "p ^f~=— h l ^— > t ■ . Prc.2 J— Y 1------1------1---- LX—|------ Pno. 1 r - i ^ . v n m ' jj-j-j j r -jr 107 F i t s . -+} Oboe ff ^ --- p p ; . ^ --- p—y—..> C l.l f t— Cl.2 l* = = I" ■m.JR ww ww —4^----1----=------BsCl. > > > i > ■ ■ ..... r r r r = ..^ - > —--...... Bssn. I--. ---- t-JiJ-- fjj >j j j j—i A.Sx. = j.-... -N—J---fJ-- 3-- j-- 5“= * “ > > >'*— > > > - => $ = ——------ft .■-H n i i — T.Sx. Eg.U> >>> > > > > ^ _^}y— }—_ ------B.Sx. H.l/3 H.2/4 Trp.l Trp.2 JTITJTJ'J Trb.l Trb.2 T Euph. Tuba Prc.l Prc.2 START | J l _ l V Tape lipew Segment 1 108 42 43 44 45 46 47 46 Fits. Cl.l T» PP Cl.2 PP BsCl. Bssn. PP A.Sx. PP T.Sx. B.Sx. H.l/3 PP B. 2/4 Trp. 1 PP Trp.2 PP Trb.l Trb.2 Euph. Tuba pUno: (m cpgfjolt) — uneven jferrot Prc.l PP Prc.2 END Tape Tape Seg.l 109 so S2 53, 54 Conducting Fits. Oboe C l.l PPP Cl.2 PPP BsCl. Bssn. A.Sx. T.Sx. B.Sx. H.l/3 PPP H.2/4 PPP Trp.l •o Trp.2 Trb.l PP Trb.2 PP Euph. PP Tuba Prc.l (h o ld piedftl d o w n ) Prc.2 T a p e 110 w 56 -5 6 - SolO Cl. Cl.l Cl.2 T» Pno. Tape eo 61 62 63 Solo Cl. % >|------Cl.l * .... J rt»i- Cl.2 Pno. |: START Tape T«pe Segmenl 2 65 66 67 66 66 70 71 Solo Cl. Cl.l Cl.2 flw) (I*.) JL END \S. Tape c«. 12" I l l 72 73 74 75 Trgl 1 PP Trgl 2 PP Trgl 3 PP Trgl 4 PP muted with towel: SI.Bells b i t 0” 7.5” 10- Tape 18” ca. 20' J =54 Tape Loud Sound ca. 38“ Tape J =120 bus drum Pipe-lilce sound: ca. 48" Tape m V.S. 112 77 Pesantc J =68 78 Fits. Oboe C l.l Cl.2 BsCl. Bssn. A.Sx. T.Sx. B.Sx. H .l/3 H. 2/4 Trp.l T rp .2 Trb. 1 Trb.2 Euph. tv buu. . t Tuba T.-Ti Prc.l P iin ^ R y t nib* line ptui Iv t »to*e BJ>. (muffled) Prc.2 113 82 83 84 65 56 87 Fits. Oboe C l.l Cl.2 BsCl. Bssn. A.Sx. T.Sx. B.Sx. dlv. H.l/3 H.2/4 Trp.l T rp .2 Ttb.l Trb.2 Euph. loco Tuba Prc.l Prc.2 m END Tape Tipe Segment 3 114 at 80 00 02 Fits. Oboe C l.l Cl.2 sim. sim. Bssn. A.Sx. T.Sx. B.Sx. H.l/3 sim. H.2/4 tin. Trp.l Trp.2 stm. Trb.l Trb.2 Euph. Tuba tan p. Prc.l Prc.2 T a p e 115 07 =120 Fits. Oboe Cl.l Cl.2 BsCl. Bssn. A.Sx. T.Sx. B.Sx. H.l/3 H. 2/4 Trp.l Trp. 2 -OjL Trb.l Trb.2 Euph. Tuba Prc.l Prc.2 T a p e 116 sg Allegro marcato 100 101 102 Fits. m a r a jln Oboe C l.l Cl.2 tim . Bssn. A.Sx. T.Sx. sim. B.Sx. H.l/3 H.2/4 Trp. 1 Trp.2 Trb.l Trb.2 Euph. Tuba Prc.l Prc.2 sim. T a p e 117 103 104 105 106 Fits. Oboe C l.l Cl.2 BsCl. Bssn. A.Sx. T.Sx. B.Sx. .> H.l/3 B.2/4 Trp. 1 Trp.2 Trb.l Trb.2 Euph. Tuba Prc.l Prc.2 Tape 118 1C?? 108 100 110 Fits. ¥ ' I L T r b i t , f f r = s = Oboe §U . .... ' J _ ^ | A I J .£ = - n - r - i — ■- —i— H 'f f n C l.l — J J J ^ J ^ frH -fr-tr-g- y ^ SJrry. M j J J " N -■ t- i— r - i — —n~ i— r - i — Cl.2 j j l J - J j l J - p 3 — "Sim. - j T ^ ~ ...... ^ = = BsCl. 'TTTTy <---- 1---- 1---- f ...- - ^ 1 ^ = j Bssn. A.Sx. T.Sx. B.Sx. H.l/3 B.2/4 Trp.l 13087482 Trp.2 Trb.l Trb.2 Euph Tuba T a to t Prc.l linp. Prc.2 Piano Tape 119 picc. only tutli Fits. Oboe Cl.l a .2 BsCl. Bssn. A.Sax T.Sx. B.Sx. H.l/3 B. 2/4 Trp. 1 Trp.2 Trb.l Trb.2 -» Euph. Tuba T» Perc. Pno. 120 Fits. Cboe C l.l Cl.2 BsCl. Bssn. A.Sax T.Sx. B.Sx. H.l/3 B.2/4 Trp. 1 Trp.2 Trb.l Trb.2 Euph. Tuba Perc. Pno. 121 F its . - g - f — ■ = r f = f Oboe W = - t r ^ - ~ M r ^ m p n marc. = r # = F 1m v , - = n - 4 # = Cl.l - t J " T ^ ------> > =4 =4 = rim. i t ; ...... 2 K. 1> 1 . ------= e p p = " f •••••— p a—EJTT55J a ^ ------± 3 — d — d J - i J — 1 ■■'...... «/ w „ ------= f c . ------1------1------— - = i — ...- — " T r — f T T T 1 BsCl. — J — J — *£— i — a _ J ------J ■ STTti = = & H y j j - k t > , ' = # — i n p - i ~ - Bssn. j j .j j ■ t j i - J ------1 5 — *<------r . L » - = y fim. A.Sax Jd .J d J - d ^ J d . $------' < U « L ^ - 1 J J JJ ]"'!^ j - ■ J J J sim. j , j « . . . . = d ^ j T.Sx. ------_ ^ s : J J B.Sx. H.l/3 H. 2/4 Trp. 1 ' t i m l Trb.l Trb.2 1 2 2 123 124 125 F its . m m L*J , >> > P i k —V- r - r Oboe ——1.r—iRt--- V-—j- —f —rLl -f =--=4=| p - _ _ C l.l Qv- =F=fr: fforf ’ ■ r j-IJ 1— Lbfr ■ ■<— 59=J 7TTT Cl.2 -. > —5=i—T-F BsCl. f j=| A r - -=feF*±> ^ -.4 y ~ Bssn. r-r-nri-'- 1 A. Sax J J JJ > -,-1------lv-1 T.Sx. H J j-J --- U f J> H tdr: ..-^ B.Sx. H.l/3 = L = ^ ^ ^ J = = i _ i V B.2/4 bjii _i.. 1 1 1 .1..... --m 3_1_=aa '_CH—1------H353E3 =3> > J J ...... = ...J3= ■ ^.-^-T-r-f _ - - ..... f : r r r r Trb.2 Y ; i, 1 y ...-U U=z > > > > >—"N > > > ,1. 1 1 ' N4-k ♦ ♦ -f- •*> It 'f' ’f' T’T' Euph. Tuba 1 - >feB > .J- > > Prc.l Pno. JJJj | j | ^ i Pno. F its . m y 4 > b i> . r ■ = ----- f— Cboe = ( = P.-— rrr-r-^ ^ 3 r* T f- Cl.l n i i | |-| »■ j-rrT> rT ^ rfy T r r V , r j t T n f - p - ----j-----r- 'T f ^ n "1 Cl.2 — »*— »-— ¥------— t z z — > — BsCl. rM —ra -==» — ^ r r 3------fcSJ t r p ^ ‘‘V j L tL Bssn. J ' ■ ■ = — A.Sax : i-*- '" rfrl _ iJ j J— F * ^ W (i." +. r P -1J r -* l - I*' -----■ ^ * k i i i . i i i'i„j T.Sx. , ■.. & --3 * ------y — i=5; > j _ j A__SA ^ *> J j B.Sx. h h .l' 1 < » 7? ill ...... — H.l/3 1 .....— Q - = » > > ^ ’ > 1 ...... i ------mm—m -■ H.2/4 § 1 * ..... O Q A _ J A ______i J A ...... j~ > > p > 7 > » t —* t h Trp.l . ■. . r ------— . T 1 - ^ — - M , j ...... - ___ - t M — ■— p = — J-fl-J— J H l-""T -TlJ Trp.2 g - * — t ...... g — i W lki " ilN . Trb.l tmfprt marc. ^ r ^ 1 Trb.2 tm qxt marc. * r t — P ------j f r r - i t — f - .. - o r r r= Euph. r .„. i ...... i — i ...... ? ? r - 1 — i - i ------1— - l l«^pr » ■ > ■ " J l ...... - - - Tuba 1 , - d i ' P j r r ^ y Prc.l 124 128- J-75 129 Fits. Cl.2 j — ■H wri ffi "P^— - Q j H J ...... & — 0 J — » -O^ dim. H.l/3 £ — H.2/4 Trp. 1 mp v i j j Trp.2 6" a. j J j J U ! J ■' J ^ Trb.l Trb.2 Euph. Tuba Prc.l Pno. W u W t t T t ^ t>,^ f S i £ , 5 M 3 L k - — — , - = s ^ Pno. Cboe a BsCl. Basn. A.Sx. T.Sx. B.Sx. H.l/3 H.2/4 Trp. 1 Trp.2 Trb.l Trb.2 Euph. Tuba 126 134 ^ 132 K ------r — F its . dum ------i>'Y Oboe ------j---3— i------ Cl.l Im - t .g rn rg-g« sCTCTTTlHS i v ? ; i : a .2 ® S-00 BsCl. !>--■■ - ---- Bssn. A.Sx. T.Sx. B.Sx. H. 1/3 B. 2/4 Trp. 1 Trb.l Trb.2 START 127 135 136 137 136 Fits. Oboe C l.l Cl.2 BsCl. Bssn. A.Sx. T.Sx. B.Sx. H.l/3 tt* ...... ~ - V /1m - -■■■ H.2/4 (1—— ^— ( fcb------=------Trp. 1 I - • . =- 1 ------=------Trp.2 ft )— ..... " 1 Trb.l Trb.2 Euph. Tuba 1.—» I _ — Pno. fyh rL ------I ------ T a p e "— 7- m — LLT^'LLl 1 H+fl 128 139 140 Fit*. Cboe C l.l Cl.2 BsCl. Bssn. A.Sx. T.Sx. B.Sx. B. 1/3 H.2/4 Trp. 1 Trp.2 Trb.l Trb.2 Euph. Tuba Pno. Tape 129 141 142 I t ■ r — — Tape Recorder sim. i / / / / ■■■ / / / / f _ - . f ------ 143 144 145 T ape r * f - f 146 147 148 14B Tape :f f f f f r r r f f r r r 150 151 152 153 T ape a------y ------y ------y ------L f - ^ U T T f ~ f ^ f r f ^ t ^ 154 155 156 157 $ = = = i g T ape / 1 " / / / 1 t - f / T ^ r .....f t r 1 ^ f r H 130 158 159 160 > T«kt tempo from tape: Fits. Oboe C l.l Cl.2 BsCl. B ssn . A.Sx. T.Sx. B.Sx. H.l/3 H.2/4 Trp. 1 Trp.2 Trb.l Trb.2 Euph. Tuba Prc.l Prc.2 c*. 122' Tape > 131 161 162 F i t s . Oboe Ugatisximo Cl.l UgaJixsuno Cl.2 BsCl. ^ Bssn. A.Sx. T.Sx. B.Sx. H.l/3 .. ~ L J — \A ---1--- — MIIIIIMM------IL._B------1------H.2/4 fe-fc — ri=J J J J — J- — *- -frj— < i------:J ... — # J ------:------Trp. 1 § * • ------JL_x ----- :------— ------Trp.2 $ -h .= -= - -=j hj—a ------Trb.l Z t ------:------ Trb.2 Euph. Tuba Prc.l Prc.2 T a p e low rollin* ftound: 132 163 164 166 166 167 Fits. C l.l Cl.2 BsCl. Bssn. A.Sx. T.Sx. B.Sx. H.l/3 H.2/4 Trp. 1 Trp.2 Trb.l Trb.2 Euph. Tuba Prc.l J" Prc.2 END Tape Segment 4 133 160 168 170 171 F i t s . k - = H Cboe 1 — ------:—— «*> ..J, -J .. -...... Cl.l E M = » r r !f r r r ' r r P_T ...... - ■ = = = * = T-- a .2 y y r = b S - - " '* ...... ~^p—*—v — f— ■ f — f f — I , ^ : = BsCl. ft - j . ------= -W1------•J* B ssn. A .Sx. T.Sx. B .Sx. H.l/3 9 ...... ; ■ - = B.2/4 Trp. 1 § — _ — t - = = Trp. 2 Trb.l Trb.2 Euph. Tuba Prc.l 4 m . a/ \itnlt Prc.2 Tope 134 174 T 175? 176 7 1777 - r \ Clarinet 1 p- — dim Clarinet 2 u p r. p PP Bass Clarinet PP Bassoon PP lo lo Horn 1/3 Horn 2/4 START (on third beu) Tape Tepe Segment S Trice tempo from tape 176 (ca. =84) 178 183 ( > ) Cl.l C l .2 B.C1. H.l/3 H.2/4 Tughmcul T u iM tin i ' m m m H.l/3 Jim. cl menu H.2/4 ■=■ n dim. cl mujce Tape z £ U )J % 9m : m m m m m poco d poco dim A PPEN D IX A LIST OF MIX FLAGS AND KEYWORDS 136 137 SZ Mix [flags] [mix_command_file] Flags: -c check mix commands; stop before mixing -d [num] global duration (leave a space between "d" and num) -f floating point mix; output float file -F floating point mix; normalize, output short intfile -h print this help synopsis -1 floating point mix; warns if overflow, short int file out -p create "mix.print" file -P put synopsis of mix run queue into output file header -s [num] global skip (leave a space between "s" and num) •S run silently. No diagnostics or warnings at all. -v verbose option. Print every possible message. Keywords (Mix commands): thos^in [] brackets are optional. input filename All keywords until next "input" affect this file. start, num Start time for this file (0). dur num How much of this file to use. (all) after [num] Start time, from last file’s end. (0) skip num How much to skip at beginning of this file.(O) ampfac num Amplitude multiplier for this file. (1) env value, tim e... Breakpoint pairs for amp env. xenv value, time, slope... Breakpoint triplets for expon. amp env. fin num Fade in duration for this file. (0) fout num Fade in duration for this file. (0) xin num "S"-shaped fade in option (for crossfading). xout num "S"-shaped fade in option (for crossfading). send mulfac [from_chan] to_chan Output channel sends. Mono files may omit second argument, loop If duration is greater than file dur, loop, gdur num Global duration. gskip num Global skip from the beginning of mix queue cdsf directory_name Reset default soundfile directory, ochans num Number of channels in output soundfile. srate num Sample rate for output soundfile. transp factor [factor,time] Breakpoint [pairs] for transposition, pan angle [angle,time] Breakpoint [pairs] for panning. (MONO ONLY). "!" shell_command Execute shell command, put Puts synposis of run queue into output file header, mix Executes a mix, returns for more. APPENDIX B PROGRAMS USED TO GENERATE MIXSCORES 138 139 m akenum s.c /* Uake numbers to be used as start times by stariem.c */ finclude FILE ^open(), *fp; float rnum, num, atof(), dur, sum = 0, mulfac = 1; char %utname[80]; register Int i; km£ randomO; srandom(i); fdr(i=0;i |printf(fp,"0.0\n"); n u m = atof(argv[l]); |f(argc = = 4) mulfac = atof(argv[3]); dur = atof(argv[2]); lf(argc = = 4) /* if randomizing is wanted */ while{(aum -f= num) < dur)| rnum = (float)(random(i)/215000000); fprintf(fp."Xf\n",sum + (rnum * .1 • mulfac)); e lse vhile((8um + = num) < dur) fprintf(fp,"Xf\n".aum); fcloae(fp); exit(0); I „ usageQ fprintf(stderr,"Xa%a", •\t makenumsmail increment duration [random scaler] \n", "\t\t creates file \"numsby.increment\"\n"); exlt(0); 140 startem.c /* Restarts a soundfile at times readfrom a file V fin d u d e fprintf(fp,"in J5s\n",inname); if(argc > = 3) vhile(fscanf(fp2"fcf'.Jest) > 0)j fprintf(fp,"in %s\n",inname); lj>rintf(fp,"st %g\n",st); fprintf(fp,"fo .04\n"); else vhileCscanff'T&f'.Jest) > 0)( fprintf(fp,"in %s\n",inname); fprintf(fp,"at %g\n",st); fprintf(fp,"fo .04\n"): fprintf(fp,"ou %s\n",outname); fcloae(fp); lf(argc = = 4) | «printf(inname,"mix -S %s\n",outname); ■ystem(inname); «xit(0); usageQ fprintf(stderr."%sJ5s". *\t atartem soundfile values_file [go)\n", "\t outputs mixacore or soundfile: aoundfile.time\n"); exit(0); frag.c /* F n g chops a smmdJUe into little pieces •/ findnde first— ; •iae { fprintf(fp,"st +Jtg\n", outstep); ^ fprintf(fp,"sk Xg\n",i); fprintf(fp,"du Xg\n",fragsize); fprintf(fp,"fi %g \n M. frags ize/4); iJ>rintf(fp,"fo J5g \n",fragsize/4); fprintf(fp,"ou %s\n",name); fcloee(fp); lf(argc > 6) | sprintf(command,"raix -S Xs\n",name); ^ system(command); «dt(0); uaageQ fprintf(stderr,"Xs/l8%s", \t frag soundfile [fragsize(.05)]\n", "\t\t[outatep(.05)] [instep(.05)J [dur(2)] [go] \n". "\t output mixscore: filename.frag \n"): «dt(0); detune, c /* Ditime a soundfile by mixing it with copies of itself */ #lnclude ifl drgc < 2) usageO; ifl |argc > 2) howmany = atoifargvf2l); if argc > 3) howmuch = atof(argv[3J); ■trcpy(n»me,argv[ 1 ]); •trcat(name/'.det"); if((fp = fopen(name.'V)) == NULL) f fprintf(stderr,"Can't open output file Xs!\n",name) { exit(l); fprintfffp/’in Xs\n",argy[l)); forintf(fp,"am %f\n",1.0/howmany); fcnr(i=0;i TAKEOUT awk ’ # take something out of a mixscore $1 !~ /am/ {print $0} ’ < $1 > $l.new m v $ l.n ew $ l PUTIN awk ’ # add something to an existing mixscore $1 !- /in/ {print SO;} $1 - /in/ { print $0; printf "am-l\n" } ’ < $1 > Sl.new mv Sl.new $1 CHANGESTART awk ’ # Spread out start times by adding fixed amount to every start time # for files that use "st +" — otherwise it would just add offset BEGIN { incr - .05 } $1 !~ /st/ {print $0} SI - /st/ { if(S2 > 0){ if(S2~A+/> {S2 += incr;print "st +"S2;} else {S2 += incr;print "st "S2;} } else {print "stO"} } ’ Abbott, Curtis. “Efficient Editing of Digital Sound on Disk.” Journal of the Audio Engineering Society XXXII (June 1984): 394-402. Banger, Colin, and Bruce Pennycook. “Gcomp: Graphic Control of Mixing and Processing,” Computer Music Journal VII (Winter 1983): 33-39. Deutsch, Diana. “Grouping Mechanisms in Music.” Chap. in The Psychology o f Music. New York: Academic Press, 1982. Erickson, Robert. Sound Structure in Music. Berkeley, California: University of California Press, 1975. Fletcher, Harvey, and William A. Munson. “Loudness, Its Definition, Measurement and Calculation.” Journal of the Acoustical Society of America V (1933). Haynes, Stanley. “The Musician-Machine Interface in Digital Sound Synthesis.” Ph.D. diss, University of Southampton, England, 1980. Haynes, Stanley. “The Computer as a Sound Processor: A Tutorial.” Computer Music Journal VI (Spring 1982): 7-17. Ingbretsen, Robert and Thomas Stockham, Jr. “Random-Access Editing of Digital Audio.” Journal o f the Audio Engineering Society XXXII (March 1984): 114- 122. Kowalski, Michael and Andrew Glassner. “The N.Y.I.T. Digital Sound Editor.” Computer Music Journal VI (Spring 1982): 66-73. Laske, Otto E. “Considering Human Memory in Designing User Interfaces for Computer Music.” Computer Music Journal U (December 1978): 39-45. McAdams, Stephen, and Bregman, Albert. “Hearing Musical Streams.” In Foundations of Computer Music, ed. C. Roads. Cambridge, Massachusetts: MIT Press, 1987. Moorer, James A. ‘The Lucasfilm Audio Signal Processor.” Computer Music Journah VI (Fall 1982): 22-32. Roads, C. and John Strawn, eds. Foundations of Computer Music. Cambridge, Massachusetts: MIT Press, 1985. 144 145 Roads, C. “Report on the IRCAM Conference: The Composer and the Computer.” Computer Music Journal V (Fall 1981): 7-27. Roederer, Juan. Introduction to the Physics and Psychophysics of Music. Second Edition. New York: Springer Verlag, 1975. Rossing, Thomas. The Science of Sound. Reading, Massachusetts: Addison-Wesley, 1982. Schoenberg, Arnold. Theory of Harmony. Translated by Roy E. Carter. London: Faber and Faber, 1978. Snell, John. “The Lucasfilm Real-Time Console for Recording Studios and Performance of Computer Music.” Computer Music Journal VI (Fall 1982): 33-45. Wells, Thomas. The Technique of Electronic Music. New York: Schirmer, 1981. Wishart, Trevor. “Sound Symbols and Landscapes.” In The Language of Electroacoustic Music, ed. Simon Emmerson, 41-60. London: Macmillan, 1986. Wishart, Trevor. “The Composition of VOX 5 at IRCAM.” Institut de Recherche et Coordination Acoustique/Musique, Paris, 1987. Yocom, Neal and Dan Timis. “The IRCAM Soundfile System.” Institut de Recherche et Coordination Acoustique/Musique, Paris, 1986.