c.~. 5' DIGITAL SAMPLING & SYNTHESIS

TEACHER UNIT GUIDE NOTES

BACKGROUND INFORMATION is considered to be the most important development in sound production in the last several decades. Experiments and research with digital synthesis began in the late 19508 with instruments like the RCA Mark 2 Electronic and other large digital . Itwas not until the 1980s that digital sampling and digital synthesis became a reality in instruments that most musicians could afford and use in live performance. Digital sampling first became widely available to musicians in 1981 when E-mu released the Emulator digital sampling keyboard. As for digital synthesis, the real revolution began in 1983 with the introduction of the Yamaha DX7 digital algorithmic FM synthesizer.

THE PROCESS Overview Electronic audio signals, exist in one of two worlds, analog or digital. An analog audio signal consists of a continuously rising and falling voltage that can be directly modified or amplified. The"shape" of the electrical signal directly corresponds to sound waves that travel through the air. In the digital world all information, including a recording of a sound, exists in the form of binary numbers.

Digital audio recording has many advantages over analog tape recording including a virtually complete lack of wow and flutter, better signal-to-noise ratio (i.e., no tape hiss), no tape generation loss, and greatly enhanced editing capabilities. Digital sampling, digital synthesis, and digital signal processing have created sonic possibilities unthinkable without the power of digital computing. Digital audio technology has revolutionized the way we can create and listen to music.

Digital Sampling The process of digitally recording a sound is called digital sampling. Simply put, digital sampling is the recording of a sound using a string of numbers to represent the sound. The string of numbers is created by measuring the amplitude (or height) of a waveform thousands of times per second. Each individual

measurement or IIsnapshot" is referred as a sample. (The word "sample" is also used to refer to the entire recording as welL) In some ways the sampling process is similar the process of creating a motion picture. The camera's film captures the action as a series of snapshots or still photographs. The projector plays back the snapshots in rapid succession, creating the illusion of continuous motion.

Fig. 5-1 When sampling, an analog signal from a or other sound source is converted to numbers, or digital information, by an analog-to-digital converter (ADC). To playa sound back, the opposite process is carried out by means of a digital-to-analog converter (DAC); that is, the string of numbers is converted into an analog waveform and played through a speaker system. Some digital instruments will refer to the internal sounds or the recording process as PCM sampled. The term PCM, Pulse Code Modulation, is the digital coding technique used in virtually all digital instruments.

The sound quality produced by the sampling process is determined by two factors: sampling rate and sample size (resolution).

Sampling Rate The number of times per second that the measurements, or samples, are taken is called the sampling rate. Typical sampling rates are between 15,000 and 52,000 samples per second. For compact discs, a sampling rate of 44,100 samples per second (44.1 KHz) is used. Thus, for every second of sound you hear on a CD, 44,100 numbers must be stored! (Actually there are 88,200 numbers stored; since most CDs are recorded in stereo.) This sampling rate of 44.1 KHz has become the standard for digital and sampling keyboards as well.

Fig. 5 - 2 Figure 5 - 2 shows how sampling rate effects how accurately a sound is represented in the sampling process. Even though thousands of individual samples are taken each second, each measurement must be a discrete number. This results in a stair­ step representation of the sound rather than a continuous change of an analog signal. The higher the sampling rate, the closer these stair steps approximate the actual analog sound. Many CD players and DAT machines advertise oversampling as a feature. Oversampling is a process that attempts to smooth out the stair-step effect by interpolating points and adding more samples between each of the steps. For example, 16x oversampling places 15 estimated samples between each actual sample point.

Sampling rate has a direct effect on the frequency response of the system. "_ , Fig. 5 - 3 Frequency response is a measure of the range of frequencies that the system is capable of producing (e.g., 30 - 22,000 Hz). As a rule, the highest frequency that can be reproduced in a sampling system is equal to one-half of the sampling rate. That number, one-half the sampling rate, is called the Nyquist frequency. Thus, for the CD standard rate of 44.1 KHz, the highest frequency that can be reproduced is just over 22 KHz, which is just above the range of human hearing. Any frequencies above the Nyquist frequency are filtered out b~fore and after the conversion process using specially designed filters.

Many samplers allow for variable sampling rates. The reason is that the highter the sampling rate, the more memory is used up in the system, both for storage and for processing. Since some sounds (particularly in lower frequency ranges) don't require as high a frequency response for reasonable quality of reproduction, setting the sample rate lower can help minimize memory allocation.

Sample Size (Resolution) Sample size or word size, refers to the number of bits used for each individual sample. The greater the number of bits used, the higher the resolution of the system. In an 8-bit system (where 8 bits are used for each individual sample), there are 256 possible numbers to represent the amplitude of the waveform at any given instant. In a 12-bit system, there are 4096 different possibilities; and with 16-bit resolution there are 65,536 possible numbers to represent each sample. Thus, with larger sample sizes, the grid of resolution is much tighter and there is greater accuracy in defining the waveform.

Fig. 5 - 4 Sample size has an effect on the signal-to-noise ratio of the system. The signal-to­ noise ratio is a comparison between the level of desired signal and the amount of undesired noise present in the sound. It is measured in decibels; the higher the dB level, the better the signal-to-noise ratio. An 8-bit resolution provides a signal-to­ noise ratio of about 48 dB, which is roughly the quality of a low grade . 8-bit digital audio is considered to be the lowest level of usable resolution and is used where memory conservation is critical. It generally has a grainy sound with noticeable hiss at low levels. A 12-bit resolution delivers about 72 dB signal­ to-noise ratio. A 16-bit system, the current digital audio standard, will produce about a 96 dB ratio. Compact discs, DAT, and most electronic music devices use the 16-bit word size for a very "clean" sound.

Low end, economical devices sometimes use a smaller sample size for the same reason as a lower sample rate: since more memory is needed to store more bits for each sample, a smaller sample size allows for a longer recording with the same amount of memory.

Memory Requirements As stated, sonic quality with digital audio comes at the price of increased memory requirements. As the sampling rate and bit resolution increase, the amount of memory to process and store the additional samples and bits also increases proportionally. The CD standard of 16 bit/44.1 kHz digital audio requires about 5 MB of memory per track per minute. That's 10 MB per minute for stereo audio and more for .

High Definition Audio The 16-bit/44.1 KHz sampling standard set by the CD has remained for over a decade. Recently, however, the audio industry has begun to work with even higher fidelity digital sound. Referred to as high-definition audio, the new high-end standard is moving toward a 24-bit sample size and a 96 kHz sample rate. A number of devices already support 20- or 24-bit resolution and sample rates greater than 44.1 kHz. With the benefits of high-definition audio, however, comes the added cost of increased processing speed, faster disk access, and greater TYlPTYlOTV TPlllliTf'TYlf'nh:L At 24-hit/9t1 kH7.. thp mf'mOTV Tf'll111Tf'mf'nt<:: illTYln In 17 MB per track per minute. While the audiophile listener with high-end equipment may perceive a noticeable difference, the average end user with a typical stereo or sound system may not notice much difference in sound quality.

DIGIT AL AUDIO APPLICATIONS (CD) The compact disc (CD) is the most common application of digital audio, and currently the standard medium for distribution of commercial recordings. Until recently it is has been a read-only format, but compact disc recorders (CD-R) are now affordable and becoming common place. The CD-R allows the average user to "burn" their own CDs which can contain digital audio tracks or computer data. Standard audio CDs use a sample rate of 44.1 KHz with a 16-bit sample size and can store about 74 minutes of stereo sound. While CDs have been the industry standard for over a decade, DVDs with a new high definition audio standard may eventually replace the current CD standard.

Digital Versatile Disc (DVD) The DVD (Digital Versatile Disc) has become the new standard for distribution of consumer movies and will likely eventually replace video tapes. The video format uses MP2 compression, the video compression standard, and supports five channel CD-quality sound along with the video. Because of its 4.7 GB storage capacity, it greatly surpasses the capabilities of the industry's current audio standard, the compact disc. As 24-bit, 96 kHz, high definition audio gradually replace the current standard, the DVD medium may eventually replace CDs for audio as well as movies. DVD players will also play standard audio CDs.

MiniDisc (MD) The MiniDisc (MD), was first introduced by Sony in 1992 and was originally intended to replace the cassette and compact disc platforms. While that has not come to pass, this means of storage has become the new standard format for many of the portable multitrack, ministudio recorders. The MiniDisc, like the CD and DVD, is a laser-based technology. Each disc has a storage capacity of 140MB, which yields about 37 minutes of four-track recording. In order to record that amount of musicwith only 140MB of storage capacity, a digitaLaudio compression scheme called Adaptive Transform Acoustic Coding (ATRAC) is used.

Digital recording technology with MiniDisc storage has enabled multitrack ministudios to migrate from the traditional analog format, to a near CD-quality, digital format. See Unit 7, Multitrack Recording, for more on multitrack ministudios. MiniDisc technology is also being used within the music industry for stereo recording and mixdown, editing decks, and for consumer playback decks. As with all tapeless systems, locating a pointin the track for editing is much faster because of its non-linear, random access format. (DAT) Digital Audio Tape (OAT) is a stereo format that has become popular for amateur and professional musicians to record their music. OAT is generally used for the final stereo mixdown tape from a multitrack recording. It is also used for direct­ to-stereo recording of a live performance. Sound is recorded onto with 16-bit resolution at a sample rate of either 48 KHz or 44.1 KHz, giving a sound quality similar to a CD. DAT machines require a special-size cassette tape, which resembles a miniature VCR tape.

Digital Multitrack Tape Until the mid-l99Os digital multitrack tape recorders were very expensive and only professional recording studios could afford them. The introduction of the Alesis ADAT recorder revolutionized digital multitrack tape recording with an affordable and expandable system. Currently both Alesis and Tascam produce eight-track, rack mountable decks that can be expanded by linking up to four decks together. Thus, when combined with the system's remote control module, a 32­ track digital recording can be assembled. The Alesis and Tascam systems are not compatible with one another - Alesis ADAT series records onto VHS tape and the Tascam DA series uses Hi-8 video cassettes.

Hard Disk Recording Systems Instead of using magnetic tape, hard disk recording systems record sound onto a hard disk drive. The process is governed by software running on a or on a stand-alone device. The major advantage of hard disk recording is its non-linear, random access, editing and playback capabilities. Recording systems that use magnetic tape, whether analog or digital, are recorded in a linear fashion. When playing back or editing, the tape must be rewound or fast forwarded to the desired point on the tape. In contrast, tapeless systems can "rewind" or "fast-forward" to any location virtually instantly. Audio segments located at different parts of the track can be seamlessly arranged and played back in any order. With random access, many alternate takes of a track can be recorded without deleting previous takes, and a single "perfect" take can easily be assembled from all the takes located anywhere on the disk. This capability is referred to as cuelist or playlist editing, because the audio segments are played ~ask~!n whatever order the user specifies in a list.

There are two common formats for hard-disk recording:

Computer-based Multitrack Recording This method relies on a personal computer with software for digital audio recording. Most computers have stereo analogs input and outputs along with analog-to-digital and digital-to-analog convertors. External hardware devices are available for additional inputs or outputs and for computers that don't have this capability. Tracks are recorded and stored directly to the computer's Now, most mid- to high-priced MIDI sequencing programs include digital audio recording capabilities. The features typically include multitrack audio complete with an on-screen mixer and digital signal processing routines. When combined with advanced MIDI sequencing capabilities, these programs provide the heart of an all-in-one recording/MIDI production studio.

Dedicated Multitrack Recording Systems These stand-alone devices have a wide range of features and capabilities, but all share the ability to record and edit multitrack digital audio, in a tape less, random access format. Recording is direct to disk, most often an internal hard disk within the unit. Capacity ranges from 4 to 16 tracks, but some systems allow two or more units to be linked together creating a larger system. While a few models are rack mountable recording decks, most of the systems are a table top design that includes a mixer and effects processor. A few even incorporate physical modeling technology for simulation of various and .

Digital Mixers Digital mixers have become popular in production studios. Typically, these mixers have 8 to 32 input channels and are primarily designed for recording and production work rather than live sound reinforcement. As with all digital devices, the analog inputs from the microphones and instruments are converted to a digital format. Some mixers will also accept digital inputs and outputs direct from a digital multitrack recorder without the need for analog conversion. All signal processing - EQ, effects such as reverb, volume levels, bus routing, etc. - is accomplished in the digital domain. Most digital mixers feature 1 to 4 built-in signal processors, each capable of a wide range of effects, thus eliminating the need for outboard signal processors. A few also utilize physical modeling technology to create a variety of specialized processing capabilities such as studio microphone and guitar simulations.

Once a "mix" for a particular session has been set, all of the parameters or settings can then be stored in memory into a IIscene" for later recall. Most mixers also feature automated mixing capabilities where changes in levels and other parameters can be recorded in real time during the mixing session. When the recording is played back, the changes in the mix are carried out automatically. These advariced types Of capabilities along with a very "clean"" sound make digital mixers a powerful option for production studios. See Unit 6, Microphones and Sound Systems, for more information on mixers.

Digital Signal Processing (DSP) The power and flexibility of digital processing allows virtually any effect to be created or simulated. Common processor effects include reverb, chorus, delays, , compression, distortion and pitch shifting. Many processors are " r_ C1 ___!~ __ 1 _____ • __~_..... ~C _cc_~ ...... _~.,:;j ~.c~~_ ~_T7___1 _CC~~..~ simultaneously. Digital signal processors are manufactured as stand alone, outboard units and are also commonly integrated into keyboards, sound modules, mixers, and dedicated digital multitrack recording systems as part of the built-in system. Software processors and "plug-in" applications for computers that work in conjunction with digital audio programs have also become common. With such integrated and software processors, the need for outboard signal processing hardware can be completely eliminated. See Unit 8 for more information on signal processing.

Sampling Keyboards & Modules A sampling keyboard (or rack-mount version of the same) is a musical instrument that allows the user to record sounds and play them back from the keyboard. Because it uses actual recordings for its timbres, the sampler is a popular way to generate reproductions of acoustic sounds. The quality of sampling varies with the price of the sampler, but 16-bit, 44.1 KHz sampling is the current standard, with stereo possible on most units.

When a sampler records a sound, the sound data is stored in RAM where itcan be played and edited. From there it can be stored to disk, but to play it back later, it must be reloaded into RAM. In order to have a variety of sounds available at any time, it is necessary to have adequate RAM. Therefore, the amount of RAM in a sampler is an important consideration. The typical range is from 2 MB to 128 MB. Like a computer, when the sampler is turned off all data in RAM will be lost unless it is saved to a disk. For long-term storage, sampling instruments usually provide an internal floppy disk drive and some include an internal hard disk. Most also include a port for connecting an external hard disk, CD-ROM, or Zip drive.

One of the criticisms of samplers, especially when they're used for imitating acoustic sounds, is that they sound lifeless or "static." For this reason, almost all sampling instruments integrate techniques such as filters, envelope generators, and modulation for modifying a sample during playback Even with such features available, effective sampling of instrumental or vocal sounds is often a time consuming process that requires a fair amount of skill. There are two sometimes tedious processes necessary to create useful sounds­ looping and multi-sampling .

. .. Looping is used to create sustained tones withcrnrusing up tremendous amounts of memory. The term comes from the tape recording technique in which a piece of recording tape is looped end to end and is played over and over again. In a sampler, looping is achieved by playing a middle segment of the sample Fig. 5 - 5 repeatedly; Le., some portion of the string of numbers is repeated over and over again until the key is released on the keyboard. The difficulty in looping comes in trying to match the beginning point of the loop with the ending point without creating a glitch in the sound from a sudden jump in amplitude or timbre.

(The term "looping" is also used by DJ s and dance music producers, and is often called beat looping. To create a beat loop, a one- to two-bar rhythm section or drum beat off of a record or CD is sampled. It is then looped and played over and over to provide the foundation for a rhythm track. Producers of styles including and hip-hop often use preproduced sampling CDs, which contain no complete songs, only hundreds of rhythm loops. Beats from these CDs are then mixed and matched to create a song arrangement.)

Multi-sampling is a technique for sampling sounds with a wide pitch range. When a sound is sampled into the keyboard, it is assigned to a particular key on the keyboard. If the sound is then played on higher or lower keys, the sound data is played back at a faster or slower rate of speed by the sampler to sound at the new pitch. Ifyou move more than a couple of steps away from the original pitch, the transformation in pitch also causes a change in the timbre. The result is very similar to that of playing a tape recording at a faster or slower speed. This effect is sometimes called the "chipmunk effect," named after the recordings in the 1960s where voices were recorded at slow speeds and played back at high speeds by the group the "Chipmunks." Therefore, in order to create a sound that is realistic over the whole range of pitches, several samples must be taken at different pitches and stored over the range.

In addition to basic looping and multisampling, most current samplers also include digital processing functions that can alter or enhance a sampled sound. For example, time compression/expansion changes the length of a sample without altering the pitch. This feature is particularly useful for allowing longer samples, such as a vocal track that was sampled at one tempo, to be played back at a different tempo. Another technique, normalization, boosts the gain of a sample that was recorded too softly so that it will be as loud as possible without distortion.

Because of the complexity in creating quality sampled sounds, most sampler manufacturers and a number of independent companies offer libraries of sounds created by experts in sampling. These"ready to go" samples can be easily loaded into a sampler from a floppy or CD-ROM.

In addition to typical musical applications, samplers are being used in other ways as well. For instance, they can be used to provide sound effects for a theater production, or to incorporate environmental sounds into an ensemble performance.

Itshould be pointed out that samplers are not limited to the reproduction of acoustic sounds. By using digital manipulation techniques on its samples, a sampler can create new and very interesting timbres. There is clearly an art to designing sounds as well as playing them in a performance.

Sample Players When sampling keyboards became widely available, many users found themselves not utilizing the instruments to their full potential because of the time necessary to create good samples. Yet many still desired the capabilities afforded by such instruments, especially the reasonably accurate reproduction of acoustic sounds. Hence the development of sample players. These instruments utilize high quality sampled sounds that are stored in ROM and are immediately available for playback. They do not allow the user to sample new sounds; only the commercially developed on-board sounds are available. Nor do they allow for much alteration of the sounds.

Basic General MIDI compatible sound modules, digital pianos, preset-only instruments, and home consumer-type keyboards fall into this category. In addition, virtually all drum machines and percussion sound modules are sample playback units.

Digital Synthesizers As mentioned earlier, nearly all synthesizers now utilize digital processes rather than analog ones. Instead of containing actual individual components such as voltage-controlled oscillators or amplifiers, all of the sound generation is accomplished by a microprocessor. There may still be buttons on the outside of the instrument that allow adjustments to oscillators, envelopes, amplifiers, and the like, but these adjustments merely instruct the digital processor in its computation of the waveform. Such buttons are still used because they are useful and familiar means for defining the sound desired. Once the waveform is computed, it is sent through a digital-to-analog converter so that it can be heard over speakers.

Regardless of the type of digital synthesis used, all digital synthesizers have certain things in common. The sound is created by a computer microprocessor which calculates a numerical representation of a waveform. The various components such as envelope generators or LFOs are also computed by the processor and are not actual components as found in an analog syntheSizer. The digital processor performs these calculations using algorithms, which are sets of mathematical procedures and formulas (Le., a "recipe" for solving a problem). The numbers computed by the processor are then converted by a digital-to-analog converter (DAC) to an analog audio signal.

Most synthesizers are manufactured as a stand-alone keyboard instrument or . sound module containing'th~·tnicroprocessor, memory and software necessary for sound production. However, recent advances in the power and speed of personal computers has made it possible to use them for synthesis. With the addition of the appropriate software, the microprocessor and memory of the PC can be used for synthesis. This "virtual" synthesizer is played from an external MIDI keyboard via the computer's MIDI interface. Such software synthesizers simulating almost all types of synthesis are now available from several companies for both the Macintosh and IBM compatibles. .1 TYPES OF DIGITAL SYNTHESIS Below are listed several of the more common types of digital synthesis and which manufacturers feature each approach.

Sampled-Based Subtractive Synthesis Since subtractive synthesis was the most common approach to synthesis prior to the digital revolution, it was only to be expected that digital techniques would be superimposed upon the subtractive modeL This began by replacing traditional analog voltage-controlled components such as VCOs, VCAs, and VCFs with digitally-controlled components- DCOs, DCAs, and DCFs. Once this step was made, it was natural to expand the set of available waveforms like sine waves and square waves to include digitally sampled waveforms. These instruments are not samplers- they do not contain analog-to-digital converters. Instead, the sampled waveforms are permanently stored in ROM and are available as choices for the oscillators as part of the overall subtractive synthesis approach. Following are some of the names used by different companies for digital subtractive synthesis:

AI and AI2 - 1/ Advanced Integrated," the name Korg used for the Ml synthesizer, the 01/W series and now the "N" family of synthesizers. AWM and AWM2 - "Advanced Wave Memory," the name Yamaha uses for their synthesizers including the TG and EX series. Transwave Synthesis - developed by Ensoniq for their Fizmo synthesizer, this method uses an advanced type "wave sequencing" for the sound source. V.A.S.T. - "Variable Architecture Synthesis Technology," the name developed by Kurzweil for the K2000 and K2500 series of synthesizer. Wavetable - Wave table Synthesis, a generic name for sampled waveform-based synthesis.

Frequency Modulation (FM) Synthesis This is the approach made famous by Yamaha's DX and TX series of synthesizers. Yamaha also adapted the FM approach in their SY77 and SY99 instruments (referred to as AFM- Advanced ), and its newest incarnation, the FSIR FormantShaping/FM Synthesis Tone Generator. While the use of FM synthesis has been almost exclusively limited to Yamaha synthesizers, some software synthesizers offer FM as one of several synthesis methods. The approach is builtllporrthe concept of modulation. When a device is used a.sa modulator, it is not heard directly; instead, the effect on what it is modulated is heard. For example, when an LFO is used to modulate an oscillator-we don't hear the LPO directly, but we hear the effect of the LFO on the oscillator in the form of vibrato.

FM synthesis is much the same. One oscillator is used to modulate another; it is not heard, but its effect is. The oscillator that is used to modulate is called the modulator and the oscillator that is affected is called the carrier. With vibrato, the T.PO (monnlator) iF; f:;pt to a VPTV low frpoupncv-somewhere betwppn 1 Hz to 12 Hz. In FM synthesis, on the other hand, the modulator is set to a frequency within Fig. 5 - 6 the range of human hearing. When this occurs, instead of causing a vibrato effect, harmonics are added to the basic waveform of the carrier.

In the Yamaha DX7, only sine waves were used. Each of the six available sine wave oscillators in the instrument is coupled with an envelope generator to make Fig. 5-7 an operator. There are several different configurations available for connecting one operator to the next-i.e., which operators will serve as modulators and which will serve as carriers. Each of these configurations is called an algorithm (not to be confused with the more general application of the term mentioned above). With later FM synths, Yamaha included other waveforms besides sine waves, and several of the models only had four operators available instead of six.

Additive Synthesis creates a timbre by combining a series sine waves at various frequencies corresponding to the harmonic series of the desired sound. The approach is based on the work ofJean Baptiste Joseph Fourier, the nineteenth century French mathematician who theorized that any complex sound can be broken down into a series of simple sounds (i.e., the harmonic series). In a sense, it is the opposite of subtractive synthesis which starts with a harmOnically rich waveform and subtracts frequencies to alter a sound. Additive synthesis is especially effective because full control of each harmonic is possible, allowing for subtle changes in timbre over time. However, additive synthesis is a painstaking and time-consuming process, since the frequency, amplitude and envelope of each harmonic needs to be set individually, and 20 - 30 harmonics are used at a time.

Additive techniques were employed in some of the earliest electronic instruments including the Telharmonium and the Hammond organ. In 1957 Max Mathews demonstrated how additive synthesis could be implemented on a digital computer. Using the power of digital processing, Kawai introduced its K5 synthesizer in 1987, one of the first commercially available synthesizers based on additive synthesis. More recently Kawai released the K5000, which uses an approach called Advanced Additive Synthesis.

Physical Modeling Physicatmodeling is one oithe newer and more powerful digitalsynthesnv"­ technologies available. In a radical departure from previous approaches, physical modeling uses complex computer algorithms to imitate acoustic sounds based on the physical characteristics and performance nuances of the acoustic instrument, not the sound itself. Such parameters as size, shape, and type of material used to make an instrument, as well as embouchure and articulation are taken into account. One of the great advantages of physical modeling is the realtime, expressive control it offers. With other synthesis methods, particularly sampling, the sound for each note is created as an isolated event. With expressive, acoustic ln~tnlTYlpnts , ~ musical context or musical phrasing. Physical modeling surpasses other synthesis methods at simulating the transitions between notes and tone colors within a musical phrase.

Because of the numbers of factors involved, physical modeling requires tremendous digital processing power. Early research and experiments with physical modeling synthesis had been accomplished using powerful computers but real-time control wasn't possible. Only in recent years have computer chips been developed that can handle such complexity in real time.

The first commercially available physical modeling synthesizer, the Yamaha VL1, was introduced in 1994. It plays two notes at a time and its modeling approach was dubbed "Virtual Acoustic Synthesis" by Yamaha. Shortly after that, Korg used physical modeling technology to create a percussion instrument called the Wavedrum. More recently, physical modeling synthesis technology has been employed in numerous applications including the following: Analog synthesis - Several manufacturers are currently producing synthesizers that use modeling to recreate the classic analog sound of the Moog and other early synths. Yamaha's version is called AN Synthesis (Analog Physical Modeling). Drum & percussion - The Roland V drums and Korg WaveDrum are examples of percussion instruments that use physical modeling synthesis for sound generation. It allows for far more musical expression than is possible with sampling technology. Guitar simulation - Roland's VG-8 Guitar System uses modeling technology to transform any electric guitar into convincing "models" of numerous guitars, amps, effects, speaker cabinets and microphones. Amplifier simulation - Modeling technology is used to simulate the sound of a number of classic guitar amplifiers, including different speaker combinations and micing techniques. Microphone - Some of the newer integrated digital recording systems such as the Roland VS series use modeling to emulate the sound of a wide variety of recording microphones.

Granular Synthesis is one of the newest types of digital synthesis and has yet to be . -"us'ed incommercialproduets.·Developed and used in academiG.drcles, the method~ .. is based on the idea that musical sounds can be created from a large number of very short segments of one or more waveforms. A sound is created by streaming (sequencing) very short sound snippets or grains in both time and space. Each grain only lasts a very brief amount of time, typically 1/1000th to 1/10th of a second in length. To create a complex sound requires a large number of grains arranged together in a larger unit called a cloud, which lasts seconds or even minutes. The main parameters for this type of synthesis are the color of the grains (frequency and waveform), the flow (dense or sparse), the grain scale (microscopic or. notpsize) and the svatial position (near, far, horizontal and vertical angle). and Sampling Hybrids Until recently individual synthesizers have been designed around a specific method of synthesis. Ongoing improvements in computing power have led the way to a generation of hybrid instruments that combine multiple synthesis methods and sampling. Hybrid production units often include built-in sounds, sampling, multitrack digital audio recording, sequencing, digital effects and a drum machine. These devices integrate numerous applications of digital audio that equip musicians with an awesome amount of sonic creative power. Here are a few hybrid examples: Korg Trinity Synthesizer - MOSS (Multi Oscillator Synthesis System) includes sampling, physical modeling, variable Phase Modulation and subtractive synthesis technology. Yamaha EX5 Synthesizer - features Advanced Wave Memory (sampled waveforms), Virtual Acoustic Synthesis (physical modeling), AN Synthesis (analog physical modeling) and Formulated Digital Sound Processing (FDSP). Ensoniq ASR-X Pro Sampling Production Studio - a desktop unit that includes a drum machine, sampling, internal ROM sounds, sequencer and effects processor. Audio Audio Input Output

Analog to Digital Digital Digital to Analog Convertor (ADC) Processing Convertor (DAC) 11011010 - ...... f1v .... 11010011 XXXXXXX 11010011 .... f1v 01101110

Figure 5 - 1. The sampling process.

A. This sample rate yields a representation with fairly prominent stair stepping.

B. A higher sample rate yields a representation with less prominent stair stepping.

-- _I<" • J'I . ____ 1'!__ __&. ______... _~ __..,.~ Sampling Nyquist Sound Rate (Highest Frequency) Quality

11 kHz 5.5 kHz Very dull sound 22 kHz 11 kHz Multimedia, games 44.1 kHz 22 kHz CD Quality

Figure 5 - 3. Some sampling rates and their relative sound quality.

, .,

Sample Numerical SIN Sound Size Size Ratio Quality

8 bit 256 48 dB Very Grainy 12 bit 4,096 72dB Cassette Tape 16 bit 65,536 96 dB CD Quality

Figure 5 - 4. Some sample sizes (resolution) and their relative sound quality.

Loop Points

Attack t Sustain ~! Release Loo Modulator 1..-..,....--' set to 440 Hz

Carrier 1...-_--' set to 440 Hz

440Hz 880Hz 1320 Hz 1760Hz etc.

Modulator set to 300 Hz 1..-...---'

Carrier set to 440 Hz 1...-_----1 440Hz 740Hz 1040 Hz 1340 Hz 1640 Hz etc.

Figure 5 - 6. Principles of FM Synthesis. When one sine wave is used to modulate another, and both are at frequencies within the audible range, what we perceive are added harmonics. The frequencies of the harmonics are determined by the formula f = c + (k • m) where c the frequency of the carrier, k = the number of the overtone, and m = the frequency of the modulator. Thus, in the top example, where the frequency of the carrier equals the frequency of the modulator, the harmonics are the same as the standard harmonic series. In the bottom example, the harmonics are at multiples of 300Hz above the fundamental.

(if' f '''YT tit ."7Wt

DX-7 Algorithm # 3 DX-7 Algorithm # 16

., I~+~ I I

.,.,," .. ,~~. = 1Op~rator I·· I~ I Out Out

Figure 5 - 7. Building Blocks of FM Synthesis. The left side shows that an operator consists of a sine wave generator and an envelope generator. The right side shows two of the algorithms from the Yamaha DX7. In Ah!Ori # erators 1 & 4 are carriers and the rest are modulators. DIGITAL SAMPLING & SYNTHESIS

ADDITIONAL RESOURCES

LISTENING: Secrets ofSynthesis Wendy Carlos CBS Records MK42333 This album demonstrates a variety of synthesis techniques with dialog provided by the composer/author. Discussion includes a comparison demonstration of analog and digital approaches.

READING: A DSP Primer: With Applications to Digital Audio and Ken Steiglitz Addison-Wesley Publication Co. 19%.

Computer Music: Synthesis; Composition and Performance Charles Dodge and Thomas A. Jerse Schirmer Books. 1997.

Computer Sound Synthesis for the Electronic Musidan ( Series) Eduardo Reck Miranda Focal Press; Book & CD edition. 1998.

Going Digital: A Musidan;s Guide to Technologl) Brad Hill Schirmer Books. 1998.

Making Music with Digital Audio: Direct to Disk Recording on the PC Ian Waugh PC Publications. 1997.

Sound Synthesis and Sampling (Music Technologl) Series) Martin Russ; francis Rurnsey(Editor) Focal Press. 1996.

CD-ROM: Computer Music: An Interactive Documentary (Mac & Windows) RonS. Nolan Digital Studios Productions. 1996. I INTERNET RESOURCES: GrainWave 2 Software Synthesizer http://www.runol.com/users/mikeb/grainw.htm GrainWcwe 2 is a realtime, software synthesizer and signal processing system for the Power Macintosh computer. GrainWave 2 is a shareware program that is capable of numerous synthesis methods.

Cloud Generator http://www.arts.ilstu.edu/'ibohn/software.html Cloud Generator is a freeware, granular synthesizer program for the Macintosh. The documentation also provides valuable information and hiStory about granular synthesis.

Syd 1.0.7 PPQ68K http://www.jbum.com/syd/ Syd, short for "Synthesis Demonstration" or "Synthesis Donut" is an instrumenteditor and software synthesizer. Itis a freeware program for the Macintosh and may be difficult to use for the novice.

AudioFusion http://www.audiofusion.com/ AudioFusion is designed to tum a PowerMac into a full function Techno workstation. Itincludes a stereo sampler, synthesizer and drum machine. This shareware program also includes a number of postproduction features. DIGITAL SAMPLING & SYNTHESIS

VOCABULARY LIST additive synthesis algorithm algorithm (Yamaha FM synthesizers) analog-to-digital convertor (ADC) compact disc (CD) digital audio tape (DAT) digital multitrack tape digital sampling digital signal processing (DSP) digital-to-analog converter (DAC) digital versatile disc (DVD) digitally-controlled amplifier (DCA) digitally-controlled filter (DCF) digitally-controlled oscillator (DCO) direct-to-disk recording

Frequency Modulation (FM) Synthesis granular synthesis looping MiniDisc (MD) multi-sampling normalization

Nyquist frequency oversampling Digital Sampling & Synthesis Vocabulary - page 2

physical modeling pulse code modulation (PCM) sample sampling rate sample size software synthesizers time compression/expansion