2018 Fall CTP431: Music and Audio Computing
Sound Synthesis (Part 1)
Graduate School of Culture Technology, KAIST Juhan Nam Outlines
• Signal model (analog / digital) – Part 1 - Additive Synthesis - Subtractive Synthesis - Modulation Synthesis - Distortion Synthesis
• Sample model (digital) – Part 2 - Sampling Synthesis - Granular Synthesis - Concatenative Synthesis
• Physical model (digital) – Part 2 - Digital Waveguide Model Signal Model
• Modeling the patterns of musical tones using elementary waveforms - Time domain: ADSR - Frequency domain: spectrum
• Types of signal models - Additive synthesis: a set of sine waveforms - Subtractive synthesis: sawtooth, square waveforms + filters - Frequency modulation synthesis: a pair of sine waveforms - Distortion synthesis: sine waveforms + nonlinear units
• These techniques date back to the analog age Additive Synthesis
• Synthesize sounds by adding multiple sine oscillators - Also called Fourier synthesis
OSC Amp (Env)
OSC Amp (Env) +
......
OSC Amp (Env) Telharmonium
• Additive synthesizer using electro-magnetic “tone wheels” (Cahill, 1897) - Transmitted through telephone lines - Subscription only but the the business failed
Tone wheel Hammond Organ
• Drawbars - Control the levels of individual tonewheels Theremin
• A sinusoidal tone generator - Two antennas are remotely controlled to adjust pitch and volume
Theremin ( by Léon Theremin, 1928) Theremin (Clara Rockmore) https://www.youtube.com/watch?v=pSzTPGlNa5U Sound Examples
• Web Audio Demo - http://femurdesign.com/theremin/ - http://www.venlabsla.com/x/additive/additive.html - http://codepen.io/anon/pen/jPGJMK
• Examples (instruments) - Kurzweil K150 - https://soundcloud.com/rosst/sets/kurzweil-k150-fs-additive - Kawai K5, K5000 Subtractive Synthesis
• Synthesize sounds by filtering wide-band oscillators - Source-Filter model
0.4
0.2 Oscillators Filter Amp 0 amplitude −0.2
−0.4 50 52 54 56 58 60 time−milliseconds
20 20 20
10 10 10
0 0 0 −10 −10 −10 −20 −20 −20 −30 −30
Magnitude (dB) −30 Magnitude (dB) −40 Magnitude (dB) −40 −40 −50 −50 −50 −60 5 10 15 20 −60 0 0.5 1 1.5 2 2.5 −60 Frequency (kHz) 5 10 15 20 Frequency (kHz) 4 x 10 Frequency (kHz) Source Filter Filtered Source Moog Synthesizers
MiniMoog (1970) Moog Synthesizers
• Architecture
Soft Envelope LFO Control
Keyboard Physical Control Wheels Slides Pedal Envelope
Parameter Parameter Parameter Audio Path Oscillators Filter Amp
Parameter = offset + depth*control (e.g. filter cut-off (static value) (dynamic value) frequency) “Switched-On-Bach” by Wendy Carlos (1968) Oscillators
• Classic waveforms
2 1 2
0 0 0
−2 −1 −2 0 50 100 150 200 0 50 100 150 200 0 50 100 150 200
20 20 20 0 −6dB/oct 0 −6dB/oct 0 −12dB/oct −20 −20 −20 −40 −40 −40 Magnitude (dB) Magnitude (dB) Magnitude (dB) −60 −60 −60 5 10 15 20 5 10 15 20 5 10 15 20 Frequency (kHz) Frequency (kHz) Frequency (kHz) Sawtooth Square Triangular • Modulation - Pulse width modulation - Hard-sync - More rich harmonics Amp Envelop Generator
• Amplitude envelope generation - ADSR curve: attack, decay, sustain and release - Each state has a pair of time and target level
Amplitude (dB) Attack Decay Sustain
Release
Note On Note Off Examples
• Web Audio Demos - http://www.google.com/doodles/robert-moogs-78th-birthday - http://webaudiodemos.appspot.com/midi-synth/index.html - http://aikelab.net/websynth/ - http://nicroto.github.io/viktor/
• Example Sounds - SuperSaw - Leads - Pad - MoogBass - 8-Bit sounds: https://www.youtube.com/watch?v=tf0-Rrm9dI0 - TR-808: https://www.youtube.com/watch?v=YeZZk2czG1c Modulation Synthesis
• Modulation is originally from communication theory - Carrier: channel signal, e.g., radio or TV channel - Modulator: information signal, e.g., voice, video
• Types of modulation synthesis - Amplitude modulation (or ring modulation) - Frequency modulation
• Decreasing the frequency of carrier to hearing range can be used to synthesize sound - Generate new sinusoidal components - Modulation is non-linear processing Ring Modulation / Amplitude Modulation
• Change the amplitude of one source with another source - Slow change: tremolo - Fast change: generate a new tone
Modulator OSC Modulator OSC
Carrier OSC x Carrier OSC x +
(1 a (t))A cos(2 f t) am (t)Ac cos(2π fct) + m c π c
Ring Modulation Amplitude Modulation Ring Modulation / Amplitude Modulation
• Frequency domain - Expressed in terms of its sideband frequencies - The sum and difference of the two frequencies are obtained according to trigonometric identity - If the modulator is a non-sinusoidal tone, a mirrored-spectrum with regard to the carrier frequency is obtained
carrier sideband sideband
am (t) = Am sin(2π fmt))
fc-fm fc fc+fm Examples
• Tone generation - SawtoothOsc x SineOsc - https://www.youtube.com/watch?v=yw7_WQmrzuk
• Ring modulation is often used as an audio effect - http://webaudio.prototyping.bbc.co.uk/ring-modulator/ Frequency Modulation
• Change the frequency of one source with another source - Slow change: vibrato - Fast change: generate a new (and rich) tone - Invented by John Chowning in 1973 à Yamaha DX7
Modulator OSC
Ac cos(2π fct + β sin(2π fmt)) frequency
Carrier OSC A β = m Index of modulation fm Frequency Modulation
• Frequency Domain - Expressed in terms of its sideband frequencies - Their amplitudes are determined by the Bessel function - The sidebands below 0 Hz or above the Nyquist frequency are folded
k=−∞ y(t) = Ac ∑ Jk (β)cos(2π( fc + kfm )t) k=−∞
carrier sideband1 sideband1 sideband2 sideband2 sideband3 sideband3
fc-3fm fc-2fm fc-fm fc fc+fm fc+2fm fc+3fm Frequency Modulation
• Bessel Function
n β k+2n ∞ (−1) ( ) 2 Jk (β) = ∑ n=0 n!(n + k)!
1 Carrier Sideband 1 Sideband 2 Sideband 3 Sideband 4
0.5 J_(k)
0
−0.5 0 50 100 150 200 250 300 350 beta The Effect of Modulation Index
1 1
0 0 Amplitude Amplitude −1 −1 0 500 1000 1500 2000 0 500 1000 1500 2000 Time (Sample) Time (Sample) 20 20 Beta = 0 Beta = 1 0 0 −20 −20 −40 −40 Magnitude (dB) −60 Magnitude (dB) −60 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 Frequency (kHz) Frequency (kHz)
1 1
0 0 Amplitude Amplitude −1 −1 0 500 1000 1500 2000 0 500 1000 1500 2000 Time (Sample) Time (Sample) 20 20 Beta = 10 Beta = 20 0 0 −20 −20 −40 −40 Magnitude (dB) −60 Magnitude (dB) −60 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 Frequency (kHz) Frequency (kHz)
fc = 500, fm = 50 FM Synthesizer
Yamaha DX7 (1983) Examples
• Web Audio Demo - http://www.taktech.org/takm/WebFMSynth/
• Sound Examples - Bell - Wood - Brass - Electric Piano - Vibraphone Non-linear Synthesis (wave-shaping)
• Generate a rich sound spectrum by distorting sine waveforms using non-linear transfer functions • Also called “distortion synthesis”
1 1
1 0 0 Amplitude 0.5 Amplitude −1 0 50 100 150 200 −1 Time (Sample) 0 50 100 150 200 0 Time (Sample) 20
Amplitude 20 0 −0.5 0 −20 −20 −40 −1 Magnitude (dB) −60 −1 −0.5 0 0.5 1 −40 5 10 15 20 Time (Sample) Magnitude (dB) −60 Frequency (kHz) 5 10 15 20 Frequency (kHz)
x’=gx: g correspond to the “gain” of the distortion Distortion Transfer Function
• Examples of transfer function: y = f(x) - y = 1.5x’ – 0.5x’3 - y = x’/(1+|x’|) - y = sin(x’)
- Chebyshev polynomial: Tk+1(x) = 2xTk(x)-Tk-1(x) T0(x)=1, T1(x)=x, 2 3 T2(x)=2x -1, T2(x)=4x -3x