History of Virtual Musical Instruments and Effects Based on Physical Modeling Principles
Total Page:16
File Type:pdf, Size:1020Kb
History of Virtual Musical Instruments and Effects Based on Physical Modeling Principles Julius Smith CCRMA, Stanford University Digital Audio Effects (DAFx-17) Conference Keynote 1 September 6, 2017 Julius Smith DAFx-17 – 1 / 50˜ Overview Physical Models Finite Differences Early History Voice Models String Models Bowed Strings Distortion Guitar iOS Guitars Recent Research Outro Overview Julius Smith DAFx-17 – 2 / 50˜ Outline Overview Virtual Instruments and Effects • Outline • • Virtual Instruments Physical Modeling Overview • Physical Models History of Virtualization • Finite Differences Virtual Voice • Early History Virtual Strings • Voice Models Selected Recent Developments String Models • Bowed Strings Distortion Guitar iOS Guitars Recent Research Outro Julius Smith DAFx-17 – 3 / 50˜ Virtualization Merriam-Webster defines virtual as • ... being on or simulated on a computer or computer network — print or virtual books — of, relating to, or existing within a virtual reality ... According to Elon Musk: • “There’s a billion to one chance we’re living in base reality” Therefore, it could be “virtual all the way down” • Let’s say virtualization involves one level of simulation • Julius Smith DAFx-17 – 4 / 50˜ Example: Virtual Duck Call A duck call is a virtual quacking device • A duck-call synthesizer is technically a virtual virtual (V 2) duck • Duck Call (Virtual Quacker) More typical unit Julius Smith DAFx-17 – 5 / 50˜ Overview Physical Models Finite Differences Early History Voice Models String Models Bowed Strings Distortion Guitar iOS Guitars Recent Research Outro Physical Models Julius Smith DAFx-17 – 6 / 50˜ Physical Models We generally follow Isaac Newton’s System of the World: No relativistic effects (yet) • No quantum effects (almost) • Newtonian mechanics sufficient • ⇒ Newton’s three laws of motion (1686) can be summarized by one classic equation: f = ma (Force = Mass times Acceleration) ∆ Expresses conservation of momentum ( f =p ˙, p = mv ) • Models based on Newton’s laws can quickly become complex • Need many simplifications that preserve both sound quality and • expressivity of control Julius Smith DAFx-17 – 7 / 50˜ Physical Modeling Formulations Our physical modeling tool box: Ordinary Differential Equations (ODE) [f(t)=p ˙(t)= m v˙(t)= ma(t)] • ′′ Partial Differential Equations (PDE) [Ky (x, t)= ǫ y¨(x, t)] • Difference Equations (DE) [p(n +1) = p(n)+ Tf(n)] • Finite Difference Schemes (FDS) [p(n +1) = p(n)+ Tf(n + 1/2)] • (Physical) State Space Models (Vector First-Order ODE) p˙ = Ap + Bu • Transfer Functions (between physical signals) [H(s)= P (s)/F (s)] • Modal Representations (Parallel Biquads) [H(s)= Hk(s)] • Pk Equivalent Circuits and their various Solvers (Node Analysis, ...) • Impedance Networks [ Lumped Models] • → Wave Digital Filters (WDF) [Masses/Inductors, Springs/Capacitors, ...] • Digital Waveguide (DW) Networks [Strings, Acoustic Tubes, ...] • Julius Smith DAFx-17 – 8 / 50˜ Recent History Numerical Sound Synthesis, Stefan Bilbao, Wiley 2009 Julius Smith DAFx-17 – 9 / 50˜ Overview Physical Models Finite Differences Early History Voice Models String Models Bowed Strings Distortion Guitar iOS Guitars Recent Research Outro Finite Differences Julius Smith DAFx-17 – 10 / 50˜ Finite-Difference Approaches (2009) Overview Physical Models Finite Differences • Wave Digital Filters (WDF), Recent Results • Bassman Example Early History Voice Models String Models Bowed Strings Distortion Guitar iOS Guitars Recent Research Outro Julius Smith DAFx-17 – 11 / 50˜ Wave Digital Filters (WDF), Recent Results Overview See 2015 Keynote talk (and the papers mentioned) for details: Physical Models WDFs can now model arbitrary circuit topologies Finite Differences • • Wave Digital Filters (not just parallel/series connections) (WDF), Recent Results • Bassman Example Any number of nonlinear elements can be included • Early History See recent PhD thesis by Kurt Werner (and recent DAFx papers) • Voice Models Nonlinear Newton solvers remain an active area of research String Models • Bowed Strings Distortion Guitar iOS Guitars Recent Research Outro Julius Smith DAFx-17 – 12 / 50˜ Fender Bassman Example Overview “The Fender Bassman 5F6-A Family of Preamplifier Circuits—A Physical Models Wave Digital Filter Case Study,” Finite Differences • Wave Digital Filters Ross Dunkel, Maximilian Rest, Kurt James Werner, Michael Jørgen (WDF), Recent Results • Bassman Example Olsen and Julius O. Smith Early History Voice Models T1B T1N T2 T3 CB2 String Models VPK VGK VPK VGK VPK VGK VPK VGK RGB2 COB CB1 Bowed Strings RIB RVBP Distortion Guitar RGN1A iOS Guitars R Recent Research PB RGN1B Outro RPN RIN RGB1A CON RVNN RVBN RGN2 RK2 RL RP2 VP CK1 RGB1B VIN RVNP RK1 Ross Dunkel’s Fender Bassman WDF (Note the 25-port R-Node) Julius Smith DAFx-17 – 13 / 50˜ Back to History Numerical Sound Synthesis, Stefan Bilbao, Wiley 2009 Julius Smith DAFx-17 – 14 / 50˜ Overview Physical Models Finite Differences Early History Voice Models String Models Bowed Strings Distortion Guitar iOS Guitars Recent Research Outro Early Virtualization of Strings Julius Smith DAFx-17 – 15 / 50˜ Early Virtualization of Strings Overview The vibrating string was a critically important object of study in the Physical Models middle-18th century, leading to Finite Differences The first wave equation (first PDE) Early History • • Virtualization Traveling-wave solution of the string wave equation • Darrigol • • Mersenne Additive synthesis solution for the terminated string • • Sauveur First glimpses of Fourier-series expansions • Taylor • • Rameau (before Fourier was born in 1768) • J Bernoulli The concept of superposition in linear systems • D Bernoulli • • Harmonics • D’Alembert • Euler • Bernoulli Replies • Math Puzzle • Paradoxes • Resolution • Fourier • Animations • Composers and Geometers Voice Models String Models ˜ BowedJulius Strings Smith DAFx-17 – 16 / 50 Main Reference Overview Excellent account of the period discussed here: Physical Models Finite Differences Olivier Darrigol Early History “The Acoustic Origins of Harmonic Analysis” • Virtualization • Darrigol Archive for History of the Exact Sciences, 2007 • Mersenne • Sauveur • Taylor • Rameau • J Bernoulli • D Bernoulli • Harmonics • D’Alembert • Euler • Bernoulli Replies • Math Puzzle • Paradoxes • Resolution • Fourier • Animations • Composers and Geometers Voice Models String Models ˜ BowedJulius Strings Smith DAFx-17 – 17 / 50 Marin Mersenne (1636) - On Audible Overtones “[Since the vibrating string] produces five or six tones..., it seems that it is entirely necessary that it beat the air five, four, three, and two times at the same time, which is impossible to imagine unless one says that half of the string beats the air twice, one third beats it three times, etc. while the whole strings beats it once. This picture runs against experience, which clearly shows that all parts of the string make the same number of returns in the same time, because the continuous string has a single motion, even though parts near the bridge move more slowly.” Plucked string video: • https://www.youtube.com/watch?v=Qr rxqwc1jE Since there was no notion of spectrum at this time, the fundamental • frequency of a sound was the periodic repetition rate of pulses in the time domain Julius Smith DAFx-17 – 18 / 50˜ Joseph Sauveur (1701) Overview “While meditating on the phenomena of sound, I was Physical Models made to observe that especially at night one may hear Finite Differences Early History from long strings not only the principal sound, but also • Virtualization other small sounds, a twelfth and a seventeenth above.... • Darrigol • Mersenne I concluded that the string in addition to the undulations • Sauveur it makes in its entire length so as to form the • Taylor • Rameau fundamental sound may divide itself in two, three, four, • J Bernoulli • D Bernoulli etc. undulations which form the octave, the twelfth, the • Harmonics fifteenth of this sound.” • D’Alembert • Euler • Bernoulli Replies • Math Puzzle • Paradoxes • Resolution • Fourier • Animations • Composers and Geometers Voice Models String Models ˜ BowedJulius Strings Smith DAFx-17 – 19 / 50 Harmonic Overtones Discovered Overview Physical Models Finite Differences Early History • Virtualization • Darrigol • Mersenne • Sauveur • Taylor • Rameau • J Bernoulli • D Bernoulli • Harmonics • D’Alembert • Euler • Bernoulli Replies • Math Puzzle • Paradoxes • Resolution Sauveur plucked a monochord having a light obstacle mounted to • Fourier • • Animations create “nodes” • Composers and Geometers He was surprised that the string did not move at the nodal points • Voice Models String Models ˜ BowedJulius Strings Smith DAFx-17 – 20 / 50 Sauveur, Continued String-playing musicians surely discovered how to play harmonics by lightly • damping a node (The Lyre dates back to perhaps 3000 BC) Sauveur coined the term node, inspired by nodes in the lunar orbit: • Lunar nodes are points where the orbit of the Moon crosses the ecliptic ◦ The ecliptic is the apparent path of the sun around the celestial sphere ◦ An eclipse can only happen at one of the two lunar node points ◦ Sauveur also coined the term harmonic, so named because they are • “harmonious” with the fundamental Julius Smith DAFx-17 – 21 / 50˜ Brook Taylor (1713) Best known for Taylor Series Expansions • First to derive the string fundamental frequency f = K/ǫ/2L, • p where K = string tension, ǫ = mass density, L = string length Derived that the string restoring force is proportional to string curvature • Approximated this by the