DOCSLIB.ORG

Enter Title Here

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Enter Title Here Integer-Based Wavetable Synthesis for Low-Computational Embedded Systems Ben Wright and Somsak Sukittanon University of Tennessee at Martin Department of Engineering Martin, TN USA [email protected], [email protected] Abstract— The evolution of digital music synthesis spans from discussed. Section B will discuss the mathematics explaining tried-and-true frequency modulation to string modeling with the components of the wavetables. Part II will examine the neural networks. This project begins from a wavetable basis. music theory used to derive the wavetables and the software The audio waveform was modeled as a superposition of formant implementation. Part III will cover the applied results. sinusoids at various frequencies relative to the fundamental frequency. Piano sounds were reverse-engineered to derive a B. Mathematics and Music Theory basis for the final formant structure, and used to create a Many periodic signals can be represented by a wavetable. The quality of reproduction hangs in a trade-off superposition of sinusoids, conventionally called Fourier between rich notes and sampling frequency. For low- series. Eigenfunction expansions, a superset of these, can computational systems, this calls for an approach that avoids represent solutions to partial differential equations (PDE) burdensome floating-point calculations. To speed up where Fourier series cannot. In these expansions, the calculations while preserving resolution, floating-point math was frequency of each wave (here called a formant) in the avoided entirely--all numbers involved are integral. The method was built into the Laser Piano. Along its 12-foot length are 24 superposition is a function of the properties of the string. This “keys,” each consisting of a laser aligned with a photoresistive is said merely to emphasize that the response of a plucked sensor connected to 8-bit MCU. The synthesis is processed in the string may need a model more complete than a Fourier series sensor array by four microcontrollers running a strictly to accurately capture its timbre as shown in equation (1). synchronous wavetable synthesis algorithm. In the laser array is 22 integrated a microcontroller that can toggle each laser, allowing uu u u( x ,0) f ( x ) ( x ,0) g ( x ) (1) the piano to play itself or limit playable keys. 22 xt t I. INTRODUCTION The function u x, t represents the displacement of a A. Background Review vibrating string as a function of position x along the string, Throughout the years, music synthesis has branched forth where 0 xa, and time t . fx() and gx() represent the from many different techniques. FM synthesis is classically initial displacement and speed, respectively. d’Alembert’s favored for simplicity but is limited in signal realism. The solution [4] to this PDE is given in the form Karplus-Strong algorithm is favored for rich harmonic content and authenticity particularly to the transient responses u(,)()() x t x ct x ct , (2) of plucked and struck strings. In [1], Karplus-Strong polynomials are used to define zeros to selectively cancel which describes the wavespeed c of travelling waves. harmonics from a traditional Karplus-Strong transfer and describe travelling waves that move in different function. The time-domain convolution associated with this directions at speed . A solution to this PDE is given by method, however, requires more RAM than may be practical for a low-computational system. In [2], the authors proposed uxt( , ) A cos ctB sin ct sin( xd ) an impressive system to synthesize music by modeling the 0 vibration of a string based on Scattering Recurrent Networks 2 a with very accurate results. To implement this system, A f( x )sin( x ) dx (3) 0 however, would still be far beyond the capacity of a typical a small embedded system. Techniques like these have been 2 a B g( x )sin( x ) dx combined in [3] by a control structure that dynamically ac 0 selects and combines synthesis techniques to benefit from the advantages of each. This control structure could prove Initial conditions fx() and gx() are used, by an appeal to invaluable to a system synthesizing a wide range of timbres and pitches but would be unnecessary for a system with orthogonality, to derive a pair of coefficients A and B for sufficiently limited scope. each eigenfunction. There exists one eigenfunction for each eigenvalue . These eigenvalues need not be associated nor For embedded systems, wavetable synthesis offers an even countable. Each eigenfunction is called a formant. attractive combination of realism and speed. In this paper, a lightweight implementation of wavetable synthesis is These formants are not as easily determined for a generalized eigenfunction expansion as for a Fourier series. These formants are assumed to relate to the fundamental Once a pattern of formant relationships and gains was frequency of each note in identical fashion. A vibrating recognized, the waveform was reconstructed in MATLAB (as string, for example, should produce the same timbre (i.e. shown by the code in Fig. 4a, and plotted with the ADSR quality of sound as described by its formant structure) even as envelope in Fig. 1) using a superposition of waves with a tuned higher or lower. The relations of formants to the similar mapping of frequencies and amplitudes. Multiple fundamental root were assumed to fit into the model of a MATLAB files were written to attempt a simulation most diatonic scale as described by Western music theory. This like on-chip synthesis as possible. Every note was sampled limited the formants to a set of the most significant few that from an array of 256 8-bit numbers and amplitude-modulated provide a skeleton into which the formants can fit. If not according to the ADSR (Attack, Decay, Sustain, Release) perfectly accurate, this assumption proved a fair envelope as illustrated in Fig. 2. approximation, and simplified the enforcement of periodicity. Adding formants at eigenvalue frequencies extends the fundamental period of the superposition. The wavetable is not easily truncated because no discontinuities must exist in it. However, the wavetable must be small enough to fit in the limited RAM capacity of a small MCU. II. ALGORITHM DESIGN A. Waveform Construction How to produce a desired timbre could be a subject of considerably deep inquiry. Applying an idealized partial differential equation to such a pursuit would be difficult enough, but easier still than so modeling a realistic physical string. The limitations of this implementation make work such precision needless. For this design, spectral analyses of professionally-recorded piano notes were studied as a first step toward reverse-engineering piano sound. To lessen RAM consumption on-chip, a wavetable only large enough to capture the note’s fundamental frequency was used. To maintain periodicity, formants were chosen that Fig. 2. A set of notes plotted in MATLAB to illustrate the effect of the satisfied periodicity within this window, most notably the amplitude modulation on the repetitive waveshape. Each enlarged section note’s perfect fifth and compound major third. This way there shows the signal within 50-ms-wide cuts. are no discontinuities in the final wave. The amplitude modulation proved to be more important to the final sound than was initially expected. The use of an Waveform 250 ADSR envelope to modulate the amplitude of the waveform provided the striking attack characteristic of a piano note. 200 Intuitively speaking, this envelope could be just as important 150 to other instruments, particularly to drums and woodwinds. 100 Amplitude B. Firmware Implementation 50 A program was written in C, using Codevision C compiler 0 50 100 150 200 250 [5], to synthesize a set of simultaneous piano notes on an Sample Index embedded system. A timer interrupt function was used to ADSR synchronize the wavetable sampling. The code controlling 250 amplitude modulation was written inside an interruptible 200 loop. The output, a superposition of all notes, was output via 150 a byte register into a DAC, from which an analog signal was filtered and sent straight to speakers. The flowchart is shown 100 Amplitude in Fig. 3b. 50 A timer built in to the microcontroller increments a byte 0 register every 32 clock cycles. Each time this register 50 100 150 200 250 Sample Index overflows, an interrupt subroutine is called and the counter is reinitialized to tune the frequency of such calls. Every timer Fig. 1. A MATLAB plot of the waveform and ADSR envelopes. The interrupt, an 8-bit number for each note is sampled from an waveform shapes the timbre of the notes and the ADSR further controls the index in the wavetable, then amplitude-modulated according amplitude of each note to realistically depict its attack and decay. to its position in the ADSR table. The sum of these numbers is normalized and output to the DAC. (a) (b) Fig. 3. (a) A system diagram of the piano hardware, (b) Flowchart describing the piano synthesis algorithm. Two counters increment each interrupt to synchronize ADSR timings for both damped and undamped notes. These indices are incremented every so often; those indices sampling is not. In addition, every interrupt must consume controlling position in the wavetable are incremented the exact same amount of CPU time to remain synchronous according to the frequency of each note. The indices for and consistent. For this reason, the use of control statements higher notes, then, step through the wavetable faster than for was avoided within the timer interrupt code. The use of lower notes. The indices controlling a note’s position in the Boolean numbers in formulae allowed the complete ADSR table is incremented slowly enough to happen within avoidance of if conditions. In this way, a logical 0 or 1 can the interruptible loop. be used as a coefficient just like a numeric 0 or 1.
Load more

Recommended publications
  • The Sonification Handbook Chapter 9 Sound Synthesis for Auditory Display
    The Sonification Handbook Edited by Thomas Hermann, Andy Hunt, John G. Neuhoff Logos Publishing House, Berlin, Germany ISBN 978-3-8325-2819-5 2011, 586 pages Online: http://sonification.de/handbook Order: http://www.logos-verlag.com Reference: Hermann, T., Hunt, A., Neuhoff, J. G., editors (2011). The Sonification Handbook. Logos Publishing House, Berlin, Germany. Chapter 9 Sound Synthesis for Auditory Display Perry R. Cook This chapter covers most means for synthesizing sounds, with an emphasis on describing the parameters available for each technique, especially as they might be useful for data sonification. The techniques are covered in progression, from the least parametric (the fewest means of modifying the resulting sound from data or controllers), to the most parametric (most flexible for manipulation). Some examples are provided of using various synthesis techniques to sonify body position, desktop GUI interactions, stock data, etc. Reference: Cook, P. R. (2011). Sound synthesis for auditory display. In Hermann, T., Hunt, A., Neuhoff, J. G., editors, The Sonification Handbook, chapter 9, pages 197–235. Logos Publishing House, Berlin, Germany. Media examples: http://sonification.de/handbook/chapters/chapter9 18 Chapter 9 Sound Synthesis for Auditory Display Perry R. Cook 9.1 Introduction and Chapter Overview Applications and research in auditory display require sound synthesis and manipulation algorithms that afford careful control over the sonic results. The long legacy of research in speech, computer music, acoustics, and human audio perception has yielded a wide variety of sound analysis/processing/synthesis algorithms that the auditory display designer may use. This chapter surveys algorithms and techniques for digital sound synthesis as related to auditory display.
    [Show full text]
  • Wavetable Synthesis 101, a Fundamental Perspective
    Wavetable Synthesis 101, A Fundamental Perspective Robert Bristow-Johnson Wave Mechanics, Inc. 45 Kilburn St., Burlington VT 05401 USA [email protected] ABSTRACT: Wavetable synthesis is both simple and straightforward in implementation and sophisticated and subtle in optimization. For the case of quasi-periodic musical tones, wavetable synthesis can be as compact in data storage requirements and as general as additive synthesis but requires much less real-time computation. This paper shows this equivalence, explores some suboptimal methods of extracting wavetable data from a recorded tone, and proposes a perceptually relevant error metric and constraint when attempting to reduce the amount of stored wavetable data. 0 INTRODUCTION Wavetable music synthesis (not to be confused with common PCM sample buffer playback) is similar to simple digital sine wave generation [1] [2] but extended at least two ways. First, the waveform lookup table contains samples for not just a single period of a sine function but for a single period of a more general waveshape. Second, a mechanism exists for dynamically changing the waveshape as the musical note evolves, thus generating a quasi-periodic function in time. This mechanism can take on a few different forms, probably the simplest being linear crossfading from one wavetable to the next sequentially. More sophisticated methods are proposed by a few authors (recently Horner, et al. [3] [4]) such as mixing a set of well chosen basis wavetables each with their corresponding envelope function as in Fig. 1. The simple linear crossfading method can be thought of as a subclass of the more general basis mixing method where the envelopes are overlapping triangular pulse functions.
    [Show full text]
  • Wavetable FM Synthesizer [RACK EXTENSION] MANUAL
    WTFM Wavetable FM Synthesizer [RACK EXTENSION] MANUAL 2021 by Turn2on Software WTFM is not an FM synthesizer in the traditional sense. Traditional Wavetable (WT) synthesis. 4 oscillators, Rather it is a hybrid synthesizer which uses the each including 450+ Wavetables sorted into flexibility of Wavetables in combination with FM categories. synthesizer Operators. Classical 4-OP FM synthesis: each operator use 450+ WTFM Wavetable FM Synthesizer produces complex Wavetables to modulate other operators in various harmonics by modulating the various selectable WT routing variations of 24 FM Algorithms. waveforms of the oscillators using further oscillators FM WT Mod Synthesis: The selected Wavetable (operators). modulates the frequency of the FM Operators (Tune / Imagine the flexibility of the FM Operators using this Ratio). method. Wavetables are a powerful way to make FM RINGMOD Synthesis: The selected Wavetable synthesis much more interesting. modulates the Levels of the FM Operators similarly to a RingMod WTFM is based on the classical Amp, Pitch and Filter FILTER FM Synthesis: The selected Wavetable Envelopes with AHDSR settings. PRE and POST filters modulates the Filter Frequency of the synthesizer. include classical HP/BP/LP modes. 6 FXs (Vocoder / EQ Band / Chorus / Delay / Reverb) plus a Limiter which This is a modern FM synthesizer with easy to program adds total control for the signal and colours of the traditional AHDSR envelopes, four LFO lines, powerful Wavetable FM synthesis. modulations, internal effects, 24 FM algorithms. Based on the internal wavetable's library with rich waveform Operators Include 450+ Wavetables (each 64 content: 32 categories, 450+ wavetables (each with 64 singlecycle waveforms) all sorted into individual single-cycle waveforms), up to 30,000 waveforms in all.
    [Show full text]
  • Product Informations Product Informations
    Product Informations Product Informations A WORD ABOUT SYNTHESIS A - Substractive (or analog) synthesis (usually called “Analog” because it was the synthesis you could find on most of the first analog synthesizers). It starts out with a waveform rich in harmonics, such as a saw or square wave, and uses filters to make the finished sound. Here are the main substractive synthesis components : Oscillators: The device creating a soundwave is usually called an oscillator. The first synthesizers used analog electronic oscillator circuits to create waveforms. These units are called VCO's (Voltage Controlled Oscillator). More modern digital synthesizers use DCO's instead (Digitally Controlled Oscillators). A simple oscillator can create one or two basic waveforms - most often a sawtooth-wave - and a squarewave. Most synthesizers can also create a completely random waveform - a noise wave. These waveforms are very simple and completely artificial - they hardly ever appear in the nature. But you would be surprised to know how many different sounds can be achieved by only using and combining these waves. 2 / 17 Product Informations Filters: To be able to vary the basic waveforms to some extent, most synthesizers use filters. A filter is an electronic circuit, which works by smoothing out the "edges" of the original waveform. The Filter section of a synthesizer may be labled as VCF (Voltage Controlled Filter) or DCF (Digitally Controlled Filter). A Filter is used to remove frequencies from the waveform so as to alter the timbre. •Low-Pass Filters allows the lower frequencies to pass through unaffected and filters out (or blocks out) the higher frequencies.
    [Show full text]
  • Fabián Esqueda Native Instruments Gmbh 1.3.2019
    SOUND SYNTHESIS FABIÁN ESQUEDA NATIVE INSTRUMENTS GMBH 1.3.2019 © 2003 – 2019 VESA VÄLIMÄKI AND FABIÁN ESQUEDA SOUND SYNTHESIS 1.3.2019 OUTLINE ‣ Introduction ‣ Introduction to Synthesis ‣ Additive Synthesis ‣ Subtractive Synthesis ‣ Wavetable Synthesis ‣ FM and Phase Distortion Synthesis ‣ Synthesis, Synthesis, Synthesis SOUND SYNTHESIS 1.3.2019 Introduction ‣ BEng in Electronic Engineering with Music Technology Systems (2012) from University of York. ‣ MSc in Acoustics and Music Technology in (2013) from University of Edinburgh. ‣ DSc in Acoustics and Audio Signal Processing in (2017) from Aalto University. ‣ Thesis topic: Aliasing reduction in nonlinear processing. ‣ Published on a variety of topics, including audio effects, circuit modeling and sound synthesis. ‣ My current role is at Native Instruments where I work as a Software Developer for our Synths & FX team. ABOUT NI SOUND SYNTHESIS 1.3.2019 About Native Instruments ‣ One of largest music technology companies in Europe. ‣ Founded in 1996. ‣ Headquarters in Berlin, offices in Los Angeles, London, Tokyo, Shenzhen and Paris. ‣ Team of ~600 people (~400 in Berlin), including ~100 developers. SOUND SYNTHESIS 1.3.2019 About Native Instruments - History ‣ First product was Generator – a software modular synthesizer. ‣ Generator became Reaktor, NI’s modular synthesis/processing environment and one of its core products to this day. ‣ The Pro-Five and B4 were NI’s first analog modeling synthesizers. SOUND SYNTHESIS 1.3.2019 About Native Instruments ‣ Pioneered software instruments and digital
    [Show full text]
  • Tutorial on MIDI and Music Synthesis
    Tutorial on MIDI and Music Synthesis Written by Jim Heckroth, Crystal Semiconductor Corp. Used with Permission. Published by: The MIDI Manufacturers Association POB 3173 La Habra CA 90632-3173 Windows is a trademark of Microsoft Corporation. MPU-401, MT-32, LAPC-1 and Sound Canvas are trademarks of Roland Corporation. Sound Blaster is a trademark of Creative Labs, Inc. All other brand or product names mentioned are trademarks or registered trademarks of their respective holders. Copyright 1995 MIDI Manufacturers Association. All rights reserved. No part of this document may be reproduced or copied without written permission of the publisher. Printed 1995 HTML coding by Scott Lehman Table of Contents • Introduction • MIDI vs. Digitized Audio • MIDI Basics • MIDI Messages • MIDI Sequencers and Standard MIDI Files • Synthesizer Basics • The General MIDI (GM) System • Synthesis Technology: FM and Wavetable • The PC to MIDI Connection • Multimedia PC (MPC) Systems • Microsoft Windows Configuration • Summary Introduction The Musical Instrument Digital Interface (MIDI) protocol has been widely accepted and utilized by musicians and composers since its conception in the 1982/1983 time frame. MIDI data is a very efficient method of representing musical performance information, and this makes MIDI an attractive protocol not only for composers or performers, but also for computer applications which produce sound, such as multimedia presentations or computer games. However, the lack of standardization of synthesizer capabilities hindered applications developers and presented new MIDI users with a rather steep learning curve to overcome. Fortunately, thanks to the publication of the General MIDI System specification, wide acceptance of the most common PC/MIDI interfaces, support for MIDI in Microsoft WINDOWS and other operating systems, and the evolution of low-cost music synthesizers, the MIDI protocol is now seeing widespread use in a growing number of applications.
    [Show full text]
  • Soft Sound: a Guide to the Tools and Techniques of Software Synthesis
    California State University, Monterey Bay Digital Commons @ CSUMB Capstone Projects and Master's Theses Spring 5-20-2016 Soft Sound: A Guide to the Tools and Techniques of Software Synthesis Quinn Powell California State University, Monterey Bay Follow this and additional works at: https://digitalcommons.csumb.edu/caps_thes Part of the Other Music Commons Recommended Citation Powell, Quinn, "Soft Sound: A Guide to the Tools and Techniques of Software Synthesis" (2016). Capstone Projects and Master's Theses. 549. https://digitalcommons.csumb.edu/caps_thes/549 This Capstone Project is brought to you for free and open access by Digital Commons @ CSUMB. It has been accepted for inclusion in Capstone Projects and Master's Theses by an authorized administrator of Digital Commons @ CSUMB. Unless otherwise indicated, this project was conducted as practicum not subject to IRB review but conducted in keeping with applicable regulatory guidance for training purposes. For more information, please contact [email protected]. California State University, Monterey Bay Soft Sound: A Guide to the Tools and Techniques of Software Synthesis Quinn Powell Professor Sammons Senior Capstone Project 23 May 2016 Powell 2 Introduction Edgard Varèse once said: “To stubbornly conditioned ears, anything new in music has always been called noise. But after all, what is music but organized noises?” (18) In today’s musical climate there is one realm that overwhelmingly leads the revolution in new forms of organized noise: the realm of software synthesis. Although Varèse did not live long enough to see the digital revolution, it is likely he would have embraced computers as so many millions have and contributed to these new forms of noise.
    [Show full text]
  • CM3106 Chapter 5: Digital Audio Synthesis
    CM3106 Chapter 5: Digital Audio Synthesis Prof David Marshall [email protected] and Dr Kirill Sidorov [email protected] www.facebook.com/kirill.sidorov School of Computer Science & Informatics Cardiff University, UK Digital Audio Synthesis Some Practical Multimedia Digital Audio Applications: Having considered the background theory to digital audio processing, let's consider some practical multimedia related examples: Digital Audio Synthesis | making some sounds Digital Audio Effects | changing sounds via some standard effects. MIDI | synthesis and effect control and compression Roadmap for Next Few Weeks of Lectures CM3106 Chapter 5: Audio Synthesis Digital Audio Synthesis 2 Digital Audio Synthesis We have talked a lot about synthesising sounds. Several Approaches: Subtractive synthesis Additive synthesis FM (Frequency Modulation) Synthesis Sample-based synthesis Wavetable synthesis Granular Synthesis Physical Modelling CM3106 Chapter 5: Audio Synthesis Digital Audio Synthesis 3 Subtractive Synthesis Basic Idea: Subtractive synthesis is a method of subtracting overtones from a sound via sound synthesis, characterised by the application of an audio filter to an audio signal. First Example: Vocoder | talking robot (1939). Popularised with Moog Synthesisers 1960-1970s CM3106 Chapter 5: Audio Synthesis Subtractive Synthesis 4 Subtractive synthesis: Simple Example Simulating a bowed string Take the output of a sawtooth generator Use a low-pass filter to dampen its higher partials generates a more natural approximation of a bowed string instrument than using a sawtooth generator alone. 0.5 0.3 0.4 0.2 0.3 0.1 0.2 0.1 0 0 −0.1 −0.1 −0.2 −0.2 −0.3 −0.3 −0.4 −0.5 −0.4 0 2 4 6 8 10 12 14 16 18 20 0 2 4 6 8 10 12 14 16 18 20 subtract synth.m MATLAB Code Example Here.
    [Show full text]
  • SOUND SYNTHESIS 01– Page 1 of 9
    FH Salzburg MMA – SOUND SYNTHESIS 01– Page 1 of 9 FH MMA SALZBURG – SOUND SYNTHESIS SOUND SYNTHESIS 01 1. Synthesizer Types 3. Analog Subtractive Synthesis ▪ Analog Subtractive Synthesizer ▪ Main Modules and Parameters ▪ DWG Subtractive Synthesizer ▪ Wavetable Synthesizer 4. FM Synthesis ▪ Vector Synthesizer ▪ Main Modules and Parameters ▪ FM Synthesizer 5. Wavetable Synthesis 2. Envelopes and LFOs ▪ Main Modules and Parameters ▪ Envelopes Types and Function ▪ Envelope Parameters 6. Subtractive vs FM Synthesis ▪ LFO Function Programming Tips ▪ LFO Parameters 1. SYNTHESIZER TYPES ANALOG SUBTRACTIVE SYNTHESIZER Figure 1: Minimoog Model D (left), Sequential Circuits Prophet-5 (right) Analog Subtractive Synthesizers use one or more VCOs (Voltage Controlled Oscillators) to generate waveforms rich of partials/overtones (such as sawtooth, pulse, square); a noise generator (pink or white noise) may also be available. The output of the VCOs is modified in timbre by a VCF (Voltage Controlled Filter) with variable cutoff frequency and (usually) adjustable resonance/emphasis. Standard filters are Low Pass, however some synths offer “multimode filters” that can be set to Low Pass, Band Pass or High Pass. The slope (or “steepness” of the filter) may also be fixed, or variable (12/18/24 dB /oct). The output of the filter is then processed by a VCA (Voltage Controlled Amplifier). Envelopes applied to the filter and amplifier can further modify the character of the sound (percussive, sustain-type, decay-type, etc.), while LFOs can add modulation effects
    [Show full text]
  • 6 Chapter 6 MIDI and Sound Synthesis
    6 Chapter 6 MIDI and Sound Synthesis ................................................ 2 6.1 Concepts .................................................................................. 2 6.1.1 The Beginnings of Sound Synthesis ........................................ 2 6.1.2 MIDI Components ................................................................ 4 6.1.3 MIDI Data Compared to Digital Audio ..................................... 7 6.1.4 Channels, Tracks, and Patches in MIDI Sequencers ................ 11 6.1.5 A Closer Look at MIDI Messages .......................................... 14 6.1.5.1 Binary, Decimal, and Hexadecimal Numbers .................... 14 6.1.5.2 MIDI Messages, Types, and Formats ............................... 15 6.1.6 Synthesizers vs. Samplers .................................................. 17 6.1.7 Synthesis Methods ............................................................. 19 6.1.8 Synthesizer Components .................................................... 21 6.1.8.1 Presets ....................................................................... 21 6.1.8.2 Sound Generator .......................................................... 21 6.1.8.3 Filters ......................................................................... 22 6.1.8.4 Signal Amplifier ............................................................ 23 6.1.8.5 Modulation .................................................................. 24 6.1.8.6 LFO ............................................................................ 24 6.1.8.7 Envelopes
    [Show full text]
  • Copyrighted Material
    1 Sound synthesis and physical modeling Before entering into the main development of this book, it is worth stepping back to get a larger picture of the history of digital sound synthesis. It is, of course, impossible to present a complete treatment of all that has come before, and unnecessary, considering that there are several books which cover the classical core of such techniques in great detail; those of Moore [240], Dodge and Jerse [107], and Roads [289], and the collections of Roads et al. [290], Roads and Strawn [291], and DePoli et al. [102], are probably the best known. For a more technical viewpoint, see the report of Tolonen, Valim¨ aki,¨ and Karjalainen [358], the text of Puckette [277], and, for physical modeling techniques, the review article of Valim¨ aki¨ et al. [376]. This chapter is intended to give the reader a basic familiarity with the development of such methods, and some of the topics will be examined in much more detail later in this book. Indeed, many of the earlier developments are perceptually intuitive, and involve only basic mathematics; this is less so in the case of physical models, but every effort will be made to keep the technical jargon in this chapter to a bare minimum. It is convenient to make a distinction between earlier, or abstract, digital sound synthesis methods, to be introduced in Section 1.1, and those built around physical modeling principles, as detailed in Section 1.2. (Other, more elaborate taxonomies have been proposed [328, 358], but the above is sufficient for the present purposes.) That this distinction is perhaps less clear-cut than it is often made out to be is a matter worthy of discussion—see Section 1.3, where some more general comments on physical modeling sound synthesis are offered, regarding the relationship among the various physical modeling methodologies and with earlier techniques, and the fundamental limitations of computational complexity.
    [Show full text]
  • Sample-Level Sound Synthesis with Recurrent Neural Networks and Conceptors
    A peer-reviewed version of this preprint was published in PeerJ on 8 July 2019. View the peer-reviewed version (peerj.com/articles/cs-205), which is the preferred citable publication unless you specifically need to cite this preprint. Kiefer C. 2019. Sample-level sound synthesis with recurrent neural networks and conceptors. PeerJ Computer Science 5:e205 https://doi.org/10.7717/peerj-cs.205 Sample-level sound synthesis with recurrent neural networks and conceptors Chris Kiefer Corresp. 1 1 Experimental Music Technologies Lab, Department of Music, University of Sussex, Brighton, United Kingdom Corresponding Author: Chris Kiefer Email address: [email protected] Conceptors are a recent development in the field of reservoir computing; they can be used to influence the dynamics of recurrent neural networks (RNNs), enabling generation of arbitrary patterns based on training data. Conceptors allow interpolation and extrapolation between patterns, and also provide a system of boolean logic for combining patterns together. Generation and manipulation of arbitrary patterns using conceptors has significant potential as a sound synthesis method for applications in computer music and procedural audio but has yet to be explored. Two novel methods of sound synthesis based on conceptors are introduced. Conceptular Synthesis is based on granular synthesis; sets of conceptors are trained to recall varying patterns from a single RNN, then a runtime mechanism switches between them, generating short patterns which are recombined into a longer sound. Conceptillators are trainable, pitch-controlled oscillators for harmonically rich waveforms, commonly used in a variety of sound synthesis applications. Both systems can exploit conceptor pattern morphing, boolean logic and manipulation of RNN dynamics, enabling new creative sonic possibilities.
    [Show full text]