FPGA-Base FM Music Synthesizer an Integration of RT-DSP and DMS for Musical Synthesis by Dominic Zucchini

FPGA-base FM Music Synthesizer An integration of RT-DSP and DMS for Musical Synthesis By Dominic Zucchini In cooperation with Missouri University of Science Technology OURE Fellows Undergraduate Research Program 2020-2021 With supervision from Dr. Rohit Dua Associate Teaching Professor of Computer and Electrical Engineering Zucchini 2 Abstract: In the field of sound design, no other form of musical synthesis is recognized for its unique timbre like frequency modulation (FM) synthesis. The versatility of this technique was used in many early keyboard synthesizers, such as the Yamaha DX7, and synthesizer chips for game consoles such as the Sega Genesis using the Yamaha YM2612. Some modern iterations of FM synthesis exist now in the form of Virtual Instruments (VSTs) in Digital Audio Workstations (DAWs) such as Image Line’s Sytrus FM synthesizer. The goal of this design was to integrate the feature set of FM synthesizers such as these in a field-programmable gate array (FGPA)- based environment using the concepts taught in courses such as digital system modeling (DSM) and real-time digital signal processing (RT-DSP). This report covers the methods discovered such as direct digital synthesis (DDS) as well as the limitations and possible improvements to the approach. The features under review will be waveform generation through a numerically controlled oscillator, envelope gain, Musical Instrument Digital Interface (MIDI) compatibility, FM using phase manipulation, mixing of signals, the unison effect, and filtering. Introduction: The FPGA-based FM Music Synthesizer is a stereo, fixed-point, digital synthesizer programmed on an Intel Cyclone IVE FPGA with a Texas Instruments MSP430 powered interface and NXP UDA1334ATS stereo digital-to-analog converter (DAC). The system was written in Verilog (hardware descriptive language) HDL code, synthesized using Intel’s Quartus Prime software suite, and tested using the Altera Modelsim simulator. The FPGA acts as the sound generating hardware and incorporates a numerically controlled oscillator (NCO), FM matrix, unison effect, envelope gain (EG) generator, and low and high pass (infinite impulse response) IIR filters as well as a serial peripheral interface (SPI) and other interfaces and a module to store patch settings. With a MIDI-connected keyboard, users can play up to 8 notes simultaneously with 5 voices per note and 4 inter-modulated oscillators per voice. The user interface allows real-time modification of the sound generating system giving users the ability to create rich sounds on the fly for song recording or live performances. The features of the synthesizer borrow concepts from other FM synthesizer implementations and much of the research was done by studying the Sytrus FM synthesizer and finding resources that can describe how its features work. The goal of this project is to address how the concepts taught in courses like Embedded Systems, RT-DSP, and DSM can be brought together to create a useful complex system. The rest of the report will cover in detail how the features are created and how they function. Overall Design: In DSM, module encapsulation defines a hierarchical structure where lower modules are contained and driven by higher-level ones. This design implements this concept with a top-level entity that contains the entire system and sits at the top of the hierarchy. The top-level includes the main modules such as the sound generating hardware (note controller), the IIR filters, memory unit, and communication unit. Below in Figure 1 is the block diagram. Zucchini 3 Figure 1: Top Level Block Diagram The main sound generating hardware is a hierarchical structure of its own starting with the NCO and moving upwards to the FMM/Voice Unit, the Unison Effect/Note Unit, and finally, the Note Controller which resides in the top level. The main sound generating hardware behaves as a multiplexed pipeline of modules. At a given level, multiple instances of the lower module take turns being processed through one piece of hardware. For example, in the note controller, eight notes can be played, but having eight note units is expensive. To avoid the cost, each note is processed one after the other through a single note unit and then mixed. This costs more time but is acceptable given the 96 kHz sample clock is much slower than the 50 MHz system clock. Design Outcomes: To demonstrate the outcomes of the system design, it will be compared with the outputs of the Sytrus FM synthesizer. The system works as designed implementing 4x4 matrix-based phase modulation, up to order 5 unison effect, and the ability to filter. Below a series of waveforms from both the Sytrus FM synth and this project will be compared to show the successful design outcomes. Figure 1 shows the result of feedback phase modulation on a sinewave oscillator. Figure 2 shows the result from Figure 1 with the addition of the unison phase shift effect at unison order 5. Figure 3 shows the result of the waveform in Figure 1 through a lowpass filter. These figures show how the audio signals compare visually. To see and hear the outcomes of the design, please view the videos below. Demo 1: https://youtu.be/yWKJBZ5DGp4 Demo 2: https://youtu.be/83Wzo1gOcug Zucchini 4 Figure 2: Sytrus (left) and FPGA (right) Comparison: Single Oscillator with Feedback Modulation. Figure 3: Sytrus (left) and FPGA (right) Comparison: Feedback with Unison Phase Shift Order 5 Figure 4: Sytrus (left) and FPGA (right) Comparison: Feedback with LPF Design Limitations: Some design limitations were not considered at the start of this project that have now come to light. These resulted in some mismatches with appropriate hardware and introduced constraints that limited the capabilities of the system. Below are the listed limitations of the design. • The low number of available DSP and memory units on the FPGA is bad for any DSP system which relies heavily on multiplications and delay lines. If the hardware had been provided with more multipliers and memory, then the number of simultaneous notes that Zucchini 5 can be played may have been greater. It also would have allowed for more complex RT- DSP algorithms to be implemented such as reverberation. • The DAC chip initially selected was not stereo and did not easily provide the proper voltage ranges for audio signals. Later, revisions to the communication module were made to integrate a more appropriate DAC with stereo capability, proper voltage levels, and a 3.5mm jack output. • To keep development at a quick pace, timing constraints were note considered deeply through the design process. Unfortunately, this means some combinational circuits introduce large delays and can cause erroneous data to be registered. At low temperatures, the FPGA chip behaves better than at high temperatures. If the chip heats up, then the system can begin introducing glitches and distorted audio. Luckily, the effects of this have not been significantly noticeable. • Not enough time was able to be spent developing a more complex interface. Because of this, users may find it difficult to configure the synthesizer's various patch settings. • The current methods by which certain notes are switched on and off are not perfect. A second note can overwrite a note that is still playing if it is playing because of the release time from the ASDR envelope that has not finished. • Most modern synthesizers have a method for saving and loading patch settings. This way if users create a sound they like, they can come back to it later at any time. No method currently exists on this system, so there is no way to recover certain sounds once they are changed or reset. • Voices are often allocated dynamically so that more notes can be played if notes are using fewer voices. In this design, there is no dynamic allocation and all notes are allocated 5 voices. When the voices aren’t in use, they simply are muted instead of being used to play more notes. Future Plans: Future plans for the system involve addressing many of the limitations and changing current functionality. • Increasing the resolution of several parameters such as frequency, frequency multiplier, FMM settings. • More envelope gain parameters such as attack level, and decay level. • Notes which are pressed again before they finish their envelope will use the same note instead of a new one. • More notes by optimizing the design and utilizing more appropriate FPGA hardware. Using the FPGA-based FM Synthesizer: The use of the synthesizer is simple. Plug in a MIDI keyboard to the SDIN MIDI port and press the keys to play a note. By default, the system will only play sinusoidal tones. To change the tone, modify the patch settings via the rotary encoder and potentiometer. Start with the rotary encoder to select a particular parameter. Then push in the encoder to enable writing to that parameter. Begin turning the potentiometer to modify this parameter’s value. Only certain parameters are changeable via the interface. Table 1 in the Memory module description shows which are user-configurable via the user interface. The provided DAC has a 3.5 mm jack output Zucchini 6 for connecting stereo speakers or headphones. The system currently has no volume control, so speakers with volume control are recommended. Module Descriptions: NCO At the heart of every musical synthesizer is an oscillator that produces a periodic waveform at a desired frequency or pitch. In this design, the oscillator is controlled numerically through a digital system and is thus known as a numerically controlled oscillator or NCO. The design is based on an analog Devices article [1] and was modified for use with the rest of the system. The block diagram for the NCO can be seen below in Figure 5. Figure 5: NCO Block Diagram The beginning of the waveform generation comes from the value in the frequency register, this is then transformed into a change-of-phase (Δ-phase) value that is accumulated every clock cycle.

Load more