Analysis and Synthesis of Expressive Guitar Performance
Total Page:16
File Type:pdf, Size:1020Kb
Analysis and Synthesis of Expressive Guitar Performance AThesis Submitted to the Faculty of Drexel University by Raymond Vincent Migneco in partial fulfillment of the requirements for the degree of Doctor of Philosophy May 2012 c Copyright 2012 Raymond Vincent Migneco. All Rights Reserved. ii Table of Contents ListofTables........................................... vi ListofFigures .......................................... vii Abstract.............................................. xi 1 INTRODUCTION ..................................... 1 1.1 Contributions........................................ 3 1.2 Overview .......................................... 4 2 COMPUTATIONAL GUITAR MODELING . 6 2.1 SoundModelingandSynthesisTechniques. 6 2.1.1 WavetableSynthesis .................................. 6 2.1.2 FMSynthesis...................................... 7 2.1.3 AdditiveSynthesis ................................... 8 2.1.4 Source-FilterModeling................................. 8 2.1.5 PhysicalModeling ................................... 9 2.2 Summary and Model Recommendation . 10 2.3 SynthesisApplications ................................... 12 2.3.1 SynthesisEngines ................................... 12 2.3.2 Description and Transmission . 12 2.3.3 New Music Interfaces . 13 3 PHYSICALLY INSPIRED GUITAR MODELING . 14 3.1 Overview .......................................... 14 3.2 WaveguideModeling .................................... 14 3.2.1 Solution for the Ideal, Plucked-String . 15 3.2.2 Digital Implementation of the Wave Solution . 15 3.2.3 Lossy Waveguide Model . 17 3.2.4 Waveguide Boundary Conditions . 18 iii 3.2.5 ExtensionstotheWaveguideModel . 20 3.3 Analysis and Synthesis Using Source-Filter Approximations . 21 3.3.1 Relation to the Karplus-Strong Model . 22 3.3.2 Plucked String Synthesis as a Source-Filter Interaction . 22 3.3.3 SDL Components . 23 3.3.4 Excitation and Body Modeling via Commuted Synthesis . 25 3.3.5 SDL Loop Filter Estimation . 27 3.4 ExtensionstotheSDLModel ............................... 31 4 SOURCE-FILTERPARAMETERESTIMATION . 32 4.1 Overview .......................................... 32 4.2 Background on Expressive Guitar Modeling . 32 4.3 Excitation Analysis . 33 4.3.1 Experiment: Expressive Variation on a Single Note . 34 4.3.2 Physicality of the SDL Excitation Signal . 36 4.3.3 Parametric Excitation Model . 38 4.4 Joint Source-Filter Estimation . 38 4.4.1 Error Minimization . 38 4.4.2 Convex Optimization . 40 5 SYSTEMFORPARAMETERESTIMATION. 43 5.1 Onset Localization . 43 5.1.1 Coarse Onset Detection . 44 5.1.2 Pitch Estimation . 45 5.1.3 Pitch Synchronous Onset Detection . 46 5.1.4 Locating the Incident and Reflected Pulse . 48 5.2 Experiment1 ........................................ 49 5.2.1 Formulation . 49 5.2.2 Problem Solution . 52 5.2.3 Results ......................................... 53 5.3 Experiment2 ........................................ 58 iv 5.3.1 Formulation . 58 5.3.2 Problem Solution . 59 5.3.3 Results ......................................... 61 5.4 Discussion.......................................... 62 6 EXCITATIONMODELING ................................ 63 6.1 Overview .......................................... 63 6.2 Previous Work on Guitar Source Signal Modeling . 64 6.3 Data Collection Overview . 66 6.3.1 Approach . 67 6.4 Excitation Signal Recovery . 68 6.4.1 Pitch Estimation and Resampling . 69 6.4.2 Residual Extraction . 69 6.4.3 Spectral Bias from Plucking Point Location . 70 6.4.4 Estimating the Plucking Point Location . 71 6.4.5 Equalization: Removing the Spectral Bias . 74 6.4.6 Residual Alignment . 76 6.5 Component-based Analysis of Excitation Signals . 77 6.5.1 Analysis of Recovered Excitation Signals . 77 6.5.2 Towards an Excitation Codebook . 78 6.5.3 Application of Principal Components Analysis . 79 6.5.4 Analysis of PC Weights and Basis Vectors . 81 6.5.5 Codebook Design . 84 6.5.6 Codebook Evaluation and Synthesis . 85 6.6 Nonlinear PCA for Expressive Guitar Synthesis . 88 6.6.1 Nonlinear Dimensionality Reduction . 89 6.6.2 Application to Guitar Data . 90 6.6.3 Expressive Control Interface . 92 6.7 Discussion.......................................... 94 7 CONCLUSIONS . 95 v 7.1 ExpressiveLimitations................................... 96 7.2 Physical Limitations . 97 7.3 FutureDirections...................................... 98 Appendix A Overview of Fractional Delay Filters . 100 A.1 Overview .......................................... 100 A.2 The Ideal Fractional Delay Filter . 100 A.3 Approximation Using FIR Filters . 102 A.3.1 Delay Approximation using Lagrange Interpolation Filters . 103 A.4 Further Considerations . 104 AppendixB PitchGlideModeling .............................. 106 B.1 Overview .......................................... 106 B.2 PitchGlideModel ..................................... 107 B.3 PitchGlideMeasurement ................................. 107 B.4 Nonlinear Modeling and Data Fitting . 108 B.4.1 Nonlinear Least Squares Formulation . 108 B.4.2 FittingandResults .................................. 109 B.5 Implementation....................................... 110 Bibliography . 113 VITA ............................................... 119 vi List of Tables 2.1 Summary of sound synthesis models including their modeling domain and applicable audio signals. Adopted from Vercoe et al. [93]. 11 2.2 Evaluating the attributes of various sound modeling techniques. The boldface tags indicate the optimal evaluation for a particular category. 11 5.1 Mean and standard deviation of the SNR computed using Equation 5.11. The joint source-filter estimation approach was used to obtain parameters for synthesizing the guitar tones based on an IIR loop filter. 58 5.2 Mean and standard deviation of the SNR computed using Equation 5.11. The joint source-filter estimation approach was used to obtain parameters for synthesizing the guitar tones using a FIR loop filter with length N = 3. 61 B.1 Pitch glide parameters of Equation B.3 for plucked guitar tones for each guitar string. p, mf and f indicate strings excited with piano, mezzo-forte and forte dynamics, respectively. ......................................... 112 vii List of Figures 3.1 Traveling wave solution of an ideal string plucked at time t = t1 and its displacement at subsequent time instances t2,t3. The string’s displacement (solid) at any position is the summation of the two disturbances (dashed) at that position. 16 3.2 Waveguide model showing the discretized solution of an ideal, plucked string. The + upper (y ) and lower (y−) signal paths represent the right and left traveling distur- + bances, respectively. The string’s displacement is obtained by summing y and y− at adesiredspatialsample. .................................. 17 3.3 Waveguide model incorporating losses due to propagation at the spatial sampling instances. The dashed lines outline a section where M gain and delay blocks are consolidated using a linear time-invariant assumption. 18 3.4 Plucked-string waveguide model as it correlates to the physical layout of the guitar. Propagation losses and boundary conditions are lumped into digital filters located at the bridge and nut positions. The delay lines are initialized with the string’s initial displacement. ........................................ 20 3.5 Single delay-loop model (right) obtained by concatenating the two delay lines from a bidirectional waveguide model (left) at the nut position. Losses from the bridge and nut filters are consolidated into a single filter in the feedback loop. 22 3.6 Plucked string synthesis using the single delay-loop (SDL) model specified by S(z). C(z) and U(z) are comb filters simulating the e↵ects of the plucking point and pickup positions along the string, respectively. 24 3.7 Components for guitar synthesis including excitation, string and body filters. The excitation and body filter’s may be consolidated for commuted synthesis. 26 3.8 Overview of the loop filter design algorithm outlined in Section 3.3.5 using short-time Fourier transform analysis on the signal. 30 4.1 Top: Plucked guitar tones representing various string articulations by the guitarist on the open, 1st string (pitch E4, 329.63 Hz). Bottom: Excitation signals for the SDL modelassociatedwitheachpluckingstyle. 35 4.2 The output of a waveguide model is observed over one period of oscillation. The top figure in each subplot shows the position of the traveling acceleration waves at di↵erent time instances. The bottom plot traces out the measured acceleration at the bridge (notedbythe’x’inthetopplots)overtime. 37 5.1 Proposed system for jointly estimating the source-filter parameters for plucked guitar tones.............................................. 43 5.2 Pitch estimation using the autocorrelation function. The lag corresponding to the global maximum indicates the fundamental frequency for a signal with f0 = 330 Hz. 46 viii 5.3 Overview of residual onset localization in the plucked-string signal. (a): Coarse onset localization using a threshold based on spectral flux with a large frame size. (b): pitch-synchronous onset detection utilizing spectral flux threshold computed with a frame size proportional to the fundamental frequency of the string. (c): Plucked-string signal with onsets coarse and pitch-synchronous onsets overlayed. 47 5.4 Detail view of the “attack” portion of the plucked-tone signal in Figure 5.3. The pitch- synchronous onset is marked as well as the incident and reflected pulses from the first period of oscillation. 48 5.5 Pole-zero and magnitude plots of a string filter S(z)withf0