No d’ordre : 3373 THESE` PRESENT´ EE´ A` L’UNIVERSITE´ BORDEAUX I ECOLE´ DOCTORALE DE MATHEMATIQUES´ ET D’INFORMATIQUE Par Martin Raspaud POUR OBTENIR LE GRADE DE DOCTEUR SPECIALIT´ E´ : INFORMATIQUE Mod`eles spectraux hi´erarchiques pour les sons et applications Soutenue le : 24 Mai 2007 Apr`es avis des rapporteurs : Daniel Arfib ....................... Directeur de Recherche Philippe Depalle .................. Professeur Devant la commission d’examen compos´ee de : Christophe Schlick ............... Professeur .................... Pr´esident et Rapporteur Daniel Arfib ....................... Directeur de Recherche Examinateur Philippe Depalle .................. Professeur .................... Examinateur Myriam Desainte–Catherine Professeur .................... Examinateur Laurent Girin ...................... Maˆıtre de Conf´erences . Examinateur Sylvain Marchand ................ Maˆıtre de Conf´erences . Examinateur 2007 Contents English Introduction 1 Introduction en Fran¸cais 7 1 Temporal Modelling 13 1.1 Temporal model ................................................... ....................................... 13 1.2 Time-scaling in the temporal model ................................................... .......... 16 Varying the playing speed ................................................... ........................... 16 Overlap-Add (OLA) ................................................... .................................. 17 SOLA and TD-PSOLA ................................................... ............................... 17 1.3 Conclusion ................................................... ............................................... 17 2 Spectral Modelling 21 2.1 The Fourier transform ................................................... .............................. 21 2.2 Discrete case ................................................... ............................................ 24 2.3 Short-time Fourier transform ................................................... .................... 26 Convolution ................................................... ............................................. 28 Analysis windows ................................................... ...................................... 28 2.4 Time-scaling using the spectral model ................................................... ....... 32 The phase vocoder ................................................... .................................... 32 Time-scaling using the phase vocoder ................................................... ............. 34 2.5 Conclusion ................................................... ............................................... 34 3 Sinusoidal Modelling 35 3.1 Description of the sinusoidal analysis process ............................................... 37 3.2 Enhancing the precision of the discrete Fourier transform ............................. 39 3.3 Enhancing the partial tracking ................................................... ................. 41 Partial trajectories as control signals ................................................... .............. 42 Predictable control signals ................................................... ........................... 43 3.4 Synthesis of sounds from the sinusoidal model .............................................. 52 3.5 Time-scaling with the classic sinusoidal model.............................................. 54 Resampling the parameters ................................................... .......................... 54 Non-uniform time scaling ................................................... ............................ 67 3.6 Conclusion ................................................... ............................................... 68 i ii CONTENTS 4 Order-2 Spectral Modelling 69 4.1 Modelling the parameters ................................................... ......................... 70 Modelling the parameters as polynomials ................................................... ........ 70 Modelling the parameters as sums of sines ................................................... ....... 73 Modelling the parameters using sums of sines and polynomials ................................ 74 High-resolution estimation method ................................................... ................ 76 4.2 Predicting partial tracks using the new model .............................................. 79 4.3 Inaudible control signals ................................................... .......................... 84 4.4 Classification of partials ................................................... ........................... 91 Context ................................................... .................................................. 91 Mid-level representation of polyphonic sounds ................................................... .. 91 The common variation cue ................................................... .......................... 92 Proposed metric ................................................... ....................................... 95 Evaluation................................................... ............................................... 98 Results ................................................... ................................................... 101 A new tool to gather partials? ................................................... ...................... 103 4.5 Considerations on the link of tremolo and vibrato ........................................ 104 4.6 Conclusion ................................................... ............................................... 105 5 Order-2 Sinusoidal Modelling 107 5.1 Previous work on vibrato and tremolo ................................................... ...... 107 5.2 Hierarchical modelling ................................................... .............................. 108 Work on frequency and amplitude control parameters and enhanced time scaling.......... 110 Work on phase and amplitude control parameters ................................................. 115 5.3 Non-uniform time scaling at order 2................................................... .......... 120 5.4 Conclusion ................................................... ............................................... 120 6 Higher Order Modelling 121 6.1 Musical modelling ................................................... .................................... 121 6.2 Finding base functions ................................................... ............................. 122 Box functions ................................................... ........................................... 124 Impulse train functions ................................................... ............................... 126 Box train functions ................................................... .................................... 126 Linearly independent system ................................................... ........................ 127 6.3 Conclusion ................................................... ............................................... 128 7 The Clicks Software 133 7.1 Constraints and general choices ................................................... ................ 133 7.2 Organisation of the program directory ................................................... ...... 134 7.3 How to install and launch Clicks ................................................... ............... 135 7.4 Basic tools ................................................... ............................................... 135 Vector operations ................................................... ...................................... 135 Matrix operations ................................................... ..................................... 137 CONTENTS iii Linear prediction ................................................... ...................................... 138 Least-square estimation ................................................... .............................. 139 Fast Fourier transform................................................... ................................ 139 Resampling ................................................... ............................................. 142 Windows................................................... ................................................. 143 7.5 Sound modelling ................................................... ...................................... 144 General sound interface ................................................... .............................. 144 PCM sounds ................................................... ............................................ 145 Spectral peaks ................................................... .......................................... 145 Spectral sounds ................................................... ........................................ 146 Spectral peak estimation ................................................... ............................. 147 Partials ................................................... .................................................. 147 Sinusoidal sounds ................................................... ...................................... 149 Partial tracking ................................................... ........................................ 149 Time-scaling of sounds ................................................... ............................... 150 7.6 Design and other features