Accurate Frequency Domain Identification of Odes with Arbitrary

arXiv:1907.04787v3 [eess.SP] 4 Dec 2020 e words: Key methods. frequency-domain of ftemdltn-ucintcnqe leigotteeffe the out filtering technique, ne modulating-function redu that the parameters considerably of of number approach the reducing the terms, application, correction in frequency-domain latter tha for the showing useful estimat For windows, is be formalism arbitrary W can The for windowed. that equations. approach are effects the transient signals to generalize if related function is transfer freq difference that system’s is the identification domain by frequency in difficulty The Abstract 359609 a +33549366000. Fax +33549366009. ⋆ de- systems linear observable fully on focus we Here [9,10]. for transients accounted spurious [11]. requires be signals techniques to arbitrary accurate of and use fast The allowing of inputs, application periodic the of for application others. identifica- the involve frequency-domain among tion for approaches functions, frequency-domain Textbook system, modulating discretized identification, the iden- of techniques: identification tification by modelled be or can obtained dynamics whose equations, by differential described of systems mathematically are systems physical Most Introduction 1 [email protected] [email protected] [email protected] rpitsbitdt Automatica to submitted Preprint b orsodn uhrEuro atn.Tel. Martini. Eduardo. addresses: Email author Corresponding ´ preetFuds hriu tCmuto,Isiu P Institut Combustion, et Thermique Département Fluides, c aoaor ’yrdnmqe NS cl Polytechnique Ecole CNRS, d’Hydrodynamique, Laboratoire cuaeFeunyDmi dnicto fOE with ODEs of Identification Domain Frequency Accurate dad Martini Eduardo dnicto ehd,Tm-nain,Sse identifica System Time-invariant, methods, Identification Ad´ .G Cavalieri), G. (André V. [email protected] a nttt Tecnológico Aeronáutica,Instituto de São Cam José dos a , Lt Lesshafft). (Lutz b PtrJordan), (Peter nreV .Cavalieri André G. , V. EuroMartini), (Eduardo rirr Signals Arbitrary France u-uptaayi fasse,adas o ytmidentifi system for also and system, a of analysis put-output t fiiilcniin hl eann h pcrlinte spectral the retaining while conditions initial of cts dt eetmtd h ytmietfiainapoc a fe has approach identification system The estimated. be to ed e laigeet n lostecmuaino h associ the of computation the allows and effects aliasing ces ec opnnso rirr nusadotusaentrel not are outputs and inputs arbitrary of components uency daogietesses aaeeswnos nti work, this In windows. parameters systems’ the alongside ed inlwnoigitoue diinltrsi h syste the in terms additional introduces windowing signal t rm,CR,Uiested otesESA 60 Poitier 86000 Poitiers,ENSMA, Université de CNRS, prime, e etnua idw r sd ti elkonta this that known well is it used, are windows rectangular hen ntttPltcnqed ai,918Plieu France Palaiseau, 91128 Paris, de Polytechnique Institut , X as equations, differential ordinary by scribed where L (1) of representation reads frequency-domain A frequencies. ihsize with n n sampling, ing evlbten0adT, and 0 between terval with sampled are j n =0 x ( a f × A )ˆ tion a n x ee Jordan Peter , j ( x x d f dt and j stesaevco ihsize with vector state the is = ) x j n u = n , R A x k X ( n f =0 × f j b s and , )ˆ n o/P Brazil pos/SP, = u B N u ( k with , f N/T d qal pcdpit natm in- time a on points spaced equally dt ˆ + ) k B k u k , n Nyquist, and , e r ytmmtie ihsizes with matrices system are n b ( t f j u uzLesshafft Lutz , ) ≤ = , n jT/N x nusadresponses and Inputs . n x , n correspond- and , u eebr2020 December 7 f h nu vector input the nyq = rpretation N/ c cation. atures (2 ated ated m’s T (2) (1) we s, ), where ditions and also avoid the need to compute derivatives of the system’s input and output. The terms Aj and Bk can ∞ be estimated from the resulting linear system obtained xˆ(f)= x(t)e−2πiftdt, (3) −∞ using several such modulating functions. Various modu- Z ∞ lating functions have been used, such as spline [12], sinu- uˆ(f)= u(t)e−2πiftdt, (4) soidal [3], Hermite polynomials [19], wavelets [13], and −∞ Poisson moments [14]. Modulating functions have been Zn a used in the identification of integer and fractional-order L(f)= ( 2πif)j A , (5) − j systems [7] and extended to identify both model param- j=0 X eters and model inputs from response observations only nb [1]. R(f)= ( 2πif)kB . (6) − k kX=0 We propose a different interpretation of transient effects in the frequency domain. Correction terms for the time An errorterm,e ˆ(f), was introduced to account for errors derivatives to compensate for windowing effects are de- due to signal noise, windowing, and finite sampling rates, rived, considerably reducing the error term in (2), with which will be discussed later. the remaining errors being due to aliasing effects and signal noise. The corrections can be understood as spu- Time-domain identification consists of estimating matri- rious inputs, which reproduces the polynomial term in- ces Aj and Bj from x(t) and u(t) data, while frequency troduced by [10] when rectangular windows are used. domain identification approaches the problem via their The proposed method however allows for the use of ar- spectral components,x ˆ(f)andu ˆ(f). These approaches, bitrary windows, with spurious terms due to signal win- although equivalent in theory, have significant practi- dowing being computed instead of estimated. This also cal differences. For instance, coloured (non-white) time- drastically reduces aliasing effects, avoiding the need to invariant noise generates signals that are correlated in account for those with artificial terms. time, and optimal time-domain identification requires the use of full correlation matrices. In the frequency The paper is structured as follows. In section 2.1 effects domain the components are uncoupled, which has im- of signal windowing on ODEs are analysed and correc- portant practical advantages [5]. If periodic signals can tion terms are derived. The approach is explored in sec- be used, measurements of the system transfer function tion 2.2 for purposes of system identification. A classi- can be easily performed and used for different system- fication of two types of aliasing effects is proposed in identification methods, e.g. [6,18]. section 3.1, with two classes of windows that minimize one type of aliasing are presented in section 3.2, being However, when non-periodic signals are used with fi- one of those a novel infinity smooth window. Numerical nite data lengths, spectral components of the inputs experiments are presented in section 4. Conclusions are and outputs lead to non-zero errors in (2). Such error presented in section 5. Details on the source of aliasing have been classically associated with spectral leakage, errors are presented in appendix A. although later [9] and [10] showed they can be understood as spurious transient effects. It was shown that, for rectangular windows, the error is a polynomial term 2 Effects of signal windowing and system iden- of order np = max(na,nb). The system parameters and tification polynomial coefficients can be estimated simultaneously, allowing for accurate identifications to be obtained. In 2.1 Signal windowing continuous-time models, finite sampling rates lead to aliasing errors, which were minimized in [10] by artifi- cially increasing the polynomial order of the correction In practice,x ˆ(f) andu ˆ(f), (3), are estimated from win- term. Reference [15] shows that systematic plant estima- dowed signals as, tion errors scale with 1/N when rectangular windows are used, or an improved convergence of 1/N 2 when Han- T −2πift ning or Diff windows are used. System identification can xˆw(f)= w(t)x(t)e dt, (7) 0 be performed as proposed by [9,10], or via variations of Z T the method, e.g. [21,18]. −2πift uˆw(f)= w(t)u(t)e dt, (8) Z0 Focusing on continuous systems, a different approach consists in multiplying (1) by modulating functions and where w(t) is a window function. For frequencies f = integrating over time. By choosing modulating functions j/T ,x ˆw(f) andu ˆw(f) coincide with Fourier-series co- that have their first n-th derivatives equal to zero at their efficients of the periodic extension of (wx) and (wu). limits, where n = max(na,nb) is the order of the system These values are typically obtained by performing a fast (1), integration by parts eliminates effects of initial con- Fourier transform (FFT) on discrete-time samples.

2 Even in the absence of aliasing eﬀects and noise, using currence relation for x{j} is obtained by noting that, xˆw andu ˆw in (2) leads non-zero errors. Multiplying (1) by the window function w(t), dj x{i} di+j (wx) dj (wdix/dti) = dtj dti+j − dtj na nb i+j (14) dj x dku k k dkw di+j−kx A w(t) = B w(t) , (9) = , j dtj k dtk i + j − j dtk dti+j−k j=0 X kX=0 kX=1 where i is the binomial of i and j, with the convention and deﬁning x{0}(t) = 0, j i that j = 0 for i< 0 or i > j. Solving j j {j} d (wx) d x x (t)= (t) w(t) (t), (10) n−1 dj x{n−j} dnwn dtj − dtj a = x, (15) j dtj dtn j=0 X for j > 0, and analogous expressions for u(t), (9) can be re-written as as a linear system

n−1 na j nb k d (wx) {j} d (wu) {k} Ai,j aj =δi,n, (16) Aj x = Bk u . dtj − dtk − j=0 j=0 k=0 X X X i i (11) A = . (17) i,j n − j Applying a Fourier transform of (12) leads to allows x{i} to be obtained as na nb i i−1 j {i−j} j {j} k {k} {i} 1 d w d x Aj ( 2πi) xˆw xˆ = Bk ( 2πi) uˆw uˆ , x = i x + aj j . (18) − − − − a0  dt dt  j=0 j=1 X Xk=0 X (12)   The first three correction terms read where the frequency dependence was omitted for clar- ity. Comparing to (2) it can be seen that the termsx ˆ{j} dw x{1} = x, (19) andu ˆ{j} are corrections that appear when Fourier trans- dt forms of windowed signals are used instead of the true d2w dx{1} x{2} = x +2 , (20) Fourier transforms of the signals. Re-arranging (12) as − dt2 dt d3w dx{2} d2x{1} x{3} = x +3 3 , (21) nb nb dt3 dt − dt2 Lxˆ = Ruˆ + A xˆ{j} B uˆ{k} . (13) w w  j − k  j=0 with corresponding frequency counterparts, X kX=0   {1} dw The term in parenthesis, which will be referred to as x˜ = x , (22) F dt spurious inputs, constitutes a significant contribution to d2w the error in (2) if not accounted for. Note also that these x˜{2} = x + 2( 2πif)˜x{1}, (23) errors are correlated with the signals of u and x, and thus −F dt2 − not accounting for them introduces a significant bias in 3 {3} d w {2} 2 {1} system estimation [9,10]. Signal noise and finite sampling x˜ = x + 3( 2πif)˜x 3( 2πif) x˜ , F dt3 − − − contribute to error terms in (13) and (12), similar to (24) those of (2). where represents Fourier transforms as in (3). We describe a procedure to compute x{j}, with the com- F putation of u{k} being analogous. It is useful to ex- A direct connection between the derivation above and {j} dm dkw that of [10] is made by writing a rectangular window as press x as a sum of terms of the form, dtm ( dtk x), as the window derivative can be obtained analytically and a sum of Heaviside step functions, H, the outer derivative obtained in the frequency domain, w (t)= H(t) H(t T ), (25) avoiding the computation of time derivatives of x. A re- rect − −

3 where T is the window length. The correction terms M1 = Ana , the unknown system’s parameters are all are a function of the window derivatives, which for the contained in M2, and satisfy Heaviside step function are the delta distribution and its derivatives. The correction terms are thus polynomi- θM2 = M1. (33) als whose coefficients are a function of the signal and it − derivatives at 0 and T. As obtaining these coefficients An estimation of θ2, θ˜2, is obtained as a least square- directly from the data can lead to large errors they were error solution of (33), as instead estimated a posteriori [9,10]. θ˜ = M M +, (34) With the proposed approach, we generalize the results − 1 2 of [9,10] to arbitrary windows, with the use of smooth windows allowing for the computation of the correction where the superscript + reefers to the Moore-Penrose terms from data, reducing the number of parameters pseudo-inverse. that need to be estimated. If signal noise is assumed, (33) becomes an error-in- variables problem, where only M 2 and M 1 are observ- 2.2 Use for system identification able, such that

′ Interpreting the correction terms as a correction for the M 2 =M2 + M2, (35) time derivatives on windowed signals, (12) can be ex- M =M + M ′ , (36) plored to obtain an estimation of the system’s parame- 1 1 1 ters. As the termsx ˆ ,x ˆ{i},u ˆ , andu ˆ{j} can be com- w w with M ′ being the contribution to signal noise in M ′ . puted directly from the inputs and outputs, the sys- 1,2 1,2 In this case, the linear-regression estimation used here tem parameters A and B can be estimated from them. i j is biased. The bias can be removed by several error- Defining the matrices in-variables estimation methods proposed in the literature, e.g. [17,18]. As the signal windowing introduces sig- θ = [A0,...,Ana ,B0,...,Bnb ], (26) nificant correlation between different frequency-domain components of the error, the resulting non-linear prob- na (f0) ... na (fn) L . L . lems that can provide an unbiased system-estimation is  . .  more complex than the one solved in [9,10], which is a trade-off for the significantly lower number of param-  (f ) ... (f )   0 0 0 n  eters that need to be estimated. In this work, we in- M =  L L  , (27)  (f ) ... (f ) stead focus on estimations using a least-square errors −Rnb 0 −Rnb 0   . .  approach, which, although biased, can be still used to  . .   . .  obtain estimations with lower computational cost, and   with high accuracy for low-noise scenarios. Note also  0(f0) ... 0(fn)  ′  −R −R  that if the sampling frequency is low M1,2, will also con-   tain aliasing effects, which, due to spectral leakage, are where correlated with M1,2: this effect can however be miti- gated with the use of low-pass filters. (f) =( 2πωf)ixˆ (f) xˆ{i}(f), (28) Li − w − i {i} i(j) =( 2πωf) uˆw(f) uˆ (f), (29) 3 Minimizing aliasing effects R − − ignoring the error terms, (12) is re-written as 3.1 Magnitude of the aliasing effects

θM =0, (30) Equations (12) and (13) are exact for noiseless signals in the continuous time domain. In practice however the To solve for θ, we fix Ana = I, and write Fourier integrals have to be computed from sampled data, with aliasing effects leading to errors in the estimation of the Fourier-series coefficients, and consequently M1 M = , (31) errors in (12) and (13). We proceed analysing the errors "M2# when estimating the Fourier coefficients of s(t), which can represent x(t) or u(t) windowed by diw/dti. θ = θ1 θ2 . (32) h i To analyse the behaviour of the errors for large number where the terms with subscript 1 contains the first of samples N, the convergence of the truncated Fourier nx lines of the M and the first nx columns of θ. As representation and of the Fourier coefficients are used.

4 Assuming s(t) Cn, the errors of truncated Fourier ∈ representation, sN (t) of s(t) , follow

s(t) s (t) < O(1/N n), (37) || − N || for large N, where represents the standard L norm. ||·|| 2 The errors in the Fourier-series coefficient, ak, of the periodic extension of s(t), when estimated using N points at finite sampling rate, is given by [20], a a˜ = O(1/N n). (38) | k − k,N | Note that for finite k, the above expressions imply a algebraic decays, while if s(t) C∞, a non-algebraic decay is obtained. Note also that∈ if s(t) is smooth every- where with the exception of the window limits, com- puting the Fourier coefficients of its periodic extension with the FFT method is equivalent to an integration Fig. 1. Proposed windows: wsinn (t) and wC∞ (t) for T = 1. using the trapezoidal rule. It can then be shown that n a a˜ = O(1/N n+1) for odd n [20]. | k − k,N | which is Cn, with its first n 1 derivatives equal to zero As typically inputs and outputs of the system are smooth − n functions, the asymptotic errors in (2) due to finite sam- at 0 and T , and corresponding to the cos windows in pling are given by the smoothness of the window func- [4], and a novel infinitely-smooth window given by tions used. Note that if w(t) Cn, then diw/dti(t) n−i ∈ ∈ C . This means that the errors in (2) due to finite sam- nT 2 − t(T −t) −4n n−max(na,nb) ∞ e /e ,, 0 < t < T, pling decays at least with 1/N . Estimates wCn (t)= (40) for ak a˜k,N are derived in appendix A, were it is ex- (0 , otherwise plicitly| − seen that| they are related to aliasing effects. which is C∞, with all derivatives equal to zero at 0 and T . As signal windowing causes spectral leakage, which may leak spectral content in frequencies above the Nyquist The two windows are shown in figure 1. These windows’ frequency, it is useful to distinguish between two types spectra exhibit algebraic and non-algebraic decay rates for large frequencies, respectively. Their spectral content of aliasing effects: type I, due to the signal content at frequencies higher than the Nyquist frequency; and type and an illustration of aliasing effects on them are shown II, due to spectral leakage of the signal above the Nyquist in figure 2. frequency due to windowing. When used to estimate signal spectra, for instance when Type I aliasing effects can be easily reduced with the performing a frequency-domain analysis of a system, the use of spatial filters. The use of filters in u and x does beam-width and dynamic-range of the window are key not affect the structure of (2), and thus does not af- parameters, as discussed in [4]. Higher-order windows fect frequency domain analysis or the identification of tend to be more compact, and thus make a poorer usage the system parameters. Filters can easily provide very of window data, leading to a lower frequency resolution, fast decay of the spectrax ˆ, and thus the decay ofw ˆ is which is a trade-off with the improved convergence rate. the dominant factor in the type II aliasing effects. In the following subsection we propose families of windowing functions with convenient properties for the present In a periodogram approach, the penalty of this trade- frequency-domain analysis. off can be alleviated by window overlap. The typical motivation for window overlap is to increase the number 3.2 Windows for algebraic and non-algebraic decay of of samples for averaging, or increase the sample length. aliasing effects This approach comes with the drawback of creating an artificial correlation between samples. It is important to Since the decay rate of the magnitude of the Fourier co- estimate this correlation: if response samples are used efficients of the window is directly related to its smooth- to estimate spectral properties, excessive overlap leads ness [16], motivating the investigation of two window to an increase in computational cost without improving families, the results.

n sin (πt/T ) , 0 < t < T, Sample correlation can be estimated assuming a Gaus- w n (t)= (39) sin 0 , otherwise sian process and a ﬂat spectral content, and the power

5 n ∞ (a) Spectral content of wsin . (b) Spectral content of wCn .

n ∞ (c) Differences for wsin . (d) Differences for wCn . ′ Fig. 2. Spectral content of the proposed windows (top) with sampling frequencies of 1024 (w ˆ, coloured lines) and 32 (w ˆ , black dots). The difference between results from the two sampling frequencies are shown on the bottom figures. Blue and red lines are curves for n = 1 and 4, respectively. spectrum standard variation can be estimated as [22] approximately converge to the same variance. For the proposed windows with n 4, a 80% overlap guarantees ≤ K−1 K−j K−1 good convergence on the estimation variance. 2 Var xˆ 1+2 j=1 K ρj 1+2 j=1 ρj 2{ } = , E xˆ P K ≈ PK In section 2.1 we derive correction terms for signal win- { } (41) dowing and explain their use in system identification, with windowing functions to be used in such framework where in order to minimise aliasing proposed in section 3.2. In the next subsection we will test the present system iden- w(t)w(t jT (1 τ))dt 2 tification method with the window families. ρ = − − , (42) j w2(t)dt R 4 Numerical experiments τ is the windowR overlap fraction,K L is the total ≈ T (1−τ) number of samples, L the length of available data and T The proposed method is compared against the approach the window length. The approximation corresponds to described in [10]. A first order system with nx = nu = 5, the limit where K 1/τ, implying ρj = 0 for j 1. na = 1 and nb = 0 is used. Assuming A0 = I the system ≫ ≫ reads, Detailed relations between correlation and window overlap, for a broad class of windows, is available in the liter- dx (t)+ A x(t)= B u(t), (43) ature [4]. Figure 3 shows the reduction in standard varia- dt 1 0 tion, for a given L, when overlap is used for the windows here studies and for W (t)=1 (t 0.5)n for 0

6 1 5

wC∞ 1 4 0.8 wC∞ 2 wC∞ 3 wC∞ 3 4 0.6 wSin1 wSin2 2

Nor. Var. wSin3 0.4 wSin4 1 W1 Nor. Var. times HPW W2 0.2 0 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Overlap Overlap

Fig. 3. Variance reduction due to overlap. On the left variance is normalized by the zero overlap value (Nor. Var.), on the left this normalized value is multiplied by each window half-power width (HPW). an are taken from a random number generator, as is the Errors in (2) and in parameter estimations exhibit an n ∞ initial condition x(0) = x0. (43) is integrated numeri- algebraic/non-algebraic decay when wsin /wCn win- cally using a fourth-order Runge-Kunta method and a dows are used, as expected from the window properties time step of 3.5 10−5. These parameters guarantee a discussed in section 3.2. Using the infinitely smooth very accurate solution,× which can be used to evaluate windows, numerical precision is obtained if the Nyquist the performance of the approaches. frequency is slightly above the maximal signal excitation. The same accuracy is only achieved if a polyno- In total, the model contains 50 parameters to be esti- mial of order 50 is used in the P&S approach, in which mated. In the following subsections, the error in (12) and the estimation of a total of 250 extra parameters is re- the accuracy of the parameter identification on a noise- quired. Figure 5 shows the computational time required less system is studied, and later parameter identification by each method, where it is seen that the estimation of on a noisy system is performed. The proposed approach, the extra parameters considerably increases the total where the spurious terms are computed, is compared to costs. A mixed approach, where polynomials terms are the approach where a rectangular window is used and estimated to reduce the impact of aliasing effects on the these terms are estimated, as described in [10]. The lat- estimation, does not provide additional gains over the ter will be referred to as P&S. To compare the methods individual approaches, as shown in figure 6. on the same basis, a least-square errors procedure is used on both approaches. Note that P&S with np = 0 cor- As described in section 3.1,the errors decay with the responds to the scenario where the spurious inputs are smoothness of the windowed signal. If the window is a ignored. n function C , and max(na,nb) = 1, Eˆ and θ θ˜ n || || || − || show a decay with 1/fs . The expected decay rate of 4.1 Noiseless system n+1 1/fs for eˆ(f) is observed for wsin3 but not for wsin1 . This is due|| to the|| fact that (dw/dtx)(0) = (dw/dtx)(T ), Figure 4 shows the frequency error norm of the system and thus the FFT transform is not equivalent6 to the given by (43) as well as the error of the estimated pa- trapezoidal rule, and the results of [20] are not applica- rameters. The norms are computed as ble. This can be remedied by averaging out the signal values at the begging and end of the window. eˆ(f) = eˆ (f) 2, (45) || || | i | i 4.2 Estimation in a noisy system sX Eˆ = eˆ(f) 2df, (46) We consider now estimations on a noisy environment. || || || || sZ A total of 500 data sets were used, with signals x and u corrupted with white noise with standard variation θ θ˜ = θ θ˜ 2, (47) || − || | i,j − i,j | σ. Parameters were estimated for each sample. The er- i j sX X ror between the mean value of the parameters and their standard variations are shown in figure 7 as a function where eˆ(f) represents the error norm in (2) for fre- of the number of samples used. || || quency f, E the L2 norm of the error, and θ θ˜ the error in|| the|| estimated system parameters. || − || The performances of the different windows in the high

7 0 100 10

-5 10-5 10

-10 10-10 10

20 40 60 80 100 120 140 101 102 103 104

(a) (b)

(e) (f)

Fig. 4. Errors in (12). Error norms at f = 2 are show in (a,b), the L2 error norms are show in (c,d), and parameter estimation 2 4 errors in (e,f). The dashed lines in (b) correspond to 1/fs (wsin1 ), 1/fs (wsin2 ), 1/fs (wsin3 and wsin4 ) trends. Results for the P&S approach are shown in (c)-(f) with dotted lines. The vertical dashed line is located at fs = 40√2, i.e. fs for which Nyquist frequency is equal to the maximum excitation frequency. noise scenario (σ = 10−2) and in the lower noise sce- pendent. Note that although the least-squares method nario (σ = 10−8) are inverted, with lower values of the used here is biased, the reduction of the estimation er- parameter n leading to more accurate estimation on the rors with the number of sample size in figure 7(a) indi- former, and higher values on the latter. This trend is cates that the variance of the accuracy of the estimation associated with the better use of the available data by is larger than the bias. Such bias can be removed using the window for low values of n (see figure 1), being thus maximum-likelihood estimations, e.g. as used in [10]. able to better account for noise, and with the smaller aliasing effects on higher order windows (see appendix 5 Conclusion A). The trade-off between lower type II aliasing and noise errors depends on the signal spectral content and the noise levels. The optimal choice is thus problem de- A new interpretation of windowing errors in frequency domain representation of ODEs has been proposed, to-

8 102 The novel infinity-smooth window results in aliasing effects orders of magnitude lower than classical windows, as shown in appendix A, leading to considerably more ac- 100 curate identification requiring only moderate sampling rates, being thus a quasi-optimal window choice for such an application. For noisy systems, it was observed that a -2 10 trade-off between a better use of the available data, i.e. lower values of n, and lower aliasing-effects, i.e. higher values of n, depends on the magnitude of these factors, 10-4 101 102 103 104 and is thus problem dependent, as illustrated in section 4.2. Fig. 5. Computational time of estimation using the proposed and P&S approaches for different sampling rates. Funding 1010

E. Martini acknowledges ﬁnancial support by CAPES grant 88881.190271/2018-01. Andr´eV. G. Cavalieri was 100 supported by CNPq grant 310523/2017-6.

A Aliasing effects 10-10 0 50 100 150 An analytical expression for the aliasing effects on the Fig. 6. Same as figure 4 (c), using a mixed approach. Fourier transform of a windowed signal is derived. Most of these results are a direct consequence of the results gether with a correction technique applicable to arbi- presented in [20], which are summarized below. For sim- trary window functions. Two types of windows were ex- plicity we assume s = (wx)(t), with results for u(t) and plored, each leading to an algebraic and a non-algebraic derivatives of w(t) being analogous. decay of errors associated with aliasing effects when sampling frequency is increased. The Fourier-series representation of the windowed signal The presented work can be used in a frequency-domain reads, investigation of systems, e.g. as in [8], where correspon- dence between the system’s inputs and outputs via the ∞ 2πikt/T linear operator is fundamental for the investigation of s(t)= ake , (A.1) the relevant physical mechanisms, or for purposes of sys- k=−∞ X tem identification. For low-noise systems, the method exhibits better performance and/or lower costs than the with P&S approach, proposed in [9,10].

T In the proposed approach, signal windowing leads to 1 −2πikt/T ak = s(t)e dt, (A.2) noise at different frequencies to be correlated. Although T Z0 the construction of a maximum-likelihood estimators in this case is considerably more complicated than the or equivalently, one proposed by [9,10], several methods are available in the literature to obtain an unbiased estimation in these ∞ cases, e.g. [17,18]. In the current work we show that k ak = wˆ(f)ˆx f df. (A.3) with the exact representation of the system obtained, −∞ T − a simple least-squares estimate is seen to provide accu- Z rate and cheap parameter estimates when the system is From this equation the contribution of windowing to noise-free, or when noise levels are small. Also, (12) can aliasing effects, i.e. type II aliasing, as defined in section be used to extend methods originally designed for peri- 3.1, is seen explicitly. This effect is illustrated in figure odic signals, e.g. [6], be used with arbitrary signals. An A.1. extension to systems of partial differential equations can be constructed using external products of the proposed ∞ ∞ windows, such as w2D(x, y)= wCn (x)wCn (y), as in [2]. The coefficientsa ˜k,N of an N-point discrete Fourier

9 102

100 101

100

-5 10-1 10

10-2 100 101 102 100 101 102

− − (a) σ = 10 2 (b) σ = 10 8

105 102 100

10-5 100

10-10

10-2 10-15 100 101 102 100 101 102

− − (c) σ = 10 2 (d) σ = 10 8

Fig. 7. Parameter estimation standard deviation and error as a function of the number of samples for a sampling frequency of fs = 80.

Fig. A.1. illustration of aliasing effects of windowed signals. A window function w is applied to a band-limited signal (blue line representing its frequency content), resulting in spectral leakage, which spreads the frequency content of the signal (shown in black). The signal is sampled with Nyquist frequency fnyq = N/2T ; blue region indicates unresolved frequencies. The frequency content at fj = j/T , aj , and its aliased components aj+N ,aj−N are indicated. transform are related to ak via, where the sum in the final line represents the aliasing effects arising from unresolved frequencies (components for which k > N/2), i.e. the blue region in figure A.1. N | | 1 jT −2πikj/N a˜k,N = s e For a band-limited signal, (A.4) shows the aliasing effects N N j=1 on the terma ˜ . Using windows for whichw ˆ(f) decays X k,N ∞ N faster will lead to smaller aliasing effects, and thus to 1 −2πi(m+k)j/N n = am e (A.4) faster convergence ofa ˜k,N to ak. As ak = O(1/k ) for N s(t) Cn, then a a˜ = O(1/kn). m=−∞ j=1 ∈ | k − k,N | X ∞ X = ak + (ak+mN + ak−mN ) , A slightly stronger results holds if s is smooth between n+1 m=1 0 and T. For this case ak a˜k,N = O(1/k ) for off X | − |

10 [4] Fredric J Harris. On the use of windows for harmonic analysis with the discrete Fourier transform. Proceedings of the IEEE, 66(1):51–83, 1978. [5] Tomas McKelvey. Frequency domain identification. In Preprints of the 12th IFAC Symposium on System Identification, Santa Barbara, USA, 2000. [6] Wen Mi and Tao Qian. Frequency-domain identification: An algorithm based on an adaptive rational orthogonal system. Automatica, 48(6):1154–1162, June 2012. [7] Peyman Nazarian, Mohammad Haeri, and Mohammad Saleh Tavazoei. Identifiability of fractional order systems using input output frequency contents. ISA Transactions, 49(2):207–214, April 2010. [8] Petrônio Nogueira, Mourra Pierluigi, Eduardo Martini, André V. G Cavalieri, and D. S. Henningson. Forcing statistics in resolvent analysis: Application in minimal Fig. A.2. Illustration of the determination of f err for the p turbulent Couette flow [accepted for publication]. Journal of wC∞ window. Black lines correspond tow ˆ(f)/S, where S is 1 Fluid Mechanics, 2020. the window area. Red line indicates the error envelope and err [9] R Pintelon, Joannes Schoukens, and G Vandersteen. dashed error levels and their corresponding fp . Frequency domain system identification using arbitrary signals. IEEE Transactions on Automatic Control, n [20], which is due to a partial cancellation between 42(12):1717–1720, 1997. ak+mN + ak−mN in (A.4). [10] Rik Pintelon and Johan Schoukens. Identification of continuous-time systems using arbitrary signals. Automatica, A.1 Estimate of aliasing effects 33(5):991–994, 1997. [11] Rik Pintelon and Johan Schoukens. System Identification: A An estimation of the aliasing effects can be ob- Frequency Domain Approach. John Wiley & Sons, 2012. tained by considering a band-limited signal with [12] H.A. Preisig and D.W.T. Rippin. Theory and application of the modulating function method—I. Review and theory xˆ( f > fmax) = 0. The coefficient of the leading term, is| max(| a , a ), where of the method and theory of the spline-type modulating | k+N | | k−N | functions. Computers & Chemical Engineering, 17(1):1–16, January 1993. fmax a = wˆ(k N f)ˆx(f)df (A.5) [13] Mahdiye Sadat Sadabadi, Masoud Shafiee, and Mehdi k±N Karrari. System identification of two-dimensional continuous- −fmax ± − Z time systems using wavelets as modulating functions. ISA Transactions, 47(3):256–266, July 2008.

err [14] Dines Chandra Saha, B. B. Prahlada Rao, and Ganti Prasada We define fp as the smallest frequency for which Rao. Structure and parameter identification in linear err w( f > fp ) /S < p, where S is the area un- continuous lumped systems—the Poisson moment functional der| | the| window.| Thus, by choosing N such that approach. International Journal of Control, 36(3):477–491, k (N f ) >f err, the influence of each frequency September 1982. | ± − max | p component ofx ˆ(f) on ak±N is smaller than p. The [15] Johan Schoukens, Yves Rolain, and Rik Pintelon. Leakage process is illustrated in figure A.2. Derivation for the Reduction in Frequency-Response Function Measurements. window derivatives is analogous. and f err values for the IEEE Transactions on Instrumentation and Measurement, p 55(6):2286–2291, December 2006. proposed windows and their derivatives are provided in table A.1. [16] Michael Schramm and Daniel Waterman. On the magnitude of Fourier coefficients. Proceedings of the American Mathematical Society, 85(3):407–407, March 1982. References [17] Torsten Söderström. A generalized instrumental variable estimation method for errors-in-variables identification problems. Automatica, 47(8):1656–1666, 2011. [1] Sharefa Asiri and Taous-Meriem Laleg-Kirati. Modulating functions-based method for parameters and source estimation [18] Torsten Söderström and Umberto Soverini. Errors-in- in one-dimensional partial differential equations. Inverse variables identification using maximum likelihood estimation Problems in Science and Engineering, 25(8):1191–1215, 2017. in the frequency domain. Automatica, 79:131–143, May 2017. [2] Sharefa [19] K Takaya. The use of Hermite functions for system Asiri and Taous-Meriem Laleg-Kirati. Source Estimation for identification. IEEE Transactions on Automatic Control, the Damped Wave Equation Using Modulating Functions 13(4):446–447, 1968. Method: Application to the Estimation of the Cerebral Blood [20] Lloyd N. Trefethen and J. A. C. Weideman. The Flow. IFAC-PapersOnLine, 50(1):7082–7088, July 2017. exponentially convergent trapezoidal rule. SIAM Review, [3] T.B. Co and B.E. Ydstie. System identification using 56(3):385–458, 2014. modulating functions and fast fourier transforms. Computers [21] Robbert van Herpen, Tom Oomen, and Maarten Steinbuch. & Chemical Engineering, 14(10):1051–1066, October 1990. Optimally conditioned instrumental variable approach

11 Table A.1 err Values of fp for the proposed windows. Window Window Derivative Window 2nd Derivative Window 3rd Derivative err err err err err err err err err err err err f10−3 f10−6 f10−12 f10−3 f10−6 f10−12 f10−3 f10−6 f10−12 f10−3 f10−6 f10−12

wsin1 16 502 >10000 1637 >10000 >10000 ------

wsin2 7 68 4911 45 1453 >10000 >10000 >10000 >10000 - - -

wsin3 5 28 867 16 153 >10000 150 >10000 >10000 >10000 >10000 >10000

wsin4 4 17 264 10 53 1683 37 369 >10000 564 >10000 >10000

wsin5 5 13 124 8 30 467 20 109 3491 96 956 >10000

wsin7 5 10 51 7 17 116 12 35 346 26 102 1607

∞ wC0.25 12 45 191 34 99 320 93 198 507 202 354 760 ∞ 7 19 64 14 33 95 28 55 136 50 88 187 wC1 ∞ wC2 7 15 41 11 22 57 19 34 77 30 50 103 ∞ 7 13 33 10 19 44 16 27 58 24 38 76 wC3 ∞ wC4 7 13 30 10 18 39 15 24 50 21 33 63

for frequency-domain system identiﬁcation. Automatica, 50(9):2281–2293, September 2014. [22] P. Welch. The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modiﬁed periodograms. IEEE Transactions on Audio and Electroacoustics, 15(2):70–73, 1967.