Working Paper N 1313

Mathematical framework for pseudo-spectra of linear stochastic diﬀerence equations

Instituto Marcos Bujosa Complutense Departamento de Fundamentos del AnálisisEconómicoII Universidad Complutense de Madrid. Campus de Somosaguas de Análisis 28223 Pozuelo de Alarcón,Spain. Económico AndrésBujosa Departamento de Matemáticaaplicada a las Tecnolog´ıasde la Información.ETSI Telecomunicación Universidad Politécnicade Madrid. Avenida Complutense, 30 28040 Madrid, Spain. Antonio Garc´ıa-Ferrer Departamento de AnálisisEconómico:Econom´ıaCuantitativa Universidad Autónomade Madrid. Campus de Cantoblanco 28049 Madrid, Spain.

Abstract Although spectral analysis of stationary stochastic processes has solid mathematical foundations, this is not always so for some non-stationary cases. Here, we establish a rigorous mathematical extension of the classic Fourier spectrum to the case in which there are AR roots in the unit circle, ie, the transfer function of the linear time-invariant filter has poles on the unit circle. To achieve it we: embed the classical problem in a wider framework, extend the Discrete Time Fourier Transform and defined a new Extended Fourier Transform pair pseudo-covariance function/pseudo-spectrum. Our approach is a proper extension of the classical spectral analysis, within which the Fourier Transform pair auto-covariance function/spectrum is a particular case. Consequently spectrum and pseudo-spectrum coincide when the first one is defined.

Keywords Spectral analysis, time series, non-stationarity, frequency domain, pseudo- covariance function, linear stochastic diﬀerence equations, partial inner product, Extended Fourier Transform.

JL Classiﬁcation C00, C22

This working paper has been accepted for publication in a future issue of IEEE Transactions on Signal Processing. Content may change prior to ﬁnal publication. Citation information: DOI:10.1109/TSP.2015.2469640 1053-587X c 2015 IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending a request to [email protected] See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

Working Paper no 1313 August, 2015 ISSN: 2341-2356 1

Abstract—Although spectral analysis of stationary stochastic processes has solid mathematical foundations, this is not always so for some non-stationary cases. Here, we establish a rigorous mathematical extension of the classic Fourier spectrum to the case in which there are AR roots in the unit circle, ie, the transfer function of the linear time-invariant filter has poles on the unit circle. To achieve it we: embed the classical problem in a wider framework, extend the Discrete Time Fourier Transform and defined a new Extended Fourier Transform pair pseudo- covariance function/pseudo-spectrum. Our approach is a proper extension of the classical spectral analysis, within which the Fourier Transform pair auto-covariance function/spectrum is a particular case. Consequently spectrum and pseudo-spectrum coincide when the first one is defined. Index Terms—Spectral analysis, time series, non-stationarity, frequency domain, pseudo-covariance function, linear stochastic difference equations, partial inner product, Extended Fourier Transform. 2

Mathematical Framework for Pseudo-spectra of Linear Stochastic Difference Equations Marcos Bujosa, Andres´ Bujosa, and Antonio Garc´ıa-Ferrer

I.INTRODUCTION tools. Nevertheless, by abuse of notation and borrowing some operation rules from the stationary case algebra, this ap- HEREAS the spectrum describes the frequential con- proach has brought several statistical methodologies to model tent of a stationary signal, the pseudo-spectrum de- W non-stationary signals.1 As a result, most national statistical scribes the frequential content of a non-stationary one. In agencies use these methods to model trends and seasonality the time series literature, two approaches to non-stationary (CENSUS Bureau in United States, Eurostat, United Nations, stochastic process representations in the frequency domain are European statistical agencies, European Central Bank, UK, found. The first one deals with the time-dependence of the Canada, New Zealand, Japan, etc.). Also, applications in other frequential content of the signal. The second, with the fre- areas are widespread, i.e, [12]–[16]. Despite the importance quential content of explosive signals. Whereas in the first case, and spread of these methods, this approach does not seem to pseudo-spectra are time dependent functions that generalize be properly grounded. Given that these methods seem to work, the traditional Fourier analysis, taking into account possible there should be a reason for that. time variations of spectral characteristics of signals; in the Here we provide an algebraic model that partly justifies second approach, spectral characteristics are time independent. most of the usual practices within this second approach. In par- The first approach to pseudo-spectra, known as time- ticular, it is a common practice to write the pseudo-spectrum frequency analysis, has a well established mathematical model. with a function that shares identical structure with the spec- It can be viewed as a time-dependent extension of classical trum. Our main contribution in this article is a definition of Fourier-based methods for finite-energy signals. Some de- pseudo-spectrum that makes this calculation rigorous, rather velopments of this approach are time-frequency, time-scale than intuitive. To do that, we need to extend some definitions and wavelet analysis, fractional Fourier and linear canonical to the non stationary case. Hence, we need a wider framework transforms. These generalizations include cyclostationary sig- that includes the Hilbert space L2 S, B,P of scalar random nal analysis, multitaper spectral estimation, and evolutionary variables with finite variance defined on the probability space spectral analysis (see [1]–[11]). These are conjoint time- S, B,P . The algebraic dual space( of L)2 S, B,P is not frequency representations that explicitly consider the time- “large enough”2. Our strategy is to consider the algebraic dual dependence of the frequential content of the signal. Here we space( of) an appropriate subspace of L2 S, B(,P . ) do do not follow this time-frequency approach. We organize our paper as follows: SectionII briefly re- The second approach to pseudo-spectra deals with models views the standard framework of spectral( analysis) for the for signals with infinite energy whose spectral characteristics stationary AutoRegressive Moving Average (ARMA) case. (like in the wide-sense stationary case) do not depend on Section III describes the state of the art for pseudo-spectral time. Such models naturally arise when the characteristic theory of Linear Stochastic Difference Equation (LSDE) with polynomial of the difference equation has roots on the unit AutoRegressive (AR) unit roots. SectionIV outlines how our circle and, therefore, the transfer function of the linear time- algebraic model for pseudo-spectra is developed step by step. invariant filter has poles on the unit circle (but note that, since There we use the Random Walk (RW) as a simple illustration. there are poles, the transfer function is not well defined in this SectionsV to VII show the technical details. In Section VIII case). This constitutes a different paradigm, since models for some properties of pseudo-spectra are reviewed. In SectionIX signals with infinite energy are outside the realm of Hilbert some examples and applications are presented. Finally, we space and, therefore, they can not be treated with the classical conclude in SectionX. Copyright (c) 2015 IEEE. Personal use of this material is permitted. Notation: Bold symbols denote either sequences of random However, permission to use this material for any other purposes must be variables or sequences of functionals of random variables. obtained from the IEEE by sending a request to [email protected]. Uppercase Greek letters denote double infinite sequences of This work was supported by the Spanish Ministerio de Econom´ıa y Z Competitividad (ECO2012-32854). numbers, and R denotes the set of all those sequences. M. Bujosa is with the Departamento de Fundamentos del Analisis´ Consequently, l1 (the set of absolutely summable sequences) Economico´ II. Universidad Complutense de Madrid. Campus de Somosaguas. and l2 (the set of squared summable sequences) are subsets 28223 Pozuelo de Alarcon,´ Spain. E-mail: [email protected]. Z A. Bujosa is with the Departamento de Matematica´ aplicada a las Tec- of R . Lowercase Greek letters denote polynomials. The only 2 nolog´ıas de la Informacion.´ ETSI Telecomunicacion.´ Universidad Politecnica´ exceptions are the variance (σ ) and the standard deviation (σ). de Madrid. Avenida Complutense, 30. 28040 Madrid, Spain. A. Garc´ıa-Ferrer is with the Departamento de Analisis´ Economico:´ 1see https://www.census.gov/srd/www/x13as/ and references therein. 2 Econom´ıa cuantitativa. Universidad Autonoma´ de Madrid. Campus de Can- although it contains the topological dual space of L2(S, B,P ), a isomor- toblanco. 28049 Madrid, Spain. phic “copy” of L2(S, B,P ). 3

We assume that the set of polynomials, R X , is contained in When φ has no roots on the unit circle there is a unique Z n R by identifying a0 a1X ... anX with the sequence stationary solution —a stationary sequence y yt t Z of α αt t where αt at if 0 t n, and[ α] t 0 if t 0 or random variables in L2 S, B,P ; see [18]. Using the abso- Z ∈ t n; that is + + + lutely summable inverse sequence 1 , the stationary≡ { solution} ∈ φ ≡ { } = ≤ ≤ = < can be written as ( ) α ... 0, 0, a , a , . . . , a , 0, 0,... 1 > 0 1 n y θ w . (4) φ where the coefficient≡ for the zero index (t 0) is boxed. As it is 3 1 = ∗ ( ∗ ) usual, if α is a polynomial and C is an element in an algebra , The sequence Θ φ θ is also absolutely summable and then n = α C a0 a1C ... anC , consequently α X α. We 1 1 use this mathematical convention with α B where B is the y ≡ θ ∗ w θ w Θ w, (5) φ φ backward( ) ≡ shift+ operator+ + and with α X 1 (where( X) =1 is the sequence in RZ that is 1 when t 1 and− ( 0)otherwise)− where the= last expression∗ ( ∗ ) = is known∗ ∗ as the= infinite∗ moving ( ) average (Wold) representation of y. In this special case, with 1 X ... 0, 1, 0= ,−0, 0,.... no AR roots on the unit circle, the LSDE is often known as − an ARMA p, q model, and the stationary solution y is often ≡ II.STANDARD FRAMEWORK FOR THE STATIONARY CASE known as an ARMA process. It should be noted that, since Θ is a square( ) summable sequence, the convolution product Here, some well known results are stated for further refer- Θ w converges in mean square. Hence, y is a well-defined ence along the paper, as well as to show the parallelisms or (second-order) stationary stochastic process with: differences between the new results, definitions or properties, ∗ and those pertaining to the standard framework. cov yi, yj yi, yj yi t, yj t (6) L2 S,B,P L2 S,B,P Let S, B,P be a probability space, where S is a nonempty for all i, j, t . The sequence( ) of+ auto-covariances+ ( ) of y, sample space, B a Borel field of subsets of S, and P a ( ) = ⟨Z ⟩ = ⟨ ⟩ ( ) known as the auto-covariance (generating) function is probability measure on B. To generate the relevant Hilbert ∈ space we use zero-mean random variables with finite variance(⋅) Γy Z R ; (7) defined on S, ,P , with the inner product x, y j y0, yj B L2 S,B,P L2 S,B,P E x y , corresponding norm x E x2 and metric x y , ∶ Ð→ ( ) ( ) ( ) ⟨ ⟩ 2 = so, cov yi, yj Γy j i→. Using⟨ the⟩ infinite moving average where E denotes the expectation operator.» If E x y 0, representation, it is easy to show that the auto-covariance we[ ⋅ say] x and y are equivalent.Y Y = Being[ equivalent] isY indeed− Y 1 2 function( is Γy) X= (Θ−X) Θ X σ and satisfies an equivalence relation, and then, the space L[(2 S,−B),P] = is 1 − 1 2 the corresponding quotient space, i.e., the collection of these φ X ( φ) =X ( )Γ∗y (θ X) θ X σ . (8) − − equivalence classes. ( ) iω Its Discrete( Time) ∗ ( Fourier) ∗ Transform= ( ) ∗ (DTFT),( ) ⋅ Γy e Consider a LSDE iω iω 2 Θ e Θ e σ , is the spectrum of y (see TableI). − p q − ( ) = aixt i bjwt j, t Z, (1) i 0 j 0 ( III.T) ( HEINCOMPLETEGENERALIZATIONTOTHE) − − NON-STATIONARY CASE Q= = Q= ∈ where a0 b0 1, and wt t Z is a white noise stochastic process with E w 0 and Var w w 2 σ2 for all t. When φ has roots on the unit circle there is no square t t t 1 ∈ p p 1 summable inverse sequence and the stationary solution (4) The characteristic= = polynomial{ } is X a1X ap 1X ap, φ 2 p does not exist, neither does the auto-covariance function nor and its reciprocal( polynomial) = φ (1 )a=1XY −aY2X= apX − is known as the AR polynomial. In+ the right+⋯+ sum in (1),+θ the spectrum. Nevertheless, in the literature we find references 2 q about spectral representation of non-stationary solutions to (3) 1 b1X b2X bqX is known= + as the+ Moving+ ⋯ Average + (MA) polynomial. = over the infinite time domain. +Using+ the discrete+ ⋯ + convolution product “ ” Within this approach, used since the late seventies, pseudo- spectra are obtained by two alternative ways. First, the pseudo- spectrum is described as the limit function of a sequence of f g t fmgt m,∗ t (2) ∞ Z m spectra of stationary ARMA models, as the modulus of an AR − root tends to one (e.g. [19]). But in the limit covariance func- we can write( (1)∗ as) =φ Q=x−∞ θ w ; where∈ x x t t t t Z tions diverge and therefore spectra are not defined. Second, an and w wt t . Sum limits in (1) are finite since coefficients Z ∈ AutoRegressive Integrated Moving Average (ARIMA) process ai are zero when i( ∗p )or= i( ∗ 0)in AR polynomials,≡ { } ∈ acted upon by a filter ϕ B that cancels out the AR roots on and coefficients≡ { } b are zero when j q or j 0 in MA j the unit circle to make it stationary. Then the spectrum of polynomials. With φ > x and θ< w denoting the whole ( ) ϕ eiwt ϕ e iwt ; > < the stationary filtered process is divided by sequences φ x t t Z and θ w t t Z respectively, ∗ ∗ so the spectrum of the filtered process is multiplied by− the we can use the following compact notation for (1): ( ) ( ) {( ∗ ) } ∈ {( ∗ ) } ∈ inverse of the power transfer function of the filter (e.g., [20]). φ x θ w. (3) However, a power transfer function is not defined for that filter, since any inverse sequence of φ is not absolutely summable 3[17, page 229, exercise 5] ∗ = ∗ when the AR polynomial φ has roots on the unit circle. In 4

TABLE I STATE OF THE ART.A MODEL THAT FILLS IN THE GAPS A, B AND C ISNEEDEDWHENTHE AR POLYNOMIALHASROOTSONTHEUNITCIRCLE.

STATIONARY CASE (No AR roots on the unit circle) Convolution type solution Covariance generating function of the stationary solution y Discrete Time Spectrum satisﬁes Fourier Transform −iω iω y 1 θ w φ X φ X−1 Γ θ X θ X−1 σ2 θ(e )θ(e ) σ2 = φ ∗ ∗ ( ) ∗ ( ) ∗ y = ( ) ∗ ( ) ⋅ Ð→ F Ð→ φ(e−iω )φ(eiω ) 1 1 sq. summable φ ∗ φ = 1 φ

NON-STATIONARY CASE (AR roots on the unit circle) Solutions? Covariance generating function? Fourier Pseudo-Spectrum Transform? −iω iω Gap A Gap B Gap C θ(e )θ(e ) σ2 φ(e−iω )φ(eiω )

many cases pseudo-spectra are simply used with no further pseudo-spectrum coincide when the former is defined. This explanation (e.g., [21]–[23]). generalization is not straightforward. Several technical steps Within this second approach the pseudo-spectrum seems are needed because, when no restrictions are imposed on the to describe “the distribution (over the frequency range) of roots of the AR polynomial φ, the discussion in SectionII 1 the energy (per unit time) or variance (possibly infinite) of is no longer valid. The symbol “ φ ” is used to denote a very the process” [24], in a similar way as the spectrum does in particular inverse, the unique absolutely summable sequence 1 the stationary case. Hence, we find that sometimes the term such that φ φ 1. But that sequence does not exist when φ “spectrum” is, in fact, used to denote this type of pseudo- has roots on the unit circle. Nevertheless, the inverse sequences spectrum [25]. But there are several major drawbacks in this. of φ are used∗ = to denote solutions to φ x θ w. So, This pseudo-spectrum cannot be the DTFT of a covariance we must consider other (non-summable) inverse sequences. function, since covariance functions of non-stationary solu- Indeed, there are an infinite number of sequences,∗ = ∗Ψ, such tions to (3) over the infinite time domain are not defined. Even that Ψ φ 1, that is, there are an infinite number of worse, pseudo-spectra are functions outside the Hilbert space inverse sequences for each non null degree polynomial. But L2 π, π , so DTFT is not applicable. the corresponding∗ = solutions are non-convergent. In addition, However practitioners and academics use pseudo-spectra when the inverse is not unique, convolution products are and[− operate] with their algebraic expressions as if they where no longer associative4: if Λ φ 1 and φ Υ 1, then spectra and, surprisingly (or not), it seems to work fine! We Λ φ Υ Λ φ Υ . Here we show how to deal with these 1 also find not well defined expressions like xt φ θ wt, issues in order to provide a mathematical∗ = model∗ for= pseudo- where φ has roots on the unit circle in seminal− papers as spectra( ∗ )∗. Below,≠ ∗( we∗ describe) the steps we take in sectionsV 1 in [21], [26]–[31]. There, φ θ wt is= ( referred∗ )∗ to as to VII along with the illustration of the RW model. “the (nonconvergent) infinite moving− average representation Step 1 Solutions to (3) are non-convergent when φ has roots of xt” [26]. Hence, since the( energy∗ ) ∗ (the variance) of x is on the unit circle. One easy example is the RW infinite (non-convergent), its physical interpretation is unclear. model xt xt 1 wt, where several formal solu- It should be noted that, even in the stationary case, there is no tions can be found. The backward causal solution clear physical interpretation for solutions of infinite duration in 0 − is ft −j w=t j, and the forward solution is the past ( . . . infinitely before “The Big Bang”!). We are used gt j 1 wt j. The problem is that those sums =−∞ + to deal with these mathematical formalisms and, therefore, we are non-convergent.= ∞∑ So we need to provide a new + don’t usually pay attention on this. In the non-stationary case framework= − ∑ = where the convergence issues are avoided. there is, indeed, a second level of abstraction since the variance To do so, we embed the standard Hilbert Space in of these solutions is also infinite. In spite of that, we often find a wider framework. This is done in the first part of in the literature that some properties from these models are SectionV. deduced, and statistical methods developed. These methods are Step 2 In RZ, discrete convolution product is not defined for applied to non-stationary finite signals; so there is no infinite any pair of sequences and, when it is defined, it is energy in practice. However, many properties of these methods not always associative. We need to check under which are deduced from these mathematical formalisms with no conditions it is possible to operate as in equation (5), physical interpretation. in order to get convolution type solutions in the new and wider framework; that is, solutions in the form5 ∗ IV. A WIDER FRAMEWORK FOR BOTH THE STATIONARY f Ψ θ w where φ Ψ 1. This is done in the ANDTHENON-STATIONARY CASES second part of SectionV. = ( ∗ ) ∗ ∗ = In this paper we generalize the spectral theory to the case 4 Although discrete convolution products are associative in l1, this is not where φ 0 is an AR polynomial either with or without true when we also consider other inverse sequences outside l1. roots on the unit circle. We also show that spectrum and 5where w∗ is the embedding of w. See SectionV. ≠ 5

TABLE II PARALLELISM BETWEENTHE STATIONARY AND NON-STATIONARY CASE. WHEN φ HASNOROOTSONTHEUNITCIRCLE, ANDTHECO-STATIONARY 1 PAIR OF STATIONARY SOLUTIONS ∗ ∗ ISUSED WHEN COVARIANCE AND COVARIANCE GENERATING FUNCTIONS ARE THE SAME (y , y ) ( Ψ = φ ), Pseudo- (Γy = Γy∗,y∗ ), ANDSOTHEYARE SPECTRUMAND Pseudo-SPECTRUM.

BOTH STATIONARY AND NON-STATIONARY CASES Convolution type solutions Pseudo-covariance generating function of any co-stationary Extended Fourier Spectrum or pair of solutions (f, g) satisﬁes Transform Pseudo-Spectrum −iω iω f Ψ θ w∗ φ X φ X−1 Γ θ X θ X−1 σ2 θ(e )θ(e ) σ2 = ∗ ∗ ( ) ∗ ( ) ∗ f,g = ( ) ∗ ( ) ⋅ Ð→ F Ð→ φ(e−iω )φ(eiω ) φ ∗ Ψ = 1 Proposition V.1 Proposition VI.1 and Theorem VI.3 Theorem VII.2

After SectionV, Gap A in TableI is filled in (see TableII, the pseudo-covariance function for the backward and forward column 1). Indeed, for the AR polynomial φ 1 X in our RW solutions pair f, g is illustration, we can consider at least two inverse sequences: Γf,g k 0 for k 0 Λ cj j , where cj 1 for j 0 and =0 otherwise;− and ( ) Z 2 ; (10) Υ cj j , where cj 1 for j 1 and 0 otherwise. With ⎧Γf,g k kσ for k 0 ∈Z ⎪ the= first{ } inverse we can= define the≤ backward causal solution ⎪ ( ) = ≥ ∈ ⎨ 1 2 ∗ it is easy to check⎪ that( )1= −X 1 X < Γf,g σ . sequence= { } of functionals=f− Λ w≥ , where each functional ⎩⎪ ∗ 0 Two more difficulties remain. First, since− pseudo-covariance ft Λ w t j wt j is well defined. Similarly, we can ∗ functions are not squared( summable− ) ∗ ( in− general,) ∗ we need= to ex- define the forward solution∗ sequence= ∗ of functionals g Υ w , ∗=−∞ + tend the DTFT outside the Hilbert space. Second, whereas, for where= ( g∗t )Υ= ∑w t w . (see Section VI-A) j 1 t j a given LSDE, the pseudo-spectrum is unique, different pairs At this point new difficulties∞ ∗ arise. Since the considered= ∗ = + pseudo-covariance solutions= sequences( ∗ ) f= −are∑ non-convergent in general, its of co-stationary solutions have different functions. For instance, when φ has all its roots outside the elements ft are functionals outside L2 S, B,P . It follows that neither the covariance nor the auto-covariance generating unit circle, the backward solution with itself form a co- y∗, y∗ function are defined for any of these non-convergent( ) solutions stationary solution pair , since the backward solution is stationary in this case. Hence, the covariance function (Γ ) f. Fortunately, it is possible to define co-stationarity for y and the pseudo-covariance( function) (Γ ∗ ∗ ) are the same. some pairs f, g . Then we define the pseudo-covariance y ,y function for co-stationary pairs sequences in a similar way But, for the same LSDE, the backward and forward solutions y∗, f as in (7). So( similar) that, when there are no AR roots on the form another co-stationary pair with a completely different (non-summable) pseudo-covariance function (Γ ∗ ). unit circle and the pair is formed by the stationary solution y ,f ( ) and itself6 y∗, y∗ , the covariance and pseudo-covariance Both difficulties are solved though the extension of the DTFT. functions coincide. Step 4 In Section VII we extend the domain outside the Hilbert space l , so it also includes the pseudo- Step 3 In Section( VI) we define the co-stationarity of pairs 2 covariance functions. In addition, the Extended Fourier of sequences and its pseudo-covariance function Transform is defined so that for any sequence that (Definitions VI.1 and VI.2). satisfies (24), that is, any sequence Ψ such that F Two more results are obtained from SectionVI: φ X φ X 1 Ψ θ X θ X 1 σ2, (11) a) as a consequence of Proposition VI.1, the pseudo- − − the image is covariance function, Γf,g , of any pair f, g of co- ( ) ∗ ( ) ∗ = ( ) ∗ ( ) ⋅ θ e iω θ eiω stationary solutions to (3) satisfies σ2. (12) ( ) φ e−iω φ eiω φ X φ X 1 Γ θ X θ X 1 σ2 (9) ( ) ( ) f,g It follows that, given a− difference equation φ x θ w, − − ( ) ( ) [note( the) ∗ parallelism( ) ∗ with= (8).( ) ∗ ( ) ⋅ there is a common image for the pseudo-covariance b) The Existence Theorem VI.3 proves that for any functions of all co-stationary solution pairs.∗ Hence,= ∗ the difference equation φ x θ w, the backward and pseudo-spectrum is unique (Theorem VII.2). the forward solutions always form a co-stationary pair. Indeed, with Section VII Gap C in TableI is finally filled in Hence, after SectionVI, Gap∗ =B in∗ TableI is also filled (see TableII, column 3). in (see TableII, column 2). Following our RW illustration, It is important to remember that when a stationary solution exists, it forms a co-stationary pair with itself. It immediately 6where y∗ is the embedding of y. See SectionV. follows that spectra and pseudo-spectra are equal when the 6

AR polynomial has no roots on the unit circle. Although since for all v L the set t Z wt v 0 is finite: n the construction of the pseudo-spectrum is different, the final indeed, if v L then v i 1 aidi, hence∗ t Z wt v n structure of the function for spectra and pseudo-spectra is, so 0 i 1 t Z∈wt(Ddi) 0 , and{ ∈t SZ w( t)d≠i } 0 ∗either = to speak, the same. So, given a difference equation, φ x is empty or∈ it has(D) only∗ one= element.∑ ∗ { ∈ S ( ) ≠ θ w, we can write the corresponding (pseudo-)spectrum } ⊂ ⋃ = { ∈ S ( ) ≠ } { ∈ S ( ) ≠ } regardless of the roots of the AR polynomial (φ 0).∗ = A. Summability in the dual space L ∗ Following our illustration, the pseudo-spectrum of a RW is: ∗ ≠ The goal of this section is to show that some solutions to σ2 σ2 ∗ (D) Γ e iω . (13) φ x θ w can be expressed via convolution products just 1 e iω 1 eiω 2 2 cos ω − like in (5). To do so, we need a minimal requirement about Due to the( pole) = on the− frequency zero,= the integral of this summability∗ = ∗ on functionals to get well defined convolution ( − )( − ) − function is infinite. So it can be viewed as a representation of products: we say that a sequence of functionals f ft t Z of L is summable if, for all v L , the subset of the infinite variance of the solutions to xt xt 1 wt; where ∈ an infinite contribution to the variance is concentrated in the indexes ∗t Z ft v 0 is finite, and its sum is the functional= { } 7 − (D) ∈ (D) zero frequency . Consequently, when a filter− whose= frequency in L given by the map v ft v 0 ft v . Note that ∗ the embedding{∗ ∈ S of( the) ≠ white} noise process, w wt t , is response has a zero in the zero frequency is applied (for ( )≠ Z summable.(D) ↦ ∑ ( ) ∗ example by taking first differences), then (as a consequence ∈ of Proposition VI.1) the output becomes stationary. In the 1) The convolution product on summable sequences:≡ { } de- following sections we describe the technical details of this spite the fact that within this framework the convolution new model for pseudo-spectra. product for any two sequences of functionals is not always defined, and when it is, it is not always associative, we V. A WIDERFRAMEWORK enumerate some useful properties of the convolution product We, first, need to give meaning to expressions like which do hold in some special cases, and which resemble a w when a is not square summable (i.e., when those of the product in (5). These will allow us to carry out, t Z t t t t Z the sequence is not in l2), so as to be able to write f Ψ w in L , operations like those involved in (5). ∈ ∈ ∗ with∑ φ Ψ θ,{ [in} the spirit of (5)] even when Ψ l2. To If we let denote the set of summable sequences of avoid convergence problems we embed the Hilbert space= of∗ the functionals,(D) and B denotes the backward shift operator, A classical∗ framework= in a wider space. This wider framework∉ B xt t Z xt 1 t Z, then: Z will be specific for each difference equation, since it is defined is∈ a subspace− of∈ L and B . Hence, if ({ } ) = { } ∗ using the white noise process w in (3). The only tools we need ● f and θ R X then θ B f θ f . are the standard scalar product , in the Hilbert space, and IfA f and Ψ RZ[, then(D) Ψ] f L(A) = AZ. ∈ A ∈ [ ] ( )( Z) = ∗ ∗ ∈ A the set of finite linear combinations of vectors belonging to ● If f , θ R X and Ψ R , then Ψ θ f ⟨⋅ ⋅⟩ ∈ A ∈ ∗ ∈ [ (D) ] a Hilbert basis that includes standardized random variables in ● Ψ θ f θ Ψ f . w. (See the appendix∈ A for∈ the[ corresponding] ∈ proofs and∗ also( ∗ for) the= Let L2 S, B,P be the Hilbert space of scalar random proofs( of∗ all) forthcoming∗ = ∗ ( propositions,∗ ) lemmas and theorems). variables with finite variance defined on the probability space 2) Solutions to φ x θ w∗ in the form of convolutions: S, B,P (, and consider) a Hilbert basis (a maximal or- we are now ready to state the main result of this SectionV: wt thonormal subset) of L2 S, B,P such that t Z , ∗ = ∗ σ Z ( wt) D Proposition V.1. If φ 0 and θ are polynomials and Ψ R where σ are the standardized random variables. The space ∗ L is the algebraic( dual space) 8 of all finite{ S linear∈ } com-⊂ D such that φ Ψ 1 then Ψ θ w is a solution of ≠ ∗ ∈ binations∗ of . The map we use to embed L2 S, B,P in ∗ = φ ( x∗ θ) ∗w . (16) L(D) is given by f f where f v f, v for all 9 Z v L∗ . SinceD this map is∗ linear and∗ injective,( then) for Note that in general there∗ is= more∗ that one sequence Ψ R such that φ Ψ 1. In particular, if φ has no roots on the unit all(Dy) L2 S, ,P Z, ↦ ( ) = ⟨ ⟩ B 1 ∗ ∈ (D) circle and Ψ l1 then Ψ θ w is the embedded∈ φ y θ w if and only if φ y∗ θ w∗; (14) φ ∈ [ ( )] version of the∗ solution= y of (5) in L Z. ∗ ∗ where y and w are the corresponding sequences of embed- = ∈ ( ∗ ) ∗ ∗ ∗ = ∗ ∗ = ∗ dings of y and w, respectively. VI.CO-STATIONARITY[ (D) ] Now, within this framework it is easy to provide a meaning By embedding the problem in L we have avoided for a w via the following definition: t Z t t convergence issues. But, is there any relation∗ between the ∗ ∈ new solutions and stationarity? Our( nextD) step is to find an ∑ atwt v atwt v , (15) ∗ expression similar to (6) in L . In this section we define t Z ∗ wt v 0 ∗ co-stationarity for pairs of sequences,∗ and we search for pairs Q∈ ( ) ≡ Q ( ) 7but this physical interpretation is outside( )≠ the mathematical framework. of co-stationary solutions to (16(D).) To do so, we shall use a so 8 ∗ although here L(D) ⊂ L2(S, B,P ) ⊂ L(D) , this should not be called [33] partial inner product in L : two functionals confused with the Gelfand triple or the Rigged Hilbert Space, since no f, g L are said compatible if d ∗ f d g d , topology in defined on L(D) (see [32]). 9 ∗ and in this case∗ f, g ∗ d f(Dd )g d is their inner since D is maximal orthonormal: f = 0, if and only if ∀d ∈ D, ⟨f, d⟩ = 0, L ∈D if and only if f = 0. product∈ .( ThisD) is a partial inner product∑sinceS it( is) only( )S defined< ∞ ⟨ ⟩ (D) = ∑ ∈D ( ) ( ) 7

for compatible functional pairs. Clearly, if f, g L2 S, B,P Those sequences are given by the following formulas: then f a d and g b d, and therefore d d d d 0 if t k ∈ ( ) 1 ∈D ∈D if t k = ∑ = ∑ φk f, g adbd f, d g, d Λt ⎧ (Backward inverse) L2 S,B,P ⎪ n k < − d d ⎪ 1 ⎪ φ Λt iφk i if t = −k ( ) = ⎨ k i−1 ⟨ ⟩ = Q f d=gQd⟨ ⟩⟨f , g ⟩ ∗ . (17) − ∈D ∈D L ⎪ − + d ⎪ (21) ∗ ∗ ∗ ∗ ⎪ ∑= > − (D) ⎩ = Q∈D ( ) ( ) = ⟨ ⟩ which is zero for all but finitely many negative indices j (a It follows that the embedding of L2 S, B,P in L is formal Laurent series); while the sequence also an isometry, and wi , wj L ∗ δij (the Kronecker∗ ( ) (D) delta). From now on, we shall∗ ∗ simply write f, g instead of 0 if t n ⟨ ⟩ (D) = f, g L ∗ . 1 if t n ⟨ ⟩ Υ φn (Forward inverse) Now we can extend the notion of stationarity to pairs of t ⎧ n k > − (D) ⎪ 1 ⟨ ⟩ Z ⎪ sequences of functionals in L and define the pseudo- ⎪ φ Υt iφn i if t = −n n i−1 covariance function for co-stationary∗ pairs: = ⎨ − ⎪ + − (22) [ (D) ] ⎪ ∑= < − Definition VI.1 (Co-stationarity). The sequences f, g is zero⎩ for all but finitely many positive indices j. From L Z are co-stationary if Proposition V.1 we know that Λ θ w∗ and Υ θ w∗ ∗ ∈ are solutions to (16), which we name the backward and the 1) fi and gj are compatible for all i, j Z, and [ (D) ] forward solutions, respectively.( ∗ ) ∗ ( ∗ ) ∗ 2) fi, gj fi t, gj t for all i, j, t Z. ∈ We then say that f, g is a co-stationary pair. Lemma VI.2. If Ψ RZ is zero for all but finitely many ⟨ ⟩ = ⟨ + + ⟩ ∈ negative indices j, and Ω RZ is zero for all but finitely many Definition VI.2. If( f and) g are co-stationary, we define their positive indices j, then∈ Ψ w∗ and Ω w∗ are co-stationary. pseudo-covariance function as: ∈ It follows then that: ∗ ∗ Γ f,g Z R , (18) Theorem VI.3. For any polynomials φ 0 and θ at least one j f0, gj pair of co-stationary solutions of ∶ Ð→ ∗ ≠ and therefore fi, gj Γf,g j →i . ⟨ ⟩ φ x θ w (23)

Moreover, a usual stochastic process y L2 S, B,P Z exists in L Z: the pair consisting of the backward and ⟨ ⟩ = ( − ) ∗ ∗ ∗ = ∗ is (second-order) stationary if and only if y , y is a co- the forward solutions.∗ stationary pair (where y∗ is the embedding∈ of[ y(in L )]). [ (D) ]

Therefore, we can also refer to Γy∗,y∗ Γy as( the usual) auto-∗ VII.THEUNIQUENESSOFTHEPSEUDO-SPECTRUM (D) covariance function of y [see (7)]. The pseudo-spectrum associated to φ x θ w∗ is defined = To complete these extensions we give one more statement in the literature as pertaining linear filters in the new framework, which we use θ e iω θ eiω ∗ = ∗ in Section VII to find the domain of the Extended Fourier σ2 . φ e−iω φ eiω Transform. ( ) ( ) Although pseudo-spectrum is− unique for each LSDE, we can ( ) ( ) Proposition VI.1. Given two polynomials θ X , φ X , if f find more than one pseudo-covariance function. For example, and g are co-stationary, then θ f and φ g are so, and backward and forward solutions form a co-stationary pair, ( ) ( ) however, when φ has all roots outside the unit circle, the Γ θ X 1 ∗ φ X Γ∗ . (19) θ∗f,φ∗g f,g backward solution with itself is also so (since backward − solution is stationary in this case). It is easy to see that pseudo- = ( ) ∗ ( ) ∗ covariance function for the backward-forward solution pair is A. Co-stationary solution pairs zero for all but finitely many positive indices, whereas auto- Our results in the previous section clearly show that if the covariance function for the backward solution is symmetric. AR polynomial φ 0 has no roots on the unit circle, there is Therefore, our target in this section is twofold: we want a co-stationary pair of solutions y∗, y∗ . But we need co- firstly to extend the DTFT outside the Hilbert space; and stationary pairs of≠ solutions to any LSDE. Indeed, we are secondly, to do so in such a way that for a given LSDE, almost ready to show (Theorem (VI.3) that) there is at least it links the same pseudo-spectrum to all pseudo-covariance one co-stationary pair of solutions for any LSDE: the pair functions of co-stationary solutions. Fortunately those pseudo- consisting of what could be interpreted as the backward and covariance functions have something in common. Indeed, ∗ the forward solutions to (16). First we need to remember that consider, g and h, two co-stationary solutions to φ x θ w ; k k 1 n for any φ X φkX φk 1X φnX with φk 0 φn, since Γφ∗g,φ∗h Γθ∗w∗,θ∗w∗ , then, by Proposition VI.1 we can find two sequences Λ and+ Υ such that 1 1 ∗2 = ∗ + φ X φ X Γ θ X θ X σ . (24) ( ) = + ⋯+ ≠ ≠ = g,h − − φ Λ 1 φ Υ. (20) [note the( similarity) ∗ ( between) ∗ (=24)( and) (8∗)].( ) ⋅ ∗ = = ∗ 8

θ π,π It follows that for any m since for any φ 0 and θ polynomials, φ C a.e , we shall choose as F( ) [− ] m 1 m 1 2 F( ) X φ X φ X Γg,h X θ X θ X σ , (25) ≠ ∈ ~( ) θ − − Q π,π a.e φ 0, θ are polynomials , but Xm φ X 1 φ X and Xm θ X 1 θ X are φ C ∗ ( )∗ ( )∗ = ∗ ( )∗ ( )⋅ [− ] polynomials, provided− that we take a large enough− m. Thus, F( ) (28) Q ≡ ∈ ~( )W ≠ ¡ pseudo-covariance∗ ( ) ∗ functions( ) are sequences∗ ( such) ∗ that( ) when and thenF the( ) extension’s co-domain will be L2 π, π . multiplied by a particular polynomial, we get another poly- c) The Extended Fourier Transform, : both and π,π nomial. L2 π, π are subspaces of C a.e. Qso+ we[ define− ] the Extended Fourier Transform as:[− ] F Q [− ] ~( ) A. The Extended Fourier Transform, l2 L2 π, π , (29) Ψ Ω θ Ω Our task is now to extend the DTFT in such a way that, φ F F ∶ S + Ð→ QF(+) [− ] for any given pair of polynomials ψ 0 and θ, it assigns where φ Ψ θ, +φ 0 .→ ThisF extension( ) + F( is) well defined since Ψ ψ Ψ θ the same image to all which verify . To retain given Ψ, Ψ and Ω, Ω l2 such that Ψ Ω Ψ Ω then ≠ θ θ′ some of the properties of the DTFT, we should require that ∗ Ω′ = ( ≠ )′ Ω (see the appendix).′ ′ ∗ = φ φ′ if φ Ψ θ then φ Ψ θ and therefore Ψ F( ) ∈ S F( ) ∈ ′ + = + θ θ . Hence, to get a meaningful extension we need to F( ) + F( ) = F( ) + F( ) φ φ B. The pseudo-spectrum F( )∗ F=( ) F( )F( ) = F( ) F( ) = include the set θ where φ 0 and θ are polynomials F( ) = F( ) φ We are now ready to state our main result: in the transform co-domainF( ) and to prove that if φ Ψ θ then θ θ′ F( ) . First, weU establish≠ the domain and′ co-domain′ Theorem VII.2. Given a pair of polynomials ψ 0 and θ, φ φ′ ∗ ofF( the) ExtendedF( ) Fourier Transform, and then we∗ define= the for any pair of co-stationary solutions to φ x θ w the extension.F( ) = F( ) Extended Fourier Transform of its pseudo-covariance≠ function is ∗ = ∗ a) Extending the domain of the DTFT: consider, in iω iω 10 2 θ e θ e addition to the Hilbert space l2, the following subspace σ . (30) φ e−iω φ eiω of RZ: ( ) ⋅ ( ) S This common image for− the Extended Fourier Transforms there exists a polynomial φ 0 ( ) ⋅ ( ) Ψ Z . of any pair of co-stationary solutions to φ x θ w∗ is the R such that φ Ψ is also a polynomial pseudo-spectrum. ≠ (26) S ≡ ∈ W ¡ ∗ = ∗ With the aid of the following∗ statement: VIII.WHICHOF LOYNES’ DESIRABLEPROPERTIESDOES Lemma VII.1. If φ Ψ θ and φ Ψ θ with φ, φ THISPSEUDO-SPECTRUM SATISFY? X 0 and θ, θ X then θ ′ φ θ ′φ, ′ R R In a seminal paper, Loynes proposed a list of eight desirable ′∗ = ∗ ′ =′ ∈ we[ can] − show{ } that the∈ map[ with] domain∗ = and∗ co-domain properties regarding the pseudo-spectrum [2]. He deals with (see below) given by representations of continuous-time processes which explicitly S Q consider the time-dependence of the frequency content of the θ Ψ , (27) signal. But we deal with discrete time processes where the φ F( ) frequency content has no time dependence. Since, as far as ↦ we know, our paper is the first formal approach to this type where Ψ is such that φ Ψ F(θ,) φ 0 and where is the DTFT11, is well defined; since it does not depend on the of pseudo-spectrum, we could not find a more appropriate list election of the pair φ, θ∗ (see= the( appendix≠ ) for the proof).F of desirable properties within our context. Hence, the extension’s domain will be l2. Let us see which of Loynes’ properties satisfies our pseudo- b) Extending the( co-domain) of the DTFT: the usual spectrum: Hilbert space L2 π, π (the DTFT co-domain)S + is a quotient A1: The pseudo-spectrum is a real function of time and of set where two functions belong to the same almost everywhere “frequency”, completely determined by the covariance (a.e.) equivalence[− class]if they differ only in a set of measure function. Loynes says this is the minimum that could zero. If we consider the same equivalence relationship over the be assumed about a spectrum. By Theorem VII.2, our π,π set of all complex functions in C , the a.e. equivalence pseudo-spectrum is a time independent real function class f of a function f has a multiplicative[− ] inverse if and completely determined by any of the pseudo-covariance only if the set of zeros of f has zero measure. As the DTFT of functions. a non-zero[ ] polynomial only has a finite number of zeros, the A2: The pseudo-spectrum describes the distribution of energy π,π DTFT has a multiplicative inverse in C a.e . Moreover, over frequency. This physical interpretation holds, but [− ] here each root (or each complex conjugate pair of roots) 10 Note that (φ ∗ Ψ ∈ R[X] and ψ ∗ Ω ∈ R[X]) ⇒ (~(φ ∗ ψ))∗ (aΨ + bΩ) ∈ on the unit circle produces a pole in the pseudo-spectrum R[X], and a, b ∈ R. Note also that, from (25), pseudo-covariance functions with an infinite contribution to the variance (energy). are vectors in this subspace. 11 A3: The pseudo-spectrum transforms reasonably, and prefer- For a sequence Φ ≡ {φt}t∈ in l2, the DTFT is defined as [F(Φ)] (ω) = ∞ −2πiωt Z ∑t=−∞ φte , belonging to L2[−π, π]. ably simply, when the process xt t Z is transformed { } ∈ 9

linearly. In particular, a knowledge of the spectrum of Γ e iω x determines the spectrum of the transformed process. − Loynes says that it would seem that one of properties ( ) A2 and A3 is essential if the name spectrum is to be justifiable; A2 describes what it is, and A3 how it can be used. Clearly property A3 is stated by Proposition VI.1. A4: The relationship between pseudo-spectrum and the covariance function is one to one. Loynes says that this is not altogether essential, but one would not wish to lose too much information in passing from covariance to σ2 spectrum. He speculates that this is probably the simplest w way of ensuring that the second part of A3 is satisfied. Although our pseudo-spectrum does not satisfy A4 (the ω Extended Fourier Transform is not invertible), Proposi- 0 π tion VI.1 holds; and the pseudo-spectrum is unique for each LSDE (Theorem VII.2). Fig. 1. Pseudo-spectrum for the white noise, a = 0 (dashed); the AR(1) model with a = 0.5 (dotted); and for the Random Walk model, a = 1 (continuous). A5: The pseudo-spectrum reduces to the ordinary spectrum, or some simple transformation if x is in fact stationary. Loynes says this is probably essential. In our case, since φ 1 aX and θ 1, these models have a pseudo-spectrum for any (second-order) stationary stochastic process y whose formula is ∗ ∗ the pair y , y is co-stationary, the pseudo-covariance = − = 2 2 σw σw function Γ ∗ ∗ Γ l is the auto-covariance function fx ω . (32) y ,y y 2 1 ae iω 1 aeiω 1 2a cos ω a2 of y (see( Section)VI), and therefore its Extended Fourier Γ= ∈ Γ Fig.1( shows) = the pseudo-spectrum− =for models in three partic- Transform y and its DTFT y are the same. ( − )( − ) − ( ) + A6: If the process is composed of a succession of stationary ular cases. parts, say xF1(t t )0 and x2t t 0, thenF( the) spectrum is also It should be noted that, when 0 a 1, pseudo-spectrum composed of the corresponding succession of (stationary) and spectrum of the stationary solution to (31) is the same ≤ > spectra. This{ property} is{ conceived} in the context of time- function. But, when a 1 only the pseudo-spectrum≤ < is defined. variable parameter models which is not our context (see Hence, pseudo-spectrum generalize the spectrum to the non- [2]). stationary case. = A7: The pseudo-spectrum is estimable in principle, probably from (infinite) length of record. In our case, a parametric B. Sum of non-stationary signals estimation is possible following the Box-Jenkins model- Suppose that Z S N, where S, and N are unobservable ing approach to identify and estimate the AR, φ, and MA, signal and noise components that follow the models, θ, polynomials. = + A8: The pseudo-spectrum is the Fourier transform, or some φS S θS b (33) related transform, of some apparently meaningful quantity. φN N θN c, (34) Loynes says that such a property would be welcome, ∗ = ∗ but it does not seem important. Our pseudo-spectrum is where each of the pairs∗ of polynomials= ∗ φS, θS , and the Extended Fourier Transform of the pseudo-covariance φN , θN have their zeros lying on or outside the unit function of any co-stationary pair of solutions of LSDE. circle and have no common zeros, and b, and{ c are} mutually In addition, it is the quotient of two positive defined independent{ } white noise processes. The non-stationary signal functions and, hence, it is also positive defined (another extraction problem in this framework has been studied since desirable feature). the sixties (see [21], [26], [28], [34]–[38]). As Bell pointed Hence, our pseudo-spectrum satisfies six out of eighth proper- out in [35]: typically, the solution for the stationary case has ties of Loynes’ list; particularly all those qualified as “essen- been borrowed and used in the nonstationary case. But this tial”. has been done without a proper definition of pseudo-spectra and pseudo-covariance functions. In this section we justify a result frequently used in the literature: the “spectrum” of Z is IX.EXAMPLESANDAPPLICATIONS the sum of the “spectra” of S and N even in the non-stationary A. The simplest example: AR(1) models case. Consider the set of models Consider φ, the least common multiple of φS and φN ; and let ϕS and ϕN be the polynomials such that φ ϕS φS xt axt 1 wt (31) ϕN φN . We can multiply (33) and (34) by ϕS and ϕN − respectively to get = ∗ = where a 0, 1 . When a is− zero, x= is the white noise process w; when a 1, it corresponds to the Random Walk model ∗ φ S ϕ θ b (35) we have∈ used[ ] as illustration in SectionIV. For 0 a 1, S S = φ N ϕN θN c. (36) there are no roots on the unit circle. Following (12), since ∗ = ∗ ∗ < < ∗ = ∗ ∗ 10

Solutions to (33) and (34) are also solutions for (35) and (36) This decomposition of a pseudo-spectrum as a sum of pseudo- respectively. Adding the last two equations we get spectra is widely used in the literature of unobserved component models, and Seasonal Adjustment of Economic Time φ S N φ Z ϑ b ϑ c S N Series (see references in SectionI). The above is a formal θ a, (37) justification for this decomposition, that it is used in the next ∗ + = ∗ = ∗ + ∗ example. where we write each product ϕk= ∗θk as ϑk; and where the last equality follows from the fact that b and c are independent white noise processes, so the right∗ hand side is a stationary C. Dynamic Harmonic Regression process with a finite moving average representation θ a. The Dynamic Harmonic Regression (DHR) model [20] is Now, lets consider a Hilbert basis of L2 S, B,P such bt ct bt ct ∗ based on a spectral approach, under the hypothesis that the that σ t Z and σ t Z , where σ , σ are the standardized random variablesD of the two mutually( indepen-) observed time series z is periodic or quasi-periodic and can dent{ whiteS ∈ noise} ⊂ D processes{ bS and∈ }c⊂. UsingD the corresponding be decomposed into several components whose variances are sequences of embeddings b∗ and c∗, we can consider the concentrated around certain frequencies: e.g. at a fundamental backward-forward co-stationary solution pairs S , S , and frequency and its associated sub-harmonics. This is an appro- N , N to ▹ priate hypothesis if the observed time series has well defined ◂ spectral peaks, which implies that its variance is concentrated ▹ ∗ ( ) ( ◂ ) φS S θS b (38) around narrow frequency bands. By ‘quasi-periodic’ we mean ∗ φN N θN c . (39) that the amplitude and the phase of the periodicity may vary ∗ = ∗ over time. The DHR model is the sum of several Unobserved By Proposition VI.1, the∗ corresponding= ∗ pseudo-covariance Components: functions verify R j 1 1 2 s e (47) ▹ φ x φ x ΓS◂,S ϑS x ϑS x σb (40) j 0 −1 − 1 2 φ x φ x ΓN N▹ ϑ x ϑ x σ . (41) Q + ( ) ∗ ( ) ∗ ◂, = N( ) ∗ N( ) ⋅ c where the irregular component,= e, is normally distributed with − − zero mean and variance σ2; and each DHR component sj has Adding( these) ∗ equations( ) ∗ we get= ( ) ∗ ( ) ⋅ e the form 1 j j j φ x φ x ΓS ,S▹ ΓN ,N▹ ◂ ◂ st at cos ωjt bt sin ωjt . (48) − ϑ x 1 ϑ x σ2 ϑ x 1 ϑ x σ2. (42) ( )S∗ ( ) ∗ (S + b N ) N c j − − Oscillations of each= DHR( component,) + (s , are) modulated by If we choose the solution pair Z S N and Z j j = ( ) ∗ ( ) ⋅ + ( ) ∗ ( ) ⋅ the stochastic processes a t Z and b t Z. Both stochastic S N to the embedding of (37), then ▹ processes, aj and bj, are solutions to the same AR(1) or AR(2) ◂ ◂ ◂ ∈ ∈ ▹ ▹ = + = difference equations with{ at} least one{ root} on the unit circle. 1 1 2 φ x φ x Γ ▹ ▹ θ x θ x σ . (43) + S◂+N◂,S +N a The frequency ωj is associated to the jth component. Usually − − j 0 corresponds to the zero frequency term, that is, the Since( b and) ∗ c(are) mutually∗ independent= ( white) ∗ noise( ) ⋅ processes, the backward-forward co-stationary pair Z , Z has a par- trend; and the other components (j 1, ..., R) correspond to = ticular property: the pseudo-covariance generating▹ function of the seasonal frequency and its harmonics. Hence, the complete ◂ = the “addition” pair Z , Z is the addition( of) the pseudo- DHR model is covariance generating functions▹ of S , S , and N , N : R j j ◂ a cos ωjt b sin ωjt et (49) ( ) ▹ ▹ j 0 t t Γ ▹ ▹ Γ ▹ ◂Γ ▹; ◂ S◂+N◂,S +N S◂,S ( N◂,N) ( (44)) This modelQ can= be considered( ) a+ straightforward( ) + extension of (see the appendix for the proof)= so, the+ left hand sides of (42) the classical harmonic regression model, in which the gain and and (43) are equal (cp. [28, Equation 1.4]). phase of the harmonic components can vary randomly due to From (44), it follows a more general result. Since the image the stochastic processes aj and bj. of the Extended Fourier Transform of the pseudo-covariance In [39] it is shown that each sj has an alternative represen- function of any pair of co-stationary solutions of a linear tation as a solution to stochastic difference equation is common, then the pseudo- j j j j spectrum associated to (37) is the sum of the pseudo-spectra φ s θ w , j 0,...,R; (50) associated to (33) and (34) j j 2 where φ has roots∗ = on the∗ unit circle,= and w N 0, σj . iω iω iω ΓZ e ΓS e ΓN e ; (45) Hence, the pseudo-spectrum of the DHR model is the sum of − − − the pseudo-spectra of is components: ∼ ( ) or ( ) = ( ) + ( ) R j iω j iω iω iω 2 2 θ e θ e 2 θ e θ e 2 fdhr ω, σ σ σ ; (51) σ j j −iω j iω e −iω iω a j 1 φ e φ e φ e φ e ( ) ( ) ( ) ⋅ ( iω ) iω iω iω = Q − + − 2 θS e θS =e 2 θN e θN e = 2 2 2 2 σ σ . where the variances in vector σ( )σ0,( . . . ,) σR, σe are the ( b ) ⋅ ( iω ) iω c iω iω (46) φS e− φS e φN e− φN e unknown hyper-parameters of the model. ( ) ⋅ ( ) ( ) ⋅ ( ) − + − = ( ) ⋅ ( ) ( ) ⋅ ( ) 11

The model is fitted in the frequency domain by seeking the Ψ θ f i v . Since v L we can choose m, m such 2 12 vector σ that minimizes the euclidean distance that fi v 0 m i m . Then ′ [( ∗ )∗ ] ( ) ∈′ (D) 2 min fz ω fdhr ω, σ , (52) Ψ ( θ) ≠ f ⇒i v ≤ ≤ 2 R+1 σ R Ψr θ f s v Ψr θpf q v [ ]∈ Z ( ) − Z [ ∗ ( r ∗s i)] ( ) r s i p q s where fz ω is the spectrum of the observed time series. This m′ strategy has an intuitive appeal but, since DHR components are = +Q= ( ∗ )) ( ) = +Q= +Q= ( ) non-stationary,( ) the corresponding pseudo-spectra have poles; Ψr θs qf q v making t s q r s i q m and therefore the norm is not defined. − = Q Qm′ ( )(n m=′ − ) In order to find the Ordinary Least Squares (OLS) solution, + = = Ψr θtf q v Ψi t q θtf q v it is needed to eliminate the unit modulus AR roots of r t q i q m t 0 q m 2 − − fdhr ω, σ . Fortunately, it is possible to exploit the algebraic ′ ′ = +n(Q+ m)= Q= ( ) = mQ= n Q= ( ) structure of the pseudo-spectra model. If (52) is multiplied Ψi t qθtf q v Ψi t qθtf q v by the function Ψ ω ϕ e iω ϕ eiω , where ϕ is the t 0 q m q m t 0 − − − − m′ n m′ minimum order polynomial with− all unit modulus AR roots = Q= Q= ( ) = Q= Q= ( ) of the complete DHR( ) model,= ( then) we( can) try the alternative Ψi t qθt f q v Ψ θ i q f q v q m t 0 q m minimization problem: − − − = Q= Q= Ψ θ(r)f=s vQ= ( Ψ∗ )θ f(i )v . 2 r s i min Ψ ω fy ω Ψ ω fdhr ω, σ . (53) σ2 R+2 R = +Q= ( ∗ ) ( ) = [( ∗ ) ∗ ] ( ) Z ∈ Z ( ) ⋅ ( ) − ( ) ⋅ 2 Z If f , Ψ R and φ R X then φ Ψ f φ Ψ f. Since Ψ ω fy ω and Ψ ω fdhr ω, σ are functions in n Proof: Let φ φ0 φnX , as before; we again only L2 π, π , it follows that (53) can be solved by OLS (see ∈ A ∈ ∈ [ ] ∗( ∗ ) = ( ∗ )∗ ( ) ⋅ ( ) ( ) ⋅ need to check that for any i Z and any v L , φ [39]). = + ⋯ + [− ] Ψ f i v φ Ψ f i v . Since v L we chose m, m such that fi v 0 m∈ i m . Then∈ (D) [ ∗ ( ∗ )] ( ) = [( ∗ ) ∗ ] ( ) ∈ (D) X.CONCLUSIONS ′ n ′ φ Ψ f v( ) ≠ φ⇒ Ψ≤ f≤ v If the spectrum is defined within the algebraic framework i k i k k 0 provided in this paper, spectrum and pseudo-spectrum are the n n − m′ [ ∗ ( ∗ )] ( ) = Q= ( ∗ ) ( ) same functions. But even when the spectrum is not-defined, φk Ψrf s v φk Ψi k sf s v the pseudo-spectrum is well defined for any LSDE. k 0 r s i k k 0 s m m′ n m′ − − Contrary to the case of the spectrum, and since the pseudo- = Q= +Q= − ( ) = Q= Q= ( ) spectrum is the Extended Fourier Transform of the pseudo- φkΨi k s f s v φ Ψ i s f s v s m k 0 s m covariance function of any pair of co-stationary solutions of − − − ∗ = Q= Q= ( ) = Q= ( ∗ ) ( ) φ x θ w , the pseudo-spectrum is not associated to φ Ψ f v φ Ψ f i v . ∞ i s s any particular solution, neither to any particular pair of co- s stationary∗ = solutions.∗ The Extended Fourier Transform is not = Q ( ∗ ) − ( ) = [( ∗ ) ∗ ] ( ) =−∞ in invertible as an operator and therefore the pseudo-spectrum is ∗ associated to the difference equation itself. Proof of Proposition V.1: φ Ψ θ wA φ Ψ ∗ ∗ The convolution type solutions to (23) that we use in θ w 1 θ w . ³¹¹¹¹¹¹¹¹¹¹¹¹¹¹·¹¹¹¹¹¹¹¹¹¹¹¹¹¹µ the paper (f Ψ θ w∗ where φ Ψ 1) closely Proof of Proposition VI.1: By∗ ( definition:∗ ( ∗ )) = ( ∗ ) ∗ ( ∗ ) = ∗ ( ∗ ) resemble the ones used in the literature (see the references n m given in Section= (III).∗ However,) ∗ we had to∗ define= them in a θkfi t k, φlgj t l different framework, the space L Z, so as to avoid the k 0 l 0 n +m− + − n m convergence issues of convolution expressions.∗ The embedding dQ= Q= i θk φl fi t k, gj t l θk φl fi k, gj l of L2 S, B,P in L is[ an( isomorphicD) ] isometry, and k 0 l 0 k 0 l 0 therefore our algebraic model∗ constitutes a generalization + − + − n m− − = Q= Q= ⟨ ⟩ = Q= Q= ⟨ ⟩ proper( of the spectral) (D theory) to the case in which the AR θkfi k, φlgj l ; k 0 l 0 polynomial φ has roots on the unit circle. − − hence θ f and φ g are co-stationary,= dQ= and thenQ= i n m APPENDIX ∗ ∗ Γθ f,θ g j θk φl f k, gj l k 0 l 0 Z If f , θ R X and Ψ R then Ψ θ f Ψ θ f. ∗n ∗ m − n − n ( ) = Q= Q= ⟨ ⟩ Proof: Let θ θ0 θnX . We only need to check θk φlΓf,g j l k θk φ X Γf,g j k that for∈ anyA i∈ Z[ and] any v∈ L , ∗(Ψ∗ )θ= (f ∗i )v∗ k 0 l 0 k 0 = + ⋯ + = n ( − + ) = [ ( ) ∗ ]( + ) Q= Q= θ X 1 φ XQ= Γ j k 12 k f,g The algorithm∈ proposed in∈ [20](D seeks) [ the∗ vector( ∗ NVR)] ( ) = 2 2 k 0 − [1,NVR0,...,NVRR], where NVRj = σj ~̂σ , using the residual − 1 2 = Q [ ( )] [ (θ X) ∗ φ](X+ )Γf g j . variance ̂σ from a fitted AR model of the observed series z. = , − = [ ( ) ∗ ( ) ∗ ]( ) 12

Thus we get on the one hand, θ φ Ω Ω and θ ′ Proof of Lemma VI.2: We know there are integers m and consequently Ω Ω φ . And on the other hand, m such as k m Ψk 0 and k m Ωk 0. Then: since ψ Ψ β and′ ψ Ψ βFF,(( it) follows= F( ) that⋅ F( − ) F( ) ′ ′ F( ) − ′F( ′) = ′ ∗ < ⇒ ∗ = > ⇒ = ∗ = ψ ∗ψ Ψ= ψ β Ψ w i t, Ω w j t ′ ′ ∗ ∗ ψ ψ Ψ ψ β ; + Ψ w +i t d Ω w j t d ∗ ′ ∗ ′ = ∗ ′ ⟨[ ∗ ] d [ ∗ ] ⟩ and therefore ψ ψ Ψ Ψ ψ β ψ β . Now, using ∗ + ∗ + ∗ ∗ = ∗ = ΨQ∈D[w ∗ i t ]wh (σ)[Ω ∗ w ]j t (w)h σ Lemma VII.1 ′ ′ ′ ′ h Z ∗ ∗ ( − ) = ∗ − ∗ 2 + 2 + ψ ψ θ ψ β ψ β φ; = Qσ∈ [ ∗Ψi t] hΩ(j t ~h )[σ ∗ Ψ] i h(′ Ωj ~h′) ′ ′ ′ ′ h Z h Z + − + − ′ − − hence ψ ψ θ ψ β ψ β 2 h i m ∗ ∗ = ( ∗ − ∗ ) ∗ = Q∈ = Q∈ ′ ′ σ h′ j m′ Ψi h Ωj h φ , and′ then ′ ′ = − F( ) ⋅ F( ) ⋅ F( ) = (F( ) ⋅ F( ) − F( ) ⋅ F( )) ⋅ − − θ ψ β ψ β β β = Q = − F( ) . φ ′ ψ ψ ′ ψ ψ′ Proof of Theorem VI.3: Let Λ and Υ be the backward F( ) F( ) ⋅ F( ) − F( ) ⋅ F( ) F( ) F( ) ′ ′ ′ and forward inverses of φ defined above in the text; from Thus =Ω Ω θ β β= , and therefore− ∗ ∗ F( ) F( ) ⋅φF( ) ψ ψ′ F( ) F( ) Proposition V.1 we know that Λ θ w and Υ θ w ′ F( ) F( ) F( ) β F( ) F( β) F( ) are solutions to (23). On the other hand, from Lemma VI.2 F( ) − F( ) = Ω = − Ω . we know that Λ w∗ and Υ w( ∗ ∗are) co-stationary,∗ ( ∗ it follows) ∗ ψ ψ′ ∗ ∗ F( ) F( ) ′ that θ Λ w and θ Υ w are also co-stationary (Propo- + F( ) = ′ + F( ) sition VI.1). Further,∗ from the∗ properties in Section V-A1 we Proof ofF Theorem( ) VII.2: FFrom( ) (25) we know Γ is ∗ ∗ ∗ g,h get θ ∗ Λ∗ w ∗θ ∗Λ w Λ θ w , which in . Therefore, since Φ Ω Φ Ω , the pseudo- it is the backward solution. A similar argument shows that spectrum, Γ , is ∗ ∗ g,h θ Υ∗ (w ∗ Υ) =θ( ∗w )gives∗ the= forward( ∗ ) solution.∗ S F( ∗ ) = F( )F( ) Proof of Lemma VII.1: Multiplying φ Ψ θ by θ ; and Xm θ X 1 θ X θ X 1 θ X σ2 F( ) σ2 φ∗ (Ψ ∗ θ by) =θ(we∗ get) ∗ ′ Xm φ X−1 φ X φ X−1 φ X ′ ′ ∗ = F( ∗ ( ) ∗ ( )) F( ( iω)) ⋅ F(iω( )) − = 2 θ e− θ e ∗ = F( ∗ ( ) ∗ ( )) Fσ( ( )) ⋅ F( ( .)) θ φ Ψ θ θ φ e−iω φ eiω θ′ φ Ψ θ′ θ , ( ) ⋅ ( ) ∗ ∗ = ∗ = − ′ ′ Proof of Equation (44): Let bt t ( ,) ⋅ ct(t ) ∗ ∗ = ∗ σ Z σ Z thus θ φ θ φ Ψ 0. Thus, there are only two , then we have and . Using the backward possibilities:′ ′ and forward recursive formulas (21{ )S and∈ (}22=);B if{ denoteS ∈ } the= ( ∗ − ∗ ) ∗ = ∗ 1) If θ φ θ φ 0, from which θ φ θ φ. backwardC solutionB ∩SC to= ∅ (38) asBΨ∪SC ⊂bD, where ΨS ΛS θS, ∗ ′ ′ ′ ′ and the forward solution S to (38) as ΩS b , where ΩS 2) If θ φ θ φ 0, since 0 φ θ φ θ φ ◂ ( ∗ − ∗ ) = ∗ = ∗ ΥS θS; and if we follow the same notation∗ convention= for∗ the ′ ′ ′ ′ ▹ Ψ ( ∗θ −φ ∗ θ ) φ≠ φ Ψ = ∗θ( φ∗ θ− φ∗ )θ∗; backward and forward solution pair N , N∗ to (39), then= we conclude′ θ 0.′ And for the same′ reason θ′ 0. ΓS +∗N ,S▹+N▹ is ▹ ◂ ◂ ◂ Consequently = ( ∗ − we∗ also) ∗ get[ θ∗ φ] = θ( ∗φ. − ∗ ′) ∗ ( ) ∗ ∗ ∗ ∗ = ′ ′ = ΨN b ΨS c i t, ΩN b ΩS c j t ∗ = ∗ ∗ ∗ ∗ ∗ Proof of “Map is well defined”: ΨN b ΨS c i+ t d ΩN b ΩS +c j t d (27) By Lemma VII.1, ⟨[ ∗ + ∗ ] [ ∗ + ∗ ] ⟩ = the fraction θ is uniquely determined by the condition φ d φ [Ψ ∗b∗ +Ψ ∗c∗ ] + (d ) ⋅ [Ω ∗b∗ +Ω ∗c∗ ] + (d ) = Ψ θ: indeed,F( if) we also had φ Ψ θ , then Q∈D N S i t N S j t F( ) d ′ ′ ∗ ∗ ∗ + ∗ ∗ + Q∈B[ΨN ∗ b + ΨS ∗ c ]i t (d) ⋅ [ΩN ∗ b + ΩS ∗ c ]j t(d) + = θ φ θ φ θ ∗φ = θ φ d ′ ′ ′ ′ ∗ + ∗ ∗ ∗ + θ φ θ φ Q[ ∗ ΨN+ b ∗ΨS] c ( i)t⋅ [d Ω∗N b+ ΩS∗ c ] j t( d) + ∗ = ∗ ⇒ F( ∗ ) = F( ∗ ) ∈C θ ′ θ ′ d . ∗ + ∗ + ⇒ F(φ) ⋅ F( φ) ′= F( ) ⋅ F( ) ∈D−Q(B∪C)[ ∗ΨN+ b ∗i t]d ( Ω)⋅N[ b∗ j+t d ∗ ] ( ) = F( ) F( ) d ⇒ = ′ ∗ + ∗ + F( ) F( ) Q∈ΨB[S c∗ i ]t d ( )Ω⋅ S[ c∗ j t] d ( )+0 d Proof of “The Extended Fourier Transform is well de- +∗ +∗ fined”: Let us assume that Ψ Ω Ψ Ω , where ψ Ψ β, Q∈C[ Ψ∗N ]b (i t) ⋅d[ Ω∗N ]b (j )t d+ = d ψ Ψ β (with ψ, ψ R X 0 ′ and′ β, β R X ) and ∗ + ∗ + Q [ΨS ∗c ]i t (d) ⋅ [ΩS ∗c ]j t (d )+ Ω′, Ω ′ l . Then,′ since ′Ψ Ψ+ Ω= Ω+ l′ , there∗ exists= ∈D 2 2 d φ ∗ ′ X= 0 and θ ∈ X[ ′ such] − {′ that} ∈ [ ] ∗ ∗ + ∗ + ∗ R R Ψ b ,[Ω ∗ b ] ( ) ⋅ [Ψ ∗c ] ,( Ω) = c ∈ − = − ∈ S ∩ N iQ∈tD N j t S i t S j t ∈ [ ] − { }φ Ψ ∈Ψ [ ]θ φ Ω Ω . So Γ ▹+ ▹ is equal to+ Γ ▹ Γ ▹+. + ⟨[ S◂∗+N◂,]S +N[ ∗ ] ⟩ S+◂⟨[,S ∗N◂,N] [ ∗ ] ⟩ ′ ′ ∗ ( − ) = = ∗ ( − ) + 13

ACKNOWLEDGMENT [20] P. C. Young, D. Pedregal, and W. Tych, “Dynamic harmonic regression,” Journal of Forecasting, vol. 18, pp. 369–394, November 1999. The authors would like to thank the anonymous reviewers [21] J. P. Burman, “Seasonal adjustment by signal extraction,” Journal of the and the associate editor Dr. A. Napolitano for their comments. Royal Statistical Society. Series A, vol. 143, no. 3, pp. 321–337, 1980. [22] W. R. Bell and S. C. Hillmer, “Issues involved with seasonal adjustment Their suggestions helped improve this article significantly. We of economic time series,” Journal of Business and Economic Statistics, are also indebted to Dr. R. Banerjee, Dr. J. Crespo, Dr. A. vol. 2, pp. 291–320, 1984. Garc´ıa-Hiernaux and Dr. M. Jerez. [23] J. Haywood and G. Tunnicliffe Wilson, “Fitting time series models by minimizing multistep-ahead errors: a frequency domain approach,” Journal of the Royal Statistical Society. Series B (Methodological), vol. 59, no. 1, pp. 237–254, 1997. REFERENCES [24] A. Maravall and C. Planas, “Estimation error and the specification of unobserved component models,” Journal of Econometrics, vol. 92, pp. [1] M. B. Priestley, “Evolutionary spectra and non-stationary processes,” 325–353, 1999. Journal of the Royal Statistical Society. Series B (Methodological), [25] A. Maravall, “Unobserved components in econometric time series,” in vol. 27, no. 2, pp. 204–237, 1965. The Handbook of Applied Econometrics, ser. Blackwell Handbooks in [2] R. M. Loynes, “On the concept of the spectrum for non-stationary Economics, H. H. Pesaran and M. Wickens, Eds. Oxford, UK: Basil processes,” Journal of the Royal Statistical Society. Series B Blackwell, 1995, ch. 1, pp. 12–72. (Methodological), vol. 30, no. 1, pp. 1–30, 1968. [Online]. Available: [26] D. A. Pierce, “Signal extraction error in nonstationary time series,” http://www.jstor.org/stable/2984457 The Annals of Statistics, vol. 7, no. 6, pp. 1303–1320, November 1979. [3] D. Tjøstheim, “Spectral generating operators for non-stationary pro- [Online]. Available: http://www.jstor.org/stable/2958546 Advances in Applied Probability cesses,” , vol. 8, no. 4, pp. 831–846, [27] G. E. P. Box, S. Hillmer, and G. C. Tiao, “Analysis and 1976. modeling of seasonal time series,” in Seasonal Analysis of Economic [4] W. Martin, “Line tracking in nonstationary processes,” Signal Process- Time Series, ser. NBER Chapters. National Bureau of Economic ing, vol. 3, no. 2, pp. 147–155, 1981. Research, Inc, August 1979, pp. 309–346. [Online]. Available: [5] W. Martin and P. Flandrin, “Wigner-Ville spectral analysis of non- http://ideas.repec.org/h/nbr/nberch/3904.html stationary processes,” Acoustics, Speech and Signal Processing, IEEE [28] S. C. Hillmer and G. C. Tiao, “An arima-model-based approach to Transactions on, vol. 33, no. 6, pp. 1461–1470, December 1985. seasonal adjustment,” Journal of the American Statistical Association, [6] C. Detka and A. El-Jaroudi, “The transitory evolutionary spectrum,” in vol. 77, no. 377, pp. 63–70, Mar 1982. [Online]. Available: Acoustics, Speech, and Signal Processing, 1994. ICASSP-94., 1994 IEEE http://www.jstor.org/stable/2287770 International Conference on, vol. 4. IEEE, 1994, pp. IV–289. [29] T. C. Mills, “Signal extraction and two illustrations of the quantity [7] R. Dahlhaus, “Fitting time series models to nonstationary processes,” theory,” The American Economic Review, vol. 72, no. 5, pp. 1162–1168, The Annals of Statistics, vol. 25, no. 1, pp. 1–37, 1997. December 1982. [8] G. Matz, F. Hlawatsch, and W. Kozek, “Generalized evolutionary spec- [30] A. C. Harvey and P. H. J. Todd, “Forecasting economic time series with tral analysis and the weyl spectrum of nonstationary random processes,” structural and box-jenkins models: A case study,” Journal of Business Signal Processing, IEEE Transactions on, vol. 45, no. 6, pp. 1520–1534, and Economic Statistics, vol. 1, no. 4, pp. 299–307, 1983. 1997. [31] C. Chen and G. C. Tiao, “Random level-shift time series models, [9] G. Matz and F. Hlawatsch, “Nonstationary spectral analysis based ARIMA approximations, and level-shift detection,” Journal of Business on time-frequency operator symbols and underspread approximations,” and Economic Statistics, vol. 8, no. 1, pp. 83–97, 1990. Information Theory, IEEE Transactions on, vol. 52, no. 3, pp. 1067– [32] I. M. Gelfand and N. J. Vilenkin, Some Applications of Harmonic 1086, 2006. Analysis. Rigged Hilbert Spaces, ser. Generalized Functions. New York: [10] W. A. Gardner, A. Napolitano, and L. Paura, “Cyclostationarity: Half Academic Press, 1964, vol. 4. a century of research,” Signal processing, vol. 86, no. 4, pp. 639–697, [33] J.-P. Antoine and A. Grossmann, “The partial inner product spaces. i. 2006. general properties,” Journal of Fuctional Analysis, vol. 23, pp. 369–378, [11] P. Flandrin, M. Amin, S. McLaughlin, and B. Torresani, “Time- 1976. frequency analysis and applications [from the guest editors],” Signal [34] G. C. Tiao and S. C. Hillmer, “Some consideration of decomposition Processing Magazine, IEEE, vol. 30, no. 6, pp. 19–150, 2013. oof a time series,” Biometrika, vol. 65, no. 3, pp. 497–502, Dec 1978. [12] W. Tych, D. J. Pedregal, P. C. Young, and J. Davies, “An unobserved [35] W. Bell, “Signal extraction for nonstationary time series,” The Annals component model for multi-rate forecasting of telephone call demand: of Statistics, vol. 12, no. 2, pp. 646–664, June 1984. the design of a forecasting support system,” International Journal of [36] E. J. Hannan, “Measurement of a wandering signal amid noise,” Forecasting, vol. 18, no. 4, pp. 673–695, Oct. 2002. [Online]. Available: Journal of Applied Probability, vol. 4, no. 1, pp. 90–102, Apr 1967. http://www.sciencedirect.com/science/article/pii/S0169207002000717 [Online]. Available: http://www.jstor.org/stable/3212301 [13] T. Vercauteren, P. Aggarwal, X. Wang, and T.-H. Li, “Hierarchical fore- [37] E. Sobel, “Prediction of noise-distorted, multivariate, non-stationary casting of web server workload using sequential monte carlo training,” signal,” Journal of Applied P, vol. 4, no. 2, pp. 330–342, Aug 1967. Signal Processing, IEEE Transactions on, vol. 55, no. 4, pp. 1286–1297, [Online]. Available: http://www.jstor.org/stable/3212027 April 2007. [38] W. P. Cleveland and G. C. Tiao, “Decomposition of seasonal time [14] S. Becker, C. Halsall, W. Tych, R. Kallenborn, Y. Su, and H. Hung, series: a model for the x-11 program,” Journal of the American “Long-term trends in atmospheric concentrations of α- and γ-hch Statistical Association, vol. 71, no. 355, pp. 581–587, Sep 1976. in the arctic provide insight into the effects of legislation and [Online]. Available: http://www.jstor.org/stable/2285586 climatic fluctuations on contaminant levels,” Atmospheric Environment, [39] M. Bujosa, A. Garc´ıa-Ferrer, and P. C. Young, “Linear dynamic vol. 42, no. 35, pp. 8225–8233, Nov. 2008. [Online]. Available: harmonic regression,” Comput. Stat. Data Anal., vol. 52, no. 2, pp. http://www.sciencedirect.com/science/article/pii/S1352231008006857 999–1024, October 2007. [Online]. Available: http://dx.doi.org/10.1016/ [15] T. Vogt, E. Hoehn, P. Schneider, A. Freund, M. Schirmer, and O. A. j.csda.2007.07.008 Cirpka, “Fluctuations of electrical conductivity as a natural tracer for bank filtration in a losing stream,” Advances in Water Resources, vol. 33, no. 11, pp. 1296–1308, Nov. 2010. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0309170810000394 [16] M. Bujosa, A. Garc´ıa-Ferrer, and A. de Juan, “Predicting recessions with factor linear dynamic harmonic regressions,” Journal of Forecasting, vol. 32, no. 6, pp. 481–499, 2013. [17] T. W. Hungerford, Algebra. Hold, Rinehart and Winston, inc, 1974. [18] P. J. Brockwell and R. A. Davis, Time Series: Theory and Methods, ser. Marcos Bujosa was born in Madrid, Spain, on March 25, 1969. He received Springer series in Statistics. New York: Springer-Verlag, 1987. the B.S. degree in Economics from Universidad Autonoma´ de Madrid, Spain, [19] J. Haywood and G. Tunnicliffe Wilson, “An improved state space in 1996 and the Ph.D. degree in Economics at the same university in 2001. representation for cyclical time series.” Biometrika, vol. 87, no. 3, pp. He is an Associate Professor of Econometrics at Universidad Complutense 724–726, 2000. [Online]. Available: http://biomet.oxfordjournals.org/ de Madrid, Spain. His research has been concerned with modelling in the cgi/content/abstract/87/3/724 frequency domain and forecasting seasonal economic time series. 14

Andres´ Bujosa was born in Leeds, UK, on March 26, 1960. He received the B.S. degree in Mathematics from Universidad Complutense de Madrid, Spain, in 1988 and the Ph.D. degree in Mathematics at Universidad Politecnica´ de Madrid, Spain, in 1993. He is an Associate Professor of Applied Mathematics at Universidad Politecnica´ de Madrid, Spain. His research interests include Artiﬁcial Intelli- gence and Computacional Logic.

Antonio Garc´ıa-Ferrer was born in La Roda (Albacete), Spain, on December 13, 1950. He received the B.S. degree in Economics from Universidad Autonoma´ de Madrid, Spain, in 1973 and the Ph.D. degree in Economics at U.C. Berkeley in 1978. From 1978 to 1980, he was Assistant Professor at the Universidad de Alcala´ de Henares, Spain. From 1984 to 1985, he was Fulbright Visiting Professor at the Booth GSB of the University of Chicago. He is currently Full Professor of Econometrics at Universidad Autonoma´ de Madrid, Spain. His research has been concerned with modeling and forecasting seasonal time series, turning point predictions, and leading indicators. Dr. Garc´ıa-Ferrer was President of the International Institute of Forecasters from 2008 to 2012.