Multivariate isotonic time series regression Konstantinos Fokianos University of Cyprus Department of Mathematics and Statistics P.O. Box 20537 CY { 1678 Nicosia Cyprus E-mail: [email protected] Anne Leucht Technische Universit¨atBraunschweig Institut f¨urMathematische Stochastik Pockelsstraße 14 D { 38106 Braunschweig Germany E-mail: [email protected] Michael H. Neumann Friedrich-Schiller-Universit¨atJena Institut f¨urMathematik Ernst-Abbe-Platz 2 D { 07743 Jena Germany E-mail: [email protected] Abstract We consider a general monotone regression estimation problem for time series with multivariate regressors that may consist of nonstationary discrete and/or continuous components as well as a deterministic trend. We propose a modification of the classical isotonic least squares estimator and establish its rate of convergence for the integrated L1-loss. The methodology captures the shape of the data without assuming additivity or a parametric form for the regression function. Some simulations and a data example complement the study of the proposed estimator. 2010 Mathematics Subject Classification: Primary 62G08; secondary 62G20, 62H12, 62M10. Keywords and Phrases: Count time series, isotonic least squares estimation, multivariate isotonic regression, rates of convergence, Rosenthal inequality. Short title: Multivariate isotonic regression. version: September 15, 2016 1 1. Motivation We consider the classical mean regression model Yt f It "t with E "t It 0 a:s:; t Z; (1.1) d where we assume that the= regression( ) + function f (R S R) =is unknown∈ and allow for dependent observations Yt;It t having a general time series structure. The primary aim of this work is to provide a nonparametric′ ′ estimator of f subject∶ → to shape constraints; in particular we will assume that(( the) function) f in (1.1) is isotonic. As we will argue, throughout the paper, the assumption of isotonicity encompasses various data generating processes, including monotone trends which are observed in several applied problems. The problem of estimating a regression function subject to shape constraints in the context of time series has not been addressed adequately in the literature, to the best of our knowledge. There exist a large body of literature on estimation and testing for situa- tions where the class of admissible functions f can be parametrized by a finite-dimensional parameter; see e.g. Seber and Wild (1989), Kedem and Fokianos (2002), Escanciano (2006), Francq and Zakoian (2010) and Shumway and Stoffer (2011) among others. There are also many results on nonparametric kernel estimators for f relying on the assumption that the covariate vector It has a Lebesgue density. For an overview, we refer the reader to the monographs by H¨ardle(1990) and Fan and Gijbels (1996). On the other hand, there are numerous applications that the covariates do not possess a density with respect to the Lebesgue measure; a case in point is various count time series models which have been employed for the analysis of financial data (e.g. modeling the number of transactions) or biomedical data (e.g. modeling infectious diseases); see Fokianos et al. (2009) for instance. The aim of this paper is therefore to provide a nonparametric estimation procedure for the regression function under less restrictive conditions on the underlying distribution such that various types of covariates can be included. Instead, we impose a shape constraint and assume f to be isotonic but not necessarily additive. This general framework also allows inclusion of a trend component. Such a covariate accommodates the case of gradual changes over time in contrast to change-point models with stationarity between these points of (abrupt) changes. Two examples below will show the usefulness of this approach and its applicability. Example 1.1 ((Non-)linear ARCH models). Since the seminal paper by Engle (1982), (non-)linear ARCH models and various generalizations are used in financial mathematics e.g. for volatility modeling. The nonlinear ARCH(p) model is given by 2 Yt σt "t; with σt f Yt 1;:::;Yt p ; t Z; − − 2 where "t t Z is a sequence of i.i.d. random variables with E"t 1. Hence, with ηt 2 2 = = ( ) ∈ σt "t 1 ; t , the squared process can be represented in form of (1.1), ∈ Z ( ) 2 = = ( − ) ∈ Yt f It ηt with It Yt 1;:::;Yt p : ′ In particular, the linear ARCH(= ( p))+ model with nonnegative= ( − coefficients− ) satisfies the mono- tonicity constraint on f and falls within the framework of (1.2). Example 1.2 (Count time series with exogenous variables). A classical model for integer- valued time series is that of that of Poisson autoregression, that is, observations Y1;:::;Yn are available with (see Ferland et al. (2006), for instance) Yt σ Yt 1;Yt 2;::: Poisson λt ; S ( − − ) ∼ ( ) 2 where λt f Yt 1;:::;Yt p; λt 1; : : : ; λt q (1.2) is the unobserved process of intensities. Special cases of the above specification are the = ( − − − − ) INGARCH(1,1) model when f Yt 1; λt 1 θ0 θ1Yt 1 θ2λt 1 and the INARCH(p) model with f Yt 1;:::;Yt p θ0 θ1Yt 1 θpYt p. For nonnegative parameters θi, the mean function f is monotonically non-decreasing( − − ) = in+ each− argument.+ − In this contribution, we − − − − will be( working with) ARCH= + type of+ ⋯ models + though. The inclusion of the λt terms poses challenging issues since it is a hidden process. The same comment applies to the GARCH model (Example 1.1); work in the context of nonparametric estimation of GARCH models has been reported by Audrino and B¨uhlmann(2009) and Meister and Kreiss (2016). Recalling model (1.2) but with past λt's omitted, consider the inclusion of a trend component and/or a covariate vector as follows (see, for example, Davis et al. (2000)) λt f Yt 1; t n; Zt : (1.3) This model can be represented as = ( − ~ ) Yt f It "t with It Yt 1; t n; Zt and for a certain choice of the white noise sequence " , falls within′ the framework of = ( ) + = ( t− t ~ ) (1.1). ( ) In model (1.3), the explanatory variables Yt 1; t n; Zt do not have a density w.r.t. Le- besgue measure. Moreover, in model (1.2) the nature of the′ stationary distribution of λt is sometimes unclear, it can be discrete or continuous( − ~ or even) a mixture thereof. Hence, ap- plication of standard nonparametric methods such as kernel estimators of the function f, as proposed e.g. by Dette et al. (2006) and references therein, does not alleviate the aforemen- tioned problems. Furthermore, with dependent errors, a data-driven choice of smoothing parameters, such as a bandwidth, is a challenging issue. While the simple leave-one-out cross-validation may fail, the method of leave-k-out cross-validation involves a choice of k, which in turn requires a difficult subjective decision; see e.g. Chu and Marron (1991). On the other hand, the isotonic least squares estimator does not require the choice of any smoothing parameter since an appropriate tuning of the degree of smoothing is done auto- matically. This estimator seems to be less sensitive to irregularities in the design and if the target function is indeed isotonic then this estimator is consistent; see e.g. Christopeit and Tosstorff (1987) and references therein. The assumption of isotonicity seems to be often appropriate in the context of models (1.2) and (1.3) and, in fact, some popular parametric models share this property. We propose a new modification of the isotonic least squares estimator and show that it attains rates of convergence that are known to be optimal in comparable settings. In sharp contrast to usual nonparametric estimators, this estimator does not require the choice of an appropriate bandwidth which could cause problems in our general setting with a possibly irregular distribution of the explanatory variables and with dependent observations. The paper is structured as follows. We introduce our estimators and present results on its rate of convergence in Section 2. Numerical examples are discussed in Section 3. All proofs as well as technical auxiliary results are deferred to Section 4. 2. The estimator and its asymptotic properties In the sequel x denotes the transpose of a vector x. We assume that Yn;1;In;1 ;:::; d Yn;n;In;n are observed,′ where Yn;t is a real-valued and In;t an R -valued random′ variable′ ′ ′ ( ) ( ) 3 such that Yn;t f In;t "n;t; t 1; : : : ; n; with E "n;t In;t 0 almost surely. We assume that the conditional mean function f is isotonic, that is, monotonously= non-decreasing( ) + in each= argument. A popular estimator is the isotonic( S least) squares= estimator fn which is given as n ̃ 2 fn arg min Yn;t g In;t : g isotonic t 1 ̃ = Q ( − ( )) It is well known that fn satisfies at all observation= points x In;1;:::;In;n the following equations: ̃ ∈ { } fn x max min AvY L U (2.1) U x U L x L ̃ min max AvY L U ; (2.2) ( ) = L∶ x∈L U ∶ x∈U ( ∩ ) where = ∶ ∈ ∶ ∈ ( ∩ ) n 1 t 1 Yn;t In;t B d AvY B ;B R ; # t n In;t B ∑ = ( ∈ ) and U and L denote upper and( ) lower= sets, respectively; see e.g.⊆ Theorem 1 in Brunk (1955) { ≤ ∶ ∈ } d and Theorem 1.4.4 in Robertson et al. (1988, p. 23). (A set U R is called an upper set d if x U and x y implies that y U; analogously, L R is called a lower set if x L and x y implies that y L. x y and x y means that xi ⊆yi or xi yi, respectively, for all∈ i 1; : : : ;⪯ d.) While fn is uniquely∈ defined at the⊆ observation points, there is some∈ arbitrariness⪰ of choosing f∈n between⪯ these points;⪰ only the postulated≤ isotonicity≥ has to be satisfied.= ̃ There are already several̃ asymptotic results for the classical isotonic least squares esti- mator fn in dimension d 1, mostly derived in the case of deterministic regressors.