CHAPTER11 Operator Methods for Continuous-Time Markov Processes Yacine Aït-Sahalia*, Lars Peter Hansen**, and José A. Scheinkman* *Department of Economics, Princeton University,Princeton, NJ **Department of Economics,The University of Chicago, Chicago, IL Contents 1. Introduction 2 2. Alternative Ways to Model a Continuous-Time Markov Process 3 2.1. Transition Functions 3 2.2. Semigroup of Conditional Expectations 4 2.3. Infinitesimal Generators 5 2.4. Quadratic Forms 7 2.5. Stochastic Differential Equations 8 2.6. Extensions 8 3. Parametrizations of the Stationary Distribution: Calibrating the Long Run 11 3.1. Wong’s Polynomial Models 12 3.2. Stationary Distributions 14 3.3. Fitting the Stationary Distribution 15 3.4. Nonparametric Methods for Inferring Drift or Diffusion Coefficients 18 4. Transition Dynamics and Spectral Decomposition 20 4.1. Quadratic Forms and Implied Generators 21 4.2. Principal Components 24 4.3. Applications 30 5. Hermite and Related Expansions of a Transition Density 36 5.1. Exponential Expansion 36 5.2. Hermite Expansion of the Transition Function 37 5.3. Local Expansions of the Log-Transition Function 40 6. Observable Implications and Tests 45 6.1. Local Characterization 45 6.2. Total Positivity and Testing for Jumps 47 6.3. Principal Component Approach 48 6.4. Testing the Specification of Transitions 49 6.5. Testing Markovianity 52 6.6. Testing Symmetry 53 6.7. Random Time Changes 54 © 2010, Elsevier B.V.All rights reserved. 1 2 Yacine Aït-Sahalia et al. 7. The Properties of Parameter Estimators 55 7.1. Maximum Likelihood Estimation 55 7.2. Estimating the Diffusion Coefficient in the Presence of Jumps 57 7.3. Maximum Likelihood Estimation with Random Sampling Times 58 8. Conclusions 61 Acknowledgments 62 References 62 Abstract This chapter surveys relevant tools, based on operator methods, to describe the evolution in time of continuous-time stochastic process, over different time horizons. Applications include modeling the long-run stationary distribution of the process, modeling the short or intermediate run transi- tion dynamics of the process, estimating parametric models via maximum-likelihood, implications of the spectral decomposition of the generator, and various observable implications and tests of the characteristics of the process. Keywords: Markov process; Infinitesimal Generator; Spectral decomposition; Transition density; Maximum-Likelihood; Stationary density; Long-run. 1. INTRODUCTION Our chapter surveys a set of mathematical and statistical tools that are valuable in understanding and characterizing nonlinear Markov processes. Such processes are used extensively as building blocks in economics and finance. In these literatures, typically the local evolution or short-run transition is specified. We concentrate on the continuous limit in which case it is the instantaneous transition that is specified. In understanding the implications of such a modeling approach we show how to infer the intermediate and long-run properties from the short-run dynamics. To accomplish this, we describe operator methods and their use in conjunction with continuous-time stochastic process models. Operator methods begin with a local characterization of the Markov process dynam- ics. This local specification takes the form of an infinitesimal generator. The infinitesimal generator is itself an operator mapping test functions into other functions. From the infinitesimal generator, we construct a family (semigroup) of conditional expectation operators. The operators exploit the time-invariant Markov structure. Each operator in this family is indexed by the forecast horizon, the interval of time between the infor- mation set used for prediction and the object that is being predicted. Operator methods allow us to ascertain global, and in particular, long-run implications from the local or infinitesimal evolution.These global implications are reflected in (a) the implied station- ary distribution, (b) the analysis of the eigenfunctions of the generator that dominate in the long run, and (c) the construction of likelihood expansions and other estimating equations. Operator Methods for Continuous-Time Markov Processes 3 The methods we describe in this chapter are designed to show how global and long-run implications follow from local characterizations of the time series evolution. This con- nection between local and global properties is particularly challenging for nonlinear time series models. Despite this complexity,the Markov structure makes characterizations of the dynamic evolution tractable. In addition to facilitating the study of a given Markov process, operator methods provide characterizations of the observable implications of potentially rich families of such processes. These methods can be incorporated into sta- tistical estimation and testing. Although many Markov processes used in practice are formally misspecificied, operator methods are useful in exploring the specific nature and consequences of this misspecification. Section 2 describes the underlying mathematical methods and notation. Section 3 studies Markov models through their implied stationary distributions. Section 4 develops some operator methods used to characterize transition dynamics including long-run behavior of Markov process. Section 5 provides approximations to transition densities that are designed to support econometric estimation. Section 6 describes the properties of some parameter estimators. Finally, Section 7 investigates alternative ways to characterize the observable implications of various Markov models, and to test those implications. 2. ALTERNATIVE WAYS TO MODEL A CONTINUOUS-TIME MARKOV PROCESS There are several different but essentially equivalent ways to parameterize continuous time Markov processes, each leading naturally to a distinct estimation strategy. In this section, we briefly describe five possible parametrizations. 2.1. Transition Functions In what follows, (, F, Pr) will denote a probability space, S a locally compact metric space with a countable basis, S a σ-field of Borelians in S, I an interval of the real line, and for each t ∈ I, Xt : (, F, Pr) → (S, S) a measurable function. We will refer to (S, S) as the state space and to X as a stochastic process. Definition 1 P : (S × S) →[0, 1) is a transition probability if, for each x ∈ S, P(x, ·) is a probability measure in S, and for each B ∈ S, P(·, B) is measurable. 2 Definition 2 A transition function is a family Ps,t, (s, t) ∈ I , s < t that satisfies for each s < t < u the Chapman–Kolmogorov equation: Ps,u(x, B) = Pt,u(y, B)Ps,t(x,dy). A transition function is time homogeneous if Ps,t = Ps,t whenever t − s = t − s . In this case we write Pt−s instead of Ps,t. 4 Yacine Aït-Sahalia et al. Definition 3 Let Ft ⊂ F be an increasing family of σ-algebras, and X a stochastic process that is adapted to Ft. X is Markov with transition function Ps,t if for each nonnegative Borel measurable φ : S → R and each (s, t) ∈ I 2, s < t, E[φ(Xt)|Fs]= φ(y)Ps,t(Xs,dy). The following standard result (for example,Revuz and Yor,1991;Chapter 3,Theorem 1.5) allows one to parameterize Markov processes using transition functions. Theorem 1 Given a transition function Ps,t on (S, S) and a probability measure Q0 on (S, S), there exists a unique probability measure Pr on S[0,∞), S[0,∞) , such that the coordinate process X is Markov with respect to σ(Xu, u ≤ t), with transition function Ps,t and the distribution of X0 given by Q0. We will interchangeably call transition function the measure Ps,t or its conditional density p (subject to regularity conditions which guarantee its existence): Ps,t(x,dy) = p(y, t|x, s)dy. In the time homogenous case, we write = t − s and p(y|x, ). In the remainder of this chapter, unless explicitly stated, we will treat only the case of time homogeneity. 2.2. Semigroup of Conditional Expectations Let Pt be a homogeneous transition function and L be a vector space of real-valued func- tions such that for each φ ∈ L, φ(y)Pt(x,dy) ∈ L. For each t define the conditional expectation operator Ttφ(x) = φ(y)Pt(x,dy). (2.1) The Chapman–Kolmogorov equation guarantees that the linear operators Tt satisfy: Tt+s = TtTs. (2.2) This suggests another parameterization for Markov processes. Let (L, ·) be a Banach space. Definition 4 A one-parameter family of linear operators in L, {Tt : t ≥ 0} is called a semigroup if (a) T0 = I and (b) Tt+s = TtTs for all s, t ≥ 0.{Tt : t ≥ 0} is a strongly continuous contraction semigroup if, in addition, (c) limt↓0Ttφ = φ, and (d) ||Tt|| ≤ 1. If a semigroup represents conditional expectations, then it must be positive, that is, if φ ≥ 0 then Ttφ ≥ 0. Operator Methods for Continuous-Time Markov Processes 5 Two useful examples of Banach spaces L to use in this context are as follows: Example 1 Let S be a locally compact and separable state space. Let L = C0 be the space of continuous functions φ : S → R, that vanish at infinity.For φ ∈ C0 define: φ∞ = sup |φ(x)|. x∈S A strongly continuous contraction positive semigroup on C0 is called a Feller semigroup. Example 2 Let Q be a measure on a locally compact subset S of Rm. Let L2(Q) be the space of all Borel measurable functions φ : S → R that are square integrable with respect to the measure Q endowed with the norm: 1 2 2 φ2 = φ dQ . In general, the semigroup of conditional expectations determine the finite- dimensional distributions of the Markov process (see e.g. Ethier and Kurtz, 1986; Proposition 1.6 of Chapter 4.) There are also many results (e.g. Revuz and Yor, 1991; Proposition 2.2 of Chapter 3) concerning whether given a contraction semigroup one can construct a homogeneous transition function such that Eq. (2.1) is satisfied. 2.3. Infinitesimal Generators Definition 5 The infinitesimal generator of a semigroup Tt on a Banach space L is the (possibly unbounded) linear operator A defined by: T φ − φ Aφ = lim t .