DOCSLIB.ORG

Some Basic Properties of Cross-Correlation Functions of N-Dimensional Vector Time Series Anthony E

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Some Basic Properties of Cross-Correlation Functions of N-Dimensional Vector Time Series Anthony E Journal of Statistical and Econometric Methods, vol.4, no.1, 2015, 63-71 ISSN: 2241-0384 (print), 2241-0376 (online) Scienpress Ltd, 2015 Some basic properties of cross-correlation functions of n-dimensional vector time series Anthony E. Usoro1 Abstract In this work, cross-correlation function of multivariate time series was the interest. The design of cross-correlation function at different lags was presented. is the matrix of the cross-covariance functions, and are the variances훾푋푖푡+푘푋푗푡 of+푙 and vectors respectively. Vector cross-correlation훾푋푖푡 훾function푋푗푡 was derived as 푖푡 푗푡 푋 , 푋 = . A statistical package was used to verify the vector cross 훾푋푖푡+푘푋푗푡+푙 푋푖푡+푘 푋푗푡+푙 휌 훾푋푖푡훾푋푗푡 correlation functions,� with trivariate analysis as a special case. From the results, some properties of vector cross-correlation functions were established. Keywords: Vector time series; cross-covariance function and cross-correlation function 1 Department of Mathematics and Statistics, Akwa Ibom State University, Mkpat Enin, Akwa Ibom State. Email: [email protected] Article Info: Received : January 17, 2015. Revised : February 28, 2015. Published online : March 31, 2015. 64 Cross-correlation functions of n-dimensional vector time series 1 Introduction In statistics, the term cross-covariance is sometimes used to refer to the covariance corr(X,Y) between two random vectors and , (where = , , … , = , , … , ). In signal processing,푋 the cross푌 -covariance푋 is 푋often1 푋2 called푋푛 푎푛푑cross푌-correlation푌1 푌2 and푌푛 is a measure of similarity of two signals, commonly used to find features in an unknown signal by comparing it to a known one. It is a function of the relative time between the signals, and it is sometimes called the sliding dot product. In univariate time series, the autocorrelation of a random process describes the correlation between values of the process at different points in time, as a function of the two times or of the time difference. Let be some repeatable process, and be some point in time after the start of that 푋process. ( may be an integer for a discrete푖 -time process or a real number for a continuous-time푖 process.) Then is the value (or realization) produced by a given run of the process at time . Suppose푋푖 that the process is further known to have defined values for mean 푖and variance for all times . Then the definition of 2 the autocorrelation between휇푖 times and 휎is푖 푖 푠 {(푡 )( )] ( , ) = , 퐸 푋푡−휇푡 푋푠−휇푠 휌 푠 푡 휎푡휎푠 where “E” is the expected value operator. It is required to note that the above expression is not well-defined for all time series or processes, because the variance may be zero. If the function is well-defined, its value must lie in the range [-1,1], with 1 indicating perfect 휌correlation and -1 indicating perfect anti- correlation. If is a second-order stationary process then the mean and the variance are푋 time-independent, and further the autocorrelation depends휇 only on 2 the difference휎 between and : the correlation depends only on the time-distance between the pair of values푡 but푠 not on their position in time. This further implies that the autocorrelation can be expressed as a function of the time-lag, and that Anthony E. Usoro 65 this would be an even function of the lag = , which implies = + . This gives the more familiar form, 푘 푠 − 푡 푠 푡 푘 [( )( )] [( )( )] = = [( ) ] [( ) ] 퐸 푋푡−휇 푋푡+푘−휇 퐸 푋푡−휇 푋푡+푘−휇 2 2 2 휌푘 √퐸 푋푡−휇 퐸 푋푡+푘−휇 휎 where and are time series process at lag k time difference. Hence, autocovariance푋푡 푋 coefficient푡+푘 at lag k, measures the covariance between two values and , a distance훾푘 k apart. The autocorrelation coefficient is defined as the 푍autocovariance푡 푍푡+푘 at lag k divided by variance ( ). The휌 푘plot of against lag k is called훾 푘the autocovariance function ( )훾,0 while푘=0 the plot of 훾푘 against lag k is called the autocorrelation function (Box 훾and푘 Jenkins 1976). 휌푘 In multivariate time series, cross-correlation or covariance involves more than one process. For instance, and are two processes of which could be cross-correlated with at lag k.푋 The푡 lag푌푡 k value return by ( , ) estimates푋푡 the ( ) correlation between 푌푡 + ( ), Venables and Ripley푐푐푓 (2002).푋 푌 Storch and Zwiers (2001) described푋 푡 cross푘 푎푛푑-correlation푌 푡 in signal processing and time series. In signal processing, cross-correlation is a measure of similarity of two waveforms as a function of a time lag applied to one of them. This is also known as a sliding dot product or sliding inner-product. It is commonly used for searching a -duration signal for a shorter known feature. It also has application in pattern recognition, signal particle analysis, electron tomographic averaging, cryptanalysis and neurophysiology. In autocorrelation, which is the cross-correlation of a signal with itself, there is always a peak at a lag of zero unless the signal is a trivial zero signal. In probability theory and Statistics, correlation is always used to include a standardising factor in such a way that correlations have values between -1 and 1. Let ( , ) represent a pair of stochastic process that are jointly wide sense stationary.푋푡 푌푡 Then the cross covariance given by Box et al (1984) is ( ) = [( ) ], 훾푥푦 휏 퐸 푋푡 − 휇푥 �푌푡+휏 − 휇푦� 66 Cross-correlation functions of n-dimensional vector time series where and are the means of and respectively. The cross-correlation function휇 푥 is휇 the푦 normalized cross-푋covariance푡 푌푡 function. Therefore, 휌푥푦 ( ) ( ) = 훾푥푦 휏 휌푥푦 휏 where and are the standard deviation 휎of푥휎 processes푦 and respectively. If = 휎for푥 all 휎t,푦 then the cross-correlation function is simply푋푡 the푌푡 autocorrelation 푋function푡 푌푡 for a discrete process of length n defined as { , … , } which known mean and variance, an estimate of the autocorrelation may푋 be1 obtained푋푛 as 1 = 푛−푘( )( ) ( ) ( ) 푅� 푘 2 � 푋푡 − 휇 푋푡+푘 − 휇 for any positive integer k<n,푛 Patrick− 푘 휎 (2005)푡=1 . When the true mean and variance are known, the estimate is unbiased. If the true mean, this estimate휇 is unbiased. 2 휎If the true mean and variance of the process are not known, there are several probabilities: i. if and are replaced by the standard formulas for sample mean and sample 2 variance,휇 then휎 this is a biased estimate. ii. if n-k in the above formula is replaced with n, the estimate is biased. However, it usually has a smaller mean square error, Priestly (1982) and Donald and Walden (1993). iii. if is stationary process, then the following are true 푡 푋 = = , for all t,s and ( , ) = ( ) = ( ), 푡 푠 푥푥 푡 푠 푥푥 푠−푡 푥푥 푇 where T=s-t,휇 is the휇 lag휇 time or the moment퐶 of time 퐶by which the퐶 signal has been shifted. As a result, the autocovariance becomes ( ) = [ ( ) ( ) ] = ( ) ( ) = ( ) , 2 2 푥푥 푡 푡+푇 푡 푡+푇 푥푥 푇 where퐶 푇 represents퐸 �푋 the− 휇autocorrelation��푋 − 휇� in the퐸� 푋signal푋 processi� − 휇ng sense.푅 − 휇 ( ) ( )푅=푥푥 , Hoel (1984). 퐶푥푥 푇 2 푅푥푥 푇 휎 Anthony E. Usoro 67 For and , the following properties hold: 푡 푡 1. 푋( ) 1푌 푥푦 ℎ 2. 휌 ( )≤= ( ) 푥푦 ℎ 푥푦 −ℎ 3. 휌 ( ) 1휌 푥푦 0 휌 ≠ ( ) 4. ( ) = 훾(푥푦) ℎ ( ) 휌푥푦 ℎ √훾푥 0 훾푦 0 Mardia and Goodall (1993) defined separable cross-correlation function as ( , ) = ( , ) , 푖푗 1 2 1 2 푖푗 where = [ ] is a ×퐶 푋positive푋 definite휌 푋 푋 matrix푎 and (. , . ) is a valid correlation퐴 function.푎푖푗 Goulard푝 푝 & Voltz (1992); Wackernage (2003);휌 Ver Hoef and Barry (1998) implied that the cross- covariance function is ( ) = ( ) , 푟 푖푗 1 2 푘=1 푘 1 2 푖푘 푗푘 for an integer 1 퐶 ,푋 where− 푋 =∑[ ] 휌is a 푋 ×− 푋 full푎 rank푎 matrix and (.) are valid stationary ≤correlation푟 ≤ 푝 functions.퐴 푎Apanasovich푖푗 푝 푟 and Genton (2010) constructed휌푘 valid parametric cross-covariance functions. Apanasovich and Genton proposed a simple methodology based on latent dimensions and existing covariance models for univariate covariance, to develop flexible, interpretable and computationally feasible classes of cross-covariance functions in closed forms. They discussed estimation of the models and performed a small simulation study to demonstrate the models. The interest in this work is to extend cross-correlation functions beyond a-two variable case, present the multivariate design of vector cross- covariance and correlation functions and therefore establish some basic properties of vector cross-correlation functions from the analysis of vector cross-correlation functions. 68 Cross-correlation functions of n-dimensional vector time series 2 The design of cross-covariance cross-correlation functions The matrix of cross-covariance functions is as shown below: ( ), ( ) ( ), ( ) ( ), ( ) . ( ), ( ) 푋 1푡+푘 푋 1푡+푙 푋 1푡+푘 푋 2푡+푙 푋 1푡+푘 푋 3푡+푙 푋 1푡+푘 푋 푛푡+푙 ( 훾 ), ( ) ( 훾 ), ( ) ( 훾 ), ( ) . ( 훾 ), ( ) 푋 2푡+푘 푋 1푡+푙 푋 2푡+푘 푋 2푡+푙 푋 2푡+푘 푋 3푡+푙 푋 2푡+푘 푋 푛푡+푙 훾 ( ), ( ) 훾 ( ), ( ) 훾 ( ), ( ) . 훾 ( ), ( ) 푋 3푡+푘 푋 1푡+푙 푋 3푡+푘 푋 2푡+푙 푋 3푡+푘 푋 3푡+푙 푋 3푡+푘 푋 푛푡+푙 훾 . 훾 훾 훾 . ( ), ( ) ( ), ( ) ( ), ( ) . ( ), ( ) 푋 푚푡+푘 푋 1푡+푙 푋 푚푡+푘 푋 2푡+푙 푋 푚푡+푘 푋 3푡+푙 푋 푚푡+푘 푋 푛푡+푙 where 훾= 0, … , , = 0훾, … , . 훾 훾 푘The above푎 matrix푙 is a 푏square matrix, and could be reduced to the form, ( ), ( ) 푋 푖푡+푘 푋 푗푡+푙 where = 1, … , , = 1, … , , 훾= 0, … , , = 0, … , , ( = ). From the푖 above cross푚 푗 -covariance푛 푘 matrix, 푎 푙 푏 푛 푚 , = , 훾푋푖푡+푘푋푗푡+푙 푋푖푡+푘 푋푗푡+푙 휌 �훾푋푖푡훾푋푗푡 where, is the matrix of the cross-covariance functions, and are the 훾variances푋푖푡+푘푋푗푡+푙 of and vectors respectively.
Load more

Recommended publications
  • Mathematical Statistics
    Mathematical Statistics MAS 713 Chapter 1.2 Previous subchapter 1 What is statistics ? 2 The statistical process 3 Population and samples 4 Random sampling Questions? Mathematical Statistics (MAS713) Ariel Neufeld 2 / 70 This subchapter Descriptive Statistics 1.2.1 Introduction 1.2.2 Types of variables 1.2.3 Graphical representations 1.2.4 Descriptive measures Mathematical Statistics (MAS713) Ariel Neufeld 3 / 70 1.2 Descriptive Statistics 1.2.1 Introduction Introduction As we said, statistics is the art of learning from data However, statistical data, obtained from surveys, experiments, or any series of measurements, are often so numerous that they are virtually useless unless they are condensed ; data should be presented in ways that facilitate their interpretation and subsequent analysis Naturally enough, the aspect of statistics which deals with organising, describing and summarising data is called descriptive statistics Mathematical Statistics (MAS713) Ariel Neufeld 4 / 70 1.2 Descriptive Statistics 1.2.2 Types of variables Types of variables Mathematical Statistics (MAS713) Ariel Neufeld 5 / 70 1.2 Descriptive Statistics 1.2.2 Types of variables Types of variables The range of available descriptive tools for a given data set depends on the type of the considered variable There are essentially two types of variables : 1 categorical (or qualitative) variables : take a value that is one of several possible categories (no numerical meaning) Ex. : gender, hair color, field of study, political affiliation, status, ... 2 numerical (or quantitative)
    [Show full text]
  • Autocovariance Function Estimation Via Penalized Regression
    Autocovariance Function Estimation via Penalized Regression Lina Liao, Cheolwoo Park Department of Statistics, University of Georgia, Athens, GA 30602, USA Jan Hannig Department of Statistics and Operations Research, University of North Carolina, Chapel Hill, NC 27599, USA Kee-Hoon Kang Department of Statistics, Hankuk University of Foreign Studies, Yongin, 449-791, Korea Abstract The work revisits the autocovariance function estimation, a fundamental problem in statistical inference for time series. We convert the function estimation problem into constrained penalized regression with a generalized penalty that provides us with flexible and accurate estimation, and study the asymptotic properties of the proposed estimator. In case of a nonzero mean time series, we apply a penalized regression technique to a differenced time series, which does not require a separate detrending procedure. In penalized regression, selection of tuning parameters is critical and we propose four different data-driven criteria to determine them. A simulation study shows effectiveness of the tuning parameter selection and that the proposed approach is superior to three existing methods. We also briefly discuss the extension of the proposed approach to interval-valued time series. Some key words: Autocovariance function; Differenced time series; Regularization; Time series; Tuning parameter selection. 1 1 Introduction Let fYt; t 2 T g be a stochastic process such that V ar(Yt) < 1, for all t 2 T . The autoco- variance function of fYtg is given as γ(s; t) = Cov(Ys;Yt) for all s; t 2 T . In this work, we consider a regularly sampled time series f(i; Yi); i = 1; ··· ; ng. Its model can be written as Yi = g(i) + i; i = 1; : : : ; n; (1.1) where g is a smooth deterministic function and the error is assumed to be a weakly stationary 2 process with E(i) = 0, V ar(i) = σ and Cov(i; j) = γ(ji − jj) for all i; j = 1; : : : ; n: The estimation of the autocovariance function γ (or autocorrelation function) is crucial to determine the degree of serial correlation in a time series.
    [Show full text]
  • LECTURES 2 - 3 : Stochastic Processes, Autocorrelation Function
    LECTURES 2 - 3 : Stochastic Processes, Autocorrelation function. Stationarity. Important points of Lecture 1: A time series fXtg is a series of observations taken sequentially over time: xt is an observation recorded at a specific time t. Characteristics of times series data: observations are dependent, become available at equally spaced time points and are time-ordered. This is a discrete time series. The purposes of time series analysis are to model and to predict or forecast future values of a series based on the history of that series. 2.2 Some descriptive techniques. (Based on [BD] x1.3 and x1.4) ......................................................................................... Take a step backwards: how do we describe a r.v. or a random vector? ² for a r.v. X: 2 d.f. FX (x) := P (X · x), mean ¹ = EX and variance σ = V ar(X). ² for a r.vector (X1;X2): joint d.f. FX1;X2 (x1; x2) := P (X1 · x1;X2 · x2), marginal d.f.FX1 (x1) := P (X1 · x1) ´ FX1;X2 (x1; 1) 2 2 mean vector (¹1; ¹2) = (EX1; EX2), variances σ1 = V ar(X1); σ2 = V ar(X2), and covariance Cov(X1;X2) = E(X1 ¡ ¹1)(X2 ¡ ¹2) ´ E(X1X2) ¡ ¹1¹2. Often we use correlation = normalized covariance: Cor(X1;X2) = Cov(X1;X2)=fσ1σ2g ......................................................................................... To describe a process X1;X2;::: we define (i) Def. Distribution function: (fi-di) d.f. Ft1:::tn (x1; : : : ; xn) = P (Xt1 · x1;:::;Xtn · xn); i.e. this is the joint d.f. for the vector (Xt1 ;:::;Xtn ). (ii) First- and Second-order moments. ² Mean: ¹X (t) = EXt 2 2 2 2 ² Variance: σX (t) = E(Xt ¡ ¹X (t)) ´ EXt ¡ ¹X (t) 1 ² Autocovariance function: γX (t; s) = Cov(Xt;Xs) = E[(Xt ¡ ¹X (t))(Xs ¡ ¹X (s))] ´ E(XtXs) ¡ ¹X (t)¹X (s) (Note: this is an infinite matrix).
    [Show full text]
  • Ph 21.5: Covariance and Principal Component Analysis (PCA)
    Ph 21.5: Covariance and Principal Component Analysis (PCA) -v20150527- Introduction Suppose we make a measurement for which each data sample consists of two measured quantities. A simple example would be temperature (T ) and pressure (P ) taken at time (t) at constant volume(V ). The data set is Ti;Pi ti N , which represents a set of N measurements. We wish to make sense of the data and determine thef dependencej g of, say, P on T . Suppose P and T were for some reason independent of each other; then the two variables would be uncorrelated. (Of course we are well aware that P and V are correlated and we know the ideal gas law: PV = nRT ). How might we infer the correlation from the data? The tools for quantifying correlations between random variables is the covariance. For two real-valued random variables (X; Y ), the covariance is defined as (under certain rather non-restrictive assumptions): Cov(X; Y ) σ2 (X X )(Y Y ) ≡ XY ≡ h − h i − h i i where ::: denotes the expectation (average) value of the quantity in brackets. For the case of P and T , we haveh i Cov(P; T ) = (P P )(T T ) h − h i − h i i = P T P T h × i − h i × h i N−1 ! N−1 ! N−1 ! 1 X 1 X 1 X = PiTi Pi Ti N − N N i=0 i=0 i=0 The extension of this to real-valued random vectors (X;~ Y~ ) is straighforward: D E Cov(X;~ Y~ ) σ2 (X~ < X~ >)(Y~ < Y~ >)T ≡ X~ Y~ ≡ − − This is a matrix, resulting from the product of a one vector and the transpose of another vector, where X~ T denotes the transpose of X~ .
    [Show full text]
  • 6 Probability Density Functions (Pdfs)
    CSC 411 / CSC D11 / CSC C11 Probability Density Functions (PDFs) 6 Probability Density Functions (PDFs) In many cases, we wish to handle data that can be represented as a real-valued random variable, T or a real-valued vector x =[x1,x2,...,xn] . Most of the intuitions from discrete variables transfer directly to the continuous case, although there are some subtleties. We describe the probabilities of a real-valued scalar variable x with a Probability Density Function (PDF), written p(x). Any real-valued function p(x) that satisfies: p(x) 0 for all x (1) ∞ ≥ p(x)dx = 1 (2) Z−∞ is a valid PDF. I will use the convention of upper-case P for discrete probabilities, and lower-case p for PDFs. With the PDF we can specify the probability that the random variable x falls within a given range: x1 P (x0 x x1)= p(x)dx (3) ≤ ≤ Zx0 This can be visualized by plotting the curve p(x). Then, to determine the probability that x falls within a range, we compute the area under the curve for that range. The PDF can be thought of as the infinite limit of a discrete distribution, i.e., a discrete dis- tribution with an infinite number of possible outcomes. Specifically, suppose we create a discrete distribution with N possible outcomes, each corresponding to a range on the real number line. Then, suppose we increase N towards infinity, so that each outcome shrinks to a single real num- ber; a PDF is defined as the limiting case of this discrete distribution.
    [Show full text]
  • This History of Modern Mathematical Statistics Retraces Their Development
    BOOK REVIEWS GORROOCHURN Prakash, 2016, Classic Topics on the History of Modern Mathematical Statistics: From Laplace to More Recent Times, Hoboken, NJ, John Wiley & Sons, Inc., 754 p. This history of modern mathematical statistics retraces their development from the “Laplacean revolution,” as the author so rightly calls it (though the beginnings are to be found in Bayes’ 1763 essay(1)), through the mid-twentieth century and Fisher’s major contribution. Up to the nineteenth century the book covers the same ground as Stigler’s history of statistics(2), though with notable differences (see infra). It then discusses developments through the first half of the twentieth century: Fisher’s synthesis but also the renewal of Bayesian methods, which implied a return to Laplace. Part I offers an in-depth, chronological account of Laplace’s approach to probability, with all the mathematical detail and deductions he drew from it. It begins with his first innovative articles and concludes with his philosophical synthesis showing that all fields of human knowledge are connected to the theory of probabilities. Here Gorrouchurn raises a problem that Stigler does not, that of induction (pp. 102-113), a notion that gives us a better understanding of probability according to Laplace. The term induction has two meanings, the first put forward by Bacon(3) in 1620, the second by Hume(4) in 1748. Gorroochurn discusses only the second. For Bacon, induction meant discovering the principles of a system by studying its properties through observation and experimentation. For Hume, induction was mere enumeration and could not lead to certainty. Laplace followed Bacon: “The surest method which can guide us in the search for truth, consists in rising by induction from phenomena to laws and from laws to forces”(5).
    [Show full text]
  • Covariance of Cross-Correlations: Towards Efficient Measures for Large-Scale Structure
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by RERO DOC Digital Library Mon. Not. R. Astron. Soc. 400, 851–865 (2009) doi:10.1111/j.1365-2966.2009.15490.x Covariance of cross-correlations: towards efficient measures for large-scale structure Robert E. Smith Institute for Theoretical Physics, University of Zurich, Zurich CH 8037, Switzerland Accepted 2009 August 4. Received 2009 July 17; in original form 2009 June 13 ABSTRACT We study the covariance of the cross-power spectrum of different tracers for the large-scale structure. We develop the counts-in-cells framework for the multitracer approach, and use this to derive expressions for the full non-Gaussian covariance matrix. We show that for the usual autopower statistic, besides the off-diagonal covariance generated through gravitational mode- coupling, the discreteness of the tracers and their associated sampling distribution can generate strong off-diagonal covariance, and that this becomes the dominant source of covariance as spatial frequencies become larger than the fundamental mode of the survey volume. On comparison with the derived expressions for the cross-power covariance, we show that the off-diagonal terms can be suppressed, if one cross-correlates a high tracer-density sample with a low one. Taking the effective estimator efficiency to be proportional to the signal-to-noise ratio (S/N), we show that, to probe clustering as a function of physical properties of the sample, i.e. cluster mass or galaxy luminosity, the cross-power approach can outperform the autopower one by factors of a few.
    [Show full text]
  • Applied Time Series Analysis
    Applied Time Series Analysis SS 2018 February 12, 2018 Dr. Marcel Dettling Institute for Data Analysis and Process Design Zurich University of Applied Sciences CH-8401 Winterthur Table of Contents 1 INTRODUCTION 1 1.1 PURPOSE 1 1.2 EXAMPLES 2 1.3 GOALS IN TIME SERIES ANALYSIS 8 2 MATHEMATICAL CONCEPTS 11 2.1 DEFINITION OF A TIME SERIES 11 2.2 STATIONARITY 11 2.3 TESTING STATIONARITY 13 3 TIME SERIES IN R 15 3.1 TIME SERIES CLASSES 15 3.2 DATES AND TIMES IN R 17 3.3 DATA IMPORT 21 4 DESCRIPTIVE ANALYSIS 23 4.1 VISUALIZATION 23 4.2 TRANSFORMATIONS 26 4.3 DECOMPOSITION 29 4.4 AUTOCORRELATION 50 4.5 PARTIAL AUTOCORRELATION 66 5 STATIONARY TIME SERIES MODELS 69 5.1 WHITE NOISE 69 5.2 ESTIMATING THE CONDITIONAL MEAN 70 5.3 AUTOREGRESSIVE MODELS 71 5.4 MOVING AVERAGE MODELS 85 5.5 ARMA(P,Q) MODELS 93 6 SARIMA AND GARCH MODELS 99 6.1 ARIMA MODELS 99 6.2 SARIMA MODELS 105 6.3 ARCH/GARCH MODELS 109 7 TIME SERIES REGRESSION 113 7.1 WHAT IS THE PROBLEM? 113 7.2 FINDING CORRELATED ERRORS 117 7.3 COCHRANE‐ORCUTT METHOD 124 7.4 GENERALIZED LEAST SQUARES 125 7.5 MISSING PREDICTOR VARIABLES 131 8 FORECASTING 137 8.1 STATIONARY TIME SERIES 138 8.2 SERIES WITH TREND AND SEASON 145 8.3 EXPONENTIAL SMOOTHING 152 9 MULTIVARIATE TIME SERIES ANALYSIS 161 9.1 PRACTICAL EXAMPLE 161 9.2 CROSS CORRELATION 165 9.3 PREWHITENING 168 9.4 TRANSFER FUNCTION MODELS 170 10 SPECTRAL ANALYSIS 175 10.1 DECOMPOSING IN THE FREQUENCY DOMAIN 175 10.2 THE SPECTRUM 179 10.3 REAL WORLD EXAMPLE 186 11 STATE SPACE MODELS 187 11.1 STATE SPACE FORMULATION 187 11.2 AR PROCESSES WITH MEASUREMENT NOISE 188 11.3 DYNAMIC LINEAR MODELS 191 ATSA 1 Introduction 1 Introduction 1.1 Purpose Time series data, i.e.
    [Show full text]
  • Lecture 4 Multivariate Normal Distribution and Multivariate CLT
    Lecture 4 Multivariate normal distribution and multivariate CLT. T We start with several simple observations. If X = (x1; : : : ; xk) is a k 1 random vector then its expectation is × T EX = (Ex1; : : : ; Exk) and its covariance matrix is Cov(X) = E(X EX)(X EX)T : − − Notice that a covariance matrix is always symmetric Cov(X)T = Cov(X) and nonnegative definite, i.e. for any k 1 vector a, × a T Cov(X)a = Ea T (X EX)(X EX)T a T = E a T (X EX) 2 0: − − j − j � We will often use that for any vector X its squared length can be written as X 2 = XT X: If we multiply a random k 1 vector X by a n k matrix A then the covariancej j of Y = AX is a n n matrix × × × Cov(Y ) = EA(X EX)(X EX)T AT = ACov(X)AT : − − T Multivariate normal distribution. Let us consider a k 1 vector g = (g1; : : : ; gk) of i.i.d. standard normal random variables. The covariance of g is,× obviously, a k k identity × matrix, Cov(g) = I: Given a n k matrix A, the covariance of Ag is a n n matrix × × � := Cov(Ag) = AIAT = AAT : Definition. The distribution of a vector Ag is called a (multivariate) normal distribution with covariance � and is denoted N(0; �): One can also shift this disrtibution, the distribution of Ag + a is called a normal distri­ bution with mean a and covariance � and is denoted N(a; �): There is one potential problem 23 with the above definition - we assume that the distribution depends only on covariance ma­ trix � and does not depend on the construction, i.e.
    [Show full text]
  • The Variance Ellipse
    The Variance Ellipse ✧ 1 / 28 The Variance Ellipse For bivariate data, like velocity, the variabililty can be spread out in not one but two dimensions. In this case, the variance is now a matrix, and the spread of the data is characterized by an ellipse. This variance ellipse eccentricity indicates the extent to which the variability is anisotropic or directional, and the orientation tells the direction in which the variability is concentrated. ✧ 2 / 28 Variance Ellipse Example Variance ellipses are a very useful way to analyze velocity data. This example compares velocities observed by a mooring array in Fram Strait with velocities in two numerical models. From Hattermann et al. (2016), “Eddy­driven recirculation of Atlantic Water in Fram Strait”, Geophysical Research Letters. Variance ellipses can be powerfully combined with lowpassing and bandpassing to reveal the geometric structure of variability in different frequency bands. ✧ 3 / 28 Understanding Ellipses This section will focus on understanding the properties of the variance ellipse. To do this, it is not really possible to avoid matrix algebra. Therefore we will first review some relevant mathematical background. ✧ 4 / 28 Review: Rotations T The most important action on a vector z ≡ [u v] is a ninety-degree rotation. This is carried out through the matrix multiplication 0 −1 0 −1 u −v z = = . [ 1 0 ] [ 1 0 ] [ v ] [ u ] Note the mathematically positive direction is counterclockwise. A general rotation is carried out by the rotation matrix cos θ − sin θ J(θ) ≡ [ sin θ cos θ ] cos θ − sin θ u u cos θ − v sin θ J(θ) z = = .
    [Show full text]
  • Mathematical Statistics the Sample Distribution of the Median
    Course Notes for Math 162: Mathematical Statistics The Sample Distribution of the Median Adam Merberg and Steven J. Miller February 15, 2008 Abstract We begin by introducing the concept of order statistics and ¯nding the density of the rth order statistic of a sample. We then consider the special case of the density of the median and provide some examples. We conclude with some appendices that describe some of the techniques and background used. Contents 1 Order Statistics 1 2 The Sample Distribution of the Median 2 3 Examples and Exercises 4 A The Multinomial Distribution 5 B Big-Oh Notation 6 C Proof That With High Probability jX~ ¡ ¹~j is Small 6 D Stirling's Approximation Formula for n! 7 E Review of the exponential function 7 1 Order Statistics Suppose that the random variables X1;X2;:::;Xn constitute a sample of size n from an in¯nite population with continuous density. Often it will be useful to reorder these random variables from smallest to largest. In reordering the variables, we will also rename them so that Y1 is a random variable whose value is the smallest of the Xi, Y2 is the next smallest, and th so on, with Yn the largest of the Xi. Yr is called the r order statistic of the sample. In considering order statistics, it is naturally convenient to know their probability density. We derive an expression for the distribution of the rth order statistic as in [MM]. Theorem 1.1. For a random sample of size n from an in¯nite population having values x and density f(x), the probability th density of the r order statistic Yr is given by ·Z ¸ ·Z ¸ n! yr r¡1 1 n¡r gr(yr) = f(x) dx f(yr) f(x) dx : (1.1) (r ¡ 1)!(n ¡ r)! ¡1 yr Proof.
    [Show full text]
  • Lecture 1: Stationary Time Series∗
    Lecture 1: Stationary Time Series∗ 1 Introduction If a random variable X is indexed to time, usually denoted by t, the observations {Xt, t ∈ T} is called a time series, where T is a time index set (for example, T = Z, the integer set). Time series data are very common in empirical economic studies. Figure 1 plots some frequently used variables. The upper left figure plots the quarterly GDP from 1947 to 2001; the upper right figure plots the the residuals after linear-detrending the logarithm of GDP; the lower left figure plots the monthly S&P 500 index data from 1990 to 2001; and the lower right figure plots the log difference of the monthly S&P. As you could see, these four series display quite different patterns over time. Investigating and modeling these different patterns is an important part of this course. In this course, you will find that many of the techniques (estimation methods, inference proce- dures, etc) you have learned in your general econometrics course are still applicable in time series analysis. However, there are something special of time series data compared to cross sectional data. For example, when working with cross-sectional data, it usually makes sense to assume that the observations are independent from each other, however, time series data are very likely to display some degree of dependence over time. More importantly, for time series data, we could observe only one history of the realizations of this variable. For example, suppose you obtain a series of US weekly stock index data for the last 50 years.
    [Show full text]