Introduction to Time Series Analysis. Lecture 1. Peter Bartlett 1. Organizational issues. 2. Objectives of time series analysis. Examples. 3. Overview of the course. 4. Time series models. 5. Time series modelling: Chasing stationarity. 1 Organizational Issues Peter Bartlett. bartlett@stat. Office hours: Tue 11-12, Thu 10-11 • (Evans 399). Joe Neeman. jneeman@stat. Office hours: Wed 1:30–2:30, Fri 2-3 • (Evans ???). http://www.stat.berkeley.edu/ bartlett/courses/153-fall2010/ • ∼ Check it for announcements, assignments, slides, ... Text: Time Series Analysis and its Applications. With R Examples, • Shumway and Stoffer. 2nd Edition. 2006. 2 Organizational Issues Classroom and Computer Lab Section: Friday 9–11, in 344 Evans. Starting tomorrow, August 27: Sign up for computer accounts. Introduction to R. Assessment: Lab/Homework Assignments (25%): posted on the website. These involve a mix of pen-and-paper and computer exercises. You may use any programming language you choose (R, Splus, Matlab, python). Midterm Exams (30%): scheduled for October 7 and November 9, at the lecture. Project (10%): Analysis of a data set that you choose. Final Exam (35%): scheduled for Friday, December 17. 3 A Time Series 400 350 300 250 200 150 100 50 0 0 1000 2000 3000 4000 5000 6000 7000 4 A Time Series 400 350 300 250 200 150 100 50 0 1960 1965 1970 1975 1980 1985 1990 year 5 A Time Series 400 350 300 250 $ 200 150 100 50 0 1960 1965 1970 1975 1980 1985 1990 year 6 A Time Series SP500: 1960−1990 400 350 300 250 $ 200 150 100 50 0 1960 1965 1970 1975 1980 1985 1990 year 7 A Time Series SP500: Jan−Jun 1987 340 320 300 $ 280 260 240 220 1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5 year 8 A Time Series SP500 Jan−Jun 1987. Histogram 30 25 20 15 10 5 0 240 250 260 270 280 290 300 310 $ 9 A Time Series SP500: Jan−Jun 1987. Permuted. 340 320 300 $ 280 260 240 220 0 20 40 60 80 100 120 10 Objectives of Time Series Analysis 1. Compact description of data. 2. Interpretation. 3. Forecasting. 4. Control. 5. Hypothesis testing. 6. Simulation. 11 Classical decomposition: An example Monthly sales for a souvenir shop at a beach resort town in Queensland. (Makridakis, Wheelwright and Hyndman, 1998) 4 x 10 12 10 8 6 4 2 0 0 10 20 30 40 50 60 70 80 90 12 Transformed data 12 11.5 11 10.5 10 9.5 9 8.5 8 7.5 7 0 10 20 30 40 50 60 70 80 90 13 Trend 12 11.5 11 10.5 10 9.5 9 8.5 8 7.5 7 0 10 20 30 40 50 60 70 80 90 14 Residuals 1.5 1 0.5 0 −0.5 −1 0 10 20 30 40 50 60 70 80 90 15 Trend and seasonal variation 12 11.5 11 10.5 10 9.5 9 8.5 8 7.5 7 0 10 20 30 40 50 60 70 80 90 16 Objectives of Time Series Analysis 1. Compact description of data. Example: Classical decomposition: Xt = Tt + St + Yt. 2. Interpretation. Example: Seasonal adjustment. 3. Forecasting. Example: Predict sales. 4. Control. 5. Hypothesis testing. 6. Simulation. 17 Unemployment data Monthly number of unemployed people in Australia. (Hipel and McLeod, 1994) 5 x 10 8 7.5 7 6.5 6 5.5 5 4.5 4 1983 1984 1985 1986 1987 1988 1989 1990 18 Trend 5 x 10 8 7.5 7 6.5 6 5.5 5 4.5 4 1983 1984 1985 1986 1987 1988 1989 1990 19 Trend plus seasonal variation 5 x 10 8 7.5 7 6.5 6 5.5 5 4.5 4 1983 1984 1985 1986 1987 1988 1989 1990 20 Residuals 4 x 10 8 6 4 2 0 −2 −4 −6 1983 1984 1985 1986 1987 1988 1989 1990 21 Predictions based on a (simulated) variable 5 x 10 8 7.5 7 6.5 6 5.5 5 4.5 4 1983 1984 1985 1986 1987 1988 1989 1990 22 Objectives of Time Series Analysis 1. Compact description of data: Xt = Tt + St + f(Yt)+ Wt. 2. Interpretation. Example: Seasonal adjustment. 3. Forecasting. Example: Predict unemployment. 4. Control. Example: Impact of monetary policy on unemployment. 5. Hypothesis testing. Example: Global warming. 6. Simulation. Example: Estimate probability of catastrophic events. 23 Overview of the Course 1. Time series models 2. Time domain methods 3. Spectral analysis 4. State space models(?) 24 Overview of the Course 1. Time series models (a) Stationarity. (b) Autocorrelation function. (c) Transforming to stationarity. 2. Time domain methods 3. Spectral analysis 4. State space models(?) 25 Overview of the Course 1. Time series models 2. Time domain methods (a) AR/MA/ARMA models. (b) ACF and partial autocorrelation function. (c) Forecasting (d) Parameter estimation (e) ARIMA models/seasonal ARIMA models 3. Spectral analysis 4. State space models(?) 26 Overview of the Course 1. Time series models 2. Time domain methods 3. Spectral analysis (a) Spectral density (b) Periodogram (c) Spectral estimation 4. State space models(?) 27 Overview of the Course 1. Time series models 2. Time domain methods 3. Spectral analysis 4. State space models(?) (a) ARMAX models. (b) Forecasting, Kalman filter. (c) Parameter estimation. 28 Time Series Models A time series model specifies the joint distribution of the se- quence X of random variables. { t} For example: P [X1 x1,...,X x ] for all t and x1,...,x . ≤ t ≤ t t Notation: X1,X2,... is a stochastic process. x1, x2,... is a single realization. We’ll mostly restrict our attention to second-order properties only: EXt, E(Xt1 ,Xt2 ). 29 Time Series Models Example: White noise: X W N(0, σ2). t ∼ i.e., X uncorrelated, EX = 0, VarX = σ2. { t} t t Example: i.i.d. noise: X independent and identically distributed. { t} P [X1 x1,...,X x ]= P [X1 x1] P [X x ]. ≤ t ≤ t ≤ ··· t ≤ t Not interesting for forecasting: P [X x X1,...,X −1]= P [X x ]. t ≤ t| t t ≤ t 30 Gaussian white noise t x 2 1 −x /2 P [Xt xt]=Φ(xt)= e dx. ≤ √2π Z−∞ 2.5 2 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2.5 0 5 10 15 20 25 30 35 40 45 50 31 Gaussian white noise 2.5 2 1.5 1 0.5 0 −0.5 −1 −1.5 −2 −2.5 0 5 10 15 20 25 30 35 40 45 50 32 Time Series Models Example: Binary i.i.d. P [X =1]= P [X = 1] = 1/2. t t − 1 0.8 0.6 0.4 0.2 0 −0.2 −0.4 −0.6 −0.8 −1 0 5 10 15 20 25 30 35 40 45 50 33 Random walk t S = X . Differences: S = S S −1 = X . t i=1 i ∇ t t − t t P 8 6 4 2 0 −2 −4 0 5 10 15 20 25 30 35 40 45 50 34 Random walk ESt? VarSt? 10 5 0 −5 −10 −15 0 5 10 15 20 25 30 35 40 45 50 35 Random Walk Recall S&P500 data. (Notice that it’s smooth) SP500: Jan−Jun 1987 340 320 300 $ 280 260 240 220 1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5 year 36 Random Walk Differences: S = S S −1 = X . ∇ t t − t t SP500, Jan−Jun 1987. first differences 10 8 6 4 2 $ 0 −2 −4 −6 −8 −10 1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5 year 37 Trend and Seasonal Models Xt = Tt + St + Et = β0 + β1t + i (βi cos(λit)+ γi sin(λit)) + Et 6 P 5.5 5 4.5 4 3.5 3 2.5 0 50 100 150 200 250 38 Trend and Seasonal Models Xt = Tt + Et = β0 + β1t + Et 6 5.5 5 4.5 4 3.5 3 2.5 0 50 100 150 200 250 39 Trend and Seasonal Models Xt = Tt + St + Et = β0 + β1t + i (βi cos(λit)+ γi sin(λit)) + Et 6 P 5.5 5 4.5 4 3.5 3 2.5 0 50 100 150 200 250 40 Trend and Seasonal Models: Residuals 0.5 0.4 0.3 0.2 0.1 0 −0.1 −0.2 −0.3 −0.4 −0.5 0 50 100 150 200 250 41 Time Series Modelling 1. Plot the time series. Look for trends, seasonal components, step changes, outliers. 2. Transform data so that residuals are stationary. (a) Estimate and subtract Tt,St. (b) Differencing. (c) Nonlinear transformations (log, √ ). · 3. Fit model to residuals. 42 Nonlinear transformations Recall: Monthly sales. (Makridakis, Wheelwright and Hyndman, 1998) 4 x 10 12 12 11.5 10 11 10.5 8 10 9.5 6 9 4 8.5 8 2 7.5 0 7 0 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90 43 Time Series Modelling 1. Plot the time series. Look for trends, seasonal components, step changes, outliers. 2. Transform data so that residuals are stationary. (a) Estimate and subtract Tt,St. (b) Differencing. (c) Nonlinear transformations (log, √ ).