Introduction to Time Series Analysis. Lecture 1. Peter Bartlett 1

Introduction to Time Series Analysis. Lecture 1. Peter Bartlett 1. Organizational issues.

2. Objectives of time series analysis. Examples. 3. Overview of the course.

4. Time series models. 5. Time series modelling: Chasing stationarity.

1 Organizational Issues

Peter Bartlett. bartlett@stat. Ofﬁce hours: Tue 11-12, Thu 10-11 • (Evans 399). Joe Neeman. jneeman@stat. Ofﬁce hours: Wed 1:30–2:30, Fri 2-3 • (Evans ???). http://www.stat.berkeley.edu/ bartlett/courses/153-fall2010/ • ∼ Check it for announcements, assignments, slides, ...

Text: Time Series Analysis and its Applications. With R Examples, • Shumway and Stoffer. 2nd Edition. 2006.

2 Organizational Issues

Classroom and Computer Lab Section: Friday 9–11, in 344 Evans. Starting tomorrow, August 27: Sign up for computer accounts. Introduction to R.

Assessment: Lab/Homework Assignments (25%): posted on the website. These involve a mix of pen-and-paper and computer exercises. You may use any programming language you choose (R, Splus, Matlab, python). Midterm Exams (30%): scheduled for October 7 and November 9, at the lecture. Project (10%): Analysis of a data set that you choose. Final Exam (35%): scheduled for Friday, December 17.

3 A Time Series

400

350

300

250

200

150

100

0 0 1000 2000 3000 4000 5000 6000 7000

4 A Time Series

400

350

300

250

200

150

100

0 1960 1965 1970 1975 1980 1985 1990 year

5 A Time Series

400

350

300

250

$ 200

150

100

0 1960 1965 1970 1975 1980 1985 1990 year

6 A Time Series

SP500: 1960−1990 400

350

300

250

$ 200

150

100

0 1960 1965 1970 1975 1980 1985 1990 year

7 A Time Series

SP500: Jan−Jun 1987 340

320

300

$ 280

260

240

220 1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5 year

8 A Time Series

SP500 Jan−Jun 1987. Histogram 30

0 240 250 260 270 280 290 300 310 $

9 A Time Series

SP500: Jan−Jun 1987. Permuted. 340

320

300

$ 280

260

240

220 0 20 40 60 80 100 120

10 Objectives of Time Series Analysis

1. Compact description of data.

2. Interpretation. 3. Forecasting.

4. Control. 5. Hypothesis testing.

6. Simulation.

11 Classical decomposition: An example

Monthly sales for a souvenir shop at a beach resort town in Queensland.

(Makridakis, Wheelwright and Hyndman, 1998)

4 x 10 12

0 0 10 20 30 40 50 60 70 80 90

12 Transformed data

11.5

10.5

9.5

8.5

7.5

7 0 10 20 30 40 50 60 70 80 90

13 Trend

11.5

10.5

9.5

8.5

7.5

7 0 10 20 30 40 50 60 70 80 90

14 Residuals

1.5

0.5

−0.5

−1 0 10 20 30 40 50 60 70 80 90

15 Trend and seasonal variation

11.5

10.5

9.5

8.5

7.5

7 0 10 20 30 40 50 60 70 80 90

16 Objectives of Time Series Analysis

1. Compact description of data.

Example: Classical decomposition: Xt = Tt + St + Yt. 2. Interpretation. Example: Seasonal adjustment. 3. Forecasting. Example: Predict sales.

4. Control. 5. Hypothesis testing.

6. Simulation.

17 Unemployment data

Monthly number of unemployed people in Australia. (Hipel and McLeod, 1994)

5 x 10 8

7.5

6.5

5.5

4.5

4 1983 1984 1985 1986 1987 1988 1989 1990

18 Trend

5 x 10 8

7.5

6.5

5.5

4.5

4 1983 1984 1985 1986 1987 1988 1989 1990

19 Trend plus seasonal variation

5 x 10 8

7.5

6.5

5.5

4.5

4 1983 1984 1985 1986 1987 1988 1989 1990

20 Residuals

4 x 10 8

−2

−4

−6 1983 1984 1985 1986 1987 1988 1989 1990

21 Predictions based on a (simulated) variable

5 x 10 8

7.5

6.5

5.5

4.5

4 1983 1984 1985 1986 1987 1988 1989 1990

22 Objectives of Time Series Analysis

1. Compact description of data:

Xt = Tt + St + f(Yt)+ Wt.

2. Interpretation. Example: Seasonal adjustment. 3. Forecasting. Example: Predict unemployment.

4. Control. Example: Impact of monetary policy on unemployment.

5. Hypothesis testing. Example: Global warming. 6. Simulation. Example: Estimate probability of catastrophic events.

23 Overview of the Course

1. Time series models

2. Time domain methods 3. Spectral analysis

4. State space models(?)

24 Overview of the Course

1. Time series models (a) Stationarity. (b) Autocorrelation function. (c) Transforming to stationarity. 2. Time domain methods

3. Spectral analysis 4. State space models(?)

25 Overview of the Course

1. Time series models 2. Time domain methods (a) AR/MA/ARMA models. (b) ACF and partial autocorrelation function. (c) Forecasting (d) Parameter estimation (e) ARIMA models/seasonal ARIMA models

3. Spectral analysis 4. State space models(?)

26 Overview of the Course

1. Time series models

2. Time domain methods 3. Spectral analysis (a) Spectral density (b) Periodogram (c) Spectral estimation 4. State space models(?)

27 Overview of the Course

1. Time series models

2. Time domain methods 3. Spectral analysis

4. State space models(?) (a) ARMAX models. (b) Forecasting, Kalman ﬁlter. (c) Parameter estimation.

28 Time Series Models

A time series model speciﬁes the joint distribution of the se- quence X of random variables. { t} For example:

P [X1 x1,...,X x ] for all t and x1,...,x . ≤ t ≤ t t

Notation: X1,X2,... is a stochastic process. x1, x2,... is a single realization. We’ll mostly restrict our attention to second-order properties only:

EXt, E(Xt1 ,Xt2 ).

29 Time Series Models

Example: White noise: X W N(0, σ2). t ∼ i.e., X uncorrelated, EX = 0, VarX = σ2. { t} t t Example: i.i.d. noise: X independent and identically distributed. { t}

P [X1 x1,...,X x ]= P [X1 x1] P [X x ]. ≤ t ≤ t ≤ ··· t ≤ t Not interesting for forecasting:

P [X x X1,...,X −1]= P [X x ]. t ≤ t| t t ≤ t

30 Gaussian white noise

t x 2 1 −x /2 P [Xt xt]=Φ(xt)= e dx. ≤ √2π Z−∞

2.5

1.5

0.5

−0.5

−1

−1.5

−2

−2.5 0 5 10 15 20 25 30 35 40 45 50

31 Gaussian white noise

2.5

1.5

0.5

−0.5

−1

−1.5

−2

−2.5 0 5 10 15 20 25 30 35 40 45 50

32 Time Series Models

Example: Binary i.i.d. P [X =1]= P [X = 1] = 1/2. t t −

0.8

0.6

0.4

0.2

−0.2

−0.4

−0.6

−0.8

−1

0 5 10 15 20 25 30 35 40 45 50

33 Random walk

t S = X . Differences: S = S S −1 = X . t i=1 i ∇ t t − t t P 8

−2

−4 0 5 10 15 20 25 30 35 40 45 50

34 Random walk

ESt? VarSt?

−5

−10

−15 0 5 10 15 20 25 30 35 40 45 50

35 Random Walk

Recall S&P500 data. (Notice that it’s smooth)

SP500: Jan−Jun 1987 340

320

300

$ 280

260

240

220 1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5 year

36 Random Walk

Differences: S = S S −1 = X . ∇ t t − t t SP500, Jan−Jun 1987. first differences 10

$ 0

−2

−4

−6

−8

−10 1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5 year

37 Trend and Seasonal Models

Xt = Tt + St + Et = β0 + β1t + i (βi cos(λit)+ γi sin(λit)) + Et

6 P

5.5

4.5

3.5

2.5 0 50 100 150 200 250

38 Trend and Seasonal Models

Xt = Tt + Et = β0 + β1t + Et

5.5

4.5

3.5

2.5 0 50 100 150 200 250

39 Trend and Seasonal Models

Xt = Tt + St + Et = β0 + β1t + i (βi cos(λit)+ γi sin(λit)) + Et

6 P

5.5

4.5

3.5

2.5 0 50 100 150 200 250

40 Trend and Seasonal Models: Residuals

0.5

0.4

0.3

0.2

0.1

−0.1

−0.2

−0.3

−0.4

−0.5 0 50 100 150 200 250

41 Time Series Modelling

1. Plot the time series. Look for trends, seasonal components, step changes, outliers. 2. Transform data so that residuals are stationary.

(a) Estimate and subtract Tt,St. (b) Differencing. (c) Nonlinear transformations (log, √ ). · 3. Fit model to residuals.

42 Nonlinear transformations

Recall: Monthly sales. (Makridakis, Wheelwright and Hyndman, 1998)

4 x 10 12 12

11.5

10 11

10.5 8 10

9.5 6

4 8.5

8 2

7.5

0 7 0 10 20 30 40 50 60 70 80 90 0 10 20 30 40 50 60 70 80 90

43 Time Series Modelling

1. Plot the time series. Look for trends, seasonal components, step changes, outliers. 2. Transform data so that residuals are stationary.

(a) Estimate and subtract Tt,St. (b) Differencing. (c) Nonlinear transformations (log, √ ). · 3. Fit model to residuals.

44 Differencing

Recall: S&P 500 data.

SP500: Jan−Jun 1987 SP500, Jan−Jun 1987. first differences 340 10

320 6

4 300

2 $ $ 280 0

−2

260 −4

−6 240

−8

220 −10 1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5 1987 1987.05 1987.1 1987.15 1987.2 1987.25 1987.3 1987.35 1987.4 1987.45 1987.5 year year

45 Differencing and Trend

Deﬁne the lag-1 difference operator, (think ‘ﬁrst derivative’)

X = X X −1 = (1 B)X , ∇ t t − t − t where B is the backshift operator, BXt = Xt−1.

If X = β0 + β1t + Y , then • t t X = β1 + Y . ∇ t ∇ t If X = k β ti + Y , then • t i=0 i t P kX = k!β + kY , ∇ t k ∇ t where kX = ( k−1X ) and 1X = X . ∇ t ∇ ∇ t ∇ t ∇ t

46 Differencing and Seasonal Variation

Deﬁne the lag-s difference operator,

s X = X X − = (1 B )X , ∇s t t − t s − t s s s−1 where B is the backshift operator applied s times, B Xt = B(B Xt) 1 and B Xt = BXt.

If Xt = Tt + St + Yt, and St has period s (that is, St = St−s for all t), then

X = T T − + Y . ∇s t t − t s ∇s t

47 Time Series Modelling

1. Plot the time series. Look for trends, seasonal components, step changes, outliers. 2. Transform data so that residuals are stationary.

(a) Estimate and subtract Tt,St. (b) Differencing. (c) Nonlinear transformations (log, √ ). · 3. Fit model to residuals.

48 Outline

1. Objectives of time series analysis. Examples.

2. Overview of the course. 3. Time series models.

4. Time series modelling: Chasing stationarity.