Forecasting with Artificial Neural Networks

Forecasting with Artificial Neural Networks EVIC 2005 Tutorial Santiago de Chile, 15 December 2005 Æ slides on www.neural-forecasting.com Sven F. Crone Centre for Forecasting Department of Management Science Lancaster University Management School email: [email protected] EVIC’05 © Sven F. Crone - www.bis-lab.com Lancaster University Management School? EVIC’05 © Sven F. Crone - www.bis-lab.com What you can expect from this session … Simple back propagation algorithm [Rumelhart et al. 1982] ∂C(t pj ,o pj ) E p = C(t pj , o pj ) o pj = f j (net pj ) Δ p w ji ∝ − ∂w ji ∂C(t ,o ) ∂C(t ,o ) ∂net pj pj = pj pj pj ∂w ji ∂net pj ∂w ji ∂C(t ,o ) Æ „How to …“ on Neural δ = − pj pj pj ∂net pj Network Forecasting ∂C(t ,o ) ∂C(t ,o ) ∂o δ = − pj pj = pj pj pj pj with limited maths! ∂net pj ∂opj ∂net pj ∂o pt ' = f j (net pj ) ∂net pj ∂C(t pj ,opj ) ' δ pj = f j (net pj ) Æ CD-Start-Up Kit for ∂o pj ∂ w o ∂C(t ,o ) ∂net ∂C(t ,o ) ∑ ki pi Neural Net Forecasting pj pj pk = pj pj i ∑ ∂net ∂o ∑ ∂net ∂o k pk pj k pk pj Æ 20+ software simulators ∂C(t pj ,opj ) = ∑ wkj = − ∑δ pj wkj k ∂net pk k Æ datasets ' δ pj = f j (net pj )∑δ pj wkj Æ literature & faq k ⎧∂C(t pj ,opj ) ' ⎪ f j (net pj ) if unit j is in the output layer ⎪ ∂opj Æ slides, data & additional info on δ pj = ⎨ ⎪ ' f j (net pj )∑δ pk wpjk if unit j is in a hidden layer www.neural-forecasting.com ⎩⎪ k EVIC’05 © Sven F. Crone - www.bis-lab.com Agenda Forecasting with Artificial Neural Networks 1. Forecasting? 2. Neural Networks? 3. Forecasting with Neural Networks … 4. How to write a good Neural Network forecasting paper! EVIC’05 © Sven F. Crone - www.bis-lab.com Agenda Forecasting with Artificial Neural Networks 1. Forecasting? 1. Forecasting as predictive Regression 2. Time series prediction vs. causal prediction 3. Why NN for Forecasting? 2. Neural Networks? 3. Forecasting with Neural Networks … 4. How to write a good Neural Network forecasting paper! EVIC’05 © Sven F. Crone - www.bis-lab.com Forecasting or Prediction? Data Mining: „ Application of data analysis algorithms & discovery algorithms that extract patterns out of the data” Æ algorithms? Data Mining TASKS Descriptive Predictive timing of dependent variable Data Mining Data Mining Categorisation: Summarisation Association Sequence Categorisation: Time Series Regression Clustering & Visualisation Analysis Discovery Classification Analysis ¾K-means ¾Feature Selection ¾Association rules ¾Temporal ¾Decision trees ¾Linear Regression ¾Exponential Clustering ¾Princip.Component ¾Link Analysis association rules ¾Logistic regress. ¾Nonlinear Regres. smoothing ¾Neural networks Analysis ¾Neural networks ¾Neural networks ¾(S)ARIMA(x) ¾K-means ¾Disciminant MLP, RBFN, GRNN ¾Neural networks Clustering Analysis ¾Class Entropy ALGORITHMS EVIC’05 © Sven F. Crone - www.bis-lab.com Forecasting or Classification Independent Metric scale Ordinal scale Nominal scale dependent Metric sale Regression Analysis of Supervised learning Variance Time Series DOWNSCALE Inputs Target Analysis ... ... Ordinal scale ... ... DOWNSCALE DOWNSCALE DOWNSCALE ... Cases ... ... ... Nominal scale Classification Contingency . .... DOWNSCALE Analysis ... Principal Clustering Unsupervised learning NONE Component Inputs Analysis ... ... ... ... ... Cases ... ... ... ..... ... Simplification from Regression (“How much”) Æ Classification (“Will event occur”) Æ FORECASTING = PREDICTIVE modelling (dependent variable is in future) Æ FORECASTING = REGRESSION modelling (dependent variable is of metric scale) EVIC’05 © Sven F. Crone - www.bis-lab.com Forecasting or Classification? What the experts say … EVIC’05 © Sven F. Crone - www.bis-lab.com International Conference on Data Mining You are welcome to contribute … www.dmin-2006.com !!! EVIC’05 © Sven F. Crone - www.bis-lab.com Agenda Forecasting with Artificial Neural Networks 1. Forecasting? 1. Forecasting as predictive Regression 2. Time series prediction vs. causal prediction 3. SARIMA-Modelling 4. Why NN for Forecasting? 2. Neural Networks? 3. Forecasting with Neural Networks … 4. How to write a good Neural Network forecasting paper! EVIC’05 © Sven F. Crone - www.bis-lab.com Forecasting Models Time series analysis vs. causal modelling Time series prediction (Univariate) Assumes that data generating process that creates patterns can be explained only from previous observations of dependent variable Causal prediction (Multivariate) Data generating process can be explained by interaction of causal (cause-and-effect) independent variables EVIC’05 © Sven F. Crone - www.bis-lab.com Classification of Forecasting Methods Forecasting Methods Objective Subjective „Prophecy“ Forecasting Methods Forecasting Methods educated guessing… Time Series Causal Methods Methods Averages Linear Regression Sales Force Composite Moving Averages Multiple Regression Analogies Naive Methods Dynamic Regression Delphi Exponential Smoothing PERT Vector Autoregression Simple ES Survey techniques Linear ES Intervention model Seasonal ES Neural Networks Dampened Trend ES Neural Networks ARE Simple Regression time series methods Autoregression ARIMA causal methods Neural Networks & CAN be used as Averages & ES Demand Planning Practice Regression … Objektive Methods + Subjektive correction EVIC’05 © Sven F. Crone - www.bis-lab.com Time Series Definition Definition Time Series is a series of timely ordered, comparable observations yt recorded in equidistant time intervals Notation Yt represents the t th period observation, t=1,2 … n EVIC’05 © Sven F. Crone - www.bis-lab.com Concept of Time Series An observed measurement is made up of a systematic part and a random part Approach Unfortunately we cannot observe either of these !!! Forecasting methods try to isolate the systematic part Forecasts are based on the systematic part The random part determines the distribution shape Assumption Data observed over time is comparable The time periods are of identical lengths (check!) The units they are measured in change (check!) The definitions of what is being measured remain unchanged (check!) They are correctly measured (check!) data errors arise from sampling, from bias in the instruments or the responses, from transcription. EVIC’05 © Sven F. Crone - www.bis-lab.com Objective Forecasting Methods – Time Series Methods of Time Series Analysis / Forecasting Class of objective Methods based on analysis of past observations of dependent variable alone Assumption there exists a cause-effect relationship, that keeps repeating itself with the yearly calendar Cause-effect relationship may be treated as a BLACK BOX TIME-STABILITY-HYPOTHESIS ASSUMES NO CHANGE: Æ Causal relationship remains intact indefinitely into the future! the time series can be explained & predicted solely from previous observations of the series Æ Time Series Methods consider only past patterns of same variable Æ Future events (no occurrence in past) are explicitly NOT considered! Æ external EVENTS relevant to the forecast must be corected MANUALLY EVIC’05 © Sven F. Crone - www.bis-lab.com Simple Time Series Patterns Diagram 1.1: Trend - Diagram 1.2: Seasonal - long-term grow th or decline occuring m ore or les s regular m ovem ents w ithin a series w ithin a year 100 120 80 100 80 60 60 40 40 regular fluctuation within a year 20 Long term movement in series 20 (or shorter period) 0 0 superimposed on trend and cycle 3 6 9 5 12 15 18 21 24 27 30 10 15 20 25 30 35 40 45 Year Year Diagram 1.3: Cycle - Diagram 1.4: Irregular - alternating upsw ings of varied length random m ovements and those w hich and intensity reflect unusual events 10 350 8 300 250 6 200 4 150 2 Regular fluctuation 100 superimposed on trend (period 50 0 5 may be random) 0 10 15 20 25 30 35 40 45 1 Year 10 19 28 37 46 55 64 73 82 EVIC’05 © Sven F. Crone - www.bis-lab.com Regular Components of Time Series A Time Series consists of superimposed components / patterns: ¾ Signal level ‘L‘ trend ‘T‘ seasonality ‘S‘ ¾ Noise irregular,error 'e' Sales = LEVEL + SEASONALITY + TREND + RANDOM ERROR EVIC’05 © Sven F. Crone - www.bis-lab.com Irregular Components of Time Series Original-Zeitreihen 45 Structural changes in systematic data 40 35 30 25 PULSE 20 15 10 one time occurrence 5 0 on top of systematic stationary / [t] M 08.2000 M 09.2000 M 10.2000 M 11.2000 M 12.2000 M 01.2001 M 02.2001 M 03.2001 M 04.2001 M 05.2001 M 06.2001 M 07.2001 M 08.2001 M 09.2001 M 10.2001 M 11.2001 M 12.2001 M 01.2002 M 02.2002 M 03.2002 M 04.2002 M 05.2002 M 06.2002 M 07.2002 M trended / seasonal development Datenwert original Korrigierter Datenwert STATIONARY time series with PULSE LEVEL SHIFT Original-Zeitreihen one time / multiple time shifts 60 50 on top of systematic stationary / trended / seasonal 40 etc. development 30 20 10 STRUCTURAL BREAKS 0 [t] Jul 01 Jul Jul 02 Jul Mai 01 Mai Okt 00 Okt Mai 02 Mai Okt 01 Okt Jun 01 Jun Jan 01 Jan Jun 02 Jun Jan 02 Jan Feb 01 Feb Mrz 01 Apr 01 Feb 02 Feb Mrz 02 Apr 02 Sep 00 Sep Nov 00 Dez 00 Sep 01 Sep Nov 01 Dez 01 Aug 00 Aug 01 Trend changes (slope, direction) Datenwert original Korrigierter Datenwert Seasonal pattern changes & shifts STATIONARY time series with level shift EVIC’05 © Sven F. Crone - www.bis-lab.com Components of Time Series Time Series Level Trend Random • Time Series Æ decomposed into Components EVIC’05 © Sven F. Crone - www.bis-lab.com Time Series Patterns Time Series Pattern REGULAR IRREGULAR Time Series Patterns Time Series Patterns STATIONARY) SEASONAL TRENDED FLUCTUATING INTERMITTANT Time Series Time Series Time Series Time Series Time Series Original-Zeitreihen

Load more