Modeling Periodic Functions for Time Series Analysis

Home , Function approximation, Periodic function

Anish Biswas Department of Mathematics and Computer Science University of Tulsa 600 South College Avenue Tulsa, OK 74014 E-mail: [email protected]

Abstract the model can be improved. Wehave also provided the convergence proof of the algorithm, which says that un- Timeseries problemsinvolve analysis of periodic functions for predicting the future. Aflexible re- der infinite sampling this method is guaranteed to give gression method should be able to dynamically accurate model and the level of accuracy is limited only select the appropriate model to fit the available by error due to truncation of the series. data. In this paper, we present a function approximation scheme that can be used for model- Chebychev polynomials ing periodic functions using a series of orthogonal polynomials, named Chebychev polynomials. In Chebychev polynomials are a family of orthogonal poly- our approach, we obtain an estimate of the error nomials (Geronimus 1961). Any function f(x) may due to neglecting higher order polynomials and approximated by a weighted sum of these polynomial thus can flexibly select a polynomialmodel of the functions with an appropriate selection of the coeffi- proper order. Wealso showthat this approxima- cients. OO tion approachis stable in the presence of noise. a0 f(x) = -~ + E ai * 7](x) Keywords: Function approximation,Orthogonal poly- i=1 nomials where Introduction Analysis of time series data plays an important role and in finance and marketing. A model derived from past 2 F f(x) * 7](x) data can help to predict future behavior of the phe- ai=-zrJ_l ~-1---’-’~ dx. nomenon under study. For example , models predicting demand for product can be used to direct capital Working with an infinite series is not feasible in allocation. For analysis of time series data, we, in gen- practice. We can, however, truncate the above series eral, start with a predetermined model and try to tune and still obtain an approximation, ](x), of the func- its parameters to fit the data. For example, we may tion (Gerald & Wheatly 1992). The Chebychev poly- choose to fit a linear model to the data. for a flexi- nomials converges faster than the Taylor series for the ble approximation scheme, however, a model should be same function (Gerald g: Wheatly 1992). Let us assume chosen dynamically, based on data. In this paper, a we are using only the first n terms of the series. For a function approximation scheme is presented, which can rapidly converging series, the error due to truncation is be used for modeling periodic functions. Wehave used approximately given by the first term of the remainder, a series of orthogonal polynomials, called Chebychev i.e., a,~Tn(x). We have chosen Chebychev polynomials polynomials for approximating time series data. We for function approximation because truncation points know that any function can be represented with an in- can be chosen to provide approximate error bounds. finite series of Chebychev polynomials. But we cannot It should be noted that the expansion of a function work with an infinite series and use only a finite number f(x) with n Chebychev polynomials, gives the least of terms to derive an approximation. This method gives squared polynomial approximation with the n polyno- an approximate model for the data using n Chebychev mials (Foxand & Parker 1968). polynomials where the number of polynomials can be To produce a viable mechanism for building a model dynamically decided from the data. The approximated based on Chebychev polynomials would require the de- model with n Chebychev polynomials gives the least velopment of an algorithm for calculating the polyno- squared error with n polynomials. mial coefficients. Wewould also need to prove that this Furthermore, our algorithm allows incremental de- algorithmic updates would result in a convergence of velopment of the model, i.e. with each piece of data, the approximated function to the actual time function 1.5 i , , , I i i i 1.2 , ) i i i , ) i i i ap~n~x(r(x)) approx(r(x)) abs(~ln(5*x)) i ~ sin(lO*x).... /~ 1 i i L 0,5 0.8 " i, i i \ i Z i o t...... ~" 0.6 i J i t 4).5 0.4

0.2 11 ~,i/ t / Lll ~ I I i I I I "l I -1.5 I I -1 -0.8 -0.6 4).4 -I).2 (I,2 114 0 6 0 8 -0,8 -0,6 -0.4 -ILl O 0.2 0,4 I},6 (ill X

Figure 1: Approximating f(x) = sin(lO*x) for Number Figure 2: Approximating f(x) = Isin(5*x)l for Number of Chebychev Polynomials used = 20 and number of of Chebychev Polynomials used = 10 and number of data points = 1000. data points = 1000.

underlying the sampled data. We provide such an al- Approximations of periodic functions gorithm and the associated convergence theorem in the We know that a periodic function S(x) can in general next section. be expressed as An algorithm to get the temporal model f(x) ---- f(x -t- T* from time series data where T is the period and n E [0, 1,2,...] . So the na- Let f(x) be the target function of x and f(x) be its ture of the function in a single period and the length approximation based on the set of samples S = {Sj}, of the period is enough for the prediction of behavior where Sj = (xj, v~:j)Vj = 1, 2,..., k and k = Numberof for unknown x values. From our approximation scheme instances and v~j = f(xj). Wemay have to change the we get these two pieces of information and can success- scale for the values x j, so that all the values are in the fully predict the behavior of the functional values for range [-1 : 1]; then we get the approximated function unseen situations. Wehave taken some typical periodic in the range [-1 : 1], which we may need to scale back functions and tested the algorithm for those functions: to get the desired value. f(x) --- sin(x 10), an d f( x) -=Isi n(x * 5)1 Let n be the number of Chebychev polynomials we are going to use for the purpose of learning. Let Weknow that the error due to truncation is given by the ~(x) i E [0..n] be the Chebychev polynomials. The first term in the series and thus we can flexibly decide steps of the algorithm are: the number of terms to be chosen for approximation. The actual functions and the polynomial approxima- 1. InitializeCi=0, Vi=0,1,2,...,n tions obtained for these two functions are presented in 2. For all j do Figures 1 and in Figures 2. We have given each data 3. Vi = 0,1,2,...,n point sequentially into the algorithm and the resulting functions generated are found to be fairly good approximations of the underlying functions. Thus we can get a 2 good measure of the period and nature of the function x/l - x in a period. We can use this informations to get the 4. End for functioal values in unseen points. 5. Set Wehave also shown the nature of improvement of the n Co ~G*Ti model with more data points for the f(x) Isin(x * 5) = s<, (-7- + in Figures 3. We have calculated error as the average i=l of the difference of approximate functional values and where K = ¢(k) , is function of number of interac- the accurate values, over a set of sampled points in the tions. interval [-1 : 1]. It shows that the approximation accuracy increases rapidly with number of points. Theorem 1 Under infinite sampling the algorithm can approximate the actual function. Effects of Noise: We have also tested the approxima- Proof: See Appendix. tion scheme under noisy data, where noise is added 0.3 0.4

0.35 1 0,25

0.3’- No m)l~-- ~.d.,~).l.... ~, ~.d.~O.3..... Em)r -- i; ~.d.-05..... 0.25

0,2 0,15 ~.j,,\ o.1

0.05 0,05

I I I I I / I I I 4) 200 400 600 R00 I(X~ 1200 1400 1604) 1800 0 2 ) ) I(X)0 I 1404) I(~10 I~.) 20{}0 Numhc¢of Data [xfints Numberofdatll [~)inl~;

Figure 3: Nature of decreasing error in approximating Figure 4: Nature of decreasing error in approximating f(x) = Isin(x 5)] Number of Chebychev Pol ynomials f(x) = sin(x, 10) under noise Number of Chebychev used is 11 and sampling resolution is 1000. Polynomials used is 20 and sampling resolution is 1000.

to each data point. Here we assume that the random of greater resolution is received. Wehave noise follow a Normal distribution. ¢ We have run experiments with random noise follow- ,fi (/(x~) ~(x~)

ing Normal distribution(# = 0,~) where g = -- = 0.1, ~ = 0.3, g = 0.5 values are applied where where ¢ is a function of no of interactions and can be value of f(x) is in the range [-1 : 1]. Wehave in- given by ¢ = 2" So,we get, vestigated the nature of decreasing error with more data points and see that the process is resilient to noise as average error do not increase rapidly with noise shownin Figures 4. j=l Now,in case of infinite sampling and infinite resolu- Conclusion tion, we have, We have presented an algorithm that can build a model from the time series data where this model is not a pre- So for Vi -- 0, 1, 2,...n i.e. the coefficient becomes, determined one. Number of Chebychev polynomials is determined automatically, as the first term in the re- Ci = lim ¢--, {fi7-_f(xJ) ¯ ~(xj) mainder gives the measure of the error for approxima- ¢’+0 71" j=l ~i/1--x~ tion. So, we can suitably select the model for better approximation. Moreover, we have tested this approach C0= lira 1 ~ f(xj) 7~(xj),¢ with noisy data and see that its performance doesn’t ¢-~0 71" j=l decrease rapidly with noise and hence stable against noise. Ci _ ¢ * / f(x) ~(x)dx Appendix As, the x range is [-1:1] and in case of infinite sam- Proof of Theorem 1 pling and resolution we have information at all the values of x, the integral limits becomes, We know, that any function can be represented as a f(x) combination of Chebychev polynomials as, c, = 1 ,j_fl , To(x)dz oo a0 f(x) : -~ + ~ al * Ti (x) ~Ci=~ i=1 2 Thus, we show that the learned coefficients approxi- Next, we are going to prove that the function ](x) mate the actual polynomial coefficients for infinite sam- tends to f(x) as more and more samples of the function, pling.

3 References Foxand, L., and Parker, I. 1968. Chebyshev Polyno- mials in Numerical Analysis. New York, NY: Oxford. Gerald, C. F., and Wheatly, P. O. 1992. Applied Nu- merical Analysis. Menlo Park, CA: Addison-Wesley Publishing Company. Geronimus, L. Y. 1961. Orthogonal Polynomials. New York, NY: Consultants Bureau.