New Methods for Spectral Estimation with Confidence Intervals for Locally Stationary Wavelet Processes
Total Page：16
File Type：pdf, Size：1020Kb
New Methods for Spectral Estimation with Confidence Intervals for Locally Stationary Wavelet Processes Kara Nicola Stevens School of Mathematics June 2013 A Dissertation Submitted to the University of Bristol in accordance with the requirements of the degree Doctor of Philosophy in the Faculty of Science Abstract In this thesis we develop new methods of estimating the second order structure of a locally stationary process with associated confidence intervals (CI) to quantify the estima- tor. In particular we focus on smoothing the estimate of the evolutionary wavelet spectrum (EWS) of a locally stationary wavelet process. Similarly to the stationary spectrum, the raw estimate of the EWS is unbiased but not consistent. Therefore, de-noising methods are required to improve our estimate. We have developed three such methods and used simulated data to test the methods. We begin by investigating the statistical properties of the raw estimate, which we use to establish theoretical models. Our first method uses a simple running mean estimator. This method produces reasonable results, but due to its strong dependence the translation invariant de-noising estimator, there are limitations in the type of spectra's which can be successfully recovered. Our next two methods uses the benefits of a posterior distribution from Bayesian analysis to produce estimators with associated CI. We use the log and Haar-Fisz transfor- mations to stabilise the variance of our raw estimates. The model for the log transformed data is more complex model than the Haar-Fisz, and required more numerical approxima- tions during the estimation process. The Haar-Fisz transformation pulls the distribution of the data closer to Gaussianity, which greatly simplified the modelling assumptions. The Haar-Fisz estimator and CI were computationally more efficient and produced superior results compared to the previous methods. It appeared to be fairly robust to wavelet selection, but again, was heavily dependent on the hyperparameters. Using different methods to determine the hyperparameters greatly influenced the re- sults. With further research we hope to improve the efficiency of determining the hyper- parameters. i ii Dedication I would like to dedicate this thesis to my Mum, Alison Stevens, gone but not forgotten. iii iv Acknowledgements Firstly I would like to thank my supervisor, Professor Guy Nason, for all the inspi- ration, guidance and encouragement given to me throughout my degree. Without his support, this PhD would not have been possible. I would like to extend my gratitude to the EPSRC and SuSTaIn initiative for funding me throughout my time at the University of Bristol. I would to thank all my friends and family for their continuous support, advice, and encouragement. In particular Linda Lietchmann for taking the time to proof read my thesis; Lesley Barwell, Becky Hunt and Lisa Giles for always believing in me and listening to my complaints. I would especially like to thank Daniel Barwell, for his enduring patient and love through some very turbulent times. Finally, I would like to thank my Mum, for all her sacrifices in life so I could even have the chance of attending University in the first place. I am what I am today because of her. Without all these people, I would not have been able to accomplish what I have. v vi Author's Declaration I declare that the work in this dissertation was carried out in accordance with the re- quirements of the University's Regulations and Code of Practice for Research Degree Programmes and that it has not been submitted for any other academic award. Except where indicated by specific reference in the text, the work is the candidate's own work. Work done in collaboration with, or with the assistance of, others, is indicated as such. Any views expressed in the dissertation are those of the author. SIGNED: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: Kara Nicola Stevens DATE: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: vii viii Contents Abstract i Dedication iii Acknowledgements v Author's Declaration vii 1 Introduction 1 2 Literature Review 4 2.1 Besov Spaces . 5 2.2 Fourier Analysis . 6 2.2.1 Properties . 8 2.2.2 Limitations . 9 2.3 Wavelet Theory . 10 2.3.1 Multi-resolution Analysis . 10 2.3.2 Function Representation . 14 2.3.3 The Discrete Wavelet Transform . 15 2.3.4 Non-decimated Wavelet Transform . 19 2.3.5 Examples of Wavelets . 19 2.4 Nonparametric Regression . 23 2.4.1 Linear Smoothing Methods . 24 2.4.2 Wavelet Shrinkage . 25 2.5 Time Series Data . 34 2.5.1 Stationary Time Series . 35 2.5.2 Non-Stationary Time Series . 38 2.5.3 Locally Stationary Fourier Processes . 40 ix 2.6 Locally Stationary Wavelet Processes . 41 2.6.1 The Evolutionary Wavelet Spectrum . 43 2.6.2 Estimation of the Evolutionary Wavelet Spectrum . 46 2.6.3 Translation-Invariant De-Noising EWS Estimator . 48 2.6.4 Haar-Fisz EWS Estimation . 49 3 Foundation Work 54 3.1 Wavelet Coefficients of the LSW Process . 54 3.2 Raw Wavelet Periodogram . 58 3.3 Asymptotic Independence . 61 3.4 Numerical Investigations . 61 3.4.1 Piecewise Spectrum . 63 3.4.2 Slowly Evolving Spectrum . 71 3.4.3 Conclusion . 79 4 Naive Spectral Confidence Interval using the Central Moving Average 80 4.1 Estimation of the Evolutionary Wavelet Spectrum . 80 4.1.1 Smoothed Wavelet Periodogram . 81 4.1.2 Smoothed EWS Estimate . 83 4.1.3 Confidence Interval . 85 4.2 Computational Details . 85 4.2.1 Simulations . 87 4.2.2 Conclusion . 91 5 Bayesian Modelling of the Log Transformed EWS 92 5.1 Regression Model for the Log-EWS . 93 5.1.1 Distribution of the Log-Error . 94 5.1.2 DWT of the Regression Model . 97 5.2 Bayesian Shrinkage Rule . 98 5.2.1 Likelihood . 98 5.2.2 Prior . 103 5.2.3 Posterior Distribution . 103 5.2.4 Posterior Mean . 104 5.2.5 Posterior Variance . 105 5.2.6 Hyperparameters . 105 x 5.2.7 Bayesian Log-EWS Estimator . 106 5.3 Choice of Tail Density Prior . 107 5.3.1 Gaussian Mixture Prior . 107 5.3.2 Laplace Mixture Prior . 110 5.4 Asymptotics for the Bayesian Log-EWS Estimator . 113 5.4.1 Asymptotic Model . 113 5.4.2 Assumptions . 114 5.4.3 Asymptotic Results . 116 6 Simulation Results of the Bayesian Log-EWS Estimator 117 6.1 Programming . 117 6.2 Comparison of the Gaussian and Laplace Prior . 120 6.3 Simulation Example . 123 6.3.1 Piecewise Constant EWS . 124 6.3.2 Slowly Evolving EWS . 127 6.3.3 Conclusion . 128 7 Bayesian Wavelet Shrinkage of the Haar-Fisz EWS 131 7.1 Haar-Fisz Transformation . 131 7.2 The Distribution and Correlation of the Haar-Fisz Periodogram . 133 7.3 Bayesian Wavelet Shrinkage of the Haar-Fisz Periodogram . 137 7.3.1 Model . 137 7.3.2 Prior . 137 7.3.3 Likelihood . 138 7.3.4 Posterior Distribution . 138 7.3.5 Posterior Mean . 139 7.3.6 Posterior Variance . 139 7.3.7 Hyperparameter Determination . 139 7.3.8 Credible Intervals . 140 7.4 Choice of Tail Density Prior . 140 7.4.1 Uniform Mixture Prior . 140 7.4.2 Laplace Mixture Prior . 142 xi 7.5 Simulation Examples . 145 7.5.1 Piecewise Constant EWS . 147 7.5.2 Slowly Evolving EWS . 149 7.5.3 Conclusion . 149 8 Conclusion 153 Bibliography 157 A Appendix of Proofs 165 A.1 Proofs for the Bayesian Log-Periodogram . 165 A.1.1 Gaussian Mixture Prior . 165 A.1.2 Laplace Mixture Prior . 168 A.1.3 Asymptotic Convergence . 173 A.2 Proofs for the Bayesian Modelling of the Haar-Fisz Periodogram . 209 B Appendix of Computational Output 211 xii List of Figures 2.1 Flow diagram of the discrete wavelet transform of an observed data set using successive applications of the low and high pass filters g and h. 17 2.2 Plots of Haar scaling (a) and wavelet (b) function. 20 2.3 Plots of Shannon scaling (a) and wavelet (b) function in the time domain. 22 2.4 Plots of the scaling function and wavelet function of Daubechies Extremal Phase wavelet with 2, 5 and 10 vanishing moments. 23 2.5 A flow diagram of wavelet shrinkage of (2.30). 26 2.6 Plots of discrete Haar autocorrelation wavelet for at the finest scale (J 1) − in (a), third finest scale (J 3) in (b) and fifth finest scale (J 5) in (c), − − over lag τ = 32;:::; 32. 44 − 3.1 Plots of the true piecewise constant spectrum defined in equation (3.9) cre- ated using the Haar synthesis wavelet, and a realised LSW process X 1023 f tgt=0 generated with Gaussian innovations. 64 3.2 Density histograms of the empirical wavelet coefficients at scales j = 9; 6; 0, obtained from the piecewise LSW process (Xt) with t5 (in a, b and c) and 2 χ1 (in d, e and f) innovations and Gaussian pdf (blue solid line). ..