Analysis of the Bitcoin Exchange Using Particle MCMC Methods

by Michael Johnson

M.Sc., University of British Columbia, 2013 B.Sc., University of Winnipeg, 2011

Project Submitted in Partial Fulﬁllment of the Requirements for the Degree of Master of Science

in the Department of Statistics and Actuarial Science Faculty of Science

c Michael Johnson 2017 SIMON FRASER UNIVERSITY Spring 2017

All rights reserved. However, in accordance with the Copyright Act of Canada, this work may be reproduced without authorization under the conditions for “Fair Dealing.” Therefore, limited reproduction of this work for the purposes of private study, research, education, satire, parody, criticism, review and news reporting is likely to be in accordance with the law, particularly if cited appropriately. Approval

Name: Michael Johnson Degree: Master of Science (Statistics) Title: Analysis of the Bitcoin Exchange Using Particle MCMC Methods Examining Committee: Chair: Dr. Jiguo Cao Associate Professor

Dr. Liangliang Wang Senior Supervisor Assistant Professor

Dr. Dave Campbell Supervisor Associate Professor

Dr. Tim Swartz Internal Examiner Professor

Date Defended: March 24, 2017

ii Abstract

Stochastic volatility models (SVM) are commonly used to model time series data. They have many applications in finance and are useful tools to describe the evolution of asset returns. The motivation for this project is to determine if stochastic volatility models can be used to model Bitcoin exchange rates in a way that can contribute to an effective trading strategy. We consider a basic SVM and several extensions that include fat tails, leverage, and covariate effects. The Bayesian approach with the particle Markov chain Monte Carlo (PMCMC) method is employed to estimate the model parameters. We assess the goodness of the estimated model using the deviance information criterion (DIC). Simulation studies are conducted to assess the performance of particle MCMC and to compare with the traditional MCMC approach. We then apply the proposed method to the Bitcoin exchange rate data and compare the effectiveness of each type of SVM. Keywords: Stochastic volatility model; hidden Markov model; sequential Monte Carlo; particle Markov chain Monte Carlo; Bitcoin.

iii Table of Contents

Approval ii

Abstract iii

Table of Contents iv

List of Tables vi

List of Figures vii

1 Introduction 1 1.1 Stochastic Volatility Models and Particle Markov Chain Monte Carlo . . . . 1 1.2 Bitcoin and Bitcoin Exchanges ...... 3 1.3 Research Objective ...... 5 1.4 Thesis Organization ...... 6

2 Stochastic Volatility Models 7 2.1 Hidden Markov Models ...... 7 2.2 Basic Stochastic Volatility Model ...... 8 2.2.1 Basic SVM Version 1 ...... 8 2.2.2 Basic SVM Version 2 ...... 9 2.3 Stochastic Volatility Model with Fat Tails ...... 9 2.4 Stochastic Volatility Model with Leverage Eﬀect ...... 10 2.5 Stochastic Volatility Model with Covariate Eﬀects ...... 11 2.6 Chapter Summary ...... 11

3 Bayesian Inference for Stochastic Volatility Models 13 3.1 Bayesian Inference ...... 13 3.2 Monte Carlo Integration ...... 14 3.3 Posterior Inference via Markov Chain Monte Carlo ...... 15 3.3.1 Markov Chain Monte Carlo (MCMC) ...... 15 3.3.2 Gibbs Sampling ...... 16 3.4 Posterior Inference via Sequential Monte Carlo (SMC) ...... 18

iv 3.4.1 Importance Sampling (IS) ...... 18 3.4.2 Sequential Importance Sampling (SIS) ...... 19 3.4.3 Sequential Monte Carlo (SMC) ...... 21 3.5 Particle Markov Chain Monte Carlo (PMCMC) ...... 22 3.6 Model Comparison ...... 24 3.7 Chapter Summary ...... 25

4 Simulation Studies 27 4.1 Gibbs Sampler and PMCMC for Basic SVM Version 1 ...... 27 4.1.1 PMCMC ...... 27 4.1.2 Gibbs Sampler ...... 30 4.1.3 Comparison of PMCMC and Gibbs Sampler ...... 31 4.2 PMCMC for Basic SVM Version 2 ...... 32 4.3 PMCMC for SVM with Fat Tails ...... 34 4.4 PMCMC for SVM with Leverage Eﬀect ...... 37 4.5 PMCMC for SVM with Covariate Eﬀects ...... 39 4.6 Chapter Summary ...... 43

5 Applications to Bitcoin Exchange Rate Data 44 5.1 Data ...... 44 5.2 Bitcoin Data Analysis with Basic SVM ...... 46 5.3 Bitcoin Data Analysis for SVM with Fat Tail ...... 48 5.4 Bitcoin Data Analysis for SVM with Leverage Eﬀect ...... 49 5.5 Bitcoin Data Analysis for SVM with Covariate Eﬀect ...... 51 5.6 Summary of Bitcoin Data Analysis ...... 54

6 Conclusion and Future work 55

Bibliography 57

v List of Tables

Table 3.1 Gibbs Sampler Algorithm...... 17 Table 3.2 Sequential Importance Sampling Algorithm...... 21 Table 3.3 Sequential Monte Carlo Algorithm...... 22 Table 3.4 Particle MCMC Algorithm...... 24

Table 4.1 Summary of Basic SVM Version 1 parameter posterior distributions resulting from PMCMC and Gibbs Sampler for simulated data. . . . 31 Table 4.2 Summary of Basic SVM Version 2 parameter posterior distributions resulting from PMCMC for simulated data...... 34 Table 4.3 Summary of SVM Fat Tails parameter posterior distributions resulting from PMCMC for simulated data...... 36 Table 4.4 Summary of SVM with Leverage Eﬀect parameter posterior distributions resulting from PMCMC for simulated data...... 39 Table 4.5 Summary of SVM with Covariate Eﬀect parameter posterior distributions resulting from PMCMC for simulated data...... 42

Table 5.1 Summary of Basic SVM Version 2 parameter posterior distributions resulting from PMCMC for Bitcoin data...... 48 Table 5.2 Summary of SVM with Fat Tails parameter posterior distributions resulting from PMCMC for Bitcoin data...... 49 Table 5.3 Summary of SVM with Leverage Effect parameter posterior distributions resulting from PMCMC for Bitcoin data...... 51 Table 5.4 Summary of SVM with Covariate Effect parameter posterior distributions resulting from PMCMC for Bitcoin data...... 53 Table 5.5 A summary of the DIC for each stochastic volatility model that was fit to the Bitcoin exchange rate data set...... 54

vi List of Figures

Figure 2.1 Illustration of a Hidden Markov Model...... 7

2 Figure 4.1 Basic SVM Version 1 trace plots of α, µx, µy and σ resulting from PMCMC for simulated data...... 28 2 Figure 4.2 Basic SVM Version 1 histograms of α, µx, µy and σ resulting from PMCMC for simulated data...... 29 Figure 4.3 Basic SVM Version 1 simulated data and PMCMC estimates, with

observations Y1:n and hidden process X1:n for simulated data. . . . 29 2 Figure 4.4 Basic SVM Version 1 trace plots of α, µx, µy and σ resulting from Gibbs Sampler for simulated data...... 30 2 Figure 4.5 Basic SVM Version 1 histograms of α, µx, µy and σ resulting from Gibbs Sampler for simulated data...... 31 2 Figure 4.6 Basic SVM Version 2 trace plots of α, µx, µy and σ resulting from PMCMC for simulated data...... 33 2 Figure 4.7 Basic SVM Version 2 histograms of α, µx, µy and σ resulting from PMCMC for simulated data...... 33 Figure 4.8 Basic SVM Version 2 simulated data and PMCMC estimates, with

observations Y1:n and hidden process X1:n for simulated data. . . . 34 2 Figure 4.9 SVM Fat Tails trace plots of α, µx, df and σ resulting from PMCMC for simulated data...... 35 2 Figure 4.10 SVM Fat Tails histograms of α, µx, df and σ resulting from PM- CMC for simulated data...... 36 Figure 4.11 SVM Fat Tails simulated data and PMCMC estimates, with obser-

vations Y1:n and hidden process X1:n for simulated data...... 37 2 Figure 4.12 SVM with Leverage Effect trace plots of α, µx, ρ and σ resulting from PMCMC for simulated data...... 38 2 Figure 4.13 SVM with Leverage Effect histograms of α, µx, ρ and σ resulting from PMCMC for simulated data...... 38 Figure 4.14 SVM with Leverage Effect simulated data and PMCMC estimates,

with observations Y1:n and hidden process X1:n for simulated data. 39 2 Figure 4.15 SVM with Covariate Eﬀect trace plots of α, µx, η1, η2 and σ resulting from PMCMC for simulated data...... 41

vii 2 Figure 4.16 SVM with Covariate Eﬀect histograms of α, µx, η1, η2 and σ resulting from PMCMC for simulated data...... 42 Figure 4.17 SVM with Covariate Eﬀect simulated data and PMCMC estimates,

with observations Y1:n and hidden process X1:n for simulated data. 43

Figure 5.1 Daily Bitcoin exchange rate...... 44 Figure 5.2 Relative change in daily Bitcoin exchange rate...... 45 Figure 5.3 Number of Bitcoin Transactions per Day...... 45 Figure 5.4 Number of Unique Bitcoin Addresses Used per Day...... 46 2 Figure 5.5 Basic SVM Version 2 trace plots of α, µx, µy and σ resulting from PMCMC for Bitcoin data...... 47 2 Figure 5.6 Basic SVM Version 2 histograms of α, µx, µy and σ resulting from PMCMC for Bitcoin data...... 47 2 Figure 5.7 SVM with Fat Tails trace plots of α, µx, df and σ resulting from PMCMC for Bitcoin data...... 48 2 Figure 5.8 SVM with Fat Tails histograms of α, µx, df and σ resulting from PMCMC for Bitcoin data...... 49 2 Figure 5.9 SVM with Leverage Effect trace plots of α, µx, ρ and σ resulting from PMCMC for Bitcoin data...... 50 2 Figure 5.10 SVM with Leverage Effect histograms of α, µx, ρ and σ resulting from PMCMC for Bitcoin data...... 50 2 Figure 5.11 SVM with Covariate Effect trace plots of α, µx, η1, η2 and σ resulting from PMCMC for Bitcoin data...... 52 2 Figure 5.12 SVM with Covariate Effect histograms of α, µx, η1, η2 and σ resulting from PMCMC for Bitcoin data...... 53

viii Chapter 1

Introduction

1.1 Stochastic Volatility Models and Particle Markov Chain Monte Carlo

Stochastic volatility models (SVM) are widely used in the field of economics and finance. They are commonly used for modeling financial time series data such as stock returns and exchange rates [11] or market indices [19]. Applied work in these fields often involves complex nonlinear relationships between many variables and the complicated models used in these situations often give rise to high dimensional integrals that cannot be solved analytically. This has led to the increased popularity of Monte Carlo methods, which can use simulation techniques to estimate complex integrals.

Monte Carlo integration [9] is a simulation technique that uses independent draws from a distribution of interest, referred to as a target distribution, to approximate integrals rather than doing them analytically. However, it is not always possible to sample independent draws directly from the target distribution. In these cases if we can sample slightly dependent draws from a Markov chain, similar approximations can still be made.

A Markov chain is a random or stochastic process that evolves over time from one state to the next. It has the property that the next state depends only on the current state and not on the sequence of states that preceded it. This memoryless characteristic is known as the Markov property. A Markov chain will eventually converge to what is known as a stationary distribution [21]. Markov chain Monte Carlo (MCMC) algorithms create a Markov chain with the target distribution as its stationary distribution. Once the chain has converged the draws are approximately the same as if they were sampled from the target distribution. The two most commonly used MCMC methods are the Gibbs sampler and

1 the Metropolis-Hastings algorithms [21]. In this project we will make use of both the Gibbs sampler and Metropolis-Hasting methods.

Importance sampling (IS) [9] is another technique which can be used when the target distribution cannot be sampled from directly. Importance sampling involves selecting an importance distribution from which it is easy to draw samples. Then importance weights are calculated and used to re-weight the draws from the important distribution so that they are approximately the same as if they were from the target distribution. If we restrict the importance distribution to a certain form that can be written recursively we can implement sequential importance sampling (SIS) [9] which can be more computationally eﬃcient. The details of these algorithms are outlined in later chapters and we see how they lead into more complicated methods such as sequential Monte Carlo and particle MCMC.

Sequential Monte Carlo (SMC) is an increasingly popular alternative to MCMC, because of its speed and scalability. In general, SMC methods [3] are algorithms that sample from a sequence of target distributions of increasing dimension. The SMC algorithm is basically sequential importance sampling that implements a resampling technique to address some of the problems with SIS. [14] contributed a pioneering paper of SMC focusing on tracking applications. Since then, various SMC algorithms have been proposed to obtain full posterior distributions for problems in nonlinear dynamic systems in science and engineering. To date, these algorithms for nonlinear and non-Gaussian state-space models have been successfully applied in various ﬁelds including computer vision, signal processing, tracking, control, econometrics, ﬁnance, robotics, and statistics [23, 3, 4, 9]. The SMC algorithm will be explained in more detail in later sections as it is an important part of the particle MCMC algorithm.

Particle Markov chain Monte Carlo (PMCMC) [9] uses sequential Monte Carlo within the MCMC algorithm [2, 15]. As with basic MCMC, both the Gibbs and Metropolis-Hastings methods can be used in PMCMC. [26] used particle Gibbs with ancestor sampling [22] for nontrivial SVMs. In this project the Metropolis-Hastings method is used for PMCMC which is known as the particle marginal Metropolis-Hastings (PMMH) method [2]. The steps of the Particle MCMC algorithm will be explained in detail in a later section.

In this project several types of stochastic volatility models are investigated and the particle Markov chain Monte Carlo method is used to estimate the posterior distributions of the model parameters. We focus on a basic SVM and several extensions that include fat or heavy tails, leverage, and covariate eﬀects. These SVMs have been considered in [26] and [1].

A basic stochastic volatility model was used by [9] to illustrate the eﬀectiveness of the PM- CMC method. The basic SVM has uncorrelated Gaussian white noise from the observation

2 (measurement) equation and the system (state) equation. Many extensions have been proposed to relax the assumption of the uncorrelated error and/or the normal distribution. For example, [17] considered correlated errors and provided an MCMC algorithm for the leverage stochastic volatility model, which extends the basic SVM to accommodate nonzero correlation between Gaussian white noise from the observation equation and the system equation.

Fat-tailed (a standard Student’s t distribution), skewed and scale mixture of normal distributions are considered in [24, 5, 13, 17]. Moreover, [10] considered SVMs with jumps. [1] extended a basic SVM to capture a leverage eﬀect, a fat-tailed distribution of asset returns and a nonlinear relationship between the current volatility and the previous volatility process. The author used the Bayesian approach with the MCMC method to estimate model parameters, and evaluate diﬀerent models with several Bayesian model selection criteria.

Besides the univariate SVMs, [16, 29] focused on the multivariate stochastic volatility models. [6] generalizes the popular stochastic volatility in mean model of [18] to allow for time-varying parameters in the conditional mean.

Since several diﬀerent SVMs will be investigated it is important that we have a way to assess their eﬀectiveness and select the best estimated model for a given application. For example, [17] applied several Bayesian model selection criteria that include the Bayes factor, the Bayesian predictive information criterion and the deviance information criterion.

1.2 Bitcoin and Bitcoin Exchanges

Bitcoin is an electronic payment system that has been growing in popularity over the last several years. It was ﬁrst introduced by Satoshi Nakamoto who published the white paper [25] in 2008 and released it as open-source software in 2009. Bitcoin is a type of cryptocur- rency which is deﬁned as an "electronic payment system based on cryptographic proof" [25]. The system allows transactions to take place directly between users, without a central payment system or any single administrator. In addition Bitcoins are not linked to any type of commodity such as gold or silver [27]. Therefore this decentralized virtual currency is controlled by its users instead of a governing body. The Bitcoin system utilizes a peer-to- peer network of all those who are involved in creating and trading Bitcoins, to process and check all transactions.

Today Bitcoins are accepted as payment for goods and services by many online e-commerce sites and by an increasing number of physical stores by way of smart phone apps. Bit- coin transactions are attractive to merchants due to their high speed and low transaction

3 fees. Simon Fraser University has recently began accepting tuition payment in Bitcoin and has introduced a Bitcoin ATM in the bookstore. Although Bitcoins are becoming more mainstream, the concept of virtual money can be confusing at ﬁrst glance to the average consumer. This section will explain the basics of Bitcoins and where they come from as well as how they are bought and sold on online exchanges.

To begin, Bitcoins are created though a process know as mining. The basic idea is that users offer their computing power for payment processing work and they are rewarded with Bitcoins. Bitcoins are exchanged over the network all the time and these transactions must be verified and recorded. A list of all the transactions made during a set period of time is called a block. Every Bitcoin transaction ever made is recorded in a public ledger made up of a long list of blocks, known as the Blockchain. When a new block of transactions is created it is the miners job to put it through a confirmation process and then add it to the Blockchain. This process is very computationally expensive as it requires finding solutions to complex mathematical problems. Miners use software and machines specifically designed for Bitcoin mining and are rewarded with new Bitcoins every time the Blockchain is updated. Bitcoin mining is a complicated process and many people do not have the means to acquire Bitcoins in this way. However, it is possible to simply purchase Bitcoins with traditional currency from miners or anyone that is looking to sell them.

Bitcoin is traded on many online exchanges where it can be bought or sold using regular government backed currencies. There are exchanges that accept Canadian Dollars (CND), Chinese Yuan (CNY) and US Dollars (USD). Exchanges such as OKCoin, BitStamp, or Kraken allow users to deposit and withdraw funds from the exchange via regular online banking services. Kraken is currently the largest exchange to accept Canadian Dollars. In addition to Bitcoin, many types of similar cryptocurrencies exists today such as Litecoin or Ethereum which have been growing in popularity, but bitcoin is by far the most widely used and largest in terms of total market value. Many Bitcoin exchanges allow users to purchase other forms of cryptocurrencies as well, or to trade Bitcoin directly for other cryptocurrencies. This project focuses speciﬁcally on Bitcoin trading, but the exchange rate of other cryptocurrencies will likely follow a similar pattern and could be an interesting future application of this work.

Bitcoin exchange rates can be extremely volatile and an effective trading strategy could po- tentially lead to large profits. The value of Bitcoin may not behave like a typical currency. Econonomic and financial theory cannot explain the large volatility in the Bitcoin price. Factors such as interest rates and inflation do not effect Bitcoin as they would a government backed currency because there is no central bank overseeing the issuing of Bitcoin. Therefore, Bitcoin price is "driven solely by the investors faith in the perpetual growth" [20]. A statistical analysis of the log-returns of the exchange rate of Bitcoin in US dollars

4 was provided by [7]. Parametric distributions that are popular in financial applications are fitted to the log-returns and the generalized hyperbolic distribution is shown to give the best fit. The links between Bitcoin price and socials signals was examined by [8]. Using data from Bitcoin exchanges, social media and Google search trends they found evidence of positive feedback loops. An increase in popularity of Bitcoin leads to an increase in Google searches for Bitcoin and social media coverage. However, their results failed to explain sudden negative changes in Bitcoin price. In this project we attempt to use SVMs from the field of finance and economics to model the exchange rate of Bitcoin. The PMCMC method will be used to estimate the SVM parameters.

1.3 Research Objective

This project begins with an examination of the use of stochastic volatility models in ﬁnancial applications. We attempt to use several types of stochastic volatility models to describe the evolution of the exchange rate of Bitcoin. We consider a basic stochastic volatility model and several extensions that include heavy tails, leverage, and covariates. The Bayesian approach with the particle Markov chain Monte Carlo (PMCMC) method is employed to estimate the model parameters. Simulation studies are conducted to assess the performance of particle MCMC and to compare with the traditional MCMC approach. We then apply the proposed method to the Bitcoin exchange rate data.

This project is focused on particle Markov chain Monte Carlo and the application of modeling Bitcoin exchange rates. Therefore the main research objectives are:

i Conduct simulation studies to evaluate the performance of PMCMC.

ii Explore several SVMs for modeling the Bitcoin exchange rate and estimate the model parameters using the proposed PMCMC method.

iii Select the most appropriate model for the Bitcoin application.

This project was motivated by a desire to understand the Bitcoin market and an interest in the extremely challenging problem of modeling and forecasting financial markets. The ultimate goal for this research is to find a way to model Bitcoin exchange rates in a way that can contribute to an effective trading strategy.

5 1.4 Thesis Organization

The rest of the thesis is organized as follows. A description of the stochastic volatility models used in this project are given in Chapter 2. A detailed description of the MCMC and PMCMC algorithms are presented in Chapter 3. Simulation studies are conducted and the performance of particle MCMC and the traditional MCMC approach are compared in Chapter 4. The proposed methods are applied to real Bitcoin exchange rate data in Chapter 5. Chapter 6 provides concluding remarks.

6 Chapter 2

Stochastic Volatility Models

2.1 Hidden Markov Models

A Markov process is a stochastic process where the future states depend only on the current state and not on the sequence of states that preceded it. This memoryless characteristic is known as the Markov property. In a hidden Markov model with unknown parameters

θ, the underlying (hidden) process X1:n is assumed to be a Markov process with initial distribution µθ(x1) and transitional distribution fθ(xt|xt−1). The observations Y1:n are assumed to be conditionally independent given the process X1:n and have the marginal distribution gθ(yt|xt). Figure 2.1 illustrates how the unobserved underlying process X1:n relates to the observed values Y1:n.

Figure 2.1: Illustration of a Hidden Markov Model.

7 In summary, a hidden Markov model is described as follows:

X1 ∼ µθ(x1), (2.1)

(Xt|Xt−1 = xt−1) ∼ fθ(xt|xt−1), (2.2)

(Yt|Xt = xt) ∼ gθ(yt|xt), (2.3) where Xt ∈ X and Yt ∈ Y. In the following sections we will consider several diﬀerent stochastic volatility models that can be written in the form of hidden Markov models.

2.2 Basic Stochastic Volatility Model

2.2.1 Basic SVM Version 1

First we will consider the following basic stochastic volatility model [9], with observations

Y1:n and underlying process X1:n. We have X = Y = R,

Xt = α·Xt−1 + σ·Vt,

Yt = β· exp{Xt/2}·Ut,

σ2 2 where X1∼N(0, 1−α2 ), Vt∼N(0, 1), Ut∼N(0, 1) and θ = (α, β, σ ) is unknown. Here, Ut and Vt are uncorrelated Gaussian white noise sequences. The scaling factor exp(Xt/2) speciﬁes the amount of volatility at time t, σ determines the volatility or log-volatility and α measures the autocorrelation [9].

Recall that if Z∼N(0, 1), then (a + bZ)∼N(a, b2). Therefore, the model can be described as follows: " # σ2 µ (x ) = N x ; 0, , θ 1 1 1 − α2

h 2i fθ(xt|xt−1) = N xt; αxt−1, σ ,

h 2 i gθ(yt|xt) = N yt; 0, β exp(xt) .

In the next section we will see another version of this basic stochastic volatility model that will also be used in this project.

8 2.2.2 Basic SVM Version 2

Consider the following alternative parameterization of the basic stochastic volatility model, with observations Y1:n and underlying process X1:n [26]. We have X = Y = R,

Xt = µx + α·(Xt−1 − µx) + σ·Vt, √ Yt = µy + γ· exp{Xt/2}·Ut,

σ2 2 where X1∼N(µx, 1−α2 ), Vt∼N(0, 1), Ut∼N(0, 1) and θ = (α, µx, µy, σ ) is unknown. Here Ut and Vt are uncorrelated Gaussian white noise sequences, and µx is the drift term in the state equation. The scaling factor exp(Xt/2) specifies the amount of volatility at time t, σ determines the volatility or log-volatility and α is the persistence parameter that measures the autocorrelation. We impose that |α| < 1 such that we have a stationary process with σ2 the initial distribution µ(x1) = N(x1; µx, 1−α2 ). To ensure identifiability, we fix γ to be 1, and leave µx unrestricted [26]. This model is just a re-parameterization of the basic model in Section 2.2.1, and if we fix µx = 0 and µy = 0, we can see that it is in the same form as the previous section.

In this case, the model can be described as follows:

" # σ2 µ (x ) = N x ; µ , , θ 1 1 x 1 − α2

h 2i fθ(xt|xt−1) = N xt; µx + α(xt−1 − µx), σ ,

gθ(yt|xt) = N [y; µy, exp(xt)] .

The basic stochastic volatility model can be too restrictive for many ﬁnancial time series [26]. In the following subsections, we will consider several extension of the basic stochastic volatility model.

2.3 Stochastic Volatility Model with Fat Tails

Consider the following stochastic volatility model with fat tails (heavy tails) [5, 13, 17]. We have observations Y1:n and underlying process X1:n with X = Y = R. The model is deﬁned

9 as follows:

Xt = µx + α·(Xt−1 − µx) + σ·Vt,

Yt = µy + exp{Xt/2}·Ut,

2 where Ut ∼ tν and θ = (α, µx, µy, ν, σ ) is unknown. Here tν denotes a Student-t distribution with ν > 2 degrees of freedom. This model can be described as follows:

" # σ2 µ (x ) = N x ; µ , , θ 1 1 x 1 − α2

h 2i fθ(xt|xt−1) = N xt; µx + α(xt−1 − µx), σ ,

gθ(yt|xt) = tν [y; µy, exp(xt), ν] .

The stochastic volatility model with fat tails can accommodate a wide range of kurtosis and is particularly important when dealing with extreme observations or outliers [24].

2.4 Stochastic Volatility Model with Leverage Eﬀect

Consider the following stochastic volatility model with leverage eﬀect [26]. We have observations Y1:n and underlying process X1:n with X = Y = R. The model is deﬁned as follows:

Xt = µx + α·(Xt−1 − µx) + σ·Vt,

Yt = µy + exp{Xt/2}·Ut,

p 2 where Ut and Vt are correlated. We write Ut = ρVt + (1 − ρ )·ξt, where ξt ∼ N(0, 1), and 2 they are uncorrelated with Vt. In this way, Yt|Vt ∼ N µˇt, σˇt , where µˇt = ρ· exp(xt/2)·Vt, 2 2 σˇt = (1 − ρ )· exp(xt) and Xt follows the one in the basic stochastic volatility model. We −1 have Vt = σ ·[(Xt − µx) − α·(Xt−1 − µx)].

In order to use similar algorithms for HMM to estimate the hidden process, we deﬁne the T state as Xt = (Xt+1,Xt) . In other words, the stochastic volatility model with leverage can be expressed in the form of a non-linear, non-Gaussian state space model with

−1 2 gθ(yt|xt) = N y; ρ· exp(xt/2)·σ ·[(xt − µx) − α·(xt−1 − µx)], (1 − ρ )· exp(xt) .

10 The state transition function is:

X ! µ (1 − α) ! α 0 ! X ! V ! t+1 = x + t + t . Xt 0 1 0 Xt−1 0

That is,

fθ(xt|xt−1) = N2(xt; µx + Axxt−1, Σx),

! 2 ! T α 0 σ 0 where µ = (µx(1 − α), 0) , Ax = , and Σx = . x 1 0 0 0

2.5 Stochastic Volatility Model with Covariate Eﬀects

Finally we consider the following stochastic volatility model which allows for covariate eﬀects. We have observations {Yn} and underlying process {Xn} with X = Y = R. The model is deﬁned as follows:

Xt = µx + α·(Xt−1 − µx) + σ·Vt, 0 Yt = Wt η + exp{Xt/2}·Ut, where Wt is a q × 1 vector of covariates, η is the associated q × 1 vector of parameters and 2 θ = (α, µx, η, σ ). This model can be described by the following functions:

! σ2 µ (x ) = N x ; µ , , θ 1 1 x 1 − α2

2 fθ(xt|xt−1) = N xt; µx + α(xt−1 − µx), σ ,

0 gθ(yt|xt) = N y; Wt η, exp(xt) .

2.6 Chapter Summary

In this section the concept of hidden Markov models was introduced and we looked at several types of stochastic volatility models that can be expressed in this form. We considered a basic SVM and several extensions that included heavy tails, leverage and covariate eﬀect. Particle Markov chain Monte Carlo algorithms will be applied to each of these models to estimate the parameter values. The models will be ﬁt to simulated data sets and Bitcoin

11 exchange rate data. In the next section we will outline the details of the particle MCMC method and show how it can be used to estimate the model parameters.

12 Chapter 3

Bayesian Inference for Stochastic Volatility Models

3.1 Bayesian Inference

Consider a stochastic volatility model with hidden process X1:n, observations Y1:n and a ﬁxed vector of parameters θ. In the Bayesian framework, equations (2.1) and (2.2) deﬁne the prior distribution of the hidden process as follows:

n Y pθ(x1:n) = µθ(x1) fθ(xt|xt−1), t=2 and equation (2.3) deﬁnes the following likelihood function,

n Y pθ(y1:n|x1:n) = gθ(yt|xt). t=1

Consequently, given θ, the posterior distribution of X1:n given the observed data Y1:n is:

pθ(x1:n, y1:n) pθ(x1:n|y1:n) = , (3.1) pθ(y1:n) where

n n Y Y pθ(x1:n, y1:n) = pθ(x1:n)pθ(y1:n|x1:n) = µθ(x1) fθ(xt|xt−1) gθ(yt|xt), t=2 t=1 and Z pθ(y1:n) = pθ(x1:n, y1:n)dx1:n. (3.2)

13 When the parameters θ are unknown, the posterior distribution of θ and X1:n is:

p(θ, x1:n|y1:n) ∝ pθ(x1:n, y1:n)·p(θ), where p(θ) is the prior for θ.

For the simplest cases, the ﬁnite state-space hidden Markov models, the integral in equation (3.2) can be computed exactly. For linear Gaussian models, the posterior distribution pθ(x1:n|y1:n) is also a Gaussian distribution whose mean and covariance can be computed using the Kalman Filter [9]. However, it is impossible to compute the integral in equation (3.2) in a closed form for most non-linear non-Gaussian models. Unfortunately, the stochastic volatility models of interest in this project belong to the latter case and we have to use numerical approximations.

3.2 Monte Carlo Integration

Monte Carlo Integration is a simulation technique that uses independent draws from a distribution to approximate integrals rather than solving them analytically. The distribution that is being approximated is known as the target distribution, denoted as πn(x1:n). In our case the target distribution is the posterior, so πn(x1:n) = pθ(x1:n|y1:n). The Monte Carlo k method involves sampling N independent draws, X1:n∼ πn(x1:n), k = 1,...,N, and then approximating πn(x1:n) by the empirical measure,

N 1 X πn(x1:n) = δXk (x1:n), b N 1:n k=1

where δx0 (x) denotes the Dirac delta mass located at x0 [3]. Consider the integral Z I = m(x1:n)πn(x1:n)dx1:n, where m(x1:n) is some function of x1:n. Then the integral I can be approximated using Monte Carlo integration by N 1 X Ib = m(Xk ). (3.3) N 1:n k=1

By the strong law of large numbers Ib−→I as N−→∞ [21]. However, in some cases it is not possible to generate independent draws from the target distribution. For example, suppose we want to sample from the posterior distribution pθ(x1:n|y1:n), but the normalizing constant pθ(y1:n) is unknown. If we can sample slightly dependent draws from the posterior

14 distribution using a Markov chain, then it is still possible to estimate the integrals or quantities of interest using equation (3.3).

3.3 Posterior Inference via Markov Chain Monte Carlo

3.3.1 Markov Chain Monte Carlo (MCMC)

A Markov chain is a stochastic process where the future states depend only on the current state and not on the sequence of states that preceded it. For example, let zt be the state of a stochastic process at time t. Then, the future state zt+1 depends only on the current state zt. In other words, a stochastic process is considered a Markov chain if it satisﬁes the Markov property:

p(zt+1|z1, z2, ..., zt) = p(zt+1|zt). (3.4)

Under mild conditions a Markov chain will eventually converge to what is known as a stationary or limiting distribution [21]. If we can create a Markov chain whose stationary distribution is the targeted posterior distribution pθ(x1:n|y1:n), then the chain can be run to get draws that are approximately from pθ(x1:n|y1:n) once it has converged.

A Markov chain should converge to the desired stationary distribution regardless of the starting point, however the time it takes to converge will vary [21]. Therefore, it is common practice to discard a certain number of the ﬁrst draws, a process known as a burn-in. This helps to assure that the draws are closer to the stationary distribution and less dependent on the starting point.

Once the Markov chain has converged the draws will be approximately the same as if they were drawn from the posterior pθ(x1:n|y1:n). However, these draws will not be independent, which is required for Monte Carlo Integration. Fortunately, the Ergodic Theorem allows the dependence between draws of the Markov chain to be ignored [21].

In summary MCMC is a simulation technique that involves taking draws from a Markov chain that has the desired posterior distribution as its stationary distribution. In general, there are two MCMC algorithms that are most commonly used: the Gibbs sampler and the Metropolis-Hastings algorithm. In the next section, we will outline the Gibbs sampling method used in this project.

15 3.3.2 Gibbs Sampling

For this project a basic MCMC algorithm using the Gibbs sampling method was imple- mented as a comparison to PMCMC. Consider the basic stochastic volatility model from the Section 2.2.1, a Gibbs MCMC algorithm can be used to estimate the model parameters θ = (α, β, σ). Let t = 1, ..., n be the number of time steps, let m be the number of iterations and θ(i) be the parameters values at the ith MCMC iteration (i = 1, ..., m).

First starting values must be selected the model parameters α, β and σ. The prior distribu- 2 tions for the parameters are: xt ∼ N(0, 1), α ∼ Uniform(−1, 1), β ∼ IG(ν0/2, γ0/2) and 2 σ ∼ IG(ν0/2, γ0/2), where IG(·, ·) is the inverse Gamma distribution with the shape and rate parameters. Then the posterior distribution of interest is:

n 2 2 Y h 2 i 2 π(α, β , σ , x1:n|y1:n) ∝ φ(yt; 0, β ·exp(xt)) ·φ(x1; 0, σ ) t=1 n Y h 2 i · φ(xt; αxt−1, σ ) ·I[α(−1, 1)] t=2 2 2 ·IG(σ ; ν0/2, γ0/2)·IG(β ; ν0/2, γ0/2).

In order to implement Gibbs sampling we must ﬁrst calculate the full conditional distributions for each parameter. A full conditional distribution is deﬁned as the distribution of a parameter conditional on the known information and all other parameters. The full conditional distributions for this model are as follows:

"Pn 2 # t=2 xt−1xt σ α|· ∼ N Pn 2 , Pn 2 ·I[α(−1, 1)], (3.5) t=2 xt−1 t=2 xt−1

" !# n + ν 1 n y2 β2|· ∼ IG 0 , γ + X t , (3.6) 2 2 0 exp(t ) t=1 n

" !# n + ν 1 n σ2|· ∼ IG 0 , x2 + γ + X(x − αx )2 . (3.7) 2 2 1 0 t t−1 t=2

Then, assuming x0 = 0, the density of the full conditional distribution for xt is:

2 ! 1/2 2 ! (xt−1 + xt+1) σ 1 −yt p(xt|·) ∝ φ xt; α 2 , 2 · 2 exp 2 , (3.8) 1 + α 1 + α β exp(xt) 2β exp(xt) for t = 1, 2, ..., n − 1. At t=n,

16 2 2 p(xt|·) ∝ φ(xt; αxt−1, σ )·φ(yt; 0, β exp(xt)). (3.9)

However, this distribution is non-standard and cannot be sampled from directly. Therefore, xt|· is proposed using an accept-reject sampler by sampling from:

h 2i ∗ q(xt) ∝ φ xt; mt, σt ·g (yt|xt, β) " # h i −x y2 ∝ φ x ; m , σ2 · exp t + t ·x · exp(m ) t t t 2 2β2 t t " ! # σ2 y2 ∝ N x ; m + t t · exp(−m ) − 1 , σ2 , t t 2 β2 t t

α(x + x ) σ2 where m = t−1 t+1 and σ2 = for t = 1, 2, ..., n−1, m = α·x and σ2 = σ2 t 1 + σ2 t 1 + σ2 t t−1 t " # x y2 for t = n, and g∗(y |x , β) ∝ exp − t + t ·x · exp(m ) . t t 2 2β2 t t

At each iteration, we propose xt from q(xt) until one of these is accepted with probability:

g(yt|xt, β) ∗ , g (yt|xt, β)

2 where g(yt|xt, β) = φ yt; 0, β exp(xt) . The steps of the Gibbs sampling algorithm are outlined in Table 3.1.

Table 3.1: Gibbs Sampler Algorithm.

(0) (0) (0) 2(0) 2(0) Draw initial parameter values x1:n and θ = α , β , σ from their prior distributions.

For i = 1, ..., m:

• Draw α(i) from its full conditional using equation (3.5).

• Draw β2(i) from its full conditional using equation (3.6).

• Draw σ2(i) from its full conditional using equation (3.7).

(i) • Update x1:n using the accept-reject sampler.

In cases it is not possible to calculate all the full conditionals necessary to implement the Gibbs sampling and a diﬀerent method such as the Metropolis-Hastings algorithm or the accept-reject sampler must be used. A Gibbs sampling algorithm is also very model speciﬁc

17 because the full conditionals must be recalculated for diﬀerent models. For this reason an MCMC algorithm with Gibbs sampling was only developed for the basic stochastic volatility model.

3.4 Posterior Inference via Sequential Monte Carlo (SMC)

The purpose of this section is to introduce sequential Monte Carlo methods which are an important part of the particle MCMC algorithm. This section focuses on estimating the hidden process X1:n and it is assumed that the parameters, θ are ﬁxed. We omit θ from the general notation for simplicity.

3.4.1 Importance Sampling (IS)

As previously mentioned a problem with Monte Carlo integration is that it might not be possible to sample directly from the target distribution. Importance sampling (IS) is another technique to address this problem. Let πn(x1:n) be the target distribution and γn(x1:n) be the unnormalized target distribution. Then

γn(x1:n) πn(x1:n) = , Zn R where Zn = γn(x1:n) dx1:n. Recall, that in this case the posterior pθ(x1:n|y1:n) is our target distribution. Therefore, we have:

pθ(x1:n, y1:n) pθ(y1:n|x1:n)·pθ(x1:n) πn(x1:n) = pθ(x1:n|y1:n) = = . pθ(y1:n) pθ(y1:n)

In order to implement importance sampling we must select an importance distribution qn(x1:n) from which it is easy to draw samples and use the following IS identities.

wn(x1:n)qn(x1:n) πn(x1:n) = , (3.10) Zn where Z Zn = wn(x1:n)qn(x1:n)dx1:n, (3.11) and the unnormalized weight function wn(x1:n) is deﬁned as

γn(x1:n) wn(x1:n) = . qn(x1:n)

18 k Then draw N independent samples X1:n∼qn(x1:n), which are commonly referred to as particles, and use the Monte Carlo approximation of qn(x1:n) in equations 3.10 and 3.11 to obtain the following estimate for the target distribution

N X k πn(x1:n) = W δ k (x1:n), (3.12) b n X1:n k=1 N 1 X Zb = w (Xk ), (3.13) n N n 1:n i=k

k where Wn are the normalized weights, which are deﬁned as

w (Xk ) W k = n 1:n . n PN j j=1 wn(X1:n)

In summary, the basic idea of importance sampling is to draw samples from the importance distribution and re-weight them using the importance weights to approximate the target distribution.

3.4.2 Sequential Importance Sampling (SIS)

Another problem with Monte Carlo methods is that even if it is possible to sample from the target distribution the computational complexity increases at least linearly with n. This problem can be addressed by using sequential importance sampling (SIS) which has a ﬁxed computational complexity at each time step [9]. Sequential importance sampling is a special case of importance sampling where the importance distribution, qn(x1:n) must be of the following form:

qn(x1:n) = qn−1(x1:n−1)qn(xn|x1:n−1) n Y = q1(x1) qt(xt|x1:t−1). t=2

i i In order to obtain N draws X1:n∼qn(x1:n) at time n, sample X1∼q1(x1) at time 1, then sam- i i ple Xk∼qk(xk|X1:k−1) at time k for k = 2, ..., n. The unnormalized weights are computed recursively as:

γn(x1:n) wn(x1:n) = qn(x1:n) γ (x ) γ (x ) = n−1 1:n−1 · n 1:n , qn−1(x1:n−1) γn−1(x1:n−1)qn(xn|x1:n−1)

19 which can be written in the form:

wn(x1:n) = wn−1(x1:n−1)·αn(x1:n) n Y = w1(x1) αk(x1:k), k=2 where αn(x1:n) is the incremental importance weight and is given by

γn(x1:n) αn(x1:n) = . γn−1(x1:n−1)qn(xn|x1:n−1)

In the case of hidden Markov Models wn(x1:n) can be simpliﬁed by selecting an importance distribution such that

q(xn|x1:n−1) = pθ(xn|xn−1) = f(xn|xn−1), n Y qn(x1:n) = pθ(x1:n) = µ(x1) f(xk|xk−1). k=2

Then wn(xn) simpliﬁes to

γn−1(x1:n−1) γn(x1:n) wn(x1:n) = · qn−1(x1:n−1) γn−1(x1:n−1)qn(xn|x1:n−1)

p(x )p(y |x ) p(x )p(y |x ) = 1:n−1 1:n−1 1:n−1 · 1:n 1:n 1:n p(x1:n−1) p(xn|xn−1)p(x1:n−1)p(y1:n−1|x1:n−1)

n−1 Y = g(yk|xk)·g(yn|xn) k=1

wn(x1:n) = wn−1(x1:n−1)·g(yn|xn), where αn(x1:n) = g(yn|xn) is the incremental importance weight.

The sequential importance sampling algorithm is outlined in Table 3.2. At any time n we can compute the estimates πbn(x1:n) and Zbn from the equations 3.12 and 3.13, respectively.

20 Table 3.2: Sequential Importance Sampling Algorithm.

At time t = 1

For k = 1, ..., N:

k • Sample X1 ∼ q1(x1). k k • Set the unnormalized weights to w1(X1 ) = g(y1|X1 ). w (Xk) • Compute the normalized weights: W k = 1 1 . 1 PN j j=1 w1(X1)

At time t = 2, ..., n

For k = 1, ..., N:

k k • Sample Xt ∼ qt(xt|X1:t−1). k k k • Compute the unnormalized weights: wt(X1:t) = wt−1(X1:t−1)·g(yt|Xt ). w (Xk ) • Compute the normalized weights: W k = t 1:t . t PN j j=1 wt(X1:t)

A sensibly chosen importance distribution will allow the time required to sample from qn(xn|x1:n−1) and to compute αn(x1:n) to be independent of n [9]. However, the variance of the estimates increase exponentially with n, which is a major drawback of the SIS method [9].

3.4.3 Sequential Monte Carlo (SMC)

Sequential Monte Carlo is essentially sequential importance sampling that implements a resampling technique to address the problem of the increasing variance of the estimates. Resampling refers to sampling from an approximation which was itself obtained by sampling [9]. In this case we are resampling from the SIS approximation πbn(x1:n) which is equivalent k k to selecting X1:n with probability Wn . The SMC algorithm is very similar to SIS except that resampling is performed at each time step. The resampling step leads to a high probability of removing the particles with low weights. In the sequential framework this means that particles with low weights are not carried forward and computational eﬀorts can be focused on regions with high probability mass [9]. The SMC algorithm is summarized in table 3.3.

21 Table 3.3: Sequential Monte Carlo Algorithm.

At time t = 1

For k = 1, ..., N:

k k • Draw X1 ∼µ(x1). k k • Set w1(X1 ) = g(y1|X1 ). w (Xk) • Normalize the importance weights: W k = 1 1 . 1 PN j j=1 w1(X1)

At time t = 2, ..., n

N n k o k 1 • Resample N particles with probabilities W and for k = 1, ..., N set wt−1(X ) = . 1:t−1 k=1 1:t−1 N For k = 1, ..., N:

k k k • Draw Xt ∼f(Xt |Xt−1). k k k • Compute the importance weights: wt(X1:t) = wt−1(X1:t−1)·g(yt|Xt ). w (Xk ) • Normalize the importance weights: W k = t 1:t . 1:t PN j j=1 w1(X1:t)

The methods discussed in this section will only provide an approximation for the hidden process Xn in a stochastic volatility model. In the next section we will see how to estimate the model parameters, θ, using particle Markov chain Monte Carlo.

3.5 Particle Markov Chain Monte Carlo (PMCMC)

Particle Markov chain Monte Carlo (PMCMC) uses the sequential Monte Carlo method within the MCMC algorithm. As with basic MCMC, both the Gibbs and Metropolis- Hastings methods can be used in PMCMC. In this project we use the particle marginal Metropolis-Hasting (PMMH) method [2]. Our goal is to provide an estimate for the stochastic volatility model parameters, θ, and the posterior distribution, pθ(x1:n|y1:n).

Let m be the number of MCMC iterations and θ(i) be the parameter values at the ith iteration (i = 1, ..., m). We start by selecting arbitrary initial values for the parameters, θ(0). Then we propose new parameter values θ∗ from a proposal or jumping distribution h(θ∗|θ(i−1)). We can also select a prior distribution for the parameters, p(θ), if we have some prior knowledge or intuition of what the values might be.

22 The Metropolis-Hastings ratio, denoted by r, is the probability of accepting the new proposed parameter values. It is deﬁned as follows:

(i−1) ∗ ∗ p ∗ (y )·h(θ |θ )·p(θ ) r = θ 1:n . ∗ (i−1) (i−1) pθ(i−1) (y1:n)·h(θ |θ )·p(θ )

Since r is an acceptance probability, if r > 1 we set r = 1. The marginal likelihoods pθ∗ (y1:n) and pθ(i−1) (y1:n) are estimated using sequential Monte Carlo as follows:

! n 1 N p (y ) = Y X w (Xk ) . (3.14) bθ 1:n N n 1:t t=1 k=1

At each iteration it must be decided whether or not to accept the proposed parameter values. Let u be a value that is drawn from a Uniform(0,1) distribution. Then the parameters are updated as follows:

• If u ≤ r, accept θ∗ and set θ(i) = θ∗.

• If u > r, reject θ∗ and set θ(i) = θ(i−1).

The particle Markov chain Monte Carlo algorithm is summarized in Table 3.4. Recall that n is the number of time steps (t = 1, ..., n), N is the number of particles (k = 1, ..., N) and m is the number of MCMC iterations (i = 1, ..., m).

23 Table 3.4: Particle MCMC Algorithm.

• Select initial parameter values θ(0).

• Run SMC algorithm with θ(0) to estimate the marginal likelihood: n Q 1 PN k pbθ(0) (y1:n) = N k=1 wt(X1:t) . t=1 For i = 1, ..., m:

• Propose new parameter values, θ∗.

• Run SMC algorithm with θ∗ to estimate the marginal likelihood: n Q 1 PN k pbθ∗ (y1:n) = N k=1 wt(X1:t) . t=1 (i−1) ∗ ∗ pˆ ∗ (y )·h(θ |θ )·p(θ ) • Calculate the Metropolis-Hastings Ratio: r = θ 1:n . ∗ (i−1) (i−1) pˆθ(i−1) (y1:n)·h(θ |θ )·p(θ ) • Draw u from a Uniform(0,1) distribution. Then update θ(i) as follows:

If u ≤ r, θ(i) = θ∗ If u > r, θ(i) = θ(i−1).

Alternatively, the log of the MH ratio can be used in the PMCMC algorithm. In this case we calculate log(r) as follows:

" # " (i−1) ∗ # ∗ p ∗ (y ) h(θ |θ ) p(θ ) log(r) = log θ 1:n + log + log , ∗ (i−1) (i−1) pθ(i−1) (y1:n) h(θ |θ ) p(θ )

h (i−1) ∗ i h ∗ i where log h(θ |θ ) is the log proposal ratio and log p(θ ) is the log prior ratio. Then h(θ∗|θ(i−1)) p(θ(i−1)) we use a similar method to update the parameters, except we comparing log(r) to log(u) to decide whether we will accept or reject θ∗. For this project we use the log MH ratio to update the parameters in the PMCMC algorithm.

3.6 Model Comparison

As previously mentioned, there are several diﬀerent stochastic volatility models that are used in this project. Using the marginal likelihood estimates from the PMCMC algorithm the deviance information criterion (DIC) [28] was calculated for each model in order to compare their eﬀectiveness.

24 n om Let θ¯ be the posterior mean or median of θ(i) and deﬁne D(θ¯) as: i=1 ¯ D(θ) = −2 log [pθ¯(y1:n)] , where the marginal likelihood, pθ¯(y1:n), is approximated using equation (3.14) by running the SMC algorithm with the posterior mean θ¯. Then the DIC is deﬁned as:

¯ DIC = D(θ) + 2pD, where pD is a penalty term that describes the complexity of the model and penalizes models with more parameters [26]. The penalty term is given by:

¯ pD = D(θ) − D(θ), where D(θ) is approximated by:

m 1 X D(θ) ≈ −2 log [p (i) (y )] , m θ 1:n i=1 where the marginal likelihood, pθ(i) (y1:n), is approximated using equation (3.14) as a byprod- uct of the SMC algorithm with the parameter θ(i). The best model will have the smallest DIC.

3.7 Chapter Summary

This chapter introduced the Bayesian approach to estimating stochastic volatility model parameters and posterior distributions. We saw how some of the required integrals cannot be calculated analytically and explored methods of estimating integrals using simulation techniques.

Monte Carlo integration can be used to approximate integrals instead of solving them analytically when it is possible to sample independent draws from the target distribution. Markov chain Monte Carlo allows us to make similar approximations using slightly dependent draws from a Markov chain that has converged to the target distribution.

Importance sampling and sequential importance sampling methods were introduced as al- ternatives to MCMC for when we cannot sample directly from the target distribution. This lead to the sequential Monte Carlo method by adding a resampling step to the SIS algorithm. However, SMC only provided an estimate for the hidden process X1:n, so the particle MCMC method was introduced to estimate the stochastic volatility model parameters.

25 The particle marginal Metropolis-Hastings (PMMH) method is used to provide an estimate for the stochastic volatility model parameters, θ, and the posterior distribution, pθ(x1:n|y1:n). The method of proposing and accepting or rejecting new parameter values was outlined and it was shown how the SMC method used to estimate the marginal likelihoods required for calculating the Metropolis-Hastings Ratio. Finally, we introduced the deviance information criterion (DIC) which was used for model comparison.

In the next section simulated data sets will be generated for each of the stochastic volatility models introduced in Chapter 2 and the particle MCMC method will be used to estimate the model parameters.

26 Chapter 4

Simulation Studies

4.1 Gibbs Sampler and PMCMC for Basic SVM Version 1

4.1.1 PMCMC

The basic stochastic volatility model from Section 2.2.1 was used to create the simulated data set shown in Figure 4.3. To generate this data set the model parameters were set to α = 0.9, β2 = 1.42 and σ2 = 0.58 for n = 500 time steps. In this section both the Gibbs sampler and PMCMC methods were used to estimate the model parameters based on the simulated observations, Y1:n.

The PMCMC algorithm was run using N = 10000 particles and m = 10000 MCMC iterations. The initial parameter values were set to α = 0.92, β2 = 1 and σ2 = 0.5. As previously mentioned the Markov chain should converge regardless of the starting values, so the starting point will only aﬀect the time required for the chain to converge. We assumed the following prior distributions for the model parameters: α ∼ Uniform(−1, 1), β2 ∼ IG(0.01, 0.01) and σ2 ∼ IG(0.01, 0.01). The parameters were updated using a random walk with the following proposal distributions:

∗ (i−1) 2 α ∼ TN α , τα, −1, 1 ,

2∗ 2 (i−1) 2 log β ∼ N log β , τβ2 ,

2∗ 2 (i−1) 2 log σ ∼ N log σ , τσ2 ,

27 where α∗ follows a truncated normal distribution over the range (−1, 1) and β2 and σ2 follow a log normal distribution. The scale parameters were set to τα = 0.01, τβ2 = 0.3 and

τσ2 = 0.3.

Figure 4.1 shows the trace plots of the parameter posterior draws from each iteration of the PMCMC algorithm. A burn-in of 2500 iterations was used to ensure that the Markov chain had converged. The histograms of the posterior draws, after the burn-in period, are displayed in Figure 4.2. The acceptance rate for the proposed parameters was 0.2514. 7 6 0.95 5 2 4 0.90 α β 3 0.85 2 1

0.80 0 2000 4000 6000 8000 10000 0 2000 4000 6000 8000 10000 MCMC Iterations MCMC Iterations 1.2 1.0 2 0.8 σ 0.6 0.4

0 2000 4000 6000 8000 10000 MCMC Iterations

2 Figure 4.1: Basic SVM Version 1 trace plots of α, µx, µy and σ resulting from PMCMC for simulated data.

28 2500 1000 600 1500 Frequency Frequency 500 200 0 0 0.80 0.85 0.90 0.95 1 2 3 4 5 6 α β2 1000 600 Frequency 200 0 0.4 0.6 0.8 1.0 1.2 σ2

2 Figure 4.2: Basic SVM Version 1 histograms of α, µx, µy and σ resulting from PMCMC for simulated data.

The posterior means of the parameters from PMCMC were used to generate the estimated X values shown in Figure 4.3.

● ● Simulated Hidden Process● (X)

10 ● Simulated Observations (Y) Estimated Hidden Process (X) ● ● ● ● ● ● ● ● ● ● ● ● ● ●

5 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ●● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●●●●●● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●●●●●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ●●● ● ● ●●● ● ● ● ●●● ●●●●● ● 0 ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●●● ● ● ●● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● −5 ● ● ● ● ●

●

● −10

●

● −15

0 100 200 300 400 500 Time

Figure 4.3: Basic SVM Version 1 simulated data and PMCMC estimates, with observations Y1:n and hidden process X1:n for simulated data.

29 4.1.2 Gibbs Sampler

Similarly, the Gibbs algorithm, outlined in Section 3.3.2, was run using m = 10000 MCMC iterations and the same initial parameter values. Figure 4.4 shows the trace plots of the parameter posterior draws from each iteration of the Gibbs Sampler algorithm. The histograms of the posterior draws, after a burn-in of 2500 iterations, are displayed in Figure 4.5. 25 0.8 20 15 2 α β 0.4 10 5 0.0 0 0 2000 4000 6000 8000 10000 0 2000 4000 6000 8000 10000 MCMC Iterations MCMC Iterations 1.2 2 σ 0.8 0.4

0 2000 4000 6000 8000 10000 MCMC Iterations

2 Figure 4.4: Basic SVM Version 1 trace plots of α, µx, µy and σ resulting from Gibbs Sampler for simulated data.

30 1200 1500 800 1000 Frequency Frequency 400 500 0 0 0.75 0.80 0.85 0.90 0.95 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 α β2 1000 600 Frequency 200 0 0.4 0.6 0.8 1.0 1.2 σ2

2 Figure 4.5: Basic SVM Version 1 histograms of α, µx, µy and σ resulting from Gibbs Sampler for simulated data.

4.1.3 Comparison of PMCMC and Gibbs Sampler

Both PMCMC and Gibbs Sampler were used to estimate the model parameters based on the simulated observations.The true parameter values and a summary of the parameter posterior distributions are given in Table 4.1.

Table 4.1: Summary of Basic SVM Version 1 parameter posterior distributions resulting from PMCMC and Gibbs Sampler for simulated data.

Gibbs Sampler PMCMC Parameters True Value Mean 95% CI Mean 95% CI α 0.9 0.875 (0.823, 0.923) 0.891 (0.847, 0.935) β2 1.42 1.875 (1.208, 2.731) 1.98 (1.134, 3.248) σ2 0.58 0.694 (0.502, 0.933) 0.644 (0.458, 0.882)

31 4.2 PMCMC for Basic SVM Version 2

The basic stochastic volatility model from Section 2.2.2 was used to create the simulated data set shown in Figure 4.8. To generate this data set the model parameters were set to 2 α = 0.88, µx = 3.5, µy = 0.35 and σ = 0.58 for n = 500 time steps.

In this section the PMCMC method was used to estimate the model parameters based on the simulated observations, Y1:n. The PMCMC algorithm was run using N = 10000 particles and m = 10000 MCMC iterations. The initial parameter values were set to α = 0.9, 2 µx = 0, µy = 0 and σ = 0.5. We assumed the following prior distributions for the model 2 parameters: α ∼ TN(0.9, 0.1, −1, 1), µx ∼ N(0, 10), µy ∼ N(0, 10) and σ ∼ IG(1, 1). The parameters were updated using a random walk with the following proposal distributions:

∗ (i−1) 2 α ∼ TN α , τα, −1, 1 ,

∗ (i−1) 2 µx ∼ N µx , τµx ,

∗ (i−1) 2 µy ∼ N µy , τµy ,

2∗ 2 (i−1) 2 log σ ∼ N log σ , τσ2 .

The scale parameters were set to τα = 0.1, τµx = 0.32, τµy = 0.03 and τσ2 = 0.1.

Figure 4.6 shows the trace plots of the parameter posterior draws from each iteration of the PMCMC algorithm. A burn-in of 2500 iterations was used to ensure that the Markov chain had converged. The histograms of the posterior draws, after the burn-in period, are displayed in Figure 4.7.

32 4 3 0.95 2 x α µ 0.85 1 0 0.75 −1 0 2000 4000 6000 8000 10000 0 2000 4000 6000 8000 10000 MCMC Iterations MCMC Iterations 0.8 0.6 0.2 0.6 y 2 µ σ −0.2 0.4 −0.6 0.2 0 2000 4000 6000 8000 10000 0 2000 4000 6000 8000 10000 MCMC Iterations MCMC Iterations

2 Figure 4.6: Basic SVM Version 2 trace plots of α, µx, µy and σ resulting from PMCMC for simulated data. 1500 1500 1000 1000 500 Frequency Frequency 500 0 0 0.75 0.80 0.85 0.90 0.95 3.0 3.5 4.0 α µx 1500 1200 1000 800 500 Frequency Frequency 400 0 0 −0.6 −0.2 0.2 0.4 0.6 0.2 0.3 0.4 0.5 0.6 0.7 0.8 2 µy σ

2 Figure 4.7: Basic SVM Version 2 histograms of α, µx, µy and σ resulting from PMCMC for simulated data.

The acceptance rate for the proposed parameters was 0.1598. Table 4.2 gives a summary of the parameter posterior distributions resulting from the PMCMC algorithm.

33 Table 4.2: Summary of Basic SVM Version 2 parameter posterior distributions resulting from PMCMC for simulated data.

Parameter True Value Mean 95% CI α 0.88 0.853 (0.799, 0.905)

µx 3.5 3.467 (3.097, 3.863)

µy 0.35 -0.032 (-0.374, 0.294) σ2 0.58 0.452 (0.305, 0.642)

The posterior means of the parameters were used to generate the estimated X values shown in Figure 4.8.

●

40 Simulated Hidden Process (X) ● Simulated Observations (Y) ●

Estimated Hidden Process (X) ● ●

● ●

● ● ● ● ● ● ● ● ●

20 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ●●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● 0 ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ●● ●● ● ● ●● ● ● ● ●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● −20 ● ● ●

●●

●

−40 0 100 200 300 400 500 Time

Figure 4.8: Basic SVM Version 2 simulated data and PMCMC estimates, with observations Y1:n and hidden process X1:n for simulated data.

4.3 PMCMC for SVM with Fat Tails

The stochastic volatility model with fat tails from Section 2.3 was used to create the simulated data set shown in Figure 4.11. To generate this data set the model parameters were 2 set to α = 0.91, µx = 3.4, df = 10 and σ = 0.32 for n = 500 time steps.

The PMCMC method was used to estimate the model parameters based on the simulated observations, Y1:n. The PMCMC algorithm was run using N = 10000 particles and m = 10000 MCMC iterations. The initial parameter values were set to α = 0.9, µx = 0, df = 10 and σ2 = 0.4. We assumed the following prior distributions for the model parame-

34 2 ters: α ∼ TN(0.9, 10, −1, 1), µx ∼ N(0, 10), df ∼ TN(20, 10, 2, ∞) and σ ∼ IG(1, 1). The parameters were updated using a random walk with the following proposal distributions:

∗ (i−1) 2 α ∼ TN α , τα, −1, 1 ,

∗ (i−1) 2 µx ∼ N µx , τµx ,

∗ (i−1) 2 df ∼ TN df , τdf , 2, ∞ ,

2∗ 2 (i−1) 2 log σ ∼ N log σ , τσ2 .

The scale parameters were set to τα = 0.05, τµx = 0.32, τdf = 3 and τσ2 = 0.1.

Figure 4.9 shows the trace plots of the parameter posterior draws from each iteration of the PMCMC algorithm. The histograms of the posterior draws, after a burn-in period of 2500 iterations, are displayed in Figure 4.10. 4 0.95 3 x α µ 2 0.85 1 0.75 0

0 5000 10000 15000 20000 0 5000 10000 15000 20000 MCMC Iterations MCMC Iterations 40 0.8 30 0.6 2 df σ 20 0.4 10 0.2

0 5000 10000 15000 20000 0 5000 10000 15000 20000 MCMC Iterations MCMC Iterations

2 Figure 4.9: SVM Fat Tails trace plots of α, µx, df and σ resulting from PMCMC for simulated data.

35 3000 4000 2000 2000 Frequency Frequency 1000 0 0 0.70 0.75 0.80 0.85 0.90 0.95 1.00 2.0 2.5 3.0 3.5 4.0 4.5 α µx 3000 1500 2000 1000 Frequency Frequency 500 1000 0 0 10 20 30 40 0.2 0.4 0.6 0.8 df σ2

2 Figure 4.10: SVM Fat Tails histograms of α, µx, df and σ resulting from PMCMC for simulated data.

The acceptance rate for the proposed parameters was 0.3514. Table 4.3 gives a summary of the parameter posterior distributions resulting from the PMCMC algorithm.

Table 4.3: Summary of SVM Fat Tails parameter posterior distributions resulting from PMCMC for simulated data.

Parameter True Value Mean 95% CI α 0.91 0.873 (0.805, 0.929)

µx 3.4 3.063 (2.609, 3.498) df 10 17.54 (6.157, 32.635) σ2 0.32 0.423 (0.263, 0.637)

The posterior means of the parameters were used to generate the estimated X values shown in Figure 4.11.

36 Simulated Hidden Process (X) ● ● Simulated Observations (Y)

60 Estimated Hidden Process (X) 40

●

● ●

●

20 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●●● ●● ● ● ●●● ●●● ●● ● ●● ● ●●● ● ● ●●●● ● ●●● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ●●● ●● ●● ● ● ● ● ● ● ● ● ● ●●●●●● ● ● ●●● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ● 0 ●●● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ●● ●● ●● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ●● ● ●●● ●● ●● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

−20 ●

●

● −40 0 100 200 300 400 500 Time

Figure 4.11: SVM Fat Tails simulated data and PMCMC estimates, with observations Y1:n and hidden process X1:n for simulated data.

4.4 PMCMC for SVM with Leverage Eﬀect

The stochastic volatility model with leverage eﬀect from Section 2.4 was used to create the simulated data set shown in Figure 4.14. To generate this data set the model parameters 2 were set to α = 0.89, µx = 3.6, ρ = −0.05 and σ = 0.47 for n = 500 time steps.

The PMCMC method was used to estimate the model parameters based on the simulated observations, Y1:n. The PMCMC algorithm was run using N = 10000 particles and m =

10000 MCMC iterations. The initial parameter values were set to α = 0.9, µx = 0, ρ = 0 and σ2 = 0.4. We assumed the following prior distributions for the model parameters: 2 α ∼ TN(0.9, 10, −1, 1), µx ∼ N(0, 10), ρ ∼ TN(0.5, 10, −1, 1) and σ ∼ IG(0.01, 0.01). The parameters were updated using a random walk with the following proposal distributions:

∗ (i−1) 2 α ∼ TN α , τα, −1, 1 ,

∗ (i−1) 2 µx ∼ N µx , τµx ,

∗ (i−1) 2 ρ ∼ TN ρ , τρ , −1, 1 ,

2∗ 2 (i−1) 2 log σ ∼ N log σ , τσ2 .

The scale parameters were set to τα = 0.03, τµx = 0.03, τρ = 0.03 and τσ2 = 0.03.

37 Figure 4.12 shows the trace plots of the parameter posterior draws from each iteration of the PMCMC algorithm. The histograms of the posterior draws, after a burn-in period of 5000 iterations, are displayed in Figure 4.13. 4 0.95 3 0.85 x α µ 2 0.75 1 0 0.65 0 2000 4000 6000 8000 10000 0 2000 4000 6000 8000 10000 MCMC Iterations MCMC Iterations 1.0 0.1 0.8 2 ρ σ 0.6 −0.1 0.4 −0.3 0.2 0 2000 4000 6000 8000 10000 0 2000 4000 6000 8000 10000 MCMC Iterations MCMC Iterations

2 Figure 4.12: SVM with Leverage Eﬀect trace plots of α, µx, ρ and σ resulting from PMCMC for simulated data. 800 1000 600 600 400 Frequency Frequency 200 200 0 0 0.70 0.75 0.80 0.85 0.90 0.95 3.0 3.5 4.0 4.5 α µx 800 1000 600 600 400 Frequency Frequency 200 200 0 0 −0.4 −0.2 0.0 0.1 0.2 0.4 0.6 0.8 1.0 ρ σ2

2 Figure 4.13: SVM with Leverage Eﬀect histograms of α, µx, ρ and σ resulting from PMCMC for simulated data.

38 The acceptance rate for the proposed parameters was 0.2441. Table 4.4 gives a summary of the parameter posterior distributions resulting from the PMCMC algorithm.

Table 4.4: Summary of SVM with Leverage Eﬀect parameter posterior distributions resulting from PMCMC for simulated data.

Parameter True Value Mean 95% CI α 0.89 0.857 (0.788, 0.914)

µx 3.6 3.783 (3.405, 4.136) ρ -0.05 -0.073 (-0.248, 0.095) σ2 0.47 0.497 (0.313, 0.766)

The posterior means of the parameters were used to generate the estimated X values shown in Figure 4.14.

Simulated Hidden Process (X) ● ● Simulated Observations (Y) ● Estimated ●Hidden Process (X) 40

●

● ●

● ● ● ● ● ● ● ● ● ● 20 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●●● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●●● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● 0 ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ●●● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

−20 ● ● ● ● ● ● ● ● ● ●

●

0 100 200 300 400 500 Time

Figure 4.14: SVM with Leverage Eﬀect simulated data and PMCMC estimates, with observations Y1:n and hidden process X1:n for simulated data.

4.5 PMCMC for SVM with Covariate Eﬀects

The stochastic volatility model with covariate eﬀects from Section 2.5 was used to create the simulated data set shown in Figure 4.17. To generate this data set the model parameters 2 were set to α = 0.88, µx = 3.6, η = (0.88, −0.25) and σ = 0.55 for n = 500 time steps.

39 The two covariates used for this simulation are Bitcoin data sets that will be introduced in the next chapter.

In this section the PMCMC method was used to estimate the model parameters based on the simulated observations, Y1:n. The PMCMC algorithm was run using N = 10000 particles and m = 10000 MCMC iterations. The initial parameter values were set to 2 α = 0.9, µx = 0.2, η = (0, 0) and σ = 0.35. We assumed the following prior distributions for the model parameters: α ∼ TN(0.9, 10, −1, 1), µx ∼ N(0, 10), η ∼ N(0, 10) and σ2 ∼ IG(0.01, 0.01). The parameters were updated using a random walk with the following proposal distributions:

∗ (i−1) 2 α ∼ TN α , τα, −1, 1 ,

∗ (i−1) 2 µx ∼ N µx , τµx ,

∗ (i−1) 2 η ∼ N η , τη ,

2∗ 2 (i−1) 2 log σ ∼ N log σ , τσ2 .

The scale parameters were set to τα = 0.05, τµx = 0.3162, τη = 0.2 and τσ2 = 0.1.

Figure 4.15 shows the trace plots of the parameter posterior draws from each iteration of the PMCMC algorithm. The histograms of the posterior draws, after a burn-in period of 5000 iterations, are displayed in Figure 4.16.

40 5 4 0.95 0.90 3 x α µ 0.85 2 0.80 1 0.75 0

0 2000 4000 6000 8000 10000 0 2000 4000 6000 8000 10000 MCMC Iterations MCMC Iterations 2 2 1 1 1 2 η η 0 0 −1 −1

0 2000 4000 6000 8000 10000 0 2000 4000 6000 8000 10000 MCMC Iterations MCMC Iterations 1.0 0.8 2 σ 0.6 0.4 0.2 0 2000 4000 6000 8000 10000 MCMC Iterations

2 Figure 4.15: SVM with Covariate Eﬀect trace plots of α, µx, η1, η2 and σ resulting from PMCMC for simulated data.

41 1200 1000 800 600 600 Frequency Frequency 400 200 200 0 0

0.75 0.80 0.85 0.90 0.95 3.0 3.5 4.0 4.5 5.0 α µx 700 600 500 500 400 300 300 Frequency Frequency 200 100 100 0 0

−1.0 −0.5 0.0 0.5 1.0 1.5 2.0 2.5 −1.5 −1.0 −0.5 0.0 0.5 1.0 1.5 2.0 η1 η2 800 600 400 Frequency 200 0

0.4 0.6 0.8 1.0 σ2

2 Figure 4.16: SVM with Covariate Eﬀect histograms of α, µx, η1, η2 and σ resulting from PMCMC for simulated data.

The acceptance rate for the proposed parameters was 0.2925. Table 4.5 gives a summary of the parameter posterior distributions resulting from the PMCMC algorithm.

Table 4.5: Summary of SVM with Covariate Eﬀect parameter posterior distributions resulting from PMCMC for simulated data.

Parameter True Value Mean 95% CI α 0.88 0.874 (0.814, 0.924)

µx 3.6 3.539 (3.041, 4.008)

η1 0.88 0.516 (-0.491, 1.694)

η2 -0.25 0.627 (-0.549, 1.59) σ2 0.55 0.526 (0.346, 0.75)

42 The posterior means of the parameters were used to generate the estimated X values shown in Figure 4.17.

Simulated Hidden Process (X) ● ● Simulated Observations (Y) 40

● Estimated Hidden Process (X) ●

●

● ● ● ● ● ● ● ● ● ● ● ● ● ● 20 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ●● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ●●● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ●● ●● ● ● ●●●● ● ● ●● ● ● ●● ●● ●●● ● ● ● ● ●● ● ● ● ● ● ● ● ●● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●●●●●● ●● ●● ● ● 0 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●● ● ●●● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ●

● −20 ● ● ●● ● ● ● ● ● ● ● ● −40

● ●

0 100 200 300 400 500 Time

Figure 4.17: SVM with Covariate Eﬀect simulated data and PMCMC estimates, with observations Y1:n and hidden process X1:n for simulated data.

4.6 Chapter Summary

In this chapter simulation studies were conducted to demonstrate the capabilities of the particle Markov chain Monte Carlo algorithm. Simulated data sets were generated using the stochastic volatility models introduced in Chapter 2. Particle MCMC was used to estimate the model parameters based on the simulated observations and compared to the true parameter values used to generate the data.

The PMCMC algorithm was found to be very sensitive to the choice of scale parameters, τ, for the proposal distributions. If a scale parameter is too large this can lead to a low acceptance rate and make the algorithm ineﬃcient. This is because the steps in the random walk are too large and too many candidate draws are being rejected. On the other hand, a scale parameter that is too small can lead to an acceptance rate that is too high because the steps in the random walk are too small and the chain is not moving around the parameter space quickly enough. Choosing the optimal scale parameters for the proposal distributions is a diﬃcult task, especially when dealing with a model that has multiple parameters such as the stochastic volatility models used in this study.

In the next chapter stochastic volatility models are used to model real Bitcoin exchange rate data and the model parameters are estimated using particle MCMC.

43 Chapter 5

Applications to Bitcoin Exchange Rate Data

5.1 Data

The data set contains daily Bitcoin exchange rates (Bitcoin versus USD) over the period of June 30, 2014 to June 30, 2016. The daily values are a weighted average of the exchange rates from the largest Bitcoin exchanges. The data was obtained from the Quandl data base, https://www.quandl.com/collections/markets/bitcoin-data. Although data is available for more than just the past two years, the Bitcoin exchange rate was extremely volatile before this period and likely followed very diﬀerent patterns than it does today.

Average Daily Bitcoin Price in USD 700 600 500 400 Bitcoin Price (USD) 300 200

2015 2016

Date

Figure 5.1: Daily Bitcoin exchange rate.

44 The particle MCMC algorithm was applied to the relative change in the exchange rate of Bitcoin. Let x(t) be a time series at time t, then the relative change, r(t), is deﬁned as:

x(t) r(t) = . x(t − 1)

The relative change in the daily Bitcoin exchange rate data is shown in Figure 5.2.

Bitcoin Relative Change in Daily Price 1.15 1.05 0.95 Relative Change Relative 0.85

2015 2016

Date

Figure 5.2: Relative change in daily Bitcoin exchange rate.

In addition to price and exchange rates, the Quandl Bitcoin database has many other data sets from various aspects of the Bitcoin network, such as market size or network activity. Some of these variables could possibly be related to Bitcoin price and could be used in a stochastic volatility model with covariate eﬀects. Speciﬁcally, we will consider two covariates: the number of Bitcoin transactions per day and the number of unique Bitcoin addresses used per day. These data sets are shown in Figures 5.3 and 5.4, respectively.

Number of Bitcoin Transactions per Day 250000 150000 Number of Transactions 50000 2015 2016

Date

Figure 5.3: Number of Bitcoin Transactions per Day.

45 Number of Unique Bitcoin Addresses Used per Day 5e+05 4e+05 3e+05 Number of Addresses 2e+05 1e+05 2015 2016

Date

Figure 5.4: Number of Unique Bitcoin Addresses Used per Day.

In the following sections the stochastic volatility models introduced in Chapter 2 are used to model the Bitcoin exchange rate data and particle Markov chain Monte Carlo was used to estimate model parameters. The Bitcoin data analysis begins with the basic stochastic volatility model. Only version 2 of the basic SVM was applied to the Bitcoin data since both versions are just diﬀerent parameterizations of the same model.

5.2 Bitcoin Data Analysis with Basic SVM

The PMCMC algorithm was run using N = 15000 particles and m = 41300 MCMC it- 2 erations. The initial parameters were set to α0 = 0.9, µx0 = 0, µy0 = 0.2 and σ0 = 0.4. The parameters are assumed to have the same prior distribution as in Section 4.2 and are updated using the same random walk method with proposal distributions of the same form.

For this application the scale parameters were set to τα = 0.1, τµx = 0.3162, τµy = 0.03 and

τσ2 = 0.1.

Figure 5.5 shows the trace plots of the parameter posterior draws from each iteration of the PMCMC algorithm. A burn-in of 2500 iterations was used to ensure that the Markov chain had converged. The histograms of the posterior draws, after the burn-in period, are displayed in Figure 5.6.

46 1.00 4 0.90 3 x α µ 2 0.80 1 0.70 0 0 10000 20000 30000 40000 0 10000 20000 30000 40000 MCMC Iterations MCMC Iterations 1.0 0.6 1.0 y 2 µ σ 0.2 0.6 −0.2 0.2 0 10000 20000 30000 40000 0 10000 20000 30000 40000 MCMC Iterations MCMC Iterations

2 Figure 5.5: Basic SVM Version 2 trace plots of α, µx, µy and σ resulting from PMCMC for Bitcoin data. 6000 6000 4000 Frequency Frequency 2000 2000 0 0 0.65 0.75 0.85 0.95 3.0 3.5 4.0 4.5 α µx 4000 6000 2000 Frequency Frequency 2000 0 0 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 0.2 0.4 0.6 0.8 1.0 1.2 2 µy σ

2 Figure 5.6: Basic SVM Version 2 histograms of α, µx, µy and σ resulting from PMCMC for Bitcoin data.

Table 5.1 gives the posterior mean and 95% credible interval resulting from the PMCMC algorithm for each of the model parameters. The acceptance rate was 0.1185 and the DIC = 4910.634.

47 Table 5.1: Summary of Basic SVM Version 2 parameter posterior distributions resulting from PMCMC for Bitcoin data.

Parameter Mean 95% CI α 0.868 (0.785, 0.917)

µx 3.621 (3.236, 3.984)

µy 0.334 (0.062, 0.609) σ2 0.571 (0.38, 0.938)

5.3 Bitcoin Data Analysis for SVM with Fat Tail

The PMCMC algorithm was run using N = 5000 particles and m = 10000 MCMC iter- 2 ations. The initial parameters were set to α0 = 0.9, µx0 = 0, df0 = 10 and σ0 = 0.4. The parameters are assumed to have the same prior distribution as in Section 4.3 and are updated using the same random walk method with proposal distributions of the same form.

For this application the scale parameters were set to τα = 0.05, τµx = 0.3162, τdf = 3 and

τσ2 = 0.1.

Figure 5.7 shows the trace plots of the parameter posterior draws from each iteration of the PMCMC algorithm.The histograms of the posterior draws, after a burn-in period of 2500 iterations, are displayed in Figure 5.8. 5 1.00 4 3 x α µ 0.90 2 1 0.80 0 2000 4000 6000 8000 10000 0 2000 4000 6000 8000 10000 MCMC Iterations MCMC Iterations 50 0.8 40 0.6 30 2 df σ 20 0.4 10 0.2

0 2000 4000 6000 8000 10000 0 2000 4000 6000 8000 10000 MCMC Iterations MCMC Iterations

2 Figure 5.7: SVM with Fat Tails trace plots of α, µx, df and σ resulting from PMCMC for Bitcoin data.

48 1500 1500 1000 1000 Frequency Frequency 500 500 0 0 0.80 0.85 0.90 0.95 1.00 2 3 4 5 α µx 1200 3000 800 2000 400 Frequency Frequency 1000 0 0 0 10 20 30 40 50 0.2 0.4 0.6 0.8 df σ2

2 Figure 5.8: SVM with Fat Tails histograms of α, µx, df and σ resulting from PMCMC for Bitcoin data.

Table 5.2 gives the posterior mean and 95% credible interval resulting from the PMCMC algorithm for each of the model parameters. The acceptance rate was 0.2648 and the DIC = 4914.64.

Table 5.2: Summary of SVM with Fat Tails parameter posterior distributions resulting from PMCMC for Bitcoin data.

Parameter Mean 95% CI α 0.936 (0.875, 0.982)

µx 3.439 (2.713, 4.142) df 11.11 (4.099, 27.172) σ2 0.3144 (0.146, 0.578)

5.4 Bitcoin Data Analysis for SVM with Leverage Eﬀect

The PMCMC algorithm was run using N = 15000 particles and m = 33400 MCMC it- 2 erations. The initial parameters were set to α0 = 0.9, µx0 = 0, ρ0 = 0 and σ0 = 0.4. The parameters are assumed to have the same prior distribution as in Section 4.4 and are updated using the same random walk method with proposal distributions of the same form.

49 For this application the scale parameters were set to τα = 0.1, τµx = 0.3162, τρ = 0.035 and

τσ2 = 0.1.

Figure 5.9 shows the trace plots of the parameter posterior draws from each iteration of the PMCMC algorithm.The histograms of the posterior draws, after a burn-in period of 5000 iterations, are displayed in Figure 5.10. 1.00 4 2 0.90 x α µ 0 0.80 −2 0.70 0 5000 15000 25000 0 5000 15000 25000 MCMC Iterations MCMC Iterations 1.0 0.1 2 ρ σ 0.6 −0.1 0.2 −0.3 0 5000 15000 25000 0 5000 15000 25000 MCMC Iterations MCMC Iterations

2 Figure 5.9: SVM with Leverage Eﬀect trace plots of α, µx, ρ and σ resulting from PMCMC for Bitcoin data. 4000 6000 2000 Frequency Frequency 2000 0 0 0.70 0.80 0.90 1.00 2.5 3.0 3.5 4.0 4.5 5.0 α µx 3000 6000 Frequency Frequency 1000 2000 0 0 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.2 0.4 0.6 0.8 1.0 1.2 ρ σ2

2 Figure 5.10: SVM with Leverage Eﬀect histograms of α, µx, ρ and σ resulting from PMCMC for Bitcoin data.

50 Table 5.3 gives the posterior mean and 95% credible interval resulting from the PMCMC algorithm for each of the model parameters. The acceptance rate was 0.173 and the DIC = 4903.698.

Table 5.3: Summary of SVM with Leverage Eﬀect parameter posterior distributions resulting from PMCMC for Bitcoin data.

Parameter Mean 95% CI α 0.886 (0.819, 0.942)

µx 3.628 (3.231, 4.05)

µy -0.041 (-0.147, 0.078) σ2 0.475 (0.256, 0.754)

5.5 Bitcoin Data Analysis for SVM with Covariate Eﬀect

The PMCMC algorithm was run using N = 5000 particles and m = 30000 MCMC iterations. The initial parameters were set to The initial parameters were set to α0 = 0.9, 2 µx0 = 0.2, η0 = (0, 0) and σ0 = 0.35. The parameters are assumed to have the same prior distribution as in Section 4.5 and are updated using the same random walk method with proposal distributions of the same form. For this application the scale parameters were set to τα = 0.05, τµx = 0.3162, τη = 0.2 and τσ2 = 0.1.

Figure 5.11 shows the trace plots of the parameter posterior draws from each iteration of the PMCMC algorithm.The histograms of the posterior draws, after a burn-in period of 5000 iterations, are displayed in Figure 5.12.

51 4 0.90 3 x α µ 2 0.80 1 0.70 0 0 5000 10000 15000 20000 25000 30000 0 5000 10000 15000 20000 25000 30000 MCMC Iterations MCMC Iterations 2.5 1 2.0 1.5 0 1.0 1 2 η η 0.5 −1 0.0 −2

−1.0 0 5000 10000 15000 20000 25000 30000 0 5000 10000 15000 20000 25000 30000 MCMC Iterations MCMC Iterations 1.2 1.0 0.8 2 σ 0.6 0.4 0.2

0 5000 10000 15000 20000 25000 30000 MCMC Iterations

2 Figure 5.11: SVM with Covariate Eﬀect trace plots of α, µx, η1, η2 and σ resulting from PMCMC for Bitcoin data.

52 4000 4000 3000 3000 2000 2000 Frequency Frequency 1000 1000 0 0

0.70 0.75 0.80 0.85 0.90 0.95 1.00 3.0 3.5 4.0 4.5 α µx 4000 4000 3000 3000 2000 2000 Frequency Frequency 1000 1000 0 0

−1.0 −0.5 0.0 0.5 1.0 1.5 2.0 2.5 −2 −1 0 1 η1 η2 2500 1500 Frequency 500 0

0.2 0.4 0.6 0.8 1.0 1.2 σ2

2 Figure 5.12: SVM with Covariate Eﬀect histograms of α, µx, η1, η2 and σ resulting from PMCMC for Bitcoin data.

Table 5.4 gives the posterior mean and 95% credible interval resulting from the PMCMC algorithm for each of the model parameters. The acceptance rate was 0.2130 and the DIC = 4905.48.

Table 5.4: Summary of SVM with Covariate Eﬀect parameter posterior distributions resulting from PMCMC for Bitcoin data.

Parameter Mean 95% CI α 0.872 (0.802, 0.931)

µx 3.614 (3.25, 4.01)

η1 0.879 (0.039, 1.644)

η2 -0.256 (-1.108, 0.679) σ2 0.545 (0.307, 0.837)

53 5.6 Summary of Bitcoin Data Analysis

In this chapter the stochastic volatility models introduced in Chapter 2 were fit to daily Bitcoin exchange rate data over the period of June 20, 2014 to June 30, 2016. The Particle MCMC algorithm was applied to these models to estimate the parameter values. The Deviance Information Criterion (DIC), outlined in Section 3.6, was calculated for each model and can be used to compare their effectiveness. Table 5.5 gives a summary of the DIC for each model. For the Bitcoin exchange rate data set the most effective model was SVM with Leverage Effect with a DIC of 4903.7.

Table 5.5: A summary of the DIC for each stochastic volatility model that was ﬁt to the Bitcoin exchange rate data set.

Model DIC Basic SVM 2 4910.63 SVM with Fat Tail 4914.64 SVM with Leverage Eﬀect 4903.70 SVM with Covariate Eﬀect 4905.48

Similar to the previous chapter, the eﬀectiveness of each model was sensitive to the choice of scale parameters (τ) which had to be speciﬁed for the proposal distributions for each model parameter.

54 Chapter 6

Conclusion and Future work

Stochastic volatility models are useful tools for modeling time series data and are commonly used in ﬁnancial applications. The motivation for this project was to determine if stochastic volatility models could be used to model the exchange rate of Bitcoin. Chapter 2 described all of the stochastic volatility models that were used in this project. We considered a basic SVM and several extensions including fat tails, leverage, and covariate eﬀects.

The Bayesian approach with the particle Markov chain Monte Carlo (PMCMC) method was employed to estimate the stochastic volatility model parameters. Chapter 3 provided a detailed description of PMCMC as well as the basic MCMC algorithm. First, we introduced the concept of Monte Carlo integration as a simulation technique that uses independent draws from the target distribution to approximate integrals rather than solving them analytically. However, it is not always possible to sample independent draws directly from the target distribution. In this case, we can use Markov chain Monte Carlo to make similar approximations using slightly dependent draws from a Markov chain.

Next, we saw that Importance Sampling is another way to address the problem of not being able to sample independent draws directly from the target distribution. Then by restricting the importance distribution to a certain form that can be written recursively, we can implement sequential importance sampling (SIS) which can be more computationally eﬃcient. A resampling step was then added to the sequential importance sampling algorithm which led to the concept of sequential Monte Carlo. The steps of the SMC algorithm were outlined in detail and we saw how this method provides an estimate of the hidden process Xn. Finally, we described the steps of the particle Markov chain Monte Carlo algorithm which can be used to estimate the SVM parameters. Sequential Monte Carlo is used within the PMCMC algorithm to estimate marginal likelihoods. In this project, the

55 Metropolis-Hastings method was used for PMCMC which is known as the particle marginal Metropolis-Hastings (PMMH) method.

Chapter 4 contains the results of simulation studies that were conducted to evaluate the performance of the PMCMC method. For each SVM we set the true parameter values and generated a simulated data set. The PMCMC method was used to estimate these model parameters based on the simulated observations Y1:n and the estimates were compared to the true parameter values that we were used to generate the data. We demonstrated that PMCMC can be used to estimate stochastic volatility model parameters, however it was diﬃcult to choose the optimal values for the scale parameters in the parameter proposal distributions.

The results of the Bitcoin data analysis were presented in Chapter 5. In this case, the true SVM parameter values were unknown and the relative change in the exchange rate of Bitcoin were the observations Y1:n. The effectiveness of each model was assessed using the deviance information criterion (DIC) and the best model was the SVM with Leverage Effect. Again we found that the effectiveness of each model was extremely sensitive to the choice of scale parameters (τ) which had to be specified for the proposal distributions for each model parameter.

Overall this approach showed some potential for modeling the exchange rate of Bitcoin, however, a method to determine the optimal scale parameters would be required before this approach could contribute to an eﬀective trading strategy. Future research on this topic could focus on the development of a better PMCMC algorithm that is not as diﬃcult to tune. We would like to use more advanced Monte Carlo methods to help improve the mixing of the Markov chain. For example, [12] introduced new latent variables and then used the MCMC moves to update the latent variables and use the SMC to propose new values for the parameters and stochastic process given the latent variables. They considered both particle Gibbs and PMMH.

More complex models could also be explored, such as a stochastic volatility model with both leverage and covariate eﬀects. Further research into additional model covariates would likely help to make this approach more useful for trading Bitcoin. In a complex ﬁnancial application such as this, there are many factors that must be considered. There are a wide variety of variables that could be related to the price of Bitcoin and this extra information could lead to us being able to more accurately model Bitcoin exchange rates.

56 Bibliography

[1] Tomohiro Ando. Bayesian inference for nonlinear and non-gaussian stochastic volatility model with leverage eﬀect. Journal of the Japan Statistical Society, 36(2):173–197, 2006.

[2] Christophe Andrieu, Arnaud Doucet, and Roman Holenstein. Particle Markov chain Monte Carlo methods. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72(3):269–342, 2010.

[3] Nando de Freitas Arnaud Doucet and Neil Gordon. An introduction to sequential Monte Carlo methods. pages 3–14, 2001.

[4] Olivier Cappé, Simon J. Godsill, and Eric Moulines. An overview of existing methods and recent advances in sequential Monte Carlo. Proceedings of the IEEE, 95(5):899– 924, May 2007.

[5] Bradley P Carlin and Nicholas G Polson. Inference for nonconjugate Bayesian models using the Gibbs sampler. Canadian Journal of statistics, 19(4):399–405, 1991.

[6] Joshua CC Chan. The stochastic volatility in mean model with time-varying parameters: An application to inﬂation modeling. Journal of Business & Economic Statistics, (just-accepted), 2015.

[7] Jeﬀrey Chu, Saralees Nadarajah, and Stephen Chan. Statistical analysis of the exchange rate of Bitcoin. PLoS ONE, 10(7), 2015.

[8] Garcia D, Tessone CJ, Mavrodiev P, and Perony N. The digital traces of bubbles: Feedback cycles between socio-economic signals in the bitcoin economy. Journal of the Royal Society Interface., 11(20140623), 2014.

[9] Arnaud Doucet and Adam M. Johansen. A tutorial on particle ﬁltering and smoothing: ﬁfteen years later. 2009.

[10] Bjørn Eraker, Michael Johannes, and Nicholas Polson. The impact of jumps in volatility and returns. The Journal of Finance, 58(3):1269–1300, 2003.

[11] Nicholas G Polson Eric Jacquier and Peter E Rossi. Bayesian analysis of stochastic volatility models. Journal of Business and Economic Statistics, 12(4), 1994.

[12] Paul Fearnhead and Loukia Meligkotsidou. Augmentation schemes for particle MCMC. Statistics and Computing, pages 1–14, 2015.

57 [13] John Geweke. Bayesian treatment of the independent student-t linear model. Journal of Applied Econometrics, 8(S1):S19–S40, 1993.

[14] N. Gordon, D. Salmond, and A. F. M. Smith. Novel approach to nonlinear/non- Gaussian Bayesian state estimation. Radar and Signal Processing, IEE Proceedings F, 140(2):107–113, April 1993.

[15] Roman Holenstein. Particle Markov Chain Monte Carlo. PhD thesis, University of British Columbia, 2009.

[16] Eric Jacquier, Nicholas G Polson, and Peter E Rossi. Models and priors for multivariate stochastic volatility. Technical report, CIRANO, 1995.

[17] Eric Jacquier, Nicholas G Polson, and Peter E Rossi. Bayesian analysis of stochastic volatility models with fat-tails and correlated errors. Journal of Econometrics, 122(1):185–212, 2004.

[18] Siem Jan Koopman and Eugenie Hol Uspensky. The stochastic volatility in mean model: empirical evidence from international stock markets. Journal of applied Econo- metrics, 17(6):667–689, 2002.

[19] Noureddine Krichene. Modeling stochastic volatility with application to stock returns. International Monetary Fund, (3-125), 2003.

[20] Ladislav Kristoufek. Bitcoin meets google trends and wikipedia: Quantifying the relationship between phenomena of the internet era. Scientiﬁc Reports, 3(3415), 2013.

[21] Patrick Lam. MCMC methods: Gibbs sampling and the metropolis-hastings algorithm. Harvard University.

[22] Fredrik Lindsten, Michael I Jordan, and Thomas B Schön. Particle Gibbs with ancestor sampling. Journal of Machine Learning Research, 15(1):2145–2184, 2014.

[23] Jun S. Liu. Monte Carlo Strategies in Scientiﬁc Computing. Springer, 2001.

[24] Hedibert F Lopes and Nicholas G Polson. Extracting sp500 and NASDAQ volatility: The credit crisis of 2007-2008. Handbook of Applied Bayesian Analysis, pages 319–342, 2010.

[25] Satoshi Nakamoto. Bitcoin: A peer-to-peer electronic cash system, 2008.

[26] Nima Nonejad. Particle Gibbs with ancestor sampling for stochastic volatility models with: heavy tails, in mean eﬀects, leverage, serial dependence and structural breaks. Studies in Nonlinear Dynamics & Econometrics, 2014.

[27] Sapuric S and Kokkinaki A. Bitcoin is volatile! isn’t that right? Business Information Systems Workshops, Lecture Notes in Business Information Processing, pages 255–265, 2014.

[28] David Spiegelhalter, Nicola G Best, Bradley P Carlin, and Angelika van der Linde. Bayesian measures of model complexity and ﬁt (with comments). Journal of the Royal Statistical Society, 64(4):583–639, 2002.

58 [29] Jun Yu and Renate Meyer. Multivariate stochastic volatility models: Bayesian estimation and model comparison. Econometric Reviews, 25(2-3):361–384, 2006.