Bayesian analysis using Stan

Lawrence McCandless Associate Professor [email protected]

Faculty of Health Sciences, Simon Fraser University, Vancouver Canada Stan Software for Bayesian Analysis

Stan is a state-of-the-art platform for Bayesian statistical modeling and high-performance computing. Developed in 2012 originally by Andrew Gelman. Widely used for data analysis in the social, biological, and physical sciences, engineering, and business.

It is implemented in R, Python, Shell, MATLAB, Stata on all major platforms (Windows, Mac, Linux)

There is also an R package for Stan called RStanArm for applied regression modelling.

Stan: is a probabilistic programming language which uses computer programs to represent probability distribution by generating data. Uses statistical sampling to fit complex models. Illustration of Stan: Bayesian linear regression

Example: Autistic behavior in 3-4 year old children (simulated example).

Y - autistic behavior measured using the Social Responsiveness Scale (SRS-2)

X - be maternal years of formal education of the mother.

Y |X ∼ Normal(α + β X, σ2)

(linear regression model for Y) Example: Autistic Behavior and Socioeconomic Status (SES)

(Simulated data)

Example: Autistic Behavior and Socioeconomic Status (SES) Different choices of priors: An absurd prior

Just for fun we could change the prior on β to be

And we obtain, as expected, Different choices of priors: An hierarchical prior

If there were several regression coefficients β1,... βk, then we could consider hierarchical priors that introduce shrinkage to share information between predictors.

E.g for multiple correlated exposure variables Other remarks

Flat priors (uninformative) are easy to implement in stan.

Generated quantities: Can generate various model outputs like predictions of Y for assessing model fit or for causal inference.

Transformed parameters: can be used to put priors on transformed parameters Dichotomize Y and use logistic regression

We can dichotomize Y using a threshold defined ASD

A = 1 if Y>60, else A=0

Then we can use the logistic regression model

∼ A |X Bernoulli{expit(α + β X)}

Log Odds(A=1|X) = α + β X

There are only <10 cases of SRS>60 out of ~600 participants, therefore we expect low power. Stan code for Bayesian logistic regression Logistic regression results

Again: Bayesian logistic regression is identical to glm() MCMC sampler traceplots using stan_trace() Posterior histograms using stan_hist() What is the advantage of Stan? JAGS, Bugs and INLA can do all this.

1. We can easily add more complex modelling structures, like hierarchical modelling

But JAGS, Bugs and INLA can do this…

So what does Stan offer…? Advantage of Stan versus JAGS, Bugs and INLA. 2. Stan allows the user to use Log Probability Increments to fit user-defined models and probability functions. y ~ normal(mu, sigma) is the same as target += normal_lpdf(y, mu, sigma) or more generally target += dist_lpdf(y | theta1, ... thetaN)

Can fit complex user-defined models (e.g. Bayesian bias models). Explore Non-standard models, poorly identified models. Example of user-defined complex Bayesian model for epidemiology

Bayesian sensitivity analysis with binary unmeasured confounder U uses the following model: Advantage of Stan versus JAGS, Bugs and INLA.

3. Sampling is done with Hamilton Monte Carlo (HMC). It is a black box procedure that rapidly explores the posterior distribution, even for poorly identified models, or models with extreme collelations in model parameters.

HMC is an MCMC method that uses the derivatives of the log-density function being sampled to generate efficient transitions spanning the posterior.

Stan also does automatic parameter tuning. It automatically optimizes the sampler to match the acceptance-rate target.

This is invaluable. Like starting a campfire with a blowtorch versus rubbing two sticks together. Advantage of Stan versus JAGS, Bugs and INLA.

4. Has 600 page manual with many great example, and many developers to provide support.

5. Stan contains extraordinary collection of probability distributions. ● All standard probability distributions (gamma, t, chi, normal, poisson etc). ● Truncated distributions, distributions on simplexes, correlation and covariance matrices. ● Not limited to conjugate models. ● User defined probability models.

One problem: Stan does not handle sampling of discrete parameters. They must be marginalized out of the model. Thank you

I hope you are persuaded to try using Stan for Bayesian analysis.

Feel free to email me if you have questions.

PS: See McCandless & Somers (2017) Stat Methods in Med Res for example of Bayesian mediation analysis using stan.