Stan User’s Guide Version 2.18 Stan Development Team Contents About this Book 7 Part 1. Example Models 9 1. Regression Models 10 1.1 Linear Regression 10 1.2 The QR Reparameterization 12 1.3 Priors for Coefficients and Scales 14 1.4 Robust Noise Models 14 1.5 Logistic and Probit Regression 15 1.6 Multi-Logit Regression 16 1.7 Parameterizing Centered Vectors 19 1.8 Ordered Logistic and Probit Regression 21 1.9 Hierarchical Logistic Regression 23 1.10 Hierarchical Priors 25 1.11 Item-Response Theory Models 26 1.12 Priors for Identifiability 30 1.13 Multivariate Priors for Hierarchical Models 31 1.14 Prediction, Forecasting, and Backcasting 38 1.15 Multivariate Outcomes 39 1.16 Applications of Pseudorandom Number Generation 45 2. Time-Series Models 48 2.1 Autoregressive Models 48 2.2 Modeling Temporal Heteroscedasticity 51 2.3 Moving Average Models 52 2.4 Autoregressive Moving Average Models 55 2.5 Stochastic Volatility Models 57 2.6 Hidden Markov Models 60 3. Missing Data and Partially Known Parameters 66 1 CONTENTS 2 3.1 Missing Data 66 3.2 Partially Known Parameters 67 3.3 Sliced Missing Data 68 3.4 Loading matrix for factor analysis 69 3.5 Missing Multivariate Data 70 4. Truncated or Censored Data 73 4.1 Truncated Distributions 73 4.2 Truncated Data 73 4.3 Censored Data 75 5. Finite Mixtures 78 5.1 Relation to Clustering 78 5.2 Latent Discrete Parameterization 78 5.3 Summing out the Responsibility Parameter 79 5.4 Vectorizing Mixtures 82 5.5 Inferences Supported by Mixtures 83 5.6 Zero-Inflated and Hurdle Models 85 5.7 Priors and Effective Data Size in Mixture Models 90 6. Measurement Error and Meta-Analysis 91 6.1 Bayesian Measurement Error Model 91 6.2 Meta-Analysis 95 7. Latent Discrete Parameters 99 7.1 The Benefits of Marginalization 99 7.2 Change Point Models 99 7.3 Mark-Recapture Models 106 7.4 Data Coding and Diagnostic Accuracy Models 115 8. Sparse and Ragged Data Structures 120 8.1 Sparse Data Structures 120 8.2 Ragged Data Structures 121 9. Clustering Models 124 9.1 Relation to Finite Mixture Models 124 9.2 Soft K-Means 124 CONTENTS 3 9.3 The Difficulty of Bayesian Inference for Clustering 127 9.4 Naive Bayes Classification and Clustering 128 9.5 Latent Dirichlet Allocation 132 10. Gaussian Processes 137 10.1 Gaussian Process Regression 137 10.2 Simulating from a Gaussian Process 139 10.3 Fitting a Gaussian Process 142 11. Directions, Rotations, and Hyperspheres 160 11.1 Unit Vectors 160 11.2 Circles, Spheres, and Hyperspheres 161 11.3 Transforming to Unconstrained Parameters 161 11.4 Unit Vectors and Rotations 162 11.5 Circular Representations of Days and Years 163 12. Solving Algebraic Equations 164 12.1 Example: System of Nonlinear Algebraic Equations 164 12.2 Coding an Algebraic System 164 12.3 Calling the Algebraic Solver 165 12.4 Control Parameters for the Algebraic Solver 166 13. Ordinary Differential Equations 168 13.1 Example: Simple Harmonic Oscillator 168 13.2 Coding an ODE System 169 13.3 Solving a System of Linear ODEs using a Matrix Exponential 170 13.4 Measurement Error Models 171 13.5 Stiff ODEs 176 13.6 Control Parameters for ODE Solving 176 Part 2. Programming Techniques 178 14. Floating Point Arithmetic 179 14.1 Floating-point representations 179 14.2 Literals: decimal and scientific notation 181 14.3 Arithmetic Precision 181 14.4 Comparing floating-point numbers 185 CONTENTS 4 15. Matrices, Vectors, and Arrays 186 15.1 Basic Motivation 186 15.2 Fixed Sizes and Indexing out of Bounds 187 15.3 Data Type and Indexing Efficiency 187 15.4 Memory Locality 189 15.5 Converting among Matrix, Vector, and Array Types 191 15.6 Aliasing in Stan Containers 191 16. Multiple Indexing and Range Indexing 192 16.1 Multiple Indexing 192 16.2 Slicing with Range Indexes 194 16.3 Multiple Indexing on the Left of Assignments 194 16.4 Multiple Indexes with Vectors and Matrices 196 16.5 Matrices with Parameters and Constants 198 17. User-Defined Functions 200 17.1 Basic Functions 200 17.2 Functions as Statements 205 17.3 Functions Accessing the Log Probability Accumulator 205 17.4 Functions Acting as Random Number Generators 206 17.5 User-Defined Probability Functions 207 17.6 Overloading Functions 208 17.7 Documenting Functions 208 17.8 Summary of Function Types 209 17.9 Recursive Functions 210 17.10 Truncated Random Number Generation 211 18. Custom Probability Functions 214 18.1 Examples 214 19. Problematic Posteriors 217 19.1 Collinearity of Predictors in Regressions 217 19.2 Label Switching in Mixture Models 224 19.3 Component Collapsing in Mixture Models 226 19.4 Posteriors with Unbounded Densities 227 19.5 Posteriors with Unbounded Parameters 228 CONTENTS 5 19.6 Uniform Posteriors 229 19.7 Sampling Difficulties with Problematic Priors 229 20. Reparameterization and Change of Variables 234 20.1 Theoretical and Practical Background 234 20.2 Reparameterizations 234 20.3 Changes of Variables 239 20.4 Vectors with Varying Bounds 243 21. Efficiency Tuning 245 21.1 What is Efficiency? 245 21.2 Efficiency for Probabilistic Models and Algorithms 245 21.3 Statistical vs. Computational Efficiency 246 21.4 Model Conditioning and Curvature 246 21.5 Well-Specified Models 248 21.6 Avoiding Validation 248 21.7 Reparameterization 249 21.8 Vectorization 263 21.9 Exploiting Sufficient Statistics 268 21.10 Aggregating Common Subexpressions 269 21.11 Exploiting Conjugacy 269 21.12 Standardizing Predictors and Outputs 270 21.13 Using Map-Reduce 273 22. Map-Reduce 274 22.1 Overview of Map-Reduce 274 22.2 Map Function 274 22.3 Example: Mapping Logistic Regression 275 22.4 Example: Hierarchical Logistic Regression 277 22.5 Ragged Inputs and Outputs 280 Appendices 282 23. Stan Program Style Guide 283 23.1 Choose a Consistent Style 283 23.2 Line Length 283 CONTENTS 6 23.3 File Extensions 283 23.4 Variable Naming 283 23.5 Local Variable Scope 284 23.6 Parentheses and Brackets 285 23.7 Conditionals 286 23.8 Functions 287 23.9 White Space 288 24. Transitioning from BUGS 291 24.1 Some Differences in How BUGS and Stan Work 291 24.2 Some Differences in the Modeling Languages 293 24.3 Some Differences in the Statistical Models that are Allowed 297 24.4 Some Differences when Running from R 298 24.5 The Stan Community 299 References 300 About this Book This book is is the official user’s guide for Stan. It provides example models and programming techniques for coding statistical models in Stan. How to use this book Part 1 gives Stan code and discussions for several important classes of models. Part 2 discusses various general Stan programming techniques that are not tied to any particular model. The appendices provide a style guide and advice for users of BUGS and JAGS. We recommend working through this book using the textbooks Bayesian Data Analysis and Statistical Rethinking: A Bayesian Course with Examples in R and Stan as references on the concepts, and using the Stan Reference Manual when necessary to clarify programming issues. Further resources are given at the end of the introductory chapter. Additional Stan manuals and guides In addition to this user’s guide, there are two reference manuals for the Stan language and algorithms. The Stan Reference Manual specifies the Stan programming language and inference algorithms. The Stan Functions Reference specifies the functions built into the Stan programming language. There is also a separate installation and getting started guide for each of the Stan interfaces (R, Python, Julia, Stata, MATLAB, Mathematica, and command line). Web resources Stan is an open-source software project, resources for which are hosted on various web sites: • The Stan Web Site organizes all of the resources for the Stan project for users and developers. It contains links to the official Stan releases, source code, installation instructions, and full documentation, including the latest version of this manual, the user’s guide and the getting started guide for each interface, tutorials, case studies, and reference materials for developers. 7 CONTENTS 8 • The Stan Forums provide structured message boards for questions, discussion, and announcements related to Stan for both users and developers. • The Stan GitHub Organization hosts all of Stan’s code, documentation, wikis, and web site, as well as the issue trackers for bug reports and feature requests and interactive code review for pull requests. Acknowledgements The Stan project could not exit without developers, users, and funding. Stan is a highly collaborative project. The individual contributions of the Stan developers to code is tracked through GitHub and to the design conversation in the Wikis and forums. Users have made extensive contributions to documentation in the way of case studies, tutorials and even books. They have also reported numerous bugs in both the code and documentation. Stan has been funded through grants for Stan and its developers, through in-kind donations in the form of companies contributing developer time to Stan and individ- uals contributing their own time to Stan, and through donations to the open-source scientific software non-profit NumFOCUS. For details of direct funding for the project, see the web site and project pages of the Stan developers. Copyright, Trademark, and Licensing This book is copyright 2011–2018, Stan Development Team and their assignees. The text content is distributed under the CC-BY ND 4.0 license. The user’s guide R and Stan programs are distributed under the BSD 3-clause license.