DOCSLIB.ORG

The Adventures of Bayesian Programming in Stan

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

The adventures of Bayesian programming in Stan Samuel Hudec Matej Bel University Faculty of Natural Sciences Department of Mathematics 9. 2. 2018 Samuel Hudec DM seminar Topics Introduction Bayesian approach MCMC Stan enviroment First model in Stan Regression model in Stan Basic Hierarchical model in Stan Conclusion Samuel Hudec DM seminar What you can get? full Bayesian statistical inference with MCMC sampling approximate Bayesian inference with variational inference penalized maximum likelihood estimation with optimization Introduction http://mc-stan.org/ Stan is a state-of-the-art platform for statistical modeling and high-performance statistical computation. Thousands of users rely on Stan for statistical modeling, data analysis, and prediction in the social, biological, and physical sciences, engineering, and business. Samuel Hudec DM seminar Introduction http://mc-stan.org/ Stan is a state-of-the-art platform for statistical modeling and high-performance statistical computation. Thousands of users rely on Stan for statistical modeling, data analysis, and prediction in the social, biological, and physical sciences, engineering, and business. What you can get? full Bayesian statistical inference with MCMC sampling approximate Bayesian inference with variational inference penalized maximum likelihood estimation with optimization Samuel Hudec DM seminar Named by Stanislaw Ulam. Who was a scientist in the fields of mathematics and nuclear physics, also he invated the Monte Carlo method to computation, ... Introduction (Why Stan?) Samuel Hudec DM seminar Introduction (Why Stan?) Named by Stanislaw Ulam. Who was a scientist in the fields of mathematics and nuclear physics, also he invated the Monte Carlo method to computation, ... Samuel Hudec DM seminar Introduction (Stanislaw Ulam) Samuel Hudec DM seminar Higher-Level Interface RStanArm provides an R formula interface for Bayesian regression modeling. etc. Introduction (Stan interfaces) RStan (R) PyStan (Phyton) CmdStan (shell, command-line terminal) MatlabStan (MATLAB) StataStan (Stata) MathematicaStan (Mathemtatica) Samuel Hudec DM seminar Introduction (Stan interfaces) RStan (R) PyStan (Phyton) CmdStan (shell, command-line terminal) MatlabStan (MATLAB) StataStan (Stata) MathematicaStan (Mathemtatica) Higher-Level Interface RStanArm provides an R formula interface for Bayesian regression modeling. etc. Samuel Hudec DM seminar p(θ D) p(D θ)p(θ) j / j Posterior predictive density (prediction, validation,...) p(D~ D) = p(D~ θ)p(θ D) d θ j j j ZΘ Bayesian approach Based on Bayesian theorem (1763) p(D θ)p(θ) p(θ D)= j j p(D θ)p(θ) d θ Θ j R Model (likelihood) = p(D θ) Prior = p(θ) (conjugate, noninformative...)j evidence = Θ p(D θ)p(θ) d θ , p(D) Posterior = p(θ D)j R j Samuel Hudec DM seminar Posterior predictive density (prediction, validation,...) p(D~ D) = p(D~ θ)p(θ D) d θ j j j ZΘ Bayesian approach Based on Bayesian theorem (1763) p(D θ)p(θ) p(θ D)= j j p(D θ)p(θ) d θ Θ j R Model (likelihood) = p(D θ) Prior = p(θ) (conjugate, noninformative...)j evidence = Θ p(D θ)p(θ) d θ , p(D) Posterior = p(θ D)j R j p(θ D) p(D θ)p(θ) j / j Samuel Hudec DM seminar Bayesian approach Based on Bayesian theorem (1763) p(D θ)p(θ) p(θ D)= j j p(D θ)p(θ) d θ Θ j R Model (likelihood) = p(D θ) Prior = p(θ) (conjugate, noninformative...)j evidence = Θ p(D θ)p(θ) d θ , p(D) Posterior = p(θ D)j R j p(θ D) p(D θ)p(θ) j / j Posterior predictive density (prediction, validation,...) p(D~ D) = p(D~ θ)p(θ D) d θ j j j ZΘ Samuel Hudec DM seminar Bayesian approach source: McElreath, R., Statistical Rethinking CRC Press 2016. Samuel Hudec DM seminar Bayesian approach Bayesian approachBayesian approach Based on Bayesian theorem (1763) Based on Bayesian theorem (1763) p(D ✓)p(✓) ??? p(D ✓)p(✓) p(✓ D)= | p(✓ D)= | | p(D ✓)p(✓)d✓ | p(D ✓)p(✓)d✓ ⇥ | ⇥ | R R Model (likelihood) = p(D ✓) Model (likelihood) = p(D ✓) | Prior = p(✓) (conjugate, noninformative...)| PriorSamuel = Hudecp(✓) DM(conjugate, seminar noninformative...) evidence = ⇥ p(D ✓)p(✓)d✓ , p(D) evidence = ⇥ p(D ✓)p(✓)d✓ , p(D) | Posterior = p(✓ D)| Posterior = p(✓ D) R | R | p(✓ D) p(D ✓)p(✓) p(✓ D) p(D ✓)p(✓) | / | | / | Posterior predictive density (prediction, validation,...) Posterior predictive density (prediction, validation,...) p(D˜ D)= p(D˜ ✓)p(✓ D)d✓ p(D˜ D)= p(D˜ ✓)p(✓ D)d✓ | ⇥ | | | | | Z Z⇥ Samuel Hudec DM seminar Samuel Hudec DM seminar Bayesian approach Bayesian approachBayesian approach Based on Bayesian theorem (1763) Based on Bayesian theorem (1763) p(D ✓)p(✓) p(D ✓)p(✓) p(✓ D)= | p(✓ D)= | | p(D ✓)p(✓)d✓ | p(D ✓)p(✓)d✓ ⇥ | ⇥ | R R Model (likelihood) = p(D ✓) Model (likelihood) = p(D ✓) | Prior = p(✓) (conjugate, noninformative...)| PriorSamuel = Hudecp(✓) DM(conjugate, seminar noninformative...) evidence = ⇥ p(D ✓)p(✓)d✓ , p(D) evidence = ⇥ p(D ✓)p(✓)d✓ , p(D) | Posterior = p(✓ D)| Posterior = p(✓ D) R | R | p(✓ D) p(D ✓)p(✓) p(✓ D) p(D ✓)p(✓) | / | | / | Posterior predictive density (prediction, validation,...) Posterior predictive density (prediction, validation,...) p(D˜ D)= p(D˜ ✓)p(✓ D)d✓ p(D˜ D)= p(D˜ ✓)p(✓ D)d✓ | ⇥ | | | | | Z Z⇥ Samuel Hudec DM seminar Samuel Hudec DM seminar How many samples do you need? How many chains we need? for reggresion motto "four short chains to check, one long chain for inference" MCMC "Markov Chain Monte Carlo methods are a class of algorithms for sampling from a probability distribution based on constructing a Markov chain that has the desired distribution as its equilibrium distribution. The state of the chain after a number of steps is then used as a sample of the desired distribution. The quality of the sample improves as a function of the number of steps." Samuel Hudec DM seminar for reggresion motto "four short chains to check, one long chain for inference" MCMC "Markov Chain Monte Carlo methods are a class of algorithms for sampling from a probability distribution based on constructing a Markov chain that has the desired distribution as its equilibrium distribution. The state of the chain after a number of steps is then used as a sample of the desired distribution. The quality of the sample improves as a function of the number of steps." How many samples do you need? How many chains we need? Samuel Hudec DM seminar MCMC "Markov Chain Monte Carlo methods are a class of algorithms for sampling from a probability distribution based on constructing a Markov chain that has the desired distribution as its equilibrium distribution. The state of the chain after a number of steps is then used as a sample of the desired distribution. The quality of the sample improves as a function of the number of steps." How many samples do you need? How many chains we need? for reggresion motto "four short chains to check, one long chain for inference" Samuel Hudec DM seminar MCMC sourse: http://heattransfer.asmedigitalcollection.asme.org/ Samuel Hudec DM seminar MCMC Samuel Hudec DM seminar Common problems with unstacionarity or "Timing a wild chain" broad, tlat regions of the posterior density non-identifiable parameters MCMC (diagnostic) The default diadnostic output from Stan includes two metrics n eff measure of the effective number of samples Rhat R^ Gelman-Rubin convergence diagnostic (above 1 usually indicates that the chain has not yet converged) Samuel Hudec DM seminar MCMC (diagnostic) The default diadnostic output from Stan includes two metrics n eff measure of the effective number of samples Rhat R^ Gelman-Rubin convergence diagnostic (above 1 usually indicates that the chain has not yet converged) Common problems with unstacionarity or "Timing a wild chain" broad, tlat regions of the posterior density non-identifiable parameters Samuel Hudec DM seminar MCMC (diagnostic) source: McElreath, R., Statistical Rethinking CRC Press 2016. Samuel Hudec DM seminar strongly recommend installing RStudio which has basic support for .stan file types and syntax highlighting for Stan Stan enviroment (before we start) build model in Stan file mymodel.stan use R (or another enviroment) for load data set, runnig Stan model, print fit, diagnostics, plotting, .... Samuel Hudec DM seminar Stan enviroment (before we start) build model in Stan file mymodel.stan use R (or another enviroment) for load data set, runnig Stan model, print fit, diagnostics, plotting, .... strongly recommend installing RStudio which has basic support for .stan file types and syntax highlighting for Stan Samuel Hudec DM seminar Stan enviroment source: Kruschke, J., K., Doing Bayesian Data Analysis Elsevier Inc. 2015 Samuel Hudec DM seminar transformed data ... declarations ...f statements ... parametersg ... declarationsf ... transformedg parameters ... declarations ... statementsf ... modelg ... declarationsf ... statements ... generatedg quantities ... declarations ...f statements ... g Stan enviroment (General structure of Stan model specification) data ... declarationsf ... g Samuel Hudec DM seminar parameters ... declarationsf ... transformedg parameters ... declarations ... statementsf ... modelg ... declarationsf ... statements ... generatedg quantities ... declarations ...f statements ... g Stan enviroment (General structure of Stan model specification) data ... declarationsf ... transformedg data ... declarations ...f statements ... g Samuel Hudec DM seminar transformed parameters ... declarations ... statementsf ... modelg ... declarationsf ... statements ... generatedg quantities ... declarations ...f statements ... g Stan enviroment (General structure of Stan model specification) data ... declarationsf ... transformedg data ... declarations ...f
Recommended publications
  • Stan: a Probabilistic Programming Language

    Stan: a Probabilistic Programming Language

    JSS Journal of Statistical Software January 2017, Volume 76, Issue 1. doi: 10.18637/jss.v076.i01 Stan: A Probabilistic Programming Language Bob Carpenter Andrew Gelman Matthew D. Hoffman Columbia University Columbia University Adobe Creative Technologies Lab Daniel Lee Ben Goodrich Michael Betancourt Columbia University Columbia University Columbia University Marcus A. Brubaker Jiqiang Guo Peter Li York University NPD Group Columbia University Allen Riddell Indiana University Abstract Stan is a probabilistic programming language for specifying statistical models. A Stan program imperatively defines a log probability function over parameters conditioned on specified data and constants. As of version 2.14.0, Stan provides full Bayesian inference for continuous-variable models through Markov chain Monte Carlo methods such as the No-U-Turn sampler, an adaptive form of Hamiltonian Monte Carlo sampling. Penalized maximum likelihood estimates are calculated using optimization methods such as the limited memory Broyden-Fletcher-Goldfarb-Shanno algorithm. Stan is also a platform for computing log densities and their gradients and Hessians, which can be used in alternative algorithms such as variational Bayes, expectation propa- gation, and marginal inference using approximate integration. To this end, Stan is set up so that the densities, gradients, and Hessians, along with intermediate quantities of the algorithm such as acceptance probabilities, are easily accessible. Stan can be called from the command line using the cmdstan package, through R using the rstan package, and through Python using the pystan package. All three interfaces sup- port sampling and optimization-based inference with diagnostics and posterior analysis. rstan and pystan also provide access to log probabilities, gradients, Hessians, parameter transforms, and specialized plotting.
  • Stan: a Probabilistic Programming Language

    Stan: a Probabilistic Programming Language

    JSS Journal of Statistical Software MMMMMM YYYY, Volume VV, Issue II. http://www.jstatsoft.org/ Stan: A Probabilistic Programming Language Bob Carpenter Andrew Gelman Matt Hoffman Columbia University Columbia University Adobe Research Daniel Lee Ben Goodrich Michael Betancourt Columbia University Columbia University University of Warwick Marcus A. Brubaker Jiqiang Guo Peter Li University of Toronto, NPD Group Columbia University Scarborough Allen Riddell Dartmouth College Abstract Stan is a probabilistic programming language for specifying statistical models. A Stan program imperatively defines a log probability function over parameters conditioned on specified data and constants. As of version 2.2.0, Stan provides full Bayesian inference for continuous-variable models through Markov chain Monte Carlo methods such as the No-U-Turn sampler, an adaptive form of Hamiltonian Monte Carlo sampling. Penalized maximum likelihood estimates are calculated using optimization methods such as the Broyden-Fletcher-Goldfarb-Shanno algorithm. Stan is also a platform for computing log densities and their gradients and Hessians, which can be used in alternative algorithms such as variational Bayes, expectation propa- gation, and marginal inference using approximate integration. To this end, Stan is set up so that the densities, gradients, and Hessians, along with intermediate quantities of the algorithm such as acceptance probabilities, are easily accessible. Stan can be called from the command line, through R using the RStan package, or through Python using the PyStan package. All three interfaces support sampling and optimization-based inference. RStan and PyStan also provide access to log probabilities, gradients, Hessians, and data I/O. Keywords: probabilistic program, Bayesian inference, algorithmic differentiation, Stan.
  • Advancedhmc.Jl: a Robust, Modular and Efficient Implementation of Advanced HMC Algorithms

    Advancedhmc.Jl: a Robust, Modular and Efficient Implementation of Advanced HMC Algorithms

    2nd Symposium on Advances in Approximate Bayesian Inference, 20191{10 AdvancedHMC.jl: A robust, modular and efficient implementation of advanced HMC algorithms Kai Xu [email protected] University of Edinburgh Hong Ge [email protected] University of Cambridge Will Tebbutt [email protected] University of Cambridge Mohamed Tarek [email protected] UNSW Canberra Martin Trapp [email protected] Graz University of Technology Zoubin Ghahramani [email protected] University of Cambridge & Uber AI Labs Abstract Stan's Hamilton Monte Carlo (HMC) has demonstrated remarkable sampling robustness and efficiency in a wide range of Bayesian inference problems through carefully crafted adaption schemes to the celebrated No-U-Turn sampler (NUTS) algorithm. It is challeng- ing to implement these adaption schemes robustly in practice, hindering wider adoption amongst practitioners who are not directly working with the Stan modelling language. AdvancedHMC.jl (AHMC) contributes a modular, well-tested, standalone implementation of NUTS that recovers and extends Stan's NUTS algorithm. AHMC is written in Julia, a modern high-level language for scientific computing, benefiting from optional hardware acceleration and interoperability with a wealth of existing software written in both Julia and other languages, such as Python. Efficacy is demonstrated empirically by comparison with Stan through a third-party Markov chain Monte Carlo benchmarking suite. 1. Introduction Hamiltonian Monte Carlo (HMC) is an efficient Markov chain Monte Carlo (MCMC) algo- rithm which avoids random walks by simulating Hamiltonian dynamics to make proposals (Duane et al., 1987; Neal et al., 2011). Due to the statistical efficiency of HMC, it has been widely applied to fields including physics (Duane et al., 1987), differential equations (Kramer et al., 2014), social science (Jackman, 2009) and Bayesian inference (e.g.
  • Maple Advanced Programming Guide

    Maple Advanced Programming Guide

    Maple 9 Advanced Programming Guide M. B. Monagan K. O. Geddes K. M. Heal G. Labahn S. M. Vorkoetter J. McCarron P. DeMarco c Maplesoft, a division of Waterloo Maple Inc. 2003. ­ ii ¯ Maple, Maplesoft, Maplet, and OpenMaple are trademarks of Water- loo Maple Inc. c Maplesoft, a division of Waterloo Maple Inc. 2003. All rights re- ­ served. The electronic version (PDF) of this book may be downloaded and printed for personal use or stored as a copy on a personal machine. The electronic version (PDF) of this book may not be distributed. Information in this document is subject to change without notice and does not rep- resent a commitment on the part of the vendor. The software described in this document is furnished under a license agreement and may be used or copied only in accordance with the agreement. It is against the law to copy the software on any medium as specifically allowed in the agreement. Windows is a registered trademark of Microsoft Corporation. Java and all Java based marks are trademarks or registered trade- marks of Sun Microsystems, Inc. in the United States and other countries. Maplesoft is independent of Sun Microsystems, Inc. All other trademarks are the property of their respective owners. This document was produced using a special version of Maple that reads and updates LATEX files. Printed in Canada ISBN 1-894511-44-1 Contents Preface 1 Audience . 1 Worksheet Graphical Interface . 2 Manual Set . 2 Conventions . 3 Customer Feedback . 3 1 Procedures, Variables, and Extending Maple 5 Prerequisite Knowledge . 5 In This Chapter .
  • End-User Probabilistic Programming

    End-User Probabilistic Programming

    End-User Probabilistic Programming Judith Borghouts, Andrew D. Gordon, Advait Sarkar, and Neil Toronto Microsoft Research Abstract. Probabilistic programming aims to help users make deci- sions under uncertainty. The user writes code representing a probabilistic model, and receives outcomes as distributions or summary statistics. We consider probabilistic programming for end-users, in particular spread- sheet users, estimated to number in tens to hundreds of millions. We examine the sources of uncertainty actually encountered by spreadsheet users, and their coping mechanisms, via an interview study. We examine spreadsheet-based interfaces and technology to help reason under uncer- tainty, via probabilistic and other means. We show how uncertain values can propagate uncertainty through spreadsheets, and how sheet-defined functions can be applied to handle uncertainty. Hence, we draw conclu- sions about the promise and limitations of probabilistic programming for end-users. 1 Introduction In this paper, we discuss the potential of bringing together two rather distinct approaches to decision making under uncertainty: spreadsheets and probabilistic programming. We start by introducing these two approaches. 1.1 Background: Spreadsheets and End-User Programming The spreadsheet is the first \killer app" of the personal computer era, starting in 1979 with Dan Bricklin and Bob Frankston's VisiCalc for the Apple II [15]. The primary interface of a spreadsheet|then and now, four decades later|is the grid, a two-dimensional array of cells. Each cell may hold a literal data value, or a formula that computes a data value (and may depend on the data in other cells, which may themselves be computed by formulas).
  • An Introduction to Data Analysis Using the Pymc3 Probabilistic Programming Framework: a Case Study with Gaussian Mixture Modeling

    An Introduction to Data Analysis Using the Pymc3 Probabilistic Programming Framework: a Case Study with Gaussian Mixture Modeling

    An introduction to data analysis using the PyMC3 probabilistic programming framework: A case study with Gaussian Mixture Modeling Shi Xian Liewa, Mohsen Afrasiabia, and Joseph L. Austerweila aDepartment of Psychology, University of Wisconsin-Madison, Madison, WI, USA August 28, 2019 Author Note Correspondence concerning this article should be addressed to: Shi Xian Liew, 1202 West Johnson Street, Madison, WI 53706. E-mail: [email protected]. This work was funded by a Vilas Life Cycle Professorship and the VCRGE at University of Wisconsin-Madison with funding from the WARF 1 RUNNING HEAD: Introduction to PyMC3 with Gaussian Mixture Models 2 Abstract Recent developments in modern probabilistic programming have offered users many practical tools of Bayesian data analysis. However, the adoption of such techniques by the general psychology community is still fairly limited. This tutorial aims to provide non-technicians with an accessible guide to PyMC3, a robust probabilistic programming language that allows for straightforward Bayesian data analysis. We focus on a series of increasingly complex Gaussian mixture models – building up from fitting basic univariate models to more complex multivariate models fit to real-world data. We also explore how PyMC3 can be configured to obtain significant increases in computational speed by taking advantage of a machine’s GPU, in addition to the conditions under which such acceleration can be expected. All example analyses are detailed with step-by-step instructions and corresponding Python code. Keywords: probabilistic programming; Bayesian data analysis; Markov chain Monte Carlo; computational modeling; Gaussian mixture modeling RUNNING HEAD: Introduction to PyMC3 with Gaussian Mixture Models 3 1 Introduction Over the last decade, there has been a shift in the norms of what counts as rigorous research in psychological science.
  • The Semantics of Subroutines and Iteration in the Bayesian Programming Language Probt

    The Semantics of Subroutines and Iteration in the Bayesian Programming Language Probt

    1 The semantics of Subroutines and Iteration in the Bayesian Programming language ProBT R. LAURENT∗, K. MEKHNACHA∗, E. MAZERy and P. BESSIERE` z ∗ProbaYes S.A.S, Grenoble, France yUniversity of Grenoble-Alpes, CNRS/LIG, Grenoble, France zUniversity Pierre et Marie Curie, CNRS/ISIR, Paris, France Abstract—Bayesian models are tools of choice when solving 8 8 8 problems with incomplete information. Bayesian networks pro- > > >Variables > > <> vide a first but limited approach to address such problems. > <> Decomposition <> Specification (π) For real world applications, additional semantics is needed to Description > P arametric construct more complex models, especially those with repetitive > :>F orms > > P rogram structures or substructures. ProBT, a Bayesian a programming > :> > Identification (based on δ) language, provides a set of constructs for developing and applying :> complex models with substructures and repetitive structures. Question The goal of this paper is to present and discuss the semantics associated to these constructs. Figure 1. A Bayesian program is constructed from a Description and a Question. The Description is given by the programmer who writes a Index Terms—Probabilistic Programming semantics , Bayesian Specification of a model π and an Identification of its parameter values, Programming, ProBT which can be set in the program or obtained through a learning process from a data set δ. The Specification is constructed from a set of relevant variables, a decomposition of the joint probability distribution over these variables and a set of parametric forms (mathematical models) for the terms I. INTRODUCTION of this decomposition. ProBT [1] was designed to translate the ideas of E.T. Jaynes [2] into an actual programming language.
  • Teaching Bayesian Behaviours to Video Game Characters

    Teaching Bayesian Behaviours to Video Game Characters

    Robotics and Autonomous Systems 47 (2004) 177–185 Teaching Bayesian behaviours to video game characters Ronan Le Hy∗, Anthony Arrigoni, Pierre Bessière, Olivier Lebeltel GRAVIR/IMAG, INRIA Rhˆone-Alpes, ZIRST, 38330 Montbonnot, France Abstract This article explores an application of Bayesian programming to behaviours for synthetic video games characters. We address the problem of real-time reactive selection of elementary behaviours for an agent playing a first person shooter game. We show how Bayesian programming can lead to condensed and easier formalisation of finite state machine-like behaviour selection, and lend itself to learning by imitation, in a fully transparent way for the player. © 2004 Published by Elsevier B.V. Keywords: Bayesian programming; Video games characters; Finite state machine; Learning by imitation 1. Introduction After listing our practical objectives, we will present our Bayesian model. We will show how we use it to Today’s video games feature synthetic characters specify by hand a behaviour, and how we use it to involved in complex interactions with human players. learn a behaviour. We will tackle learning by exam- A synthetic character may have one of many different ple using a high-level interface, and then the natural roles: tactical enemy, partner for the human, strategic controls of the game. We will show that it is possible opponent, simple unit amongst many, commenter, etc. to map the player’s actions onto bot states, and use In all of these cases, the game developer’s ultimate this reconstruction to learn our model. Finally, we will objective is for the synthetic character to act like a come back to our objectives as a conclusion.
  • GNU Octave a High-Level Interactive Language for Numerical Computations Edition 3 for Octave Version 2.0.13 February 1997

    GNU Octave a High-Level Interactive Language for Numerical Computations Edition 3 for Octave Version 2.0.13 February 1997

    GNU Octave A high-level interactive language for numerical computations Edition 3 for Octave version 2.0.13 February 1997 John W. Eaton Published by Network Theory Limited. 15 Royal Park Clifton Bristol BS8 3AL United Kingdom Email: [email protected] ISBN 0-9541617-2-6 Cover design by David Nicholls. Errata for this book will be available from http://www.network-theory.co.uk/octave/manual/ Copyright c 1996, 1997John W. Eaton. This is the third edition of the Octave documentation, and is consistent with version 2.0.13 of Octave. Permission is granted to make and distribute verbatim copies of this man- ual provided the copyright notice and this permission notice are preserved on all copies. Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided that the en- tire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this manual into another language, under the same conditions as for modified versions. Portions of this document have been adapted from the gawk, readline, gcc, and C library manuals, published by the Free Software Foundation, 59 Temple Place—Suite 330, Boston, MA 02111–1307, USA. i Table of Contents Publisher’s Preface ...................... 1 Author’s Preface ........................ 3 Acknowledgements ........................................ 3 How You Can Contribute to Octave ........................ 5 Distribution .............................................. 6 1 A Brief Introduction to Octave ....... 7 1.1 Running Octave...................................... 7 1.2 Simple Examples ..................................... 7 Creating a Matrix ................................. 7 Matrix Arithmetic ................................. 8 Solving Linear Equations..........................
  • Stan Modeling Language

    Stan Modeling Language

    Stan Modeling Language User’s Guide and Reference Manual Stan Development Team Stan Version 2.12.0 Tuesday 6th September, 2016 mc-stan.org Stan Development Team (2016) Stan Modeling Language: User’s Guide and Reference Manual. Version 2.12.0. Copyright © 2011–2016, Stan Development Team. This document is distributed under the Creative Commons Attribution 4.0 International License (CC BY 4.0). For full details, see https://creativecommons.org/licenses/by/4.0/legalcode The Stan logo is distributed under the Creative Commons Attribution- NoDerivatives 4.0 International License (CC BY-ND 4.0). For full details, see https://creativecommons.org/licenses/by-nd/4.0/legalcode Stan Development Team Currently Active Developers This is the list of current developers in order of joining the development team (see the next section for former development team members). • Andrew Gelman (Columbia University) chief of staff, chief of marketing, chief of fundraising, chief of modeling, max marginal likelihood, expectation propagation, posterior analysis, RStan, RStanARM, Stan • Bob Carpenter (Columbia University) language design, parsing, code generation, autodiff, templating, ODEs, probability functions, con- straint transforms, manual, web design / maintenance, fundraising, support, training, Stan, Stan Math, CmdStan • Matt Hoffman (Adobe Creative Technologies Lab) NUTS, adaptation, autodiff, memory management, (re)parameterization, optimization C++ • Daniel Lee (Columbia University) chief of engineering, CmdStan, builds, continuous integration, testing, templates,
  • Maple 7 Programming Guide

    Maple 7 Programming Guide

    Maple 7 Programming Guide M. B. Monagan K. O. Geddes K. M. Heal G. Labahn S. M. Vorkoetter J. McCarron P. DeMarco c 2001 by Waterloo Maple Inc. ii • Waterloo Maple Inc. 57 Erb Street West Waterloo, ON N2L 6C2 Canada Maple and Maple V are registered trademarks of Waterloo Maple Inc. c 2001, 2000, 1998, 1996 by Waterloo Maple Inc. All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the copyright holder, except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the former are not especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone. Contents 1 Introduction 1 1.1 Getting Started ........................ 2 Locals and Globals ...................... 7 Inputs, Parameters, Arguments ............... 8 1.2 Basic Programming Constructs ............... 11 The Assignment Statement ................. 11 The for Loop......................... 13 The Conditional Statement ................. 15 The while Loop....................... 19 Modularization ........................ 20 Recursive Procedures ..................... 22 Exercise ............................ 24 1.3 Basic Data Structures .................... 25 Exercise ............................ 26 Exercise ............................ 28 A MEMBER Procedure ..................... 28 Exercise ............................ 29 Binary Search ......................... 29 Exercises ........................... 30 Plotting the Roots of a Polynomial ............. 31 1.4 Computing with Formulæ .................. 33 The Height of a Polynomial ................. 34 Exercise ...........................
  • Bayesian Linear Mixed Models with Polygenic Effects

    Bayesian Linear Mixed Models with Polygenic Effects

    JSS Journal of Statistical Software June 2018, Volume 85, Issue 6. doi: 10.18637/jss.v085.i06 Bayesian Linear Mixed Models with Polygenic Effects Jing Hua Zhao Jian’an Luan Peter Congdon University of Cambridge University of Cambridge University of London Abstract We considered Bayesian estimation of polygenic effects, in particular heritability in relation to a class of linear mixed models implemented in R (R Core Team 2018). Our ap- proach is applicable to both family-based and population-based studies in human genetics with which a genetic relationship matrix can be derived either from family structure or genome-wide data. Using a simulated and a real data, we demonstrate our implementa- tion of the models in the generic statistical software systems JAGS (Plummer 2017) and Stan (Carpenter et al. 2017) as well as several R packages. In doing so, we have not only provided facilities in R linking standalone programs such as GCTA (Yang, Lee, Goddard, and Visscher 2011) and other packages in R but also addressed some technical issues in the analysis. Our experience with a host of general and special software systems will facilitate investigation into more complex models for both human and nonhuman genetics. Keywords: Bayesian linear mixed models, heritability, polygenic effects, relationship matrix, family-based design, genomewide association study. 1. Introduction The genetic basis of quantitative phenotypes has been a long-standing research problem as- sociated with a large and growing literature, and one of the earliest was by Fisher(1918) on additive effects of genetic variants (the polygenic effects). In human genetics it is common to estimate heritability, the proportion of polygenic variance to the total phenotypic variance, through twin and family studies.