An Introduction to Winbugs

Home , OpenBUGS, S (programming language), WinBUGS

WinBUGS

UBC Summer School

An Introduction to WinBUGS ∗ PIMS Collaborative Research Group Outline ∗ Summer School on Bayesian Modeling and Computation Goals and Scope Preliminaries Lung cancer data Gibbs sampling

The WinBUGS 14th July, 2008 Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Sebastien Haneuse, PhD Miscellaneous notes Group Health Center for Health Studies Department of Biostatistics, University of Washington

.1 WinBUGS

UBC Summer School

Outline

• Goals and Scope

Outline

• Preliminaries Goals and Scope

Preliminaries Lung cancer data • The WinBUGS Software Gibbs sampling The WinBUGS Software • Navigating the WinBUGS Menu System Graphical models WinBUGS syntax Data • Additional Topics Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.2 Goals and Scope WinBUGS UBC Summer School

Bayesian analyses

• Formulate the scientiﬁc question ∗ understanding the association between two variables ∗ prediction of some future event Outline Goals and Scope

Preliminaries • Identify a corresponding parameterization Lung cancer data Gibbs sampling ∗ ‘translation’ of the scientific question into statistical terms The WinBUGS Software and quantities Graphical models WinBUGS syntax Data • Specify prior distributions for the unknown parameters Navigating WinBUGS Specification Tool Update & Sample Monitor ∗ translation of substantive knowledge Tools ∗ identification of ‘non-informative’ priors Convergence and analysis ∗ establish several options towards a sensitivity analyses Additional Topics Errors and traps Miscellaneous notes

.3 Goals and Scope WinBUGS UBC Summer School

Bayesian analyses (cont)

• Characterize linkage between the parameters and the observed data Outline ∗ speciﬁcation of the likelihood Goals and Scope Preliminaries Lung cancer data Gibbs sampling

• Apply Bayes Theorem The WinBUGS Software ∗ posterior ∝ prior × likelihood Graphical models ∗ turn the Bayesian ‘handle’ WinBUGS syntax Data ∗ examine features of the posterior Navigating WinBUGS Specification Tool Update & Sample Monitor Tools • Interpret the results and refine the scientific question Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.4 Goals and Scope WinBUGS UBC Summer School

Where does WinBUGS ﬁt in?

• We’ve seen that there are many steps in a Bayesian analysis

• WinBUGS facilitates turning the Bayesian handle ∗ computational tool Outline ∗ via Markov Chain Monte Carlo (MCMC) Goals and Scope Preliminaries Lung cancer data Gibbs sampling • Cuts out much of the hard work for a very general class of The WinBUGS models Software Graphical models ∗ model: likelihood/prior combinations WinBUGS syntax Data

Navigating WinBUGS Specification Tool • Sole purpose is to implement MCMC-based analyses Update & Sample Monitor Tools ∗ perform tasks with a few lines of code that would take Convergence and analysis (potentially) hundreds of lines of ‘home-grown’ code Additional Topics Errors and traps ∗ highly efficient, both in terms of model specification but also Miscellaneous notes in the MCMC implementation

.5 Goals and Scope WinBUGS UBC Summer School

Caveats

• Underlying WinBUGS is a language

∗ even within a single language, there are often many ways of Outline ‘saying’ the same thing! Goals and Scope ∗ many idiosycracies Preliminaries Lung cancer data Gibbs sampling

The WinBUGS • Cannot accommodate all analyses Software Graphical models ∗ OpenBUGS is an open source version WinBUGS syntax Data

Navigating WinBUGS • WinBUGS User Manual: Speciﬁcation Tool Update & Sample Monitor Tools Beware: MCMC sampling can be dangerous! Convergence and analysis Additional Topics Errors and traps Miscellaneous notes

.6 Goals and Scope WinBUGS UBC Summer School

Scope

• A good way to learn, and implement your own analyses within WinBUGS, is to build on work by others ∗ hundreds of examples out there Outline ∗ WinBUGS manual and website Goals and Scope ∗ various books Preliminaries Lung cancer data Gibbs sampling

The WinBUGS • A reasonable strategy is to adapt existing code to your speciﬁc Software Graphical models setting WinBUGS syntax Data

Navigating WinBUGS • The main purpose here is to provide an overview of WinBUGS Speciﬁcation Tool Update & Sample Monitor functionality and syntax Tools Convergence and analysis ∗ understanding models and adapting them to your own Additional Topics purposes Errors and traps Miscellaneous notes

.7 Goals and Scope WinBUGS UBC Summer School

Scope (cont)

• Aimed at a new WinBUGS user ∗ assume some familiarity with the Bayesian philosophy and Outline theory Goals and Scope ∗ underlying premise of using draws from a distribution as a Preliminaries means to approximate its features Lung cancer data Gibbs sampling

The WinBUGS • Software Not a tutorial on the intricacies of Bayesian reasoning or MCMC Graphical models WinBUGS syntax ∗ model and prior speciﬁcation Data

∗ convergence, optimal sampling strategies, etc Navigating WinBUGS ∗ plenty of good texts for this Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis • An introductory WinBUGS book is on the way Additional Topics Errors and traps Miscellaneous notes

.8 WinBUGS

UBC Summer School

Outline

• Goals and Scope

• Preliminaries Outline ∗ Lung cancer data from Ohio Goals and Scope ∗ Gibbs sampling algorithm Preliminaries Lung cancer data Gibbs sampling

The WinBUGS • The WinBUGS Software Software Graphical models WinBUGS syntax • Navigating the WinBUGS Menu System Data Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor • Additional Topics Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.9 Preliminaries WinBUGS UBC Summer School

Lung cancer data from Ohio

• Population estimates and lung cancer death counts in 1988 ∗ 88 counties ∗ 2,220,177 individuals Outline ∗ 5,533 deaths Goals and Scope Preliminaries Lung cancer data Gibbs sampling

• Covariates include The WinBUGS Software ∗ gender Graphical models ∗ race (white vs. non-white) WinBUGS syntax Data

∗ age (10-year bands) Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools • National Center for Health Statistics Compressed Mortality ﬁle Convergence and analysis ∗ Waller et al. (1997) Additional Topics Errors and traps Miscellaneous notes

.10 Preliminaries WinBUGS UBC Summer School Scientiﬁc interest

• Estimate the effects of gender and race on lung cancer mortality ∗ marginal associations 50 50 Outline

● ● ● ●

40 40 Goals and Scope

● ● ●● ● ● ● ● ● ● ●● ● ● Preliminaries ● ● ●● ● ● ●● ● ● ● ● 30 ● 30 ● Lung cancer data ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● Gibbs sampling ● ●● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ●● ●● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ● ● ●● ●●● ● ● ● ● ● ● ● ● ● ● ● The WinBUGS 20 ● ● 20 ● ● ● ● ● ● ● ● ● ● ● ● ●● ● Software ●● ● ● ● ● ●● ● ● Mortality rate, x 10,000 ●● Mortality rate, x 10,000 ● ● ● ● ● ● Graphical models ●● ● ● ● ● 10 10 WinBUGS syntax Data

0 0 Navigating WinBUGS Speciﬁcation Tool 0 5 10 15 20 50 52 54 56 58 Update & Sample Monitor Percent non−white Percent female Tools Convergence and analysis

Additional Topics • To assess their full joint effect, we might be interested in a Errors and traps potential interaction Miscellaneous notes ∗ consider a modeling framework

.11 Preliminaries WinBUGS UBC Summer School Data

• Four unique covariate patterns ∗ indexed using j, where j = 1, ..., 4

th th Outline • Nkj denotes population size in the k county with the j covariate pattern Goals and Scope Preliminaries Lung cancer data Gibbs sampling • Ykj corresponding number of lung cancer deaths The WinBUGS Software Graphical models • Dataframe in R WinBUGS syntax Data

> ohio Navigating WinBUGS county sex race pattern N Y Speciﬁcation Tool [1,] 39001 0 0 1 2631 9 Update & Sample Monitor Tools [2,] 39001 1 0 2 2883 1 Convergence and analysis [3,] 39001 0 1 3 16 0 [4,] 39001 1 1 4 23 0 Additional Topics [5,] 39003 0 0 1 9315 31 Errors and traps Miscellaneous notes [6,] 39003 1 0 2 12095 19 ...

.12 Preliminaries WinBUGS UBC Summer School

Likelihood/Model

• Observed data: Ykj ∼ Binomial(Nkj , πkj )

• Logistic disease model Outline Goals and Scope

logit πkj = β0 + βSSj + βRRj + βSRSj ∗ Rj + Vk Preliminaries Lung cancer data Gibbs sampling ∗ R = 0/1: white vs. non-white The WinBUGS Software ∗ S = 0/1: male vs. female Graphical models WinBUGS syntax ∗ Vk , county-speciﬁc random effect Data

Navigating WinBUGS Speciﬁcation Tool • Assume the V are normally distributed across the 88 counties Update & Sample Monitor k Tools Convergence and analysis 2 Vk ∼ Normal(0, σ ) Additional Topics Errors and traps Miscellaneous notes

.13 Preliminaries WinBUGS UBC Summer School Prior speciﬁcation

• In the absence of substantive scientiﬁc input, adopt vague, non-informative priors

• Regression coefﬁcients Outline β ∼ Normal(0, 1e6) Goals and Scope Preliminaries Lung cancer data • Random effect variance Gibbs sampling The WinBUGS ∗ often parameterized in terms of the precision Software Graphical models 1 WinBUGS syntax τ = ∼ ( . , . ) Data 2 Gamma 0 5 0 01 σ Navigating WinBUGS Speciﬁcation Tool ∗ mainly to exploit conjugacy Update & Sample Monitor Tools Convergence and analysis

Additional Topics • Consider the induced prior for σ Errors and traps Miscellaneous notes ∗ median: 0.21 ∗ (2.5th, 97.5th) quantiles: (0.06, 4.51)

.14 A WinBUGS Session WinBUGS UBC Summer School

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.15 A WinBUGS Session WinBUGS UBC Summer School

Components/Windows

• Code file ∗ model specification ∗ data Outline ∗ initial values Goals and Scope Preliminaries Lung cancer data • WinBUGS tools Gibbs sampling The WinBUGS ∗ specification tool Software Graphical models ∗ update tool WinBUGS syntax ∗ sample monitor tool Data Navigating WinBUGS Specification Tool Update & Sample Monitor • Convergence diagnostics Tools Convergence and analysis ∗ time series plot Additional Topics ∗ Errors and traps Brooks-Gelman-Rubin statistic Miscellaneous notes

.16 A WinBUGS Session WinBUGS UBC Summer School

Components/Windows (cont)

• Posterior summary statistics

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data • Based on 2 chains of 10,000 reps each Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor ∗ burnin of 1,000 Tools Convergence and analysis

Additional Topics • These are just the default statistics Errors and traps Miscellaneous notes ∗ could extract the samples themselves using ‘coda’

.17 Gibbs sampling algorithm WinBUGS UBC Summer School

Gibbs sampler

• Goal is to generate samples from the posterior distribution

π(θ| Y) ∝ L(Y| θ) × π(θ) Outline • In many problems, θ consists of several parameters, often with Goals and Scope very different properties Preliminaries Lung cancer data ∗ probabilities, means, variances, log-odds ratios, ... Gibbs sampling + The WinBUGS ∗ may be deﬁned on [0,1], R and R , ... Software Graphical models WinBUGS syntax Data • Can be challenging to generate draws from the full joint posterior Navigating WinBUGS distribution Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis • Suppose θ can be divided into d components or subvectors Additional Topics Errors and traps ∗ θ = (θ1, ..., θd ) Miscellaneous notes

.18 Gibbs sampling algorithm WinBUGS UBC Summer School

Gibbs sampler (cont)

• Gibbs sampling scheme is deﬁned in terms of these subvectors ∗ cycle the d subvectors, drawing each subset conditional on the value of the other (d-1) subvectors Outline ∗ for the jth subvector, generate a random draw from the Goals and Scope conditional posterior Preliminaries Lung cancer data Gibbs sampling

π(θj | Y, θ−j ) The WinBUGS Software Graphical models where θ−j = (θ1, . . . , θj−1, θj+1, . . . , θd ) WinBUGS syntax Data

Navigating WinBUGS • It is often the case that subvectors can be identiﬁed that yield full Speciﬁcation Tool Update & Sample Monitor conditionals which are convenient to draw samples from Tools Convergence and analysis

∗ bivariate normal (see below) Additional Topics ∗ hierarchical models Errors and traps Miscellaneous notes

.19 Gibbs sampling algorithm WinBUGS UBC Summer School

Simple example

• Bivariate normal distribution with unknown mean „ « „„ « „ «« Y1 θ1 1 ρ ∼ Normal , Outline Y2 θ2 ρ 1 Goals and Scope

Preliminaries Lung cancer data ∗ assume ρ is known Gibbs sampling

The WinBUGS Software • Single observation y = (y , y ) Graphical models 1 2 WinBUGS syntax Data

Navigating WinBUGS • With a uniform prior on (θ1, θ2), the posterior is Speciﬁcation Tool Update & Sample Monitor „ «˛ „„ « „ «« Tools θ1 ˛ y1 1 ρ Convergence and analysis ˛ y ∼ Normal , θ2 ˛ y2 ρ 1 Additional Topics Errors and traps Miscellaneous notes

.20 Gibbs sampling algorithm WinBUGS UBC Summer School

Simple example (cont)

• Algorithms exist for sampling directly from a multivariate normal

distribution Outline

Goals and Scope • Consider an application of Gibbs sampling Preliminaries Lung cancer data Gibbs sampling

The WinBUGS • Require two full conditionals: Software Graphical models 2 WinBUGS syntax θ1| y, θ2 ∼ Normal(y1 + ρ(θ2 − y2), 1 − ρ ) Data

2 Navigating WinBUGS θ2| y, θ1 ∼ Normal(y2 + ρ(θ1 − y1), 1 − ρ ) Speciﬁcation Tool Update & Sample Monitor Tools • In the following y = (0, 0) and ρ = 0.6 Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.21 Gibbs sampling algorithm WinBUGS UBC Summer School Single run

(0) (0) • Start at (θ1 , θ2 ) = (3, 3) ∗ 2 complete iterations

(1) (0) (0) (0) (θ , θ ) (θ , θ ) Outline 3 1 2 1 2 Goals and Scope

2 Preliminaries Lung cancer data ( ) ( ) θ 2 θ 2 Gibbs sampling ( 1 , 2 ) 1 The WinBUGS Software

2 Graphical models θ 0 WinBUGS syntax ( ) ( ) ( ) ( ) θ 2 θ 1 θ 1 θ 1 ( 1 , 2 ) ( 1 , 2 ) Data Navigating WinBUGS −1 Speciﬁcation Tool Update & Sample Monitor Tools −2 Convergence and analysis

Additional Topics

−3 Errors and traps Miscellaneous notes −3 −2 −1 0 1 2 3

θ1

.22 Gibbs sampling algorithm WinBUGS UBC Summer School Four runs

• Dispersed starting points ∗ ﬁrst 10 complete iterations for each

3 Outline

Goals and Scope

2 Preliminaries Lung cancer data Gibbs sampling 1 The WinBUGS Software 2

θ Graphical models 0 WinBUGS syntax Data

−1 Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor

−2 Tools Convergence and analysis

Additional Topics −3 Errors and traps Miscellaneous notes −3 −2 −1 0 1 2 3

θ1

.23 Gibbs sampling algorithm WinBUGS UBC Summer School Four runs

• Dispersed starting points ∗ ﬁrst 10 complete iterations for each

3 Outline

Goals and Scope

● 2 ● Preliminaries Lung cancer data ● ● Gibbs sampling ●

1 ● ● ●● ● ● ● The WinBUGS ● ● Software ● 2 ● ● ● ● ● θ ● Graphical models 0 ● ● ● ● ● ● ● ● ● WinBUGS syntax ● ● Data ● ● −1 Navigating WinBUGS ● Speciﬁcation Tool ● Update & Sample Monitor

−2 Tools Convergence and analysis

Additional Topics −3 Errors and traps Miscellaneous notes −3 −2 −1 0 1 2 3

θ1

.24 Gibbs sampling algorithm WinBUGS UBC Summer School Four runs

• 2,500 iterations for each run 5

Outline

3 Goals and Scope

Preliminaries Lung cancer data Gibbs sampling 1

2 The WinBUGS

θ Software Graphical models

−1 WinBUGS syntax Data

Navigating WinBUGS

−3 Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis −5 Additional Topics Errors and traps −5 −3 −1 1 3 5 Miscellaneous notes

θ1

.25 Gibbs sampling algorithm WinBUGS UBC Summer School

Notes

• Each iteration now consists of d steps ∗ can be inefﬁcient, especially for highly correlated Outline parameters Goals and Scope Preliminaries ∗ block-updating can sometimes be exploited and WinBUGS Lung cancer data has some functionality Gibbs sampling The WinBUGS Software Graphical models • If a full conditional is not of a convenient form, use a Metropolis WinBUGS syntax update Data Navigating WinBUGS ∗ Metropolis-within-Gibbs Speciﬁcation Tool Update & Sample Monitor ∗ chain no longer guaranteed to move with each iteration Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.26 WinBUGS

UBC Summer School

Outline

• Goals and Scope

• Preliminaries Outline

Goals and Scope

• The WinBUGS Software Preliminaries Lung cancer data ∗ Graphical models Gibbs sampling ∗ WinBUGS syntax The WinBUGS Software ∗ Data Graphical models WinBUGS syntax Data • Navigating the WinBUGS Menu System Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools • Additional Topics Convergence and analysis Additional Topics Errors and traps Miscellaneous notes

.27 The WinBUGS Software WinBUGS UBC Summer School

BUGS

• Bayesian inference Using Gibbs Sampling

• WinBUGS runs under Windows 95/98/NT ∗ provides a graphical interface to the BUGS language Outline ∗ jointly developed by Biostatistics Unit of the Medical Goals and Scope Research Council, Cambridge and Imperial College School Preliminaries Lung cancer data of Medicine, St. Mary’s, London Gibbs sampling The WinBUGS Software Graphical models • Freely available for download from the WinBUGS website: WinBUGS syntax Data http://www.mrc-bsu.cam.ac.uk/bugs/ Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor ∗ key is required to unlock the educational version Tools Convergence and analysis

Additional Topics Errors and traps • Versions of BUGS for a variety of other platforms Miscellaneous notes ∗ e.g., several approaches to running WinBUGS on a Mac

.28 The WinBUGS Software WinBUGS UBC Summer School

WinBUGS

• With WinBUGS you both make and use software

• Each analysis results in the creation of a new software system ∗ you provide the plan Outline ∗ WinBUGS provides the building materials and tools Goals and Scope Preliminaries Lung cancer data Gibbs sampling • Two user interfaces for the plan (i.e., model speciﬁcation) The WinBUGS Software ∗ GUI Graphical models ∗ script language WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool • Planning/structure of the software built on principles of graphical Update & Sample Monitor Tools models Convergence and analysis ∗ graphical representations of Bayesian models Additional Topics Errors and traps ∗ framework for calculating full conditional distributions, Miscellaneous notes required for Gibbs sampling

.29 The WinBUGS Software WinBUGS UBC Summer School

WinBUGS

• Modular design written in Component Pascal Outline ∗ consists of over 200 modules Goals and Scope ∗ each has a well-deﬁned interface Preliminaries Lung cancer data ∗ consistency checks, across modules, are performed at Gibbs sampling compilation The WinBUGS Software Graphical models WinBUGS syntax • OpenBUGS permits development of new modules Data Navigating WinBUGS http://mathstat.helsinki.fi/openbugs/ Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.30 Graphical representations of models WinBUGS UBC Summer School Graphs as a formal language

• A graph is a set of vertices and edges ∗ vertices are also called nodes

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools • The structure of a graph can be used to represent a wide range Convergence and analysis

of systems Additional Topics Errors and traps Miscellaneous notes • Additional complexity can be incorporated by assigning attributes to the nodes and edges

.31 Graphical representations of models WinBUGS UBC Summer School Bayesian systems

• Bayesian systems can represented via graphs ∗ structure of the graph is a representation of the joint posterior probability distribution

Outline • Nodes represent Goals and Scope Preliminaries ∗ parameters Lung cancer data ∗ data Gibbs sampling The WinBUGS Software Graphical models • Edges represent relationships WinBUGS syntax Data ∗ dependencies among data/parameters Navigating WinBUGS ∗ all are ‘directed’ Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

• Useful to distinguish ‘child’ and ‘parent’ nodes Additional Topics Errors and traps Miscellaneous notes • A good introduction to graphical models can be found in Fryback et al. (2001).

.32 Linear regression with a single predictor WinBUGS UBC Summer School

Mathematical representation

• Consider a simple linear regression

Stage I: Likelihood Outline

−1 Goals and Scope Yi ∼iid Normal(µi , τ ), i = 1,..., N Preliminaries µi = β0 + βXXi Lung cancer data Gibbs sampling

The WinBUGS Stage II: Priors Software Graphical models 6 WinBUGS syntax β0 ∼ Normal(0, 1e ) Data 6 βX ∼ Normal(0, 1e ) Navigating WinBUGS Speciﬁcation Tool τ ∼ Gamma(0.5, 0.01) Update & Sample Monitor Tools Convergence and analysis • τ is the precision (i.e., inverse variance) Additional Topics Errors and traps √1 Miscellaneous notes ∗ interest may also lie in the standard deviation, σ = τ

.33 Linear regression with a single predictor WinBUGS UBC Summer School

Graphical representation

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.34 Linear regression with a single predictor WinBUGS UBC Summer School

WinBUGS representation

model { ## Model and likelihood Outline for(i in 1:N) Goals and Scope { Y[i] ~ dnorm(mu[i], tau) Preliminaries mu[i] <- beta0 + betaX X[i] Lung cancer data * Gibbs sampling } The WinBUGS Software ## Priors Graphical models beta0 ~ dnorm(0.0, 1.0E-6) WinBUGS syntax betaX ~ dnorm(0.0, 1.0E-6) Data

tau ~ dgamma(0.5, 0.01) Navigating WinBUGS Speciﬁcation Tool ## Additional parameter of interest Update & Sample Monitor sigma <- 1 / sqrt(tau) Tools } Convergence and analysis Additional Topics Errors and traps Miscellaneous notes

.35 Building blocks WinBUGS UBC Summer School

Structures

• Three types of nodes ∗ stochastic ∗ logical Outline ∗ constant Goals and Scope Preliminaries Lung cancer data Gibbs sampling

• Two types of edges The WinBUGS Software ∗ single arrow Graphical models ∗ double arrow WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool • Plate Update & Sample Monitor Tools ∗ permits the concise representation of repetition Convergence and analysis ∗ convenient for structured models Additional Topics Errors and traps Miscellaneous notes

.36 Building blocks WinBUGS UBC Summer School

Stochastic nodes

• Stochastic nodes are data/parameters that are assigned a distribution

∗ e.g., Yi , β0, βX, and τ Outline ∗ distribution → likelihood or prior Goals and Scope

Preliminaries Lung cancer data • Appear once on the LHS of the mathematical representation Gibbs sampling

The WinBUGS ∗ accompanied by a ‘∼’ Software Graphical models WinBUGS syntax • Appear once in the graphical representation Data Navigating WinBUGS ∗ as an oval Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis • Appear once on the LHS in the WinBUGS representation Additional Topics Errors and traps ∗ again, accompanied by a ‘∼’ Miscellaneous notes

.37 Building blocks WinBUGS UBC Summer School

Stochastic nodes (cont)

Outline

• Each stochastic node in each representation requires Goals and Scope ∗ a name Preliminaries Lung cancer data ∗ a distribution Gibbs sampling ∗ speciﬁcation of parameters for the distribution The WinBUGS Software Graphical models WinBUGS syntax • In a graph, these are referred to as ‘attributes’ Data Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.38 Building blocks WinBUGS UBC Summer School

Stochastic node Yi

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.39 Building blocks WinBUGS UBC Summer School

Stochastic node τ

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.40 Building blocks WinBUGS UBC Summer School

Logical nodes

• Logical nodes are deterministic functions of other parameters/data

∗ e.g., µi , σ Outline Goals and Scope

Preliminaries • Appear once on the LHS of the mathematical representation Lung cancer data Gibbs sampling

∗ accompanied by an ‘=’ The WinBUGS Software Graphical models WinBUGS syntax • Appear once in the graphical representation Data ∗ as an oval, although the ‘attributes’ are different (see below) Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools • Appear once on the LHS in the WinBUGS representation Convergence and analysis Additional Topics ∗ accompanied by a ‘<-’ Errors and traps Miscellaneous notes

.41 Building blocks WinBUGS UBC Summer School

Logical node µi

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.42 Building blocks WinBUGS UBC Summer School

Logical node σ

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.43 Building blocks WinBUGS UBC Summer School

Logical nodes (nodes)

• Logical nodes require speciﬁcation of ∗ a name ∗ a value Outline Goals and Scope

Preliminaries • ‘link’ option is to facilitate speciﬁcation of generalized linear Lung cancer data models Gibbs sampling The WinBUGS ∗ e.g., log, logit Software Graphical models WinBUGS syntax Data

• Logical node ‘deviance’ is automatically created and monitored Navigating WinBUGS Speciﬁcation Tool ∗ −2×log(likelihood) Update & Sample Monitor Tools ∗ contributes to the calculation of the Deviance Information Convergence and analysis

Criterion (DIC); Additional Topics ∗ developed as a measure of model comparison and criticism Errors and traps Miscellaneous notes

.44 Building blocks WinBUGS UBC Summer School

Constants

• Constants are just that Outline

∗ include data such as Xi ,N Goals and Scope

∗ ﬁxed parameters, such as the variances in the normal priors Preliminaries Lung cancer data Gibbs sampling • Can either be incorporated as nodes (rectangles), or as part of The WinBUGS Software the data Graphical models WinBUGS syntax Data ∗ Have no ‘parents’ in the graph Navigating WinBUGS Speciﬁcation Tool ∗ not updated in the MCMC scheme Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.45 Building blocks WinBUGS UBC Summer School

Edges

• Edges represent dependencies between nodes ∗ nature of the dependency depends on the characteristics of the child node Outline Goals and Scope

Preliminaries • Single arrow denotes a stochastic dependence Lung cancer data Gibbs sampling

∗ child node is stochastic The WinBUGS ∗ parental node is a parameter for the probability distribution Software Graphical models associated with the child node WinBUGS syntax Data

Navigating WinBUGS • Double arrow denotes a deterministic dependence Speciﬁcation Tool Update & Sample Monitor Tools ∗ child node is logical Convergence and analysis ∗ parental node directly determines (at least in part) the value Additional Topics Errors and traps of the child node Miscellaneous notes

.46 Ohio lung cancer example WinBUGS UBC Summer School

Three representations

• Consider the three representations of the Ohio lung data model

Outline • Data consist of over 2 million records Goals and Scope Preliminaries ∗ aggregated (in R) according to county, sex and race Lung cancer data Gibbs sampling

The WinBUGS • County-speciﬁc index: k = 1, ..., K Software Graphical models WinBUGS syntax ∗ K = 88 Data

Navigating WinBUGS Speciﬁcation Tool • Covariate (sex/race)-speciﬁc index: j = 1, ..., J Update & Sample Monitor Tools ∗ J = 4 Convergence and analysis Additional Topics Errors and traps Miscellaneous notes

.47 Ohio lung cancer example WinBUGS UBC Summer School

Mathematical representation

Stage I: Likelihood

Ykj ∼iid Binomial(Nkj , πkj ), k = 1, . . . , K, j = 1, . . . , J Outline

Goals and Scope πkj = β0 + βSSj + βRRj + βSRSj ∗ Rj + Vk Preliminaries θ = exp{βSR} Lung cancer data Gibbs sampling

−1 The WinBUGS Vk ∼ Normal(0, τ ) Software Graphical models WinBUGS syntax √1 σ = τ Data Navigating WinBUGS Speciﬁcation Tool Stage II: Priors Update & Sample Monitor Tools Convergence and analysis 6 (β0, βS, βR, βSR) ∼iid Normal(0, 1e ) Additional Topics τ ∼ Gamma(0.5, 0.01) Errors and traps Miscellaneous notes

.48 Ohio lung cancer example WinBUGS UBC Summer School

Graphical representation: Stochastic node Ykj

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.49 Ohio lung cancer example WinBUGS UBC Summer School

Graphical representation: Logical node πkj

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.50 Ohio lung cancer example WinBUGS UBC Summer School

WinBUGS representation

model { ## Model and likelihood for(k in 1:K){

V[k] ~ dnorm(0, tau) Outline for(j in 1:J){ Y[k,j] ~ dbin(p[k,j], N[k,j]) Goals and Scope logit(p[k,j]) <- beta0 + betaS*sex[j] + betaR*race[j] Preliminaries + betaSR*sex[j]*race[j] + V[k] Lung cancer data } Gibbs sampling } The WinBUGS Software ## Priors Graphical models beta0 ~ dnorm(0.0, 1.0E-6) WinBUGS syntax Data betaS ~ dnorm(0.0, 1.0E-6) betaR ~ dnorm(0.0, 1.0E-6) Navigating WinBUGS betaSR ~ dnorm(0.0, 1.0E-6) Speciﬁcation Tool Update & Sample Monitor tau ~ dgamma(0.5, 0.01) Tools Convergence and analysis

## Additional parameters of interest Additional Topics theta <- exp(betaSR) Errors and traps sigma <- 1 / sqrt(tau) Miscellaneous notes }

.51 WinBUGS and graphical models WinBUGS UBC Summer School Doodles

• In WinBUGS, graphical representations of Bayesian systems are referred to as Doodles

• DoodleBUGS is a built-in utility for creating/editing Doodles Outline

∗ accessed via Doodle Menu Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps • The utility is well-documented in the WinBUGS manual Miscellaneous notes ∗ creation and manipulation of Doodles

.52 WinBUGS and graphical models WinBUGS UBC Summer School

Moving beyond graphical models

• Understanding graphical models is helpful in terms of understanding how WinBUGS works ∗ calculate the full conditional distribution Outline ∗ examine each factor in the full conditional and classify its’ Goals and Scope Preliminaries functional form Lung cancer data ∗ combine functional forms for each factor Gibbs sampling ∗ choose suitable MCMC simulation algorithm The WinBUGS Software Graphical models WinBUGS syntax • For simple model, also provides a convenient means of Data communication Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools • Quickly become a cumbersome as models get more and more Convergence and analysis Additional Topics complex Errors and traps Miscellaneous notes

.53 WinBUGS language and syntax WinBUGS UBC Summer School WinBUGS script language

• Script, text-based language provides a second interface for model speciﬁcation

• WinBUGS has functionality to convert a doodle into a script Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes • With experience, it becomes more convenient to write your own code

.54 WinBUGS language and syntax WinBUGS UBC Summer School

Building blocks

• Each representation consists of three main types of building blocks Outline

Goals and Scope • Nodes Preliminaries Lung cancer data ∗ stochastic/logical/constants Gibbs sampling

The WinBUGS Software • Edges Graphical models WinBUGS syntax ∗ stochastic/deterministic dependencies or relations Data Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor • Plates Tools Convergence and analysis

∗ characterize repetition via the use of indices Additional Topics Errors and traps Miscellaneous notes

.55 WinBUGS language and syntax WinBUGS UBC Summer School

Distributions

• Stochastic nodes are characterized by a name and distribution

∗ parameters associated with the distribution Outline

Goals and Scope

Preliminaries • Assignment is via a ‘twiddle’ Lung cancer data Gibbs sampling

Y ~ dnorm(mu, tau) The WinBUGS Software Graphical models WinBUGS syntax • Generally appear once and only once on the LHS of an Data expression Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor ∗ exception is when one performs a data transformation within Tools WinBUGS (see later on) Convergence and analysis Additional Topics Errors and traps Miscellaneous notes

.56 WinBUGS language and syntax WinBUGS UBC Summer School

Distributions (cont)

• Parameters that index distributions must be explicit nodes ∗ cannot be functional expressions ∗ a check of the syntax returns an error Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.57 WinBUGS language and syntax WinBUGS UBC Summer School

Distributions (cont)

• Broad range of distributional choices

Outline • Univariate Goals and Scope Preliminaries ∗ Bernoulli, binomial, categorical, negative binomial, poisson Lung cancer data 2 ∗ beta, χ , double exponential, exponential, gamma, Gibbs sampling generalized gamma, log-normal, logistic, normal, pareto, The WinBUGS Software Student-t, uniform, Weibull Graphical models WinBUGS syntax Data • Navigating WinBUGS Multivariate Speciﬁcation Tool Update & Sample Monitor ∗ multinomial Tools ∗ Dirichlet, mutivariate normal, mutivariate Student-t, Wishart Convergence and analysis Additional Topics Errors and traps Miscellaneous notes

.58 WinBUGS language and syntax WinBUGS UBC Summer School

Distributions (cont)

• Should consult the WinBUGS manual for the speciﬁc parameterization

∗ for example, the normal distribution is parameterized in Outline

terms of the precision, as opposed to the variance: Goals and Scope

Preliminaries Y ~ dnorm(mu, tau) Lung cancer data Gibbs sampling

The WinBUGS has the density Software Graphical models WinBUGS syntax r Data τ n τ 2o fY(y) = exp − (y − µ) Navigating WinBUGS 2π 2 Speciﬁcation Tool Update & Sample Monitor Tools • Additional caveats and restrictions mainly apply to multivariate Convergence and analysis

distributions Additional Topics Errors and traps ∗ e.g., restrictions on their use as priors Miscellaneous notes

.59 WinBUGS language and syntax WinBUGS UBC Summer School

Functional expressions

• Logical nodes are characterized by a name and a value ∗ value is determined by a logical expression, represented as a function of the parent nodes Outline

Goals and Scope • Assignment is via a left-pointing arrow Preliminaries Lung cancer data ∗ for example, Gibbs sampling The WinBUGS Software mu[i] <- beta0 + betaX[i] Graphical models sigma <- 1 / sqrt(tau) WinBUGS syntax Data

Navigating WinBUGS ∗ similar to R Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis • Only appear once on the LHS of an expression Additional Topics Errors and traps ∗ data transformation exception (again, see later on) Miscellaneous notes

.60 WinBUGS language and syntax WinBUGS UBC Summer School

Functional expressions (cont)

• Functional arguments can be ∗ expressions, e Outline exp(beta), exp(beta0 + betaX) log(e) Goals and Scope min(e1, e2) Preliminaries e Lung cancer data pow(e1, e2) 2 = e1 Gibbs sampling The WinBUGS ∗ v Software vector-valued nodes, Graphical models mean() WinBUGS syntax sd(v) Data inprod(v1, v2) Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor ∗ scalar-values, s Tools Convergence and analysis th ranked(v, s) returns the s smallest component of v Additional Topics Errors and traps Miscellaneous notes

.61 WinBUGS language and syntax WinBUGS UBC Summer School

Functional expressions (cont)

• Extensive list of functions ∗ details in the WinBUGS manual Outline

Goals and Scope

• Generally, functions live on the RHS of the ‘<-’ Preliminaries Lung cancer data Gibbs sampling • Certain functions can serve as link functions on the LHS The WinBUGS Software ∗ as in a generalized linear model Graphical models WinBUGS syntax ∗ for example, the following are equivalent representations Data

Navigating WinBUGS logit(pi) <- beta0 Speciﬁcation Tool Update & Sample Monitor pi <- exp(beta0) / (1 + exp(beta0)) Tools Convergence and analysis

Additional Topics ∗ others include log, cloglog, and probit, Errors and traps Miscellaneous notes

.62 WinBUGS language and syntax WinBUGS UBC Summer School

Functional expressions (cont)

• Logical nodes cannot be given data or initial values ∗ they are determined, as functions of other quantities Outline

Goals and Scope

• A consequence of this is that one cannot model observed data Preliminaries as a sum of two random variables Lung cancer data Gibbs sampling model{ The WinBUGS Software Y <- U + V Graphical models WinBUGS syntax U ~ dnorm(muU, tauU) Data

V ~ dnorm(muV, tauV) Navigating WinBUGS } Speciﬁcation Tool Update & Sample Monitor Tools ## data Convergence and analysis Additional Topics list(Y = ...) Errors and traps Miscellaneous notes

.63 Data in WinBUGS WinBUGS UBC Summer School

Data format

• Generally follows the S-Plus and R conventions Outline ∗ loaded as part of a list() Goals and Scope ∗ assignment of names via an ‘=’ symbol Preliminaries Lung cancer data Gibbs sampling

The WinBUGS • Vectors use the ‘c()’ format Software Graphical models list(N = 6, WinBUGS syntax Data Y = c(10.3, 4,5 7.4, 12.3, 11.9, 15.6), Navigating WinBUGS X = c(0, 0, 0, 1, 1, 1)) Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.64 Data in WinBUGS WinBUGS UBC Summer School

Data format (cont)

• Arrays use the ‘structure()’ format list(N = 6,

dMat = structure(.Data = c(10.3, 0, Outline

4.5, 0, Goals and Scope 7.4, 0, Preliminaries Lung cancer data 12.3, 1, Gibbs sampling

11.9, 1, The WinBUGS 15.6, 1), Software Graphical models .Dim = c(6,2))) WinBUGS syntax Data

Navigating WinBUGS Specification Tool • Notice how WinBUGS reads the vector into the matrix by filling Update & Sample Monitor Tools the right-most index first Convergence and analysis ∗ fills in the 6×2 matrix by rows Additional Topics Errors and traps ∗ S-Plus and R fill it in by columns Miscellaneous notes

.65 Data in WinBUGS WinBUGS UBC Summer School

Data format (cont)

• Data in arrays (or vectors) can also be represented in a rectangular format

Y[] X[] Outline

10.3 0 Goals and Scope

4.5 0 Preliminaries 7.4 0 Lung cancer data Gibbs sampling 12.3 1 The WinBUGS 11.9 1 Software Graphical models 15.6 1 WinBUGS syntax END Data Navigating WinBUGS Speciﬁcation Tool ∗ each column is headed by the vector name, each explicitly Update & Sample Monitor Tools ending with brackets Convergence and analysis

Additional Topics Errors and traps • File must end with an ‘END’ statement Miscellaneous notes

.66 Data in WinBUGS WinBUGS UBC Summer School

Data format (cont)

• Multi-dimensional arrays can be deﬁned by explicit indexing

dMat[,1] dMat[,2] Outline 10.3 0 Goals and Scope 4.5 0 Preliminaries Lung cancer data 7.4 0 Gibbs sampling 12.3 1 The WinBUGS Software 11.9 1 Graphical models 15.6 1 WinBUGS syntax Data END Navigating WinBUGS Speciﬁcation Tool • First index position for the array name should always be empty Update & Sample Monitor Tools ∗ including vectors (previous slide) Convergence and analysis Additional Topics Errors and traps Miscellaneous notes

.67 Data in WinBUGS WinBUGS UBC Summer School

Data format (cont)

• Can use a mixture of the list() and rectangular formats ∗ matter of convenience Outline

∗ see the Rats example in the WinBUGS manual Goals and Scope

Preliminaries Lung cancer data • Missing data are represented as NA Gibbs sampling The WinBUGS Software • Various resources exist for converting data from other packages Graphical models WinBUGS syntax into a format conducive to WinBUGS Data Navigating WinBUGS ∗ e.g., S-plus, R, SAS, Excel Speciﬁcation Tool ∗ WinBUGS website has a comprehensive list under Update & Sample Monitor Tools Convergence and analysis

.../weblinks/webresource.shtml Additional Topics Errors and traps Miscellaneous notes

.68 Data in WinBUGS WinBUGS UBC Summer School

Unbalanced data

• Repeated measures or longitudinal studies often have differing numbers of observations per cluster ∗ community-based study of breast cancer screening ∗ different radiologists interpret varying numbers of Outline mammograms over the course of a year Goals and Scope Preliminaries Lung cancer data • Resulting data form a ‘ragged array’ Gibbs sampling The WinBUGS cluster 1: 13.2 16.5 Software Graphical models cluster 2: 12.1 WinBUGS syntax cluster 3: 15.5 9.9 6.6 Data Navigating WinBUGS ... Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis • Several ways to handle this Additional Topics Errors and traps ∗ 1. ﬁll in the ‘missing’ data Miscellaneous notes ∗ 2. nested indexing

.69 Data in WinBUGS WinBUGS UBC Summer School

Unbalanced data: Option 1

• Suppose there are 100 clusters, the largest of which is of size 10 model{ for(i in 1:N){ for(j in 1:nMax){ Outline Y[i,j] ~ dnorm(mu[i], tau) Goals and Scope } mu[i] ~ dnorm(0, 1.0E-6) Preliminaries } Lung cancer data tau ~ dgamma(0.5, 0.01) Gibbs sampling } The WinBUGS Software Graphical models ## data WinBUGS syntax list(N = 100, nMax = 10, Data Y = structure(.Data = c(13.2, 16.5, NA, NA, ...), Navigating WinBUGS .Dim=c(100, 10))) Speciﬁcation Tool Update & Sample Monitor • Can be slow Tools Convergence and analysis ∗ missing are treated as unknown parameters Additional Topics Errors and traps ∗ require updating in the MCMC scheme Miscellaneous notes

.70 Data in WinBUGS WinBUGS UBC Summer School

Unbalanced data: Option 2

• An alternative is to consider nested indexing

• For each observation provide an index for its cluster membership Outline Goals and Scope

Y cluster Preliminaries 13.2 1 Lung cancer data 16.5 1 Gibbs sampling The WinBUGS 12.1 2 Software 15.5 3 Graphical models WinBUGS syntax 9.9 3 Data 6.6 3 Navigating WinBUGS Speciﬁcation Tool ...... Update & Sample Monitor Tools Convergence and analysis • Only required to loop through the observed data Additional Topics ∗ rather than the 100×10 observed + unobserved elements Errors and traps Miscellaneous notes

.71 Data in WinBUGS WinBUGS UBC Summer School

Unbalanced data: Option 2

• Suppose there are 567 observations from the 100 clusters model{ Outline for(j in 1:nTot){ Goals and Scope Y[j] ~ dnorm(mu[cluster[j]], tau) } Preliminaries Lung cancer data for(i in 1:N){ Gibbs sampling mu[i] ~ dnorm(0, 1.0E-6) The WinBUGS } Software tau ~ dgamma(0.5, 0.01) Graphical models } WinBUGS syntax Data

## data Navigating WinBUGS list(N = 100, nTot = 567, Speciﬁcation Tool Y = c(13.2, 16.5, 12.1, ...), Update & Sample Monitor cluster = c(1, 1, 2, ...)) Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.72 Data in WinBUGS WinBUGS UBC Summer School

Data transformation

• Data manipulations are best done prior to using WinBUGS

• Might be interested in comparing models with varying transformations of the outcome variable Outline Goals and Scope

for(i in 1:N){ Preliminaries z[i] <- sqrt(y[i]) Lung cancer data z[i] ~ dnorm(mu, tau) Gibbs sampling } The WinBUGS Software Graphical models ∗ WinBUGS has built-in checks to permit this type of WinBUGS syntax declaration Data Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor • May be useful to place the transformation at the beginning of the Tools Convergence and analysis

script Additional Topics Errors and traps ∗ separate transformation from main model speciﬁcation Miscellaneous notes

.73 WinBUGS

UBC Summer School

Outline

• Goals and Scope

• Preliminaries Outline

Goals and Scope

• The WinBUGS Software Preliminaries Lung cancer data Gibbs sampling

• Navigating the WinBUGS Menu System The WinBUGS Software ∗ Speciﬁcation Tool Graphical models WinBUGS syntax ∗ Update & Sample Monitor Tools Data

∗ Convergence and analysis Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools • Additional Topics Convergence and analysis Additional Topics Errors and traps Miscellaneous notes

.74 Navigating WinBUGS WinBUGS UBC Summer School

WinBUGS menu system

• Extensive menu system designed to facilitate and compliment the analysis

Outline • Two key pull-down menu lists are Goals and Scope Preliminaries ∗ Model Menu Lung cancer data ∗ Inference Menu Gibbs sampling The WinBUGS Software Graphical models • We’ll focus on these WinBUGS syntax Data

Navigating WinBUGS • Others that may be of immediate interest include: Speciﬁcation Tool Update & Sample Monitor Tools ∗ Info Menu; status/error log, node information Convergence and analysis

∗ Options Menu; how output is returned, blocking and Additional Topics updating options Errors and traps Miscellaneous notes

.75 Navigating WinBUGS WinBUGS UBC Summer School

Model Menu

• Provides access to the Speciﬁcation Tool window ∗ WinBUGS reads in model, data and starting values ∗ compiles the code to ‘create’ new software Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.76 Navigating WinBUGS WinBUGS UBC Summer School Speciﬁcation Tool

• Position the cursor anywhere within the curly {} brackets and press ‘check model’ ∗ message bar reports on any syntactic problems

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.77 Navigating WinBUGS WinBUGS UBC Summer School Speciﬁcation Tool

• To enter in data, highlight any part of the word ‘list’ and press ‘load data’ ∗ highlighting beyond the word ‘list’ generates an error

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.78 Navigating WinBUGS WinBUGS UBC Summer School Speciﬁcation Tool

• Enter the number of chains you want to run and press ‘compile’ ∗ builds the structures required to carry out the Gibbs sampling ∗ model is checked for completeness and consistency Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.79 Navigating WinBUGS WinBUGS UBC Summer School Speciﬁcation Tool

• Highlight the word ‘list’ and press ‘load inits’ to establish starting values for each chain ∗ ‘gen inits’ will use the prior distribution (or an approximation) for any parameters not already provided Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.80 Navigating WinBUGS WinBUGS UBC Summer School

Speciﬁcation Tool

• Once all the components have been correctly loaded, the message ‘model is initialized’ should appear

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.81 Navigating WinBUGS WinBUGS UBC Summer School Inference Menu

• After initialization, we need to tell WinBUGS which parameters we want to monitor ∗ ‘Samples...’ provides access to the ‘Sample Monitor Tool’ window Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.82 Navigating WinBUGS WinBUGS UBC Summer School

Sample Monitor Tool

• Enter the name of each node and press ‘set’ ∗ if a node is an array, all components will be monitored

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.83 Navigating WinBUGS WinBUGS UBC Summer School

Sample Monitor Tool

• Check which nodes have been entered

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.84 Navigating WinBUGS WinBUGS UBC Summer School Model Menu

• ‘Update...’ becomes active once the model has been compiled and initialized ∗ provides access to the ‘Update Tool’ window

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.85 Navigating WinBUGS WinBUGS UBC Summer School

Update Tool

• Enter the number of MCMC updates and press ‘update’ ∗ ‘thin’ = k indicates that every k th sample is stored

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.86 Navigating WinBUGS WinBUGS UBC Summer School Update Tool

• Progress is reported in the ‘iteration’ box ∗ ‘refresh’ lets you control how often the screen is updated ∗ WinBUGS lets you know how long it took

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.87 Navigating WinBUGS WinBUGS UBC Summer School Convergence and analysis

• ‘Sample Monitor Tool’ window provides functionality for assessing convergence of the MCMC scheme and analysis of the posterior samples ∗ all nodes are updated ∗ only those entered prior to updating are stored Outline Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics • Choose a speciﬁc node by entering the name or by accessing Errors and traps Miscellaneous notes the drop-down list ∗ take all nodes by entering a ‘?’

.88 Navigating WinBUGS WinBUGS UBC Summer School

Convergence diagnostics

• ‘history’ generates a time series-type plot ∗ chains separated by color

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS • Others include Speciﬁcation Tool Update & Sample Monitor Tools ∗ ‘trace’ provides a dynamic version Convergence and analysis

∗ ‘quantiles’ plots the running mean, with running 95% Additional Topics conﬁdence intervals Errors and traps ∗ ‘auto cor’ plots the autocorrelation function out to lag-50 Miscellaneous notes

.89 Navigating WinBUGS WinBUGS UBC Summer School Convergence diagnostics

• ‘bgr diag’ calculates the Gelman-Rubin convergence statistic ∗ Brooks and Gelman (1998) modiﬁcation

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.90 Navigating WinBUGS WinBUGS UBC Summer School Convergence diagnostics

• Consideration of the posterior central 80% interval for a given parameter ∗ green line: width of central 80% interval of the pooled runs ∗ blue line: average width of the 80% intervals within the individual runs Outline ∗ red line: their ratio R = pooled/within Goals and Scope Preliminaries Lung cancer data Gibbs sampling

• Original statistic based on the posterior variance The WinBUGS Software Graphical models WinBUGS syntax • For plotting purposes, pooled and within interval widths are Data

normalized to have a maximum of 1 Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools • Looking for Convergence and analysis ∗ R > 1 at the outset indicates suitably over-dispersed Additional Topics Errors and traps starting points Miscellaneous notes ∗ stability in the pooled and within interval estimates ∗ convergence to R = 1

.91 Navigating WinBUGS WinBUGS UBC Summer School

WinBUGS graphics

• Note that the plot two slides ago was modiﬁed from what was automatically generated by WinBUGS ∗ ‘right-click’ to open up a ‘Plot Properties’ window

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.92 Navigating WinBUGS WinBUGS UBC Summer School

Posterior summaries

• Pressing ‘stat’ returns posterior summary statistics ∗ pooled over the selected chains

∗ ‘MC error’ is the Monte Carlo standard error of the mean Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data • ‘density’ returns a smoothed kernel density estimate for Navigating WinBUGS continuous nodes and a histogram for discrete nodes Speciﬁcation Tool Update & Sample Monitor ∗ not particularly good looking though Tools Convergence and analysis

∗ may want a little more graphical sophistication for journal Additional Topics articles or presentations Errors and traps Miscellaneous notes

.93 Navigating WinBUGS WinBUGS UBC Summer School CODA

• Move beyond the diagnostic and graphical options of WinBUGS

• Press to ‘coda’ dump out an ascii representation of the samples ∗ coda: Convergence Diagnostics and Output Analysis Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.94 Navigating WinBUGS WinBUGS UBC Summer School

CODA

• Generates separate ﬁles for each chain, as well as a ﬁle Outline

indicating the indices for each node Goals and Scope ∗ save each as a ‘.txt’ ﬁle from the File Menu Preliminaries Lung cancer data Gibbs sampling

The WinBUGS • Exploit software specifically written to assess convergence and Software Graphical models generate interesting figures WinBUGS syntax Data ∗ e.g., coda, an S-Plus and R package written by Martyn Navigating WinBUGS Plummer and colleagues Specification Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.95 WinBUGS

UBC Summer School

Outline

• Goals and Scope

• Preliminaries Outline

Goals and Scope

Preliminaries • The WinBUGS Software Lung cancer data Gibbs sampling

The WinBUGS • Navigating the WinBUGS Menu System Software Graphical models WinBUGS syntax • Additional Topics Data Navigating WinBUGS ∗ Errors and traps Speciﬁcation Tool Update & Sample Monitor ∗ Miscellaneous notes Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.96 Errors and traps WinBUGS UBC Summer School Error messages

• If an error is encountered, a message is displayed at the bottom of the WinBUGS window ∗ if it can, WinBUGS will place the cursor somewhere after

the offending location Outline

∗ change the preferences to make the cursor more visible Goals and Scope

∗ also recorded in a log which can be accessed via the Info Preliminaries Menu Lung cancer data Gibbs sampling

The WinBUGS Software • WinBUGS error and trap messages are fairly notorious Graphical models WinBUGS syntax ∗ some are self-explanatory Data ∗ many are not Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools • Worth taking into context the breadth of models that WinBUGS Convergence and analysis Additional Topics can accomodate Errors and traps ∗ given the ﬂexibility of WinBUGS, it’s challenging thinking up Miscellaneous notes all the ways in which an error can be generated

.97 Errors and traps WinBUGS UBC Summer School

Error messages (cont)

• Errors can be the result of improper syntax ∗ Use of an ‘=’ sign

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.98 Errors and traps WinBUGS UBC Summer School Error messages (cont)

• Others become evident at compilation

• For example, consider attempting to model observed data as a sum of two random variables Outline ∗ syntactically OK Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.99 Errors and traps WinBUGS UBC Summer School

Error messages (cont)

• Error at compilation

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.100 Errors and traps WinBUGS UBC Summer School Error messages (cont)

• Other, similar errors include ∗ ‘undeﬁned variable’; variable is in the data but not deﬁned in the model ∗ note that WinBUGS is case-sensitive ∗ ‘index out of range’ Outline Goals and Scope

Preliminaries • Limitations and restrictions Lung cancer data Gibbs sampling ∗ ‘logical expression too complex’; too many The WinBUGS Software parameters/constants or too many operators Graphical models ∗ linear predictors can take advantage of the inprod(,) WinBUGS syntax Data function Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools • WinBUGS manual has greater detail Convergence and analysis

Additional Topics Errors and traps • Debugging can often take longer than actually performing the Miscellaneous notes MCMC. ∗ phenomenon not restricted to WinBUGS though!

.101 Errors and traps WinBUGS UBC Summer School

Trap messages

• Traps that occur during the MCMC process can be particularly difﬁcult to diagnose

∗ ‘undeﬁned real result’ indicates numerical overﬂow Outline

∗ can happen if all the initial values are equal Goals and Scope

Preliminaries Lung cancer data • Can sometimes restart the simulation by pressing the ‘update’ Gibbs sampling The WinBUGS button twice. Software Graphical models WinBUGS syntax • If this doesn’t work, possible solutions include Data Navigating WinBUGS ∗ better initial values Speciﬁcation Tool Update & Sample Monitor ∗ more informative priors Tools ∗ re-parameterization to improve orthogonality Convergence and analysis ∗ standardization of covariates to have mean 0 and standard Additional Topics Errors and traps deviation 1 Miscellaneous notes

.102 Miscellaneous notes WinBUGS UBC Summer School Specifying new distributions

• In some instances one might need to specify a distribution not yet implemented ∗ e.g., a truncated distribution

Outline • ‘ones’ trick for specifying a new sampling distribution Goals and Scope Preliminaries C <- 10000 # ensure all p[i]’s < 1 Lung cancer data for(i in 1:N){ Gibbs sampling ones ~ dbern(pi[i]) The WinBUGS p[i] <- L[i] / C Software ones[i] <- 1 Graphical models } WinBUGS syntax Data L[i] Navigating WinBUGS ∗ is the likelihood contribution for the new sampling Speciﬁcation Tool distribution Update & Sample Monitor Tools ∗ function of observed data and parameters of interest Convergence and analysis Additional Topics Errors and traps Miscellaneous notes • WinBUGS manual also has tricks for specifying a new prior distribution as well as a discrete prior on a set of values

.103 Miscellaneous notes WinBUGS UBC Summer School

Compound documents

• WinBUGS allows documents to be embedded in a compund document

• Convenient way to Outline Goals and Scope

∗ save a doodle, the code and data Preliminaries ∗ store output, including posterior summaries and graphics Lung cancer data Gibbs sampling

The WinBUGS Software • All in a single document Graphical models WinBUGS syntax ∗ helpful for keeping a track of analyses Data Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor • The use of ‘folds’ is helpful in making the document concise Tools Convergence and analysis

∗ ‘hide’ sections of the document (including graphics), to be Additional Topics revealed when needed. Errors and traps Miscellaneous notes ∗ WinBUGS manual has documentation

.104 Gibbs sampling algorithm WinBUGS UBC Summer School

Choice of sampling method

• The Gibbs sampling algorithm is the underlying framework for MCMC within WinBUGS ∗ cycling through a series of full conditional posterior Outline distributions Goals and Scope Preliminaries Lung cancer data Gibbs sampling

• When WinBUGS can identify a full conditional as being a known The WinBUGS distribution, it will sample directly Software Graphical models ∗ modules implemented for a range of distributions WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool • When the full conditional is not of a known form, WinBUGS Update & Sample Monitor Tools employs a Metropolis step Convergence and analysis ∗ Metropolis-within-Gibbs Additional Topics Errors and traps Miscellaneous notes

.105 Gibbs sampling algorithm WinBUGS UBC Summer School Choice of sampling method (cont)

• Towards this, WinBUGS has a variety of MCMC methods at its disposal ∗ vary according to characteristics of the parameter, and its’

posterior distribution Outline

Goals and Scope • Continuous θ Preliminaries Lung cancer data Distribution Method Gibbs sampling Conjugate Direct sampling The WinBUGS Software Log-concave Derivative-free adaptive rejection Graphical models WinBUGS syntax sampling Data

Restricted range Slice sampling Navigating WinBUGS Unrestricted range Current-point Metropolis Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

• Discrete θ Additional Topics Distribution Method Errors and traps Miscellaneous notes Finite upper bound Inversion Shifted Poisson Direct sampling

.106 Gibbs sampling algorithm WinBUGS UBC Summer School

Choice of sampling method (cont) Outline Goals and Scope • WinBUGS tries increasingly complex methods, until one works Preliminaries Lung cancer data ∗ if it is unable to identify a method, an error message is Gibbs sampling returned The WinBUGS Software ∗ introduction to the WinBUGS manual has more detail Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.107 Miscellaneous notes WinBUGS UBC Summer School

Software resources

• Add-ons speciﬁcally written for WinBUGS ∗ GeoBUGS for spatial analyses Outline ∗ PKBUGS for pharmacokinetic analyses Goals and Scope Preliminaries Lung cancer data Gibbs sampling • Packages written for other software programs The WinBUGS Software ∗ facilitate exporting data into a WinBUGS format Graphical models ∗ convergence and analysis (e.g., coda, boa) WinBUGS syntax Data ∗ setting up simulations/batches Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools • WinBUGS website Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.108 Miscellaneous notes WinBUGS UBC Summer School

Mac OS X

• It is possible to directly run WinBUGS on a Mac ∗ OS X 10.4 or higher ∗ Intel-based architecture Outline

Goals and Scope • Requires installation of Preliminaries Lung cancer data ∗ X11 windowing system Gibbs sampling The WinBUGS ∗ Darwine; a pre-compiled version is available at Software http://www.kronenberg.org/darwine/ Graphical models WinBUGS syntax Data

Navigating WinBUGS • Download the zip archive installation of WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

• More detail is available at Additional Topics Errors and traps http://idiom.ucsd.edu/∼rlevy/winbugsonmacosx.pdf Miscellaneous notes

.109 Miscellaneous notes WinBUGS UBC Summer School

Mac OS X screenshot

Outline

Goals and Scope

Preliminaries Lung cancer data Gibbs sampling

The WinBUGS Software Graphical models WinBUGS syntax Data

Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools Convergence and analysis

Additional Topics Errors and traps Miscellaneous notes

.110 Miscellaneous notes WinBUGS UBC Summer School

Additional references and resources

• Good introductory papers ∗ Scollnik, D.P. Actuarial modeling with MCMC and BUGS. North American Actuarial Journal, Vol 5, No 2, (2001),

pp.96-125. Outline

∗ Fryback, D.G., Stout, N.K., and Rosenberg, M.A. An Goals and Scope

elementary introduction to Bayesian computing using Preliminaries WinBUGS. International Journal of Technology Assessment Lung cancer data Gibbs sampling in Health Care, Vol 17, No 1 (Winter 2001), pp. 98-113. The WinBUGS Software Graphical models • WinBUGS manual is very comprehensive and worth reading WinBUGS syntax Data ∗ doesn’t provide guidance on Bayesian analyses Navigating WinBUGS Speciﬁcation Tool Update & Sample Monitor Tools • WinBUGS website has links to a huge number of resources Convergence and analysis Additional Topics ∗ additional worked examples Errors and traps ∗ slides for other short courses/presentations/classes Miscellaneous notes ∗ software

.111