December 2005
Total Page:16
File Type:pdf, Size:1020Kb
THE ISBA BULLETIN Vol. 12 No. 4 December 2005 The official bulletin of the International Society for Bayesian Analysis AMESSAGE FROM THE PRESIDENT This year has passed too quickly (don’t they all?) and I look forward to carrying on working with our by Sylvia Richardson new President, Alan Gelfand, to advance some of ISBA President the unfinished business, in particular concerning [email protected] the arrangements for the upkeep of Bayes’ grave. I have enjoyed working with the ISBA community This year has been very productive for ISBA with and I would like to thank all members of the Board a strong society engaged worldwide, the establish- and of the Executive, in particular the past Presi- ment of Sections to encourage diversity, many suc- dent Jim Berger and the treasurer Bruno Sanso,´ for cessful meetings (my personal highlight was the being so responsive. I would also like to thank all MCMski meeting in Bormio), and the launch of our of you who have accepted new responsibilities on electronic journal Bayesian Analysis under the stew- both the prize committees and programme com- ardship of Rob Kass. Following on from the launch, mittees, and congratulate our new elected board a proposal to distribute paper copies under the um- members: Marilena Barbieri, Wes Johnson, Steve brella of the IMS is under discussion by both Soci- MacEachern and Jim Zideck, as well as our new eties. Both the Bulletin and Bayesian Analysis are President Elect Peter Green. flagships for our Society and we are indebted to I wish you all a happy and successful year in Rob Kass and Andres´ Christen for their work on 2006 and look forward to seeing you at our next the editorial side. ISBA 2006 conference in Valencia.▲ AMESSAGE FROM THE EDITOR our Bayesian friends in the southern hemisphere. ▲ by J. Andr´esChristen Contents [email protected] ' $ The last issue of the year presents an assorted se- ➤ ANNOTATED lection of interesting articles, from regular sections BIBLIOGRAPHY to a contribution from Tony O’Hagan; I hope you ☛Page 2 enjoy reading it as much as I did. ➤ The treasurer of ISBA, Bruno Sanso,´ wishes to APPLICATIONS ☛ thank a donation to the Valencia 8 meeting (to be Page 6 held next June) by BEST, LLC. We thank BEST, LLC, ➤ SOFTWARE again for this generous donation. ☛Page 7 Finally, please do not hesitate to send me any suggestions about articles that you may wish to see ➤ CONTRIBUTIONS published in the Bulletin, or send me any free con- ☛Page 9 tribution you might feel is of general interest for the ➤ ISBA community. Taking the opportunity of this NEWS FROM THE WORLD ☛ December issue, I wish you all a great and exiting Page 10 Bayesian 2006, and also a terrific summer ... to all & % ISBA Bulletin, 12(4), December 2005 ANNOTATED BIBLIOGRAPHY ADVANCED MARKOV CHAIN ergy variable is approximately uniformly dis- MONTE CARLO METHODS tributed, and propose an iterative procedure for constructing such a trial distribution. by Faming Liang [email protected] • Chen, M.-H. and Schmeiser, B.W. (1993). Per- formances of the Gibbs, hit-and-run, and Markov chain Monte Carlo (MCMC) methods Metropolis samplers. Journal of Computational are rooted in the work of physicists such as and Graphical Statistics, 2, 251-272. Metropolis and von Neumann during the period The authors propose a general form of the 1945-1955 when they employed modern electronic hit-and-run algorithm, which behaves like a computers for the simulation of some probabilis- random-direction Gibbs sampler and allows tic problems in atomic bomb designs. After five for a complete exploration of a randomly cho- decades of continual development, they have be- sen direction. The hit-and-run algorithm is come the dominant methodology in the solution of particularly useful when the sample space is many classes of computational problems of central sharply constrained. importance to science and technology. The MCMC • methods have numerous application areas such as Duane, S., Kennedy, A.D., Pendleton, B.J. Bayesian statistical inference, spin-glasses simula- and Roweth, D. (1987). Hybrid Monte Carlo. Physics Letters B 195 tions, chip design, image processing, economics , , 216-222. and finance, signal processing, machine learning, The authors propose the hybrid Monte Carlo biological sequence analysis, phylogeny inference, algorithm which combines the basic idea protein structure prediction, microarray data anal- of molecular dynamics and the Metropolis ysis, among others. acceptance-rejection rule to produce Monte In brief, a MCMC method simulates a Markov Carlo samples for a complex distribution. chain to draw samples proportionally (with respect • Edwards, R.G. and Sokal, A.D. (1988). Gener- to the invariant distribution) from each part of the alization of the Fortuin-Kasteleyn-Swendsen- sample space, and then conducts statistical infer- Wang representation and Monte Carlo algo- ences based on the samples drawn during the sim- rithm. Physical Review D, 38, 2009-2012. ulation process. The local trap phenomenon oc- curs when the energy function, or the negative The authors propose the slice sampler which log-posterior density function in Bayesian statis- seeks to generate samples which are uni- tics, has multiple local minima separated by high formly distributed in a region under the energy barriers. In this situation the Markov chain surface of the target density function. A will be trapped into a local energy minimum indef- marginal distribution of the sample is iden- initely. Consequently, the simulation process may tical to the target distribution. fail to sample from the relevant parts of the sam- • Gelfand, A.E. and Smith, A.F.M. (1990). ple space, and the quantities of interest can not Sampling-based approaches to calculating be estimated correctly. Many applications of the marginal densities. Journal of the American MCMC methods, such as protein folding, combi- Statistical Association, 85, 398-409. natorial optimization, and spin-glasses, could be The authors demonstrate that the conditional dramatically enhanced if we had better algorithms distributions needed in the Gibbs sampler are which allowed the process to avoid being trapped commonly available in many Bayesian and into local minima. Developing advanced MCMC likelihood computations. methods that are immune to the local trap problem has long been considered as one of the most im- • Geman, S. and Geman, D. (1984). Stochas- portant research topics in scientific computing. A tic relaxation, Gibbs distributions and the non-exhaustive list of the works in this direction is Bayesian restoration of Images. IEEE Trans- as follows. action on Pattern Analysis and Machine Intelli- gence, 6, 721-741. • Berg, B.A. and Neuhaus, T. (1991). Mul- The authors propose the Gibbs sampler ticanonical algorithms for 1st order phase- which turns out to be a special scheme of the transitions. Physics Letters B, 267, 249-253. Metropolis-Hastings algorithm with the pro- The authors propose the multicanonical al- posal distributions being the conditional dis- gorithm which seeks to generate samples tributions derived from the target distribu- from a trial distribution under which the en- tion. In the Gibbs sampler, the components of 2 the parameter vector (multidimensional) can The authors propose the Langevin algorithm be updated in a systematic or random order. which produces Monte Carlo samples by sim- ulating a diffusion process with the target • Geyer, C.J. (1991). Markov chain Monte Carlo distribution being its stationary distribution. maximum likelihood. Computing Science and The diffusion process can be discretized and Statistics: proceedings of the 23rd Symposium on moderated by the Metropolis-Hastings algo- the Interface (ed. E.M. Keramigas), 156-163, In- rithm. terface Foundations, Fairfax. The author proposes the parallel temper- • Hastings, W.K. (1970). Monte Carlo sampling ing algorithm which falls into the class of methods using Markov chains and their ap- multiple-chain MCMC algorithms. The in- plications. Biometrika, 57, 97-109. variant distributions of the multiple Markov The author generalizes the Metropolis algo- chains are constructed by scaling (or temper- rithm to the case that the proposal distribu- ing) the target distribution along a given tem- tion is asymmetric. perature ladder. The swapping operation, exchange of samples between neighbouring • Hesselbo, B. and Stinchcombe, R.B. (1995). Markov chains, accelerates the convergences Monte Carlo simulation and global optimiza- of the Markov chains at low temperature lev- tion without parameters. Physics Review Let- els. ters, 74, 2151-2155. • Geyer, C.J. and Thompson, E.A. (1995). An- The authors propose the 1/k-ensemble sam- nealing Markov chain Monte Carlo with ap- pling algorithm, which is similar in spirit plications to pedigree analysis. Journal of the to the multicanonical algorithm (Berg and American Statistical Association, 90, 909-920. Neuhaus, 1991) and seeks to produce sam- ples from a trial distribution under which The authors consider practical issues of sim- the configuration entropy variable is approx- ulated tempering (Marinari and Parisi, 1992), imately uniformly distributed. The trial dis- for example, how to set the temperature tribution can be constructed in the same pro- ladder and how to estimate the pseudo- cedure as that used in the multicanonical al- normalizing constants for each of the trial dis- gorithm. tributions constructed by scaling (or temper- ing) the target distribution along a given tem- • Hukushima K. and Nemoto, K. (1996). Ex- perature ladder. The authors also demon- change Monte Carlo method and application strate the usefulness of the algorithm through to spin glass simulations. Journal of the Physi- a biomedical example. cal Society of Japan, 65, 1604-1608. • Gilks, W.R., Roberts, G.O. and George, E.I. The authors propose the exchange Monte (1994). Adaptive direction sampling. Statis- Carlo algorithm which is a reinvention of par- tician, 43, 179-189. allel tempering (Geyer, 1991). The authors propose adaptive direction sam- • Kirkpatrick, S., Gelatt, C.D.