THE ISBABULLETIN

Vol. 18 No. 1 March 2011

The official bulletin of the International Society for Bayesian Analysis

A MESSAGE FROM THE PRESIDENT

WHAT ARETHE OPEN que, I am afraid. In particular, the individuals as- PROBLEMSIN BAYESIAN sembled are a highly non-random sample—they STATISTICS? are a set of people who have the misfortune of being in the intersection of two sets: (a) highly- respected senior statisticians and (b) entries in - Michael I. Jordan - my email address book. ISBA President, 2011 The question that I asked was “What do you [email protected] view as the top two or three open problems in Bayesian statistics?” The focus on Bayes is due From time to time, I am approached by young to the ISBA context of course, but I also think students who are considering a career in statistics that frequentist statisticians are more accustomed and who ask “What are the open problems in sta- to thinking in terms of “open problems,” and I tistics?” While I’m often tempted to respond that wanted to make the question challenging (given “we don’t tend to think that way in statistics,” that I didn’t have to answer it myself). Here is the nature of the question tends to imply a stu- the distinguished group that I wrote to: Andrew dent with mathematical training of the kind that Barron, Susie Bayarri, Jim Berger, Jose´ Bernardo, I usually look for in a prospective student, and Peter Bickel, Larry Brown, Brad Carlin, George so I do my best to give a thoughtful response and Casella, Ming-Hui Chen, Merlise Clyde, Phil Da- cast our activities in terms of “open problems.” wid, Persi Diaconis, David Draper, David Dun- This scenario came to my mind as I sat down son, Brad Efron, . . . Continued on page 2. to contemplate writing this column. It had alrea- dy occurred to me that one of the consolations of being President of ISBA (or President of any In this issue society) is that one can ask others to do the actu- al work that’s attributed to the President of the ® A MESSAGE FROM THE BA EDITOR society. It was but a short step to realize that this Page 5 might also apply to the writing of my column. In- ® ISBA - SECTIONS deed, inspired by the notion of “crowd-sourcing” Page 5 that is all the rage, I realized that as ISBA Presi- ® ANNOTATED BIBLIOGRAPHY dent I had been given an unparalleled opportu- Page 6 nity for “statistician-sourcing.” People might re- ® INTERVIEWS spond to the ISBA President in ways that they Page 8 might not respond to humble old me. And so I thought that I would seize the opportunity, as- ® STUDENTS’ CORNER semble a distinguished panel of statisticians and Page 10 see what their views on the “open problems of ® NEWS FROM THE WORLD statistics” might be. I imagined that this might be Page 14 of interest beyond my recruiting scenario. ® ISBA DISCOUNTS My polling methodology is rather open to criti- Page 19 ISBA Bulletin, 18(1), March 2011 A MESSAGE FROM THE PRESIDENT

MESSAGE FROM THE PRESIDENT, Continued ginal inference’ of a function of the para- from page 1. . . . Steve Fienberg, Peter Green, Alan meter can be viewed as ‘optimal’ in some Gelfand, Andrew Gelman, Ed George, Malay sense? Must the prior depend on the func- Ghosh, Nils Hjort, Peter Hoff, Jay Kadane, Rob tion?” Larry Wasserman: “Find a full non- Kass, Jun Liu, Steve MacEachern, Xiao-Li Meng, parametric prior on a function space such Peter Mueller, Tony O’Hagan, Luis Pericchi, So- that the (1 α) posterior probability regi- − nia Petrone, Fernando Quintana, Adrian Rafte- on has frequentist coverage (approximate- ry, Sylvia Richardson, Thomas Richardson, Chri- ly/asymptotically) equal to (1 α).” − stian Robert, Judith Rousseau, Fabrizio Rugge- Many of the problems listed in the other ca- ri, Mark Schervish, David Spiegelhalter, Terry tegories below were also raised in the non- Speed, Steve Stigler, Aad van der Vaart, Stephen parametric context. Indeed, problems sur- Walker, Larry Wasserman, Mike West, and Wing rounding prior specification and identifia- Wong. I (amazingly) had a response rate not too bility were viewed as particularly virulent far from 100%, and the responses were invigora- in the nonparametric setting. David Dun- ting. son: “Nonparametric Bayes models involve I note parenthetically that one person didn’t infinitely many parameters and priors are answer my question but instead conducted his typically chosen for convenience with hy- own mini-poll of colleagues the results of which perparameters set at seemingly reasonable he transmitted to me; impressed by this skill in values with no proper objective or subjecti- delegation of responsibility, I intend to nominate ve justification.” And Stephen Walker: “De- this person in the next ISBA presidential election. spite a lot of recent work on Bayesian non- I turn to the results of my poll. I have organi- parametric regression I am far from convin- zed the results into categories, with examples of ced that the current presented models will open problems listed within each category. In se- stand the test of time. The models are too veral cases I have used quotes from individuals big and too unidentifiable.” when I felt that a paraphrase would be less clear than the original text. I organize my results as a Finally, it was noted by several people “top-five list.” that one of the appealing applications of frequentist nonparametrics is to semipa- 5. Nonparametrics and semiparametrics. Bayesi- rametric inference, where the nonparame- an nonparametrics is viewed by some of tric component of the model is a nui- my respondents as a class of methods loo- sance parameter. These people felt that it king for a problem, and so the main open would be desirable to flesh out the (fre- problem in Bayesian nonparametrics is (for quentist) theory of Bayesian semiparame- some people) that of finding a characteriza- trics. For example, Thomas Richardson as- tion of classes of problems for which these ked for “Bayesian approaches to dealing tools are worth the trouble. with mis-specification, e.g., when will a (1 α) posterior credible region for a pa- − But the success stories in frequentist nonpa- rameter have (1 α) frequentist coverage − rametrics are alluring to many in my group even if some (‘nuisance’) parts of the like- of respondents, and the concrete open pro- lihood are mis-specified?”. blems raised for nonparametrics by the group are generally frequentist in charac- 4. Priors. Not surprisingly, priors were on the ter. From Andrew Barron: “Suppose in an minds of many. Elicitation remains a ma- i.i.d. sampling model that the parameter jor source of open problems. Tony O’Hagan value of the distribution from which the da- avers: “When it comes to eliciting distribu- ta are sampled has the property that the pri- tions for two or more uncertain quantities or probability of Kullback neighborhoods we are working more or less in the dark.” of that value are given positive probabili- Mike West pointed to the fact that many ty. Then, from that condition alone, does scientific fields express their prior know- it follow that the risk of the Bayes proce- ledge in terms of “scientifically predictive dure at that parameter value will converge models,” and using these models in a sta- to zero?” Wing Wong: “Can we construct tistical setting involves the quintessentially priors on a very large parameter space (e.g., Bayesian tasks of understanding assump- the space of all densities) so that a ‘mar- tions and conducting detailed sensitivity

Content 2 www.bayesian.org ISBA Bulletin, 18(1), March 2011 A MESSAGE FROM THE PRESIDENT

analyses. Aad van der Vaart turned objec- variational methods, of ABC approaches?”. tive Bayes on its head and pointed to a lack Several respondents asked for a more tho- of theory for “situations where one wants rough integration of computational science the prior to come through in the posterior” and statistical science, noting that the set of as opposed to “merely providing a Bayesi- inferences that one can reach in any given an approach to smoothing.” And Sonia Pe- situation are jointly a function of the mo- trone noted that we often wish to model da- del, the prior, the data and the computatio- ta that arise from human behavior and hu- nal resources, and wishing for more explicit man beliefs, and in such settings the mode- management of the tradeoffs among these ling of human beliefs thus arises (implicitly quantities. Indeed, Rob Kass raised the pos- at least) in both the likelihood and the pri- sibility of a notion of “inferential solvabili- or, and there should be some consistency in ty,” where some problems are understood our approaches to these specifications. to be beyond hope (e.g., model selection in regression where “for modest amounts 3. Bayesian/frequentist relationships. As already mentioned in the nonparametrics section, of data subject to nontrivial noise it is im- many respondents expressed a desire to possible to get useful confidence intervals further hammer out Bayesian/frequentist about regression coefficients when there are relationships. This was most commonly large numbers of variables whose presence evinced in the context of high-dimensional or absence in the model is unspecified a models and data, where not only are sub- priori”) and where there are other problems jective approaches to specification of priors (“certain functionals for which useful con- difficult to implement but priors of con- fidence intervals exist”) for which there venience can be (highly) misleading. Open is hope. Terry Speed raised the intriguing problems discussed here often are cou- possibility of a connection between the no- ched as statements about frequentist cover- tion of “inference being possible” when age of Bayesian procedures. More broadly, (and only when) simulation from a model Brad Efron reminds us that “two connec- is possible (and this may well be the subject ting technologies are empirical Bayes and of a future column; not mine, but Terry’s). the bootstrap.” Some respondents pined for Several respondents, while apologizing for non-asymptotic theory that might reveal a certain vagueness, expressed a feeling more fully the putative advantages of Baye- that a large amount of data does not ne- sian methods; e.g., David Dunson: “Often, cessarily imply a large amount of compu- the frequentist optimal rate is obtained by tation; rather, that somehow the inferential procedures that clearly do much worse in strength present in large data should trans- finite samples than Bayesian approaches.” fer to the algorithm and make it possible Finally, some respondents, whose names I to make do with fewer computational steps will not reveal for their own protection, to achieve a satisfactory (approximate) in- asked whether there might be a sense in ferential solution. which it is worthwhile to give up some Other respondents were concerned with in- Bayesian coherence in return for some of teractions between model complexity and the advantages of the frequentist paradigm, algorithmic complexity; for example Jun including simplicity of implementation and Liu referred to a notion of “weak identi- computational tractability. fiability” in complex latent variable models where even though parameters might be 2. Computation and statistics. It was interesting identifiable via a proper posterior the in- to see some disagreement on the subject ference algorithm might run aground (e.g., of computation, with some people feeling MCMC failing to mix). that MCMC has tamed the issue, and with others (the majority by my count) opining 1. Model selection and hypothesis testing. I ha- that many open problems remain. E.g., ve placed this topic as number one not on- Alan Gelfand: “Arguably the biggest chal- ly for the large numbers of respondents lenge is in computation. If MCMC is no lon- mentioning it, but also for the urgency that ger viable for the problems people want to was transmitted. From Jim Berger: “We just address, then what is the role of INLA, of don’t have any agreed upon methods, and

Content 3 www.bayesian.org ISBA Bulletin, 18(1), March 2011 A MESSAGE FROM THE PRESIDENT

the problem is especially important becau- views Bayesian decision theory for group se the Bayesian and frequentist methods decision-making as entirely open). Christi- can differ so much. This is also crucially im- an Robert holds out for some radical new portant because science is choking on the framework. multiplicity problem, and Bayesian model On a more practical note, many people no- selection is likely the way forward to its so- ted the lack of off-the-shelf methods for lution.” George Casella is concerned about model criticism and diagnostics. Steve Ma- lack of theory for inference after selection: cEachern: “Our current diagnostics are in “We now do model selection but Bayesians a sorry state.” And David Spiegelhalter: don’t seem to worry about the properties “How best to make checks for prior/data of basing inference on the selected model. conflict an integral part of Bayesian analy- What if it is wrong? What are the conse- sis?” And the last word on the matter goes quences of setting up credible regions for a to Andrew Gelman: “For model checking, certain parameter β1 when you have selec- a key open problem is developing graphi- ted the wrong model? Can we have proce- cal tools for understanding and comparing dures with some sort of guarantee?”. And models. Graphics is not just for raw data; many people feel that prior specification for rather, complex Bayesian models give op- model selection is still wide open. portunity for better and more effective ex- ploratory data analysis.” There are also open problems at the foun- dations of model selection. Jose´ Bernardo: And thus ends my statistician-sourced column, “My favorite problem is to reach some form which I’ve quite enjoyed “writing.” I will forgo of agreement on hypothesis testing and drawing any grander conclusions at this point, model selection. There are two rather diffe- for at least two reasons: (1) I am past my dead- rent Bayesian attitudes: to compute a poste- line and am being pursued by the Editor of the rior probability for the hypotheses (which Bulletin, and (2) I am well over my page li- needs a sharp prior, very different from tho- mit. I do wish to take the opportunity, howe- se commonly used for estimation) or to use ver, to solicit reactions from the larger commu- decision analysis to minimize an expected nity. I’d enjoy hearing from anyone who feels loss (which may be done with conventio- that my panel of experts has missed a fundamen- nal, possibly noninformative, priors).” Da- tal “open problem” or otherwise wishes to com- vid Draper agrees for the need for mo- ment on the material presented here. My email re work on decision-theoretic foundations is [email protected]. With any luck I’ll in model selection (and he adds that he get enough responses to fill my second column.L

AMESSAGEFROMTHE EDITOR sting and useful sections. In particular, I call your attention to the Annotated Bibliography Section where the Editor, Beatrix Jones, asked Nicholas Manuel Mendoza Cummings to write an article on the use of the [email protected] Bayesian methods in Ecology. He focusses on the capture-recapture problem and provides a wide Spring has arrived (in the northern hemisfe- list of interesting references. In addition, and fol- re) and with the new season we are witness of lowing the idea he started in the December issue, a variety of changes. For example, in this issue our Student’s Corner Editor, Luke Bornn, poses of the Bulletin we have the first MESSAGE from another question to his panel of distiguished col- our 2011 ISBA President, Michael Jordan. The leagues. The result is a revealing set of answers. list of distiguished Bayesians that have collabo- The Interviews Section presents the conversation rated with him to produce this list of Open Pro- of our Editor, Donatello Telesca, with Jeff Rosen- blems in Bayesian Statistics is impressive. Hopeful- thal where some aspects of the use of MCMC me- ly, this contribution will trigger a fruitful discus- thods in Bayesian Statistics are discussed. sion among our members! In this issue, we also introduce the ISBA - Also in this issue, you will find other intere- SECTIONS Section. There, you will find rele-

Content 4 www.bayesian.org