STATISTICAL SCIENCE Volume 31, Number 4 November 2016

Home , Engineering statistics, Richard Peto

ApproximateModelsandRobustDecisions...... James Watson and Chris Holmes 465 ModelUncertaintyFirst,NotAfterwards...... Ingrid Glad and Nils Lid Hjort 490 ContextualityofMisspecificationandData-DependentLosses...... Peter Grünwald 495 SelectionofKLNeighbourhoodinRobustBayesianInference...... Natalia A. Bochkina 499 IssuesinRobustnessAnalysis...... Michael Goldstein 503 Nonparametric Bayesian Clay for Robust Decision Bricks ...... Christian P. Robert and Judith Rousseau 506 Ambiguity Aversion and Model Misspecification: An Economic Perspective ...... Lars Peter Hansen and Massimo Marinacci 511 Rejoinder: Approximate Models and Robust Decisions. . . . .James Watson and Chris Holmes 516 Filtering and Tracking Survival Propensity (Reconsidering the Foundations of Reliability) ...... Nozer D. Singpurwalla 521 On Software and System Reliability Growth and Testing...... FrankP.A.Coolen 541 HowAboutWearingTwoHats,FirstPopper’sandthendeFinetti’s?...... Elja Arjas 545 WhatDoes“Propensity”Add?...... Jane Hutton 549 Reconciling the Subjective and Objective Aspects of Probability ...... Glenn Shafer 552 Rejoinder:ConcertUnlikely,“Jugalbandi”Perhaps...... Nozer D. Singpurwalla 555 ChaosCommunication:ACaseofStatisticalEngineering...... Anthony J. Lawrance 558 Bayes, Reproducibility and the Quest for Truth ...... D. A. S. Fraser, M. Bédard, A. Wong, Wei Lin and A. M. Fraser 578 A Review and Comparison of Age–Period–Cohort Models for Cancer Incidence ...... Theresa R. Smith and Jon Wakefield 591 Multiple Change-Point Detection: A Selective Overview ...... Yue S. Niu, Ning Hao and Heping Zhang 611 AConversationwithJeffWu...... Hugh A. Chipman and V. Roshan Joseph 624 AConversationwithSamadHedayat...... Ryan Martin, John Stufken and Min Yang 637

Statistical Science [ISSN 0883-4237 (print); ISSN 2168-8745 (online)], Volume 31, Number 4, November 2016. Published quarterly by the Institute of Mathematical Statistics, 3163 Somerset Drive, Cleveland, OH 44122, USA. Periodicals postage paid at Cleveland, Ohio and at additional mailing ofﬁces. POSTMASTER: Send address changes to Statistical Science, Institute of Mathematical Statistics, Dues and Subscriptions Ofﬁce, 9650 Rockville Pike—Suite L2310, Bethesda, MD 20814-3998, USA. Copyright © 2016 by the Institute of Mathematical Statistics Printed in the United States of America EDITOR Peter Green University of Bristol and University of Technology, Sydney

ASSOCIATE EDITORS Steve Buckland Byron Morgan Eric Tchetgen Tchetgen University of St. Andrews University of Kent Harvard School of Public Jiahua Chen Peter Müller Health University of British Columbia University of Texas Yee Whye Teh Rong Chen Sonia Petrone University of Oxford Rutgers University Bocconi University Jon Wakeﬁeld Rainer Dahlhaus Nancy Reid University of Washington University of Heidelberg University of Toronto Guenther Walther Peter J. Diggle Glenn Shafer Stanford University Lancaster University Rutgers Business Jon Wellner Robin Evans School–Newark and University of Washington University of Oxford New Brunswick Martin Wells Michael Friendly Royal Holloway College, Cornell University York University University of London Tong Zhang Edward I. George Michael Stein Rutgers University University of Pennsylvania University of Chicago

MANAGING EDITOR T. N. Sriram University of Georgia

PRODUCTION EDITOR Patrick Kelly

EDITORIAL COORDINATOR Kristina Mattson

PAST EXECUTIVE EDITORS Morris H. DeGroot, 1986–1988 Morris Eaton, 2001 Carl N. Morris, 1989–1991 George Casella, 2002–2004 Robert E. Kass, 1992–1994 Edward I. George, 2005–2007 Paul Switzer, 1995–1997 David Madigan, 2008–2010 Leon J. Gleser, 1998–2000 Jon A. Wellner, 2011–2013 Richard Tweedie, 2001 Statistical Science 2016, Vol. 31, No. 4, 465–489 DOI: 10.1214/16-STS592 © Institute of Mathematical Statistics, 2016 Approximate Models and Robust Decisions James Watson and Chris Holmes

Abstract. Decisions based partly or solely on predictions from probabilistic models may be sensitive to model misspecification. Statisticians are taught from an early stage that “all models are wrong, but some are useful”; however, little formal guidance exists on how to assess the impact of model ap- proximation on decision making, or how to proceed when optimal actions appear sensitive to model fidelity. This article presents an overview of recent developments across different disciplines to address this. We review diagnostic techniques, including graphical approaches and summary statistics, to help highlight decisions made through minimised expected loss that are sensitive to model misspecification. We then consider formal methods for decision making under model misspecification by quantifying stability of optimal actions to perturbations to the model within a neighbourhood of model space. This neighbourhood is defined in either one of two ways. First, in a strong sense via an information (Kullback–Leibler) divergence around the approximating model. Second, using a Bayesian nonparametric model (prior) centred on the approximating model, in order to “average out” over possible misspecifications. This is presented in the context of recent work in the robust control, macroeconomics and financial mathematics literature. We adopt a Bayesian approach throughout although the presentation is agnostic to this position. Key words and phrases: Computational decision theory, model misspecification, D-open problem, Kullback–Leibler divergence, robustness, Bayesian nonparametrics.

REFERENCES BEAUMONT,M.A.,ZHANG,W.andBALDING, D. J. (2002). Ap- proximate Bayesian computation in population genetics. Genet- AHMADI-JAV I D , A. (2011). An information-theoretic approach ics 162 2025–2035. to constructing coherent risk measures. In IEEE International BELSLEY,D.A.,KUH,E.andWELSCH, R. E. (1980). Regres- Symposium on Information Theory Proceedings (ISIT) 2125– sion Diagnostics: Identifying Inﬂuential Data and Sources of 2127. IEEE, New York. Collinearity. Wiley, New York. MR0576408 AHMADI-JAV I D , A. (2012). Entropic value-at-risk: A new coherent risk measure. J. Optim. Theory Appl. 155 1105–1123. BERGER, J. O. (1984). The robust Bayesian viewpoint. In Ro- MR3000633 bustness of Bayesian Analyses (J. Kadane, ed.) 63–144. North- Holland, Amsterdam. MR0785367 ARTZNER,P.,DELBAEN,F.,EBER,J.-M.andHEATH,D. (1999). Coherent measures of risk. Math. Finance 9 203–228. BERGER, J. O. (1985). Statistical Decision Theory and Bayesian MR1850791 Analysis, 2nd ed. Springer, New York. MR0804611 AUTIER, P. (2015). Breast cancer: Doubtful health beneﬁt of BERGER, J. O. (1994). An overview of robust Bayesian analysis. screening from 40 years of age. Nat. Rev. Clin. Oncol. 12 570– TEST 3 5–124. MR1293110 572. BERGER,J.andBERLINER, L. M. (1986). Robust Bayes and em- BAIO,G.andDAWID, A. P. (2015). Probabilistic sensitivity anal- pirical Bayes analysis with ε-contaminated priors. Ann. Statist. ysis in health economics. Stat. Methods Med. Res. 24 615–634. 14 461–486. MR0840509

James Watson is a Postdoctoral Researcher at the Mahidol–Oxford Research Unit, Centre for Tropical Medicine and Global Health, Nuffield Department of Clinical Medicine, University of Oxford, Oxford, United Kingdom (e-mail: [email protected]). Chris Holmes is Professor of Biostatistics, Department of Statistics, University of Oxford, United Kingdom (e-mail: [email protected]). BERNARDO,J.-M.andSMITH, A. F. M. (1994). Bayesian The- GRÜNWALD,P.andVA N OMMEN, T. (2014). Inconsistency of ory. Wiley, Chichester. MR1274699 Bayesian inference for misspecified linear models, and a pro- BISSIRI,P.G.,HOLMES,C.C.andWALKER, S. G. (2013). posal for repairing it. Preprint. Available at arXiv:1412.3730. A general framework for updating belief distributions. J. R. HAND, D. J. (2006). Classifier technology and the illusion of Stat. Soc. Ser. B. Stat. Methodol. Preprint. Available at progress. Statist. Sci. 21 1–34. MR2275965 arXiv:1306.6430. HANSEN,L.P.andSARGENT, T. J. (2001a). Acknowledging mis- BISSIRI,P.G.andWALKER, S. G. (2012). Converting informa- specification in macroeconomic theory. Rev. Econ. Dyn. 4 519– tion into probability measures with the Kullback-Leibler diver- 535. gence. Ann. Inst. Statist. Math. 64 1139–1160. MR2981617 HANSEN,L.P.andSARGENT, T. J. (2001b). Robust control and model uncertainty. Am. Econ. Rev. 91 60–66. BOX,G.E.P.andDRAPER, N. R. (1987). Empirical HANSEN,L.P.andSARGENT, T. J. (2008). Robustness. Princeton Model-Building and Response Surfaces. Wiley, New York. Univ. Press, Princeton, NJ. MR0861118 HANSEN,L.P.,SARGENT,T.J.,TURMUHAMBETOVA,G.and BREUER,T.andCSISZÁR, I. (2013). Systematic stress tests with WILLIAMS, N. (2006). Robust control and model misspecifica- entropic plausibility constraints. J. Bank. Financ. 37 1552– tion. J. Econom. Theory 128 45–90. MR2229772 1559. HASTIE,T.andTIBSHIRANI, R. (1993). Varying-coefficient mod- BREUER,T.andCSISZÁR, I. (2016). Measuring distribution els. J. R. Stat. Soc. Ser. B. Stat. Methodol. 55 757–796. model risk. Math. Finance 26 395–411. MR3481309 MR1229881 CAROTA,C.,PARMIGIANI,G.andPOLSON, N. G. (1996). Diag- HJORT,N.L.,HOLMES,C.C.,MÜLLER,P.andWALKER,S.G. nostic measures for model criticism. J. Amer. Statist. Assoc. 91 (2010). Bayesian Nonparametrics. Cambridge Univ. Press, 753–762. MR1395742 Cambridge, MA. MR2722987 CHIPMAN,H.A.,GEORGE,E.I.andMCCULLOCH,R.E. HUBER, P. J. (2011). Robust Statistics. Springer, Berlin. (1998). Bayesian CART model search. J. Amer. Statist. Assoc. KADANE,J.B.,ED. (1984). Robustness of Bayesian Analyses. 443 935–948. North-Holland, Amsterdam. MR0785365 BASLE COMMITTEE (1996). Amendment to the capital accord to KADANE,J.B.andCHUANG, D. T. (1978). Stable decision prob- incorporate market risks. Basle Committee on banking supervi- lems. Ann. Statist. 6 1095–1110. MR0494601 sion. KADANE,J.B.andSRINIVASAN, C. (1994). Discussion of Berger, J. O., An overview of robust Bayesian analysis. TEST NATIONAL RESEARCH COUNCIL,COMMITTEE ON THE ANALY- 3 116–120. SIS OF MASSIVE DATA,COMMITTEE ON APPLIED AND THE- KERMAN,J.,GELMAN,A.,ZHENG,T.andDING, Y. (2008). Vi- ORETICAL STATISTICS,BOARD ON MATHEMATICAL SCI- sualization in Bayesian data analysis. In Handbook of Data Vi- ENCES AND THEIR APPLICATIONS and DIVISION ON ENGI- sualization 709–724. Springer, Berlin. NEERING AND PHYSICAL SCIENCES (2013). Frontiers in Mas- LØBERG,M.,LOUSDAL,M.L.,BRETTHAUER,M.and sive Data Analysis. The National Academies Press, Washington, KALAGER, M. (2015). Benefits and harms of mammography DC. screening. Breast Cancer Res. Treat. 17 63. DALALYAN,A.andTSYBAKOV, A. B. (2008). Aggregation by ex- MARIN,J.-M.,PUDLO,P.,ROBERT,C.P.andRYDER,R.J. ponential weighting, sharp PAC-Bayesian bounds and sparsity. (2012). Approximate Bayesian computational methods. Stat. Mach. Learn. 72 39–61. Comput. 22 1167–1180. MR2992292 DALALYAN,A.S.andTSYBAKOV, A. B. (2012). Sparse re- MARJORAM,P.,MOLITOR,J.,PLAGNOL,V.andTAVA R É ,S. gression learning by aggregation and Langevin Monte-Carlo. (2003). Markov chain Monte Carlo without likelihoods. Proc. J. Comput. System Sci. 78 1423–1443. MR2926142 Natl. Acad. Sci. USA 100 15324–15328. DEL MORAL,P.,DOUCET,A.andJASRA, A. (2006). Sequential MARMOT,M.G.ET AL. (2012). The benefits and harms of breast Monte Carlo samplers. J. R. Stat. Soc. Ser. B. Stat. Methodol. 68 cancer screening: An independent review. Lancet 380 1778– 411–436. MR2278333 1786. DEMPSTER, A. P. (1975). A subjectivist look at robustness. Bull. MCCULLOCH, R. E. (1989). Local model influence. J. Amer. Int. Stat. Inst. 46 349–374. Statist. Assoc. 84 473–478. ILLER UNSON DENISON,D.G.T.,HOLMES,C.C.,MALLICK,B.K.and M ,J.W.andD , D. B. (2015). Robust Bayesian inference via coarsening. Preprint. Available at SMITH, A. F. M. (2002). Bayesian Methods for Nonlinear Classification and Regression. Wiley, Chichester. MR1962778 arXiv:1506.06101. MINKA, T. P. (2001). Expectation propagation for approximate FEARNHEAD,P.andPRANGLE, D. (2012). Constructing sum- Bayesian inference. In Proceedings of the Seventeenth Confer- mary statistics for approximate Bayesian computation: Semi- ence on Uncertainty in Artificial Intelligence 362–369. Morgan automatic approximate Bayesian computation. J. R. Stat. Soc. Kaufmann, San Mateo, CA. Ser. B. Stat. Methodol. 74 419–474. MR2925370 MOSS,S.M.,WALE,S.,SMITH,R.,EVANS,A.,CUCKLE,H. GELMAN, A. (2007). Data Analysis Using Regression and Multi- and DUFFY, S. W. (2015). Effect of mammographic screening level/Hierarchical Models. Cambridge Univ. Press, Cambridge, from age 40 years on breast cancer mortality in the UK age trial MA. at 17 years’ follow-up: A randomised controlled trial. Lancet GILBOA,I.andSCHMEIDLER, D. (1989). Maxmin expected Oncol. 16 1123–1132. utility with nonunique prior. J. Math. Econom. 18 141–153. PARMIGIANI, G. (1993). On optimal screening ages. J. Amer. MR1000102 Statist. Assoc. 88 622–628. MR1224388 GOOD, I. J. (1952). Rational decisions. J. R. Stat. Soc. Ser. B. Stat. PARMIGIANI,G.andINOUE, L. Y. T. (2009). Decision Theory. Methodol. 14 107–114. MR0077033 Wiley, Chichester. MR2604978 PRITSKER, M. (1997). Evaluating value at risk methodologies: Ac- SIVAGANESAN, S. (2000). Global and local robustness ap- curacy versus computational time. J. Financ. Serv. Res. 12 201– proaches: Uses and limitations. In Robust Bayesian Analysis (D. 242. Rios Insua and F. Ruggeri, eds.) 89–108. Springer, New York. RASMUSSEN,C.E.andWILLIAMS, C. K. I. (2006). Gaussian MR1795211 Processes for Machine Learning. MIT Press, Cambridge, MA. VARIN,C.,REID,N.andFIRTH, D. (2011). An overview of com- MR2514435 posite likelihood methods. Statist. Sinica 21 5–42. MR2796852 RATMANN,O.,ANDRIEU,C.,WIUF,C.andRICHARDSON,S. VICKERS,A.J.andELKIN, E. B. (2006). Decision curve analysis: (2009). Model criticism based on likelihood-free inference, with A novel method for evaluating prediction models. Med. Decis. an application to protein network evolution. Proc. Natl. Acad. Mak. 26 565–574. Sci. USA 106 10576–10581. VIDAKOVIC, B. (2000). -minimax: A paradigm for conservative RÍOS INSUA,D.andRUGGERI,F.,EDS. (2000). Robust Bayesian robust Bayesians. In Robust Bayesian Analysis (D. Rios In- Analysis. Springer, New York. MR1795206 sua and F. Ruggeri, eds.) 241–259. Springer, New York. ROBBINS, H. (1951). Asymptotically subminimax solutions of MR1795219 compound statistical decision problems. In Proceedings of the VON NEUMANN,J.andMORGENSTERN, O. (1947). Theory of Second Berkeley Symposium on Mathematical Statistics and Games and Economic Behavior, 2nd ed. Princeton Univ. Press, Probability 131–148. University of California Press, Berkeley Princeton, NJ. MR0021298 and Los Angeles. MR0044803 WAINWRIGHT,M.andJORDAN, M. I. (2003). Graphical models, OBERT ASELLA Monte Carlo Statistical R ,C.P.andC , G. (2004). exponential families and variational inference. Faund. Trends Methods, 2nd ed. Springer, New York. MR2080278 Mach. Learn. 1–305. ROCKAFELLAR,R.T.andURYASEV, S. (2000). Optimization of WALD, A. (1950). Statistical Decision Functions. Wiley, New conditional value-at-risk. The Journal of Risk 2 21–42. York. MR0036976 ROSTEK, M. (2010). Quantile maximization in decision theory. WALKER,S.andHJORT, N. L. (2001). On Bayesian consistency. Rev. Econ. Stud. 77 339–371. MR2599290 J. R. Stat. Soc. Ser. B. Stat. Methodol. 63 811–821. MR1872068 RUE,H.,MARTINO,S.andCHOPIN, N. (2009). Approximate Bayesian inference for latent Gaussian models by using inte- WASSERMAN, L. (1992). Recent methodological advances in ro- grated nested Laplace approximations. J. R. Stat. Soc. Ser. B. bust Bayesian inference. In Bayesian Statistics,4(PeñíScola, Stat. Methodol. 71 319–392. MR2649602 1991) 483–502. Oxford Univ. Press, New York. MR1380293 ATSON IETO ARAJAS OLMES RUGGERI,F.,INSUA,D.R.andMARTÍN, J. (2005). Robust W ,J.,N -B ,L.andH , C. (2016). Char- Bayesian analysis. In Bayesian Thinking: Modeling and Com- acterising variation of nonparametric random probability mod- putation. Handbook of Statistics 25 623–667. Elsevier, Amster- els using the Kullback–Leibler divergence. Statistics. To appear. dam. MR2490541 Available at 1411.6578. RUGGERI,F.andWASSERMAN, L. (1993). Infinitesimal sensi- WHITTLE, P. (1990). Risk-Sensitive Optimal Control.Wiley, tivity of posterior distributions. Canad. J. Statist. 21 195–203. Chichester. MR1093001 MR1234761 WU,D.,ROSNER,G.L.andBROEMELING, L. D. (2007). SAVAG E , L. J. (1954). The Foundations of Statistics. Wiley, New Bayesian inference for the lead time in periodic cancer screen- York. MR0063582 ing. Biometrics 63 873–880. MR2395806 SHAPIRO,S.,VENET,W.,STRAX,P.andVENET, L. (1988). Pe- ZHANG, T. (2006a). From ε-entropy to KL-entropy: Analysis riodic Screening for Breast Cancer: The Health Insurance Plan of minimum information complexity density estimation. Ann. Project and Its Sequelae, 1963–1986. The John Hopkins Univ. Statist. 34 2180–2210. MR2291497 Press, Baltimore, MD. ZHANG, T. (2006b). Information-theoretic upper and lower bounds SIVAGANESAN, S. (1994). Discussion of Berger, J. O., An for statistical estimation. IEEE Trans. Inform. Theory 52 1307– overview of robust Bayesian analysis. TEST 3 116–120. 1321. MR2241190 Statistical Science 2016, Vol. 31, No. 4, 490–494 DOI: 10.1214/16-STS559 © Institute of Mathematical Statistics, 2016

Model Uncertainty First, Not Afterwards Ingrid Glad and Nils Lid Hjort

Abstract. Watson and Holmes propose ways of investigating robustness of statistical decisions by examining certain neighbourhoods around a posterior distribution. This may partly amount to ad hoc modelling of extra uncertainty. Instead of creating neighbourhoods around the posterior a posteriori, we argue that it might be more fruitful to model a layer of extra uncertainty ﬁrst, in the model building process, and then allow the data to determine how big the resulting neighbourhoods ought to be. We develop and brieﬂy illustrate a general strategy along such lines. Key words and phrases: Envelopes, Kullback–Leibler distance, local neighbourhoods, model robustness.

REFERENCES Oxford Univ. Press, Oxford. With discussion. MR2082419 HJORT,N.L.andCLAESKENS, G. (2003). Frequentist model av- CLAESKENS,G.andHJORT, N. L. (2008). Model Selection erage estimators (with discussion). J. Amer. Statist. Assoc. 98 and Model Averaging. Cambridge Univ. Press, Cambridge. MR2431297 879–899. MR2041481 HJORT, N. L. (2003). Topics in non-parametric Bayesian statistics. SCHWEDER,T.andHJORT, N. L. (2016). Conﬁdence, Likelihood, In Highly Structured Stochastic Systems (P.J.Green,N.L.Hjort Probability: Statistical Inference with Conﬁdence Distributions. and S. Richardson, eds.). Oxford Statist. Sci. Ser. 27 455–487. Cambridge Univ. Press, Cambridge.

Ingrid Glad and Nils Lid Hjort are Professors of Statistics, Department of Mathematics, University of Oslo, P.B.1053, Blindern, N-0316 Oslo, Norway (e-mail: [email protected]; [email protected]). Statistical Science 2016, Vol. 31, No. 4, 495–498 DOI: 10.1214/16-STS561 © Institute of Mathematical Statistics, 2016

Contextuality of Misspeciﬁcation and Data-Dependent Losses Peter Grünwald

Abstract. We elaborate on Watson and Holmes’ observation that misspec- iﬁcation is contextual: a model that is wrong can still be adequate in one prediction context, yet grossly inadequate in another. One can incorporate such phenomena by adopting a generalized posterior, in which the likelihood is multiplied by an exponentiated loss. We argue that Watson and Holmes’ characterization of such generalized posteriors does not really explain their good practical performance, and we provide an alternative explanation which suggests a further extension of the method.

REFERENCES GRÜNWALD,P.D.andVAN OMMEN, T. (2014). Inconsistency of Bayesian inference for misspecified linear models, and a pro- BERGER, J. O. (1985). Statistical Decision Theory and Bayesian posal for repairing it. Technical report No. abs/1502.08009. Analysis, 2nd ed. Springer, New York. MR0804611 VOVK, V. (2001). Competitive on-line statistics. Int. Stat. Rev. 69 GRÜNWALD, P. (1999). Viewing all models as ‘probabilistic’. In 213–248. Proceedings of the Twelfth Annual Conference on Computa- ZHANG, T. (2006a). From -entropy to KL-entropy: Analysis tional Learning Theory (Santa Cruz, CA, 1999) 171–182 (elec- of minimum information complexity density estimation. Ann. tronic). ACM, New York. MR1811613 Statist. 34 2180–2210. MR2291497 GRÜNWALD,P.andLANGFORD, J. (2007). Suboptimal behav- ZHANG, T. (2006b). Information-theoretic upper and lower bounds ior of Bayes and MDL in classification under misspecification. for statistical estimation. IEEE Trans. Inform. Theory 52 1307– Mach. Learn. 66 119–149. DOI 10.1007/s10994-007-0716-7. 1321. MR2241190

Peter Grünwald is Senior Researcher, Mathematical Institute, CWI and Professor of Statistical Learning, Leiden University, Science Park 123, 1098 XG Amsterdam, The Netherlands (e-mail: [email protected]). Statistical Science 2016, Vol. 31, No. 4, 499–502 DOI: 10.1214/16-STS562 © Institute of Mathematical Statistics, 2016

Selection of KL Neighbourhood in Robust Bayesian Inference Natalia A. Bochkina

REFERENCES PANOV,M.andSPOKOINY, V. (2015). Finite sample Bernstein– von Mises theorem for semiparametric problems. Bayesian BOCHKINA,N.A.andGREEN, P. J. (2014). The Bernstein–von Anal. 10 665–710. MR3420819 Mises theorem and nonregular models. Ann. Statist. 42 1850– RIBATET,M.,COOLEY,D.andDAV I S O N , A. C. (2012). Bayesian 1878. MR3262470 inference from composite likelihoods, with an application to CHERNOZHUKOV,V.andHONG, H. (2004). Likelihood estimation and inference in a class of nonregular econometric models. spatial extremes. Statist. Sinica 22 813–845. MR2954363 Econometrica 72 1445–1480. MR2077489 ROYALL,R.andTSOU, T.-S. (2003). Interpreting statistical ev- CHOI, H. (2016). Expert information and nonparametric Bayesian idence by using imperfect models: Robust adjusted likelihood inference of rare events. Bayesian Anal. 11 421–445. functions. J. R. Stat. Soc. Ser. B. Stat. Methodol. 65 391–404. MR3471997 MR1983754 DALALYAN,A.S.andTSYBAKOV, A. B. (2012). Sparse regres- STOEHR,J.andFRIEL, N. (2015). Calibration of conditional sion learning by aggregation and Langevin Monte-Carlo. J. composite likelihood for Bayesian inference on Gibbs random Comput. System Sci. 78 1423–1443. MR2926142 fields. J. Mach. Learn. Res. Workshop Conf. Proc. 38 921–929. MÜLLER, U. K. (2013). Risk of Bayesian inference in misspeci- VIELE, K. (2007). Nonparametric estimation of Kullback-Leibler fied models, and the sandwich covariance matrix. Econometrica information illustrated by evaluating goodness of fit. Bayesian 81 1805–1849. MR3117034 Anal. 2 239–280. MR2312281

Natalia A. Bochkina is a Lecturer, University of Edinburgh and Maxwell Institute, United Kingdom, Peter Guthrie Tait Road, Kings Buildings, Edinburgh EH9 3FD, United Kingdom (e-mail: [email protected]). Statistical Science 2016, Vol. 31, No. 4, 503–505 DOI: 10.1214/16-STS563 © Institute of Mathematical Statistics, 2016

Issues in Robustness Analysis Michael Goldstein

Abstract. How may we develop methods of analysis which address the con- sequences of the mismatch between the formal structural requirements of Bayesian analysis and the actual assessments that are carried out in practice? A paper by Watson and Holmes provides an overview of methods developed to address such issues and makes suggestions as to how such analyses might be carried out. This article adds commentary on the principles and practices which should guide us in such problems. Key words and phrases: Robustness, axiomatics, coherence.

REFERENCES robust decisions (with discussion). Statist. Sci. 31 465–557. ILLIAMSON OLDSTEIN GOLDSTEIN,M.andROUGIER, J. (2009). Reiﬁed Bayesian mod- W ,D.andG , M. (2015). Posterior belief elling and inference for physical systems. J. Statist. Plann. In- assessment: Extracting meaningful subjective judgements from ference 139 1221–1239. MR2479863 Bayesian analyses with complex statistical models. Bayesian WATSON,J.andHOLMES, C. (2016). Approximate models and Anal. 10 877–908. MR3432243

Michael Goldstein is Professor of Statistics, Department of Mathematical Sciences, Durham University, Science Laboratories, Stockton Road, Durham DH1 3LE, UK (e-mail: [email protected]). Statistical Science 2016, Vol. 31, No. 4, 506–510 DOI: 10.1214/16-STS567 © Institute of Mathematical Statistics, 2016

Nonparametric Bayesian Clay for Robust Decision Bricks Christian P. Robert and Judith Rousseau

Abstract. This note discusses Watson and Holmes [Statist. Sci. (2016) 31 465–489] and their proposals towards more robust Bayesian decisions. While we acknowledge and commend the authors for setting new and all- encompassing principles of Bayesian robustness, and while we appreciate the strong anchoring of these within a decision-theoretic framework, we remain uncertain as to what extent such principles can be applied outside binary decisions. We also wonder at the ultimate relevance of Kullback–Leibler neighbourhoods into characterising robustness and we instead favour extensions along nonparametric axes. Key words and phrases: Decision-theory, Gamma-minimaxity, misspeciﬁ- cation, prior selection, robust methodology.

REFERENCES DEGROOT, M. H. (1970). Optimal Statistical Decisions.McGraw- Hill, New York. MR0356303 BARDENET,R.,DOUCET,A.andHOLMES, C. (2014). Towards MARIN,J.,PUDLO,P.,ROBERT,C.andRYDER, R. (2011). Ap- scaling up Markov chain Monte Carlo: An adaptive subsam- proximate Bayesian computational methods. Stat. Comput. 21 pling approach. In Proc.31st Intern. Conf. Machine Learning 279–291. (ICML) 405–413. RAIFFA, H. (1968). Decision Analysis: Introductory Lectures on BERGER, J. O. (1985). Statistical Decision Theory and Bayesian Choices Under Uncertainty. Addison-Wesley, Reading, MA. Analysis, 2nd ed. Springer, New York. MR0804611 ROBERT,C.andCASELLA, G. (2011). A short history of Markov BERGER,J.,INSUA,D.andRUGGERI, F. (1996). Bayesian robust- chain Monte Carlo: Subjective recollections from incomplete ness. In Proceedings of the 2nd International Workshop Held in data. Statist. Sci. 26 102–115. MR2849912 Rimini, May 22–25, 1995. IMS, Hayward, CA. MR1478676 RUBIN, H. (1987). A weak system of axioms for “rational” behav- BISSIRI,P.,HOLMES,C.andWALKER, S. (2013). A general ior and the nonseparability of utility from prior. Statist. Deci- framework for updating belief distributions. Technical report, sions 5 47–58. MR0886877 available at arXiv:1306.6430v1. SAMANIEGO, F. J. (2010). A Comparison of the Bayesian and CAPPÉ,O.andROBERT, C. P. (2000). Markov chain Monte Carlo: Frequentist Approaches to Estimation. Springer, New York. 10 years and still running. J. Amer. Statist. Assoc. 95 1282– MR2664350 1286. MR1825276 SIMPSON,D.P.,MARTINS,T.G.,RIEBLER,A.,FUGLSTAD,G.- CHOPIN,N.andROBERT, C. (2007). Comments on “Estimating A., RUE,H.andSØRBYE, S. H. (2014). Penalising model the integrated likelihood via posterior simulation using the har- component complexity: A principled, practical approach to con- monic mean identity”. In Bayesian Statistics (O.U.P.Bernardo structing priors. Available at arXiv:1403.4630. et al., eds.). Oxford Sci. Publ. 8 371–416. Oxford Univ. Press, WATSON,J.andHOLMES, C. (2016). Approximate models and Oxford. MR2433201 robust decisions. Statist. Sci. 31 465–489.

Christian P. Robert is Professor, and Judith Rousseau is Professor, CEREMADE, Université Paris-Dauphine, PSL, 75775 Paris cedex 16, France (e-mail: [email protected]; [email protected]). Both authors are members of Laboratoire de Statistique, CREST, Paris. C. P. Robert is also afﬁliated as a part-time professor in the Department of Statistics of the University of Warwick, Coventry, UK. Statistical Science 2016, Vol. 31, No. 4, 511–515 DOI: 10.1214/16-STS570 © Institute of Mathematical Statistics, 2016

Ambiguity Aversion and Model Misspeciﬁcation: An Economic Perspective Lars Peter Hansen and Massimo Marinacci

REFERENCES HANSEN,L.P.andSARGENT, T. J. (2016). Sets of models and prices of uncertainty. Working Paper No. 22000. ANDERSON,E.,HANSEN,L.P.andSARGENT, T. J. (2003). HANSEN,L.P.,SARGENT,T.J.,TURMUHAMBETOVA,G.and A quartet of semigroups for model speciﬁcation, robustness, WILLIAMS, N. (2006). Robust control and model misspeciﬁca- prices of risk, and model detection. J. Eur. Econ. Assoc. 1 68– tion. J. Econom. Theory 128 45–90. MR2229772 123. KLIBANOFF,P.,MARINACCI,M.andMUKERJI, S. (2005). ARROW, K. J. (1951). Alternative approaches to the theory of A smooth model of decision making under ambiguity. Econo- choice in risk-taking situtations. Econometrica 19 404–437. metrica 73 1849–1892. MR2171327 MR0046020 KLIBANOFF,P.,MARINACCI,M.andMUKERJI, S. (2009). Re- CHAMBERLAIN, G. (2000). Econometric applications of maxmin cursive smooth ambiguity preferences. J. Econom. Theory 144 expected utility. J. Appl. Econometrics 15 625–644. 930–976. MR2887278 CHEN,Z.andEPSTEIN, L. (2002). Ambiguity, risk, and as- set returns in continuous time. Econometrica 70 1403–1443. KOOPMANS, T. C. (1960). Stationary ordinal utility and impa- MR1929974 tience. Econometrica 28 287–309. MR0127422 MACCHERONI,F.,MARINACCI,M.andRUSTICHINI,A. DE FINETTI, B. (1937). La prévision: Ses lois logiques, ses sources subjectives. Ann. Inst. Henri Poincaré 7 1–68. MR1508036 (2006a). Dynamic variational preferences. J. Econom. Theory ELLSBERG, D. (1961). Risk, ambiguity, and the Savage axioms. Q. 128 4–44. MR2229771 J. Econ. 75 643–669. MACCHERONI,F.,MARINACCI,M.andRUSTICHINI,A. EPSTEIN,L.G.andSCHNEIDER, M. (2003). Recursive multiple- (2006b). Ambiguity aversion, robustness, and the variational priors. J. Econom. Theory 113 1–31. MR2017864 representation of preferences. Econometrica 74 1447–1498. GILBOA,I.andMARINACCI, M. (2013). Ambiguity and the MR2268407 Bayesian Paradigm, in Advances in Economics and Economet- MARINACCI, M. (2015). Model uncertainty. J. Eur. Econ. Assoc. rics: Theory and Applications (D. ACEMOGLU,M.ARELLANO 13 998–1076. and E. DEKEL, eds.). Cambridge Univ. Press, Cambridge. PETERSEN,I.R.,JAMES,M.R.andDUPUIS, P. (2000). Minimax GILBOA,I.andSCHMEIDLER, D. (1989). Maxmin expected optimal control of stochastic uncertain systems with relative en- utility with nonunique prior. J. Math. Econom. 18 141–153. tropy constraints. IEEE Trans. Automat. Control 45 398–412. MR1000102 MR1762853 GOOD, I. J. (1952). Rational decisions. J. Roy. Statist. Soc. Ser. B. SAVAG E , L. J. (1954). The Foundations of Statistics. Wiley, New 14 107–114. MR0077033 York. MR0063582 HANSEN, L. (2014). Nobel lecture: Uncertainty outside and inside VON NEUMANN,J.andMORGENSTERN, O. (1944). Theory of economic models. J. Polit. Econ. 122 945–987. Games and Economic Behavior. Princeton Univ. Press, Prince- HANSEN,L.P.andSARGENT, T. J. (2001). Robust control and ton, NJ. MR0011937 model uncertainty. Am. Econ. Rev. 91 60–66. WALD, A. (1950). Statistical Decision Functions. Wiley, New HANSEN,L.P.andSARGENT, T. J. (2007). Recursive robust es- York. MR0036976 timation and control without commitment. J. Econom. Theory WATSON,J.andHOLMES, C. (2016). Approximate models and 136 1–27. MR2888400 robust decisions. Statist. Sci. 31 465–589.

Lars Peter Hansen is the David Rockefeller Distinguished Service Professor in Economics, Statistics and the College, University of Chicago, Chicago, Illinois 60637, USA (e-mail: [email protected]). Massimo Marinacci holds the AXA-Bocconi Chair in Risk, Department of Decision Sciences and IGIER, Università Bocconi, 20136 Milano, Italy (e-mail: [email protected]). Statistical Science 2016, Vol. 31, No. 4, 516–520 DOI: 10.1214/16-STS596 © Institute of Mathematical Statistics, 2016

Rejoinder: Approximate Models and Robust Decisions James Watson and Chris Holmes

REFERENCES GRÜNWALD,P.andVA N OMMEN, T. (2014). Inconsistency of Bayesian inference for misspeciﬁed linear models, and a pro- AHMADI-JAV I D , A. (2011). An information-theoretic approach posal for repairing it. Preprint. Available at arXiv:1412.3730. to constructing coherent risk measures. In IEEE International HANSEN, L. P. (2014). Nobel lecture: Uncertainty outside and in- Symposium on Information Theory Proceedings (ISIT) 2125– side economic models. J. Polit. Econ. 122 945–987. 2127. IEEE, New York. HANSEN,L.P.andSARGENT, T. J. (2008). Robustness. Princeton ARTZNER,P.,DELBAEN,F.,EBER,J.-M.andHEATH,D. University Press, Princeton, NJ. (1999). Coherent measures of risk. Math. Finance 9 203–228. MACCHERONI,F.,MARINACCI,M.andRUSTICHINI, A. (2006). MR1850791 Ambiguity aversion, robustness, and the variational representa- BERRY,S.M.,CARLIN,B.P.,LEE,J.J.andMÜLLER, P. (2011). tion of preferences. Econometrica 74 1447–1498. MR2268407 Bayesian Adaptive Methods for Clinical Trials. CRC Press, SIMPSON,D.P.,RUE,H.,MARTINS,T.G.,RIEBLER,A.and Boca Raton, FL. MR2723582 SØRBYE, S. H. (2014). Penalising model component com- BISSIRI,P.G.,HOLMES,C.C.andWALKER, S. G. (2016). plexity: A principled, practical approach to constructing priors. A general framework for updating belief distributions. J. R. Stat. Preprint. Available at arXiv:1403.4630. Soc. Ser. B. Stat. Methodol. To appear. SPIEGELHALTER,D.J.,FREEDMAN,L.S.andPAR- BOX, G. E. P. (1980). Sampling and Bayes’ inference in scientiﬁc MAR, M. K. B. (1994). Bayesian approaches to randomized modelling and robustness. J. Roy. Statist. Soc. Ser. A 143 383– trials. J. Roy. Statist. Soc. Ser. A 157 357–416. MR1321308 430. MR0603745 WATSON,J.,NIETO-BARAJAS,L.andHOLMES, C. (2016). Char- CAROTA,C.,PARMIGIANI,G.andPOLSON, N. G. (1996). Diag- acterising variation of nonparametric random probability mod- nostic measures for model criticism. J. Amer. Statist. Assoc. 91 els using the Kullback–Leibler divergence. Statistics. To appear. 753–762. MR1395742 Available at arXiv:1411.6578. ELLSBERG, D. (1961). Risk, ambiguity, and the Savage axioms. WHITTLE, P. (1990). Risk-Sensitive Optimal Control.Wiley, Q. J. Econ. 643–669. Chichester. MR1093001

James Watson is a Postdoctoral Researcher at the Mahidol-Oxford Research Unit, Centre for Tropical Medicine and Global Health, Nufﬁeld Department of Clinical Medicine, University of Oxford, Oxford, United Kingdom (e-mail: [email protected]). Chris Holmes is Professor of Biostatistics, Department of Statistics, University of Oxford, United Kingdom (e-mail: [email protected]). Statistical Science 2016, Vol. 31, No. 4, 521–540 DOI: 10.1214/16-STS565 © Institute of Mathematical Statistics, 2016 Filtering and Tracking Survival Propensity (Reconsidering the Foundations of Reliability) Nozer D. Singpurwalla

Abstract. The work described here was motivated by the need to address a long standing problem in engineering, namely, the tracking of reliability growth. An archetypal scenario is the performance of software as it evolves over time. Computer scientists are challenged by the task of when to release software. The same is also true for complex engineered systems like aircraft, automobiles and ballistic missiles. Tracking problems also arise in actuarial science, biostatistics, cancer research and mathematical finance. A natural approach for addressing such problems is via the control theory methods of filtering, smoothing and prediction. But to invoke such methods, one needs a proper philosophical foundation, and this has been lacking. The first three sections of this paper endeavour to fill this gap. A consequence is the point of view proposed here, namely, that reliability not be interpreted as a probability. Rather, reliability should be conceptualized as a dynamically evolving propensity in the sense of Pierce and Popper. Whereas propensity is to be taken as an undefined primitive, it manifests as a chance (or frequency) in the sense of de Finetti. The idea of looking at reliability as a propensity also appears in the philosophical writings of Kolmogorov. Furthermore, survivability which quantifies ones uncertainty about a propensity should be the metric of performance that needs to be tracked. The first part of this paper is thus a proposal for a paradigm shift in the manner in which one concep- tualizes reliability, and by extension, survival analysis. This message is also germane to other areas of applied probability and statistics, like queueing, inventory and time series analysis. The second part of this paper is technical. Its purpose is to show how the philosophical material of the first part can be incorporated into a framework that leads to a methodological package. To do so, we focus on the problem which motivated the first part, and develop a mathematical model for describing the evolution of an item’s propensity to survive. The item could be a component, a system or a biological entity. The proposed model is based on the notion of competing risks. Models like this also appear in biostatistics under the label of cure models. Whereas the competing risks scenario is in- structive, it is not the only way to describe the phenomenon of growth; its use here is illustrative. All the same, one of its virtues is that it paves the path towards a contribution to the state of the art of filtering by considering the case of censored observations. Even though censoring is the hallmark of survival analysis, it could also arise in time series analysis and control theory, making the development here of a broader and more general appeal.

Nozer D. Singpurwalla is Chair Professor, Department of Systems Engineering and Engineering Management, City University of Hong Kong, Hong Kong, China (e-mail: [email protected]). Key words and phrases: Chance, competing risks, cure models, exchangeability, ﬁltering censored observations, frailty, propensity, proportional hazards, time series analysis.

REFERENCES HAMADA,M.S.,WILSON,A.G.,REESE,C.S.and MARTZ, H. F. (2008). Bayesian Reliability. Springer, New AALEN, O. O. (1988). Heterogenity in survival analysis. Stat. Med. York. MR2724437 7 1121–1137. HEWITT,E.andSAVAG E , L. J. (1955). Symmetric measures BARLOW,R.E.andMENDEL, M. B. (1992). De Finetti-type rep- on Cartesian products. Trans. Amer. Math. Soc. 80 470–501. resentations for life distributions. J. Amer. Statist. Assoc. 87 MR0076206 1116–1122. MR1209569 HOWSON,C.andURBACH, P. (2006). Scientific Reasoning: The BARLOW,R.E.andPROSCHAN, F. (1965). Mathematical Theory Bayesian Approach. Open Court, Chicago, IL. of Reliability . Wiley, New York-London-Sydney. With Contri- HUMPHREYS, P. (1985). Why propensities cannot be probabilities. butions by Larry C. Hunter. MR0195566 Philos. Rev. 94 557–570. BARLOW,R.E.andPROSCHAN, F. (1975). Statistical Theory of KENDALL, M. G. (1949). On the reconciliation of theories of prob- Reliability and Life Testing: Probability Models. Holt, Rinehart ability. Biometrika 36 101–116. MR0035407 and Winston, Inc., New York. MR0438625 KOLMOGOROV, A. N. (1969). The theory of probability. In Math- BATHER, J. A. (1965). Invariant conditional distributions. Ann. ematics: Its Content, Methods and Meaning,Vol.2,Part3. Math. Stat. 36 829–846. MR0179840 (A. D. Aleksandrov, A. N. Kolmogorov and M. A. Lavrentav, BLOCK,H.W.andSAV I T S , T. H. (1997). Burn-in. Statist. Sci. 12 eds.). MIT Press, Cambridge, MA. 1–13. LINDLEY, D. V. (1990). The 1988 Wald memorial lectures: The CHEN,M.-H.,IBRAHIM,J.G.andSINHA, D. (1999). A new present position in Bayesian statistics. Statist. Sci. 5 44–89. Bayesian model for survival data with a surviving fraction. J. With comments and a rejoinder by the author. MR1054857 Amer. Statist. Assoc. 94 909–919. MR1723307 LINDLEY,D.V.andSINGPURWALLA, N. D. (1986). Multivari- CLAYTON, D. G. (1991). A Monte Carlo method for Bayesian in- ate distributions for the life lengths of components of a system ference in frailty models. Biometrics 47 467–485. sharing a common environment. J. Appl. Probab. 23 418–431. CLAYTON,D.andCUZICK, J. (1985). Multivariate generalizations MR0839996 of the proportional hazards model. J. Roy. Statist. Soc. Ser. A 148 LINDLEY,D.V.andSINGPURWALLA, N. D. (2002). On ex- 82–117. MR0806480 changeable, causal and cascading failures. Statist. Sci. 17 209– COX, D. R. (1972). Regression models and life-tables. J. R. Stat. 219. MR1939340 Soc. Ser. B. Stat. Methodol. 34 187–220. With discussion by F. MARTIN, O. S. (2012). A Dynamic Competing Risk Model for Downton, Richard Peto, D. J. Bartholomew, D. V. Lindley, P. Filtering Reliability and Tracking Survivability. ProQuest LLC, W. Glassborow, D. E. Barton, Susannah Howard, B. Benjamin, Ann Arbor, MI. Ph.D. thesis, The George Washington Univer- John J. Gart, L. D. Meshalkin, A. R. Kagan, M. Zelen, R. E. sity. MR2995920 Barlow, Jack Kalbfleisch, R. L. Prentice and Norman Breslow, MEEKER,W.Q.andESCOBAR, L. A. (1998). Statistical Methods andareplybyD.R.Cox.MR0341758 for Reliability Data. Wiley, New York. DE FINETTI, B. (1937). La prévision : Ses lois logiques, MEINHOLD,R.andSINGPURWALLA, N. D. (1983). Understand- ses sources subjectives. Ann. Inst. Henri Poincaré 7 1–68. ing the Kalman filter. Amer. Statist. 37 123–127. MR1508036 MILLER, R. W. (1975). Propensity: Popper or peirce? British J. DE GROOT, M. H. (1988). A Bayesian view of assessing uncer- Philos. Sci. 123–132. tainty and comparing expert opinion. J. Statist. Plann. Inference POPPER, K. R. (1957). The propensity interpretation of the cal- 20 295–306. MR0976182 culus of probability, and the quantum theory. In Observation DIACONIS,P.andFREEDMAN, D. (1987). A dozen de Finetti-style and Interpretation in the Philosophy of Physics (S. Korner, ed.). results in search of a theory. Ann. Inst. Henri Poincaré Probab. Dover, Mineola, NY. Stat. 23 397–423. MR0898502 POPPER, K. R. (1959). The propensity interpretation of probabil- DUBEY, S. D. (1966). Compound Pascal distributions. Ann. Inst. ity. British J. Philos. Sci. 10 25–42. Statist. Math. 18 357–365. MR0208712 SINGPURWALLA, N. D. (1988). Foundational issues in reliability ESCOBAR,L.A.andMEEKER, W. Q. (2006). A review of accel- andriskanalysis.SIAM Rev. 30 264–282. MR0941113 erated test models. Statist. Sci. 21 552–577. MR2380715 SINGPURWALLA, N. D. (1995). Survival in dynamic environ- FETZER, J. H. (1981). Scientific Knowledge: Causation, Explana- ments. Statist. Sci. 10 86–113. tion, and Corroboration. Boston Stud. Philos. Sci. 69.D.Reidel, SINGPURWALLA, N. D. (2002). Some cracks in the empire of Dordrecht. chance (flaws in the foundations of reliability). Int. Stat. Rev. GILLIES, D. (2000). Varieties of propensity. British J. Philos. Sci. 70 53–65. 51 807–835. MR1813908 SINGPURWALLA, N. D. (2006). Reliability and Risk: A Bayesian GOOD, I. J. (1950). Probability and the Weighing of Evidence. Perspective. Wiley, Chichester. MR2265917 Charles Griffin & Co., Ltd., London. MR0041366 SINGPURWALLA, N. D. (2007). Betting on residual life: The GOOD,I.J.andCARD, W. I. (1971). The diagnostic process with caveats of conditioning. Statist. Probab. Lett. 77 1354–1361. special reference to errors. Methods Inf. Med. 10 176–188. MR2392806 SINGPURWALLA, N. D. (2010). Comment: “Reliability growth 209. MR0626767 management metrics and statistical methods for discrete- SPIZZICHINO, F. (2001). Subjective Probability Models for Life- use systems” [MR2789242]. Technometrics 52 396–397. times. Monographs on Statistics and Applied Probability 91. MR2779425 Chapman & Hall/CRC, Boca Raton, FL. MR1980207 SINGPURWALLA, N. D. (2016). The indicative and irrealis moods TSODIKOV,A.D.,IBRAHIM,J.G.andYAKOVLEV, A. Y. (2003). in Bayesian inference. Under review. Estimating cure rates from survival data: An alternative to two- SINGPURWALLA,N.D.andWILSON, P. (2004). When can fi- J Amer Statist Assoc 98 nite testing ensure infinite trustworthiness? J. Iran. Stat. Soc. component mixture models. . . . . 1063– (JIRSS) 3 1–37. 1078. MR2055496 SMITH, A. F. M. (1981). On random sequences with centred spher- VON MISES, R. (1941). On the foundations of probability and ical symmetry. J. R. Stat. Soc. Ser. B. Stat. Methodol. 43 208– statistics. Ann. Math. Stat. 12 191–205. MR0004386 Statistical Science 2016, Vol. 31, No. 4, 541–544 DOI: 10.1214/16-STS583 © Institute of Mathematical Statistics, 2016

On Software and System Reliability Growth and Testing Frank P. A. Coolen

Abstract. Singpurwalla presents an insightful proposal on foundations of reliability [Statist. Sci. 31 (2016) 521–540], suggesting to consider reliability not as a probability but as a propensity, in particular as the unobservable parameter in De Finetti’s famous representation theorem. One speciﬁc issue considered is reliability growth, with example scenario the performance of software as it evolves over time. We brieﬂy discuss some related aspects, mainly based on applied research on statistical methods to support software testing and insights from our research on system reliability. Key words and phrases: Reliability growth, software testing, system reliability.

REFERENCES [5] COOLEN,F.P.A.,GOLDSTEIN,M.andWOOFF,D.A. (2007). Using Bayesian statistics to support testing of software [1] COOLEN, F. P. A. (2012). On some statistical aspects of soft- systems. J. Risk Reliab. 221 85–93. ware testing and reliability. In Complex Systems and Depend- [6] COOLEN-MATURI,T.andCOOLEN, F. P. A. (2011). Unob- ability (W. Zamojski, J. Mazurkiewicz, J. Sugier, T. Walkowiak served, re-deﬁned, unknown or removed failure modes in com- and J. Kacprzyk, eds.) 103–113. Springer, Berlin. peting risks. J. Risk Reliab. 225 461–474. [2] COOLEN,F.P.A.andCOOLEN-MATURI, T. (2012). On gen- eralizing the signature to systems with multiple types of com- [7] SAMANIEGO, F. J. (2007). System Signatures and Their Appli- ponents. In Complex Systems and Dependability (W. Zamojski, cations in Engineering Reliability. International Series in Op- J. Mazurkiewicz, J. Sugier, T. Walkowiak and J. Kacprzyk, erations Research & Management Science 110. Springer, New eds.) 115–130. Springer, Berlin. York. MR2380178 [3] COOLEN,F.P.A.andCOOLEN-MATURI, T. (2016). The [8] SINGPURWALLA, N. D. (2016). Filtering and tracking sur- structure function for system reliability as predictive (impre- vival propensity (reconsidering the foundations of reliability). cise) probability. Reliab. Eng. Syst. Saf. 154 180–187. Statist. Sci. 31 521–540. [4] COOLEN,F.P.A.,GOLDSTEIN,M.andMUNRO, M. (2001). [9] WOOFF,D.A.,GOLDSTEIN,M.andCOOLEN,F.P.A. Generalized partition testing via Bayes linear methods. Inf. (2002). Bayesian graphical models for software testing. IEEE Softw. Technol. 43 783–793. Trans. Softw. Eng. 28 510–525.

Frank P. A. Coolen is Professor of Statistics, Department of Mathematical Sciences, Durham University, Durham, DH1 3LE, United Kingdom (e-mail: [email protected]). Statistical Science 2016, Vol. 31, No. 4, 545–548 DOI: 10.1214/16-STS577 © Institute of Mathematical Statistics, 2016

How About Wearing Two Hats, First Popper’s and then de Finetti’s? Elja Arjas

REFERENCES GEISSER, S. (1993). Predictive Inference. Monographs on Statis- tics and Applied Probability 55. Chapman & Hall, New York. ARJAS,E.,HAARA,P.andNORROS, I. (1992). Filtering the his- MR1252174 tories of a partially observed marked point process. Stochastic JAYNES, E. T. (1990). Maximum entropy and Bayesian methods. Process. Appl. 40 225–250. MR1158025 In Proceedings of the Ninth Annual Workshop Held at Dart- ARJAS,E.andHOLMBERG, J. (1995). Marked point process mouth College, Hanover, New Hampshire, August 14–18, 1989. framework for living probabilistic safety assessment and risk Fundamental Theories of Physics 39 (Paul F. Fougère, ed.). follow-up. Reliab. Eng. Syst. Saf. 49 59–73. Kluwer Academic, Dordrecht. MR1093675 NATVIG,B.andEIDE, H. (1987). Bayesian estimation of system BERGMAN, B. (1985). On reliability theory and its applications. reliability. Scand. J. Stat. 14 319–327. MR0943292 Scand. J. Stat. 12 1–41. MR0804223 PITMAN, J. (1996). Some developments of the Blackwell– BERLINER,L.M.andHILL, B. M. (1988). Bayesian nonpara- MacQueen urn scheme. In Statistics, Probability and metric survival analysis. J. Amer. Statist. Assoc. 83 772–784. Game Theory. Institute of Mathematical Statistics Lecture MR0963805 Notes—Monograph Series 30 245–267. IMS, Hayward, CA. DAWID, A. P. (2004). Probability, causality and the empirical MR1481784 world: A Bayes–de Finetti–Popper–Borel synthesis. Statist. Sci. SPIZZICHINO, F. (2001). Subjective Probability Models for Life- 19 44–57. MR2082146 times. Chapman & Hall/CRC, Boca Raton, FL. MR1980207

Elja Arjas is Professor Emeritus, Department of Mathematics and Statistics, P.O. Box 68 (Gustaf Hällströmin katu 2b), FI-00014 University of Helsinki, Finland, and Part Time Researcher, Oslo Centre for Biostatistics and Epidemiology (OCBE), University of Oslo, Norway (e-mail: elja.arjas@helsinki.ﬁ). Statistical Science 2016, Vol. 31, No. 4, 549–551 DOI: 10.1214/16-STS585 © Institute of Mathematical Statistics, 2016

What Does “Propensity” Add? Jane Hutton

Abstract. Singpurwalla addresses the important challenge of modelling a unique individual. He proposes “propensity” as an approach to describing the reliability or life time of “one of a kind”. My view is that mathematical modelling is only possible when we assume that nonunique features provide sufﬁcient information for statistical prediction to be useful. As far as possible, we should test our assumptions. However, contrary to a popular perception of Hume, we always rely on some beliefs.

REFERENCES RIZOPOULOS, D. (2012). Joint Models for Longitudinal and Time- to-Event Data. Chapman & Hall/CRC, Boca Raton, FL. CLARK, S. R. L. (2016). Atheism considered as a Christian sect. Philosophy 90 277–303. ROGERS,J.K.andHUTTON, J. L. (2013). Joint modelling of pre- HUTTON, J. L. (1995). Statistics is essential for professional randomisation event counts and multiple post-randomisation ethics. J. Appl. Philos. 12 253–261. survival times with cure rates: Application to data for early PEARL, J. (2009). Causality: Models, Reasoning, and Inference, epilepsy and single seizures. J. Appl. Stat. 40 546–562. 2nd ed. Cambridge Univ. Press, Cambridge. MR2548166 MR3047301

Jane Hutton is Professor, Department of Statistics, The University of Warwick, Coventry, CV4 7AL, United Kingdom (e-mail: [email protected]). Statistical Science 2016, Vol. 31, No. 4, 552–554 DOI: 10.1214/16-STS575 © Institute of Mathematical Statistics, 2016

Reconciling the Subjective and Objective Aspects of Probability Glenn Shafer

Abstract. Since the early nineteenth century, the concept of objective probability has been dynamic. As we recognize this history, we can strengthen Pro- fessor Nozer Singpuwalla’s vision of reliability of survival analysis by align- ing it with earlier conceptions elaborated by Laplace, Borel, Kolmogorov, Ville and Neyman. By emphasizing testing and recognizing the generality of the vision of Kolmogorov and Neyman, we gain a perspective that does not rely on exchangeability. Key words and phrases: Objective probability, subjective probability, game-theoretic probability, martingale testing, defensive forecasting.

REFERENCES NEYMAN, J. (1960). Indeterminism in science and new de- mands on statisticians. J. Amer. Statist. Assoc. 55 625–639. AALEN,O.O.,ANDERSEN,P.K.,BORGAN,Ø.,GILL,R.D. MR0116393 and KEIDING, N. (2009). History of applications of martin- SHAFER, G. (1990). The unity and diversity of probability. Statist. gales in survival analysis. J. Électron. Hist. Probab. Stat. 5 1–28. Sci. 5 435–462. With comments and a rejoinder by the author. MR2520671 MR1092984 BIENVENU,L.,SHAFER,G.andSHEN, A. (2009). On the history of martingales in the study of randomness. J. Électron. Hist. SHAFER, G. (2008). Defensive forecasting: How to use similarity Probab. Stat. 5 1–40. MR2520666 to make forecasts that pass statistical tests. In Preferences and DAWID, A. P. (2004). Probability, causality and the empirical Similarities (G. D. Riccia, D. Dubois, R. Kruse and H.-J. Lenz, world: A Bayes–de Finetti–Popper–Borel synthesis. Statist. Sci. eds.) 215–247. Springer, New York. 19 44–57. MR2082146 SHAFER,G.andVOVK, V. (2001). Probability and Finance: It’s FISCHER, H. (2011). A History of the Central Limit Theorem: From Only a Game! Wiley-Interscience, New York. MR1852450 Classical to Modern Probability Theory. Springer, New York. VILLE, J. (1939). Étude Critique de la Notion de Collectif. MR2743162 Gauthier-Villars, Paris. HEYDE,C.C.andSENETA, E. (1977). I.J. Bienaymé: Statistical VOVK,V.,TAKEMURA,A.andSHAFER, G. (2005). Defensive Theory Anticipated. Springer, New York. MR462888 forecasting. In AISTATS 2005: Proceedings of the Tenth In- KLEIN, J. L. (1997). Statistical Visions in Time: A History of ternational Workshop on Artiﬁcial Intelligence and Statistics Time Series Analysis, 1662–1938. Cambridge Univ. Press, Cam- (R. Cowell and Z. Ghahramani, eds.) 365–372. The Society for bridge. MR1479409 Artiﬁcial Intelligence and Statistics.

Glenn Shafer is University Professor, Rutgers Business School, 1 Washington Park, Newark, New Jersey 07102, USA (e-mail: [email protected]). Statistical Science 2016, Vol. 31, No. 4, 555–557 DOI: 10.1214/16-STS593 © Institute of Mathematical Statistics, 2016

Rejoinder: Concert Unlikely, “Jugalbandi” Perhaps Nozer D. Singpurwalla

Abstract. This rejoinder to the discussants of Filtering and Tracking Sur- vival Propensity begins with a brief history of the statistical aspects of reliability and its impact on survival analysis and responds to the several issues raised by the discussants, some of which are conceptual and some pragmatic. Key words and phrases: Military standards, Weibull distribution, Wash 1400.

REFERENCES GNEDENKO,B.V.,BELYAEV,Y.K.andSOLOVIEW,A.D. (1965) Mathematical Methods in Reliability Theory. Nauka, ARJAS, E. (1981). The failure and hazard processes in multivariate Moscow. reliability systems. Math. Oper. Res. 6 551–562. MR0703096 HE,W.,WILLIARD,N.,OSTERMAN,M.andPECHT, M. (2011). BARLOW,R.E.andPROSCHAN, F. (1965). Mathematical Theory Pronostics of lithium-ion batteries on Dempster–Shafer theory of Reliability. Wiley, New York. MR0195566 and the Bayesian Monte Carlo method. J. Power Sources 196 BARLOW,R.E.andPROSCHAN, F. (1975). Statistical Theory of 10314–10321. Reliability and Life Testing: Probability Models. Holt, Rinehart KAO, J. H. K. (1956). A new life quality measure for electron and Winston, New York. MR0438625 tubes. IRE Trans. Reliab. Qual. Control 7 1–11. BIENVENU,L.,SHAFER,G.andSHEN, A. (2009). On the history KAO, J. H. K. (1959). A graphical estimation of mixed Weibull of martingales in the study of randomness. J. Électron. Hist. parameters in life testing of electronic tubes. Technometrics 1 Probab. Stat. 5 1–40. MR2520666 389–407. COX, D. R. (1972). Regression models and life-tables. J. R. Stat. KAPLAN,E.L.andMEIER, P. (1958). Nonparametric estimation Soc. Ser. B. Stat. Methodol. 34 187–220. MR0341758 from incomplete observations. J. Amer. Statist. Assoc. 53 457– DAV I S , D. J. (1952). An analysis of some failure data. J. Amer. 481. MR0093867 Statist. Assoc. 47 113–150. SHAFER, G. (1990). The unity and diversity of probability. Statist. DAWID, A. P. (2004). Probability, causality and the empirical Sci. 5 435–462. With comments and a rejoinder by the author. world: A Bayes–de Finetti–Popper–Borel synthesis. Statist. Sci. MR1092984 19 44–57. MR2082146 VILLE, J. (1939). Étude Critique de la Notion de Collectif. EPSTEIN,B.andSOBEL, M. (1953). Life testing. J. Amer. Statist. Gauthier-Villars, Paris. Assoc. 48 486–502. MR0056898 WASH 1400 (1975). Reactor Safety Study. NUREG-75/014. U.S. EPSTEIN,B.andSOBEL, M. (1954). Some theorems relevant to Nuclear Regulatory Commission, Washinton, D.C. life testing from an exponential distribution. Ann. Math. Stat. WEIBULL, W. (1951). A statistical distribution functions of wide 25 373–381. MR0061339 applicabiity. J. Appl. Mech. 293–297.

Nozer D. Singpurwalla is Chair Professor, Department of Systems Engineering and Engineering Management, City University of Hong Kong, Hong Kong, China (e-mail: [email protected]). Statistical Science 2016, Vol. 31, No. 4, 558–577 DOI: 10.1214/16-STS572 © Institute of Mathematical Statistics, 2016 Chaos Communication: A Case of Statistical Engineering Anthony J. Lawrance

Abstract. The paper gives a statistically focused selective view of chaos- based communication which uses segments of noise-like chaotic waves as carriers of messages, replacing the traditional sinusoidal radio waves. The presentation concerns joint statistical and dynamical modelling of the binary communication system known as “chaos shift-keying”, representative of the area, and leverages the statistical properties of chaos. Practically, such systems apply to both wireless and optical laser communication channels. The- oretically, the chaotic waves are generated iteratively by chaotic maps, and practically, by electronic circuits or lasers. Both single-user and multiple- user systems are covered. The focus is on likelihood-based decoding of messages, essentially estimation of binary-valued parameters and efﬁciency of the system is in terms of the probability of bit decoding error. The empha- sis is on exact theoretical results for bit error rate, their structured approximations and engineering interpretations. Design issues, optimality of performance, interference and fading are other topics considered. The statistical aspects of chaotic synchronization are involved in the modelling of optical systems. Empirical illustrations from an experimental laser-based system are presented. The overall aim is to show the use of statistical methodology in unifying and advancing the area. Key words and phrases: Bit error rate, chaos communications, chaos modelling, chaos shift-keying, communications engineering, decoding as likelihood-based statistical inference, empirical communication system analysis, Gaussian approximations, laser chaos, nonlinear dynamics, optical noise, statistical modelling, statistical time series, synchronization error.

REFERENCES HASLER,M.andSCHIMMING, T. (2002). Optimal and suboptimal chaos receivers. Proc. IEEE 50 733–746. ABEL,A.,SCHWARZ,W.andGOTZ, M. (2000). Noise perfor- HASLER,M.,MAZZINI,G.,OGORZALEK,M.,ROVATTI,M.and mance of chaotic communication systems. IEEE Trans. Circuits SETTI, G. (2002). Scanning the special issue—special issue on Syst. I, Fundam. Theory Appl. 47 1726–1732. applications of nonlinear dynamics to electronic and informa- BERLINER, L. M. (1992). Statistics, probability and chaos. Statist. Sci. 7 69–122. MR1173418 tion engineering. Proc. IEEE 90 631–640. CARROLL,T.L.andPECORA, L. M. (1992). A circuit for study- HEIL,T.,MULET,J.,FISCHER,I.,MIRASSO,C.R.,PEIL,M., ing the synchronization of chaotic systems. Internat. J. Bifur. COLET,P.andELSABER, W. (2002). ON/OFF phase shift key- Chaos Appl. Sci. Engrg. 2 659–667. MR1192702 ing for chaos-encrypted communication using external-cavity DONATI,S.andMIRASSO, C. M. (2002). Introduction to the fea- semiconductor lasers. IEEE J. Quantum Electron. 38 1162– ture section on optical chaos and applications to cryptography. 1170. IEEE J. Quantum Electron. 38 1138–1140. HOMER,M.E.,HOGAN,S.J.,BERNADO,M.D.and GALIAS,Z.andMAGGIO, G. M. (2001). Quadrature chaos-shift WILLIAMS, C. (2004). The importance of choosing attractors keying: Theory and performance analysis. IEEE Trans. Circuits for optimizing chaotic communications. IEEE Trans. Circuits Syst. I, Fundam. Theory Appl. 48 1510–1519. MR1873101 Syst. II, Express Briefs 51 511–516.

Anthony J. Lawrance is Professor, Department of Statistics, University of Warwick, Coventry, United Kingdom (e-mail: [email protected]). HUA,Y.andPING, J. G. (2012). High-efﬁciency differential- LAWRANCE,A.J.andBALAKRISHNA, N. (2008). Statistical de- chaos-shift-keying scheme for chaos-based non-coherent com- pendency in chaos. Internat. J. Bifur. Chaos Appl. Sci. Engrg. munication. IEEE Trans. Circuits Syst. II, Express Briefs 59 18 3207–3219. MR2487910 312–316. LAWRANCE,A.J.andOHAMA, G. (2003). Exact calculation of KADDOUM,G.andGAGNON, F. (2012). Design of a high-data- bit error rates in communication systems with chaotic modu- rate chaos-shift system. IEEE Trans. Circuits Syst. II, Express lation. IEEE Trans. Circuits Syst. I, Fundam. Theory Appl. 50 Briefs 59 448–452. 1391–1400. MR2024566 KADDOUM,G.andGAGNON, F. (2013a). Performance analysis LAWRANCE,A.J.andOHAMA, G. (2005). Bit error probability of STBC-CSK communication system over slow fading chan- and bit outage rate in chaos communication. Circuits Systems nel. Signal Process. 93 2055–2060. Signal Process. 24 519–533. MR2187036 KADDOUM,G.andGAGNON, F. (2013b). Lower bound on the LAWRANCE,A.J.,PAPAMARKOU,T.andUCHIDA, A. (2017). BER of a decode-and-forward relay network under chaos shift Synchronized laser choas communication: Statistical investiga- keying communication system. IET Commun. COM-2013- tion of an experimental system. IEEE J. Quantum Electronics 0421.R2 1–16. 53. To appear. KADDOUM,G.,SOUJERI,E.andARCILA, C. A. (2015). I- LAWRANCE,A.J.andYAO, J. (2008). Likelihood-based demodu- DCSK: An improved non-coherent communication system ar- lation in multi-user chaos shift keying communication. Circuits chitecture. IEEE Transactionson Circuits, Systems II: Express Systems Signal Process. 27 847–864. MR2466043 Briefs 1–5. MIRASSO,C.R.,COLET,P.andGARCIA-FERNANDEZ,P. KADDOUM,G.,COULON,M.,ROVIRAS,D.andCHARGE,P. (1996). Synchronization of chaotic semiconductor lasers: Ap- (2010). Theoretical performance for asychronous multi-user plication to encoded communications. IEEE Photonics Technol. chaos-based communication systems on fading channels. Sig- Lett. 8 299–301. nal Processing 90 2923–2933. PAPAMARKOU,T.andLAWRANCE, A. J. (2007). Optimal spread- KENNEDY,M.P.,ROVATTI,R.andSETTI, G., eds. (2000). ing sequences for chaos-based communications. In 2007 Inter- Chaotic Electronics in Telecommunications. CRC Press, Boca national Symposium on Nonlinear Theory and Its Applications Raton. (NOLTA’07). Research Society of Nonlinear Theory and Its Ap- KOHDA,T.andMURAO, K. (1990). Approach to time sereis anal- plications 208–211. IEICE, Vancouver. ysis for one-dimnsionaal chaos based on Frobenius–Perron op- PAPAMARKOU,T.andLAWRANCE, A. J. (2013). Paired Bernoulli erator. Transactions Institute of Electronics, Information and circular spreading: Attaining the BER lower bound in a Communication Engineers of Japam (IEICE), E 73 793–800. CSK setting. Circuits Systems Signal Process. 32 143–166. KOLUMBÁN, G. (2000). Theoretical noise performance of MR3018767 correlator-based chaotic communications schemes. IEEE Trans. PARLITZ,U.andERGEZINGER, S. (1994). Robust communica- Circuits Syst. I, Fundam. Theory Appl. 47 1692–1701. tion based on chaotic spreading sequences. Phys. Lett. A 188 MR1816148 146–150. KOLUMBÁN,G.andKENNEDY, M. P. (2000). Special Issue on PECORA,L.M.andCARROLL, T. L. (1990). Synchronization in “Non-coherent chaotic communications”. IEEE Trans. Circuits chaotic systems. Phys. Rev. Lett. 64 821–824. MR1038263 Syst. I, Fundam. Theory Appl. 47 1661–1662. PROAKIS, J. G. (2001). Digital Communication. McGraw Hill, KOLUMBÁN,G.,VIZVARI,B.,SCHWARZ,W.andABEL,A. Boston, MA. (1996). Differential chaos shift keying: A robust coding for SUSHCHIK,M.,TSIMRING,L.S.andVOLKOVSKII,A.R. chaos communication. In Proc. NDES’96 87–92. Seville, Spain. (2000). Performance analysis of correlation-based communica- KOLUMBÁN,G.,KIS,G.,JAKO,Z.andKENNEDY, M. P. (1998). tion schemes utilizing chaos. IEEE Trans. Circuits Syst. I, Fun- FM-DCSK: A robust modulation scheme for chaos communica- dam. Theory Appl. 47 1684–1691. tion. IEICE Transactions on Fundamentals of Electronics, Com- TAM,W.M.,LAU,F.C.M.andTSE, C. K. (2003). The analysis munications and Computer Sciences E 81-A 1798–1802. of bit error rates for multiple access CSK and DCSK communi- LARSON,L.E.,LIU,J.-M.andTSIMRING, L. S., eds. (2006). cation. IEEE Trans. Circuits Syst. I, Fundam. Theory Appl. 50 Digital Communications Using Chaos and Nonlinear Dynam- 702–707. ics. Springer, New York. TAM,W.M.,LAU,F.C.M.andTSE, C. K. (2007). Digital Com- LASOTA,A.andMACKEY, M. C. (1994). Chaos, Fractals, and munications with Chaos. Elsevier, Oxford. Noise: Stochastic Aspects of Dynamics, 2nd ed. Applied Mathe- TAM,W.M.,LAU,F.C.M.,TSE,C.K.andYIP, M. (2002). An matical Sciences 97. Springer, New York. MR1244104 approach to calculating the bit-error rate of a coherent chaos- LAU,F.C.M.,CHEONG,K.Y.andTSE, C. K. (2003). shift-keying communication system under a noisy multiuser en- Permutation-based DCSK and multiple-access DCSK systems. vironment. IEEE Trans. Circuits Syst. I, Fundam. Theory Appl. IEEE Trans. Circuits Syst. I, Fundam. Theory Appl. 50 733–742. 49 210–233. MR2003801 TAM,W.M.,LAU,F.C.M.,TSE,C.K.andLAWRANCE,A.J. LAU,F.C.M.andTSE, C. K. (2003). Chaos-Based Digital Com- (2004). Exact analytical of bit error rates for multiple ac- munications Systems. Springer, Heidelberg. cess chaos-based communication systems. IEEE Trans. Circuits LAWRANCE,A.J.andBALAKRISHNA, N. (2001). Statistical as- Syst. I 51 473–481. pects of chaotic maps with negative dependence in a communi- UCHIDA, A. (2012). Optical Communications with Lasers— cations setting. J. R. Stat. Soc. Ser. B. Stat. Methodol. 63 843– Applications of Nonlinear Dynamics and Synchronization. 853. MR1872070 Wiley-VCH, Weinheim. cuits and Systems (ISCAS2004) 593–596, Vancouver, Canada. UCHIDA,A.,YOSHIMORI,S.,SHINOZUKA,M.,OGAWA,T.and KANNARI, F. (2001). Chaotic on off keying for secure commu- ZHOU,Z.,WANG,J.andYE, Y. (2010). Exact BER analysis of nications. Opt. Lett. 26 866–868. YAO, J. (2004). Optimal chaos shift keying communications with differential chaos shift keying communication system in fading correlation decoding. In IEEE International Symposium on Cir- channels. Wirel. Pers. Commun. 53 299–310. Statistical Science 2016, Vol. 31, No. 4, 578–590 DOI: 10.1214/16-STS573 © Institute of Mathematical Statistics, 2016 Bayes, Reproducibility and the Quest for Truth D. A. S. Fraser, M. Bédard, A. Wong, Wei Lin and A. M. Fraser

Abstract. We consider the use of default priors in the Bayes methodology for seeking information concerning the true value of a parameter. By default prior, we mean the mathematical prior as initiated by Bayes [Philos. Trans. R. Soc. Lond. 53 (1763) 370–418] and pursued by Laplace [Théorie Analytique des Probabilités (1812) Courcier], Jeffreys [Theory of Probabil- ity (1961) Clarendon Press], Bernardo [J. Roy. Statist. Soc. Ser. B 41 (1979) 113–147] and many more, and then recently viewed as “potentially dan- gerous” [Science 340 (2013) 1177–1178] and “potentially useful” [Science 341 (2013) 1452]. We do not mean, however, the genuine prior [Science 340 (2013) 1177–1178] that has an empirical reference and would invoke stan- dard frequency modelling. And we do not mean the subjective or opinion prior that an individual might have and would be viewed as specific to that individual. A mathematical prior has no referenced frequency information, but on occasion is known otherwise to lead to repetition properties called confidence. We investigate the presence of such supportive property, and ask can Bayes give reliability for other than the particular parameter weightings chosen for the conditional calculation. Thus, does the methodology have reproducibility? Or is it a leap of faith. For sample-space analysis, recent higher-order likelihood methods with regular models show that third-order accuracy is widely available using profile contours [In Past, Present and Future of Statistical Science (2014) 237– 252 CRC Press]. But for parameter-space analysis, accuracy is widely limited to first order. An exception arises with a scalar full parameter and the use of the scalar Jeffreys [J. Roy. Statist. Soc. Ser. B 25 (1963) 318–329]. But for vector full parameter even with a scalar interest parameter, difficulties have long been known [J. Roy. Statist. Soc. Ser. B 35 (1973) 189–233] and with parameter curvature, accuracy beyond first order can be unavailable [Statist. Sci. 26 (2011) 299–316]. We show, however, that calculations on the parameter space can give full second-order information for a chosen scalar interest parameter; these calculations, however, require a Jeffreys prior that is used fully restricted to the one-dimensional profile for that interest parameter. Such a prior is effectively data-dependent and parameter-dependent and is focally restricted to the one-dimensional contour; these priors fall outside the usual Bayes approach and yet with substantial calculations can still give less than frequency analysis.

D. A. S. Fraser is Professor Emeritus, Department of Statistical Sciences, University of Toronto, Toronto, Canada (e-mail: [email protected]). M. Bédard is Associate Professor, Département de mathématiques et de statistique, Université de Montréal, Montréal, Canada (e-mail: [email protected]). A. Wong is Professor, Department of Mathematics and Statistics, York University, Toronto, Canada (e-mail: [email protected]). Wei Lin is Assistant Professor, Department of Statistical Sciences, University of Toronto, Toronto, Canada (e-mail: [email protected]). A. M. Fraser is Professor, Department of Mathematics, University of British Columbia, Vancouver, Canada (e-mail: [email protected]). We provide simple examples using discrete extensions of Jeffreys prior. These serve as counter-examples to general claims that Bayes can offer accuracy for statistical inference. To obtain this accuracy with Bayes, more effort is required compared to recent likelihood methods, which still remain more accurate. And with vector full parameters, accuracy beyond first order is routinely not available, as a change in parameter curvature causes Bayes and frequentist values to change in opposite direction, yet frequentist has full reproducibility. An alternative is to view default Bayes as an exploratory technique and then ask does it do as it overtly claims? Is it reproducible as understood in contemporary science? The posterior gives a distribution for an interest parameter and, thereby, a quantile for the interest parameter; an oracle could record whether it was left or right of the true value. If the average split in evaluative repetitions is in accord with the nominal level, then the approach is providing accuracy. And if not, then what is up, other than performance specific to the parameter frequencies in the prior. No one has answers although speculative claims abound. Key words and phrases: Confidence, curved parameter, exponential model, gamma mean, genuine prior, Jeffreys, L’Aquila, linear parameter, opinion prior, regular model, reproducibility, risks, rotating parameter, two theories, Vioxx, Welch–Peers.

REFERENCES C. Genest, D. L. Banks, G. Molenberghs, D. W. Scott and J.- L. Wang, eds.) 237–252. CRC Press, Boca Raton, FL. BAYES, T. (1763). An essay towards solving a problem in the doc- FRASER, D. A. S. (2016). Definitive testing of an interest param- trine of chances. Philos. Trans. R. Soc. Lond. 53 370–418. eter: Using parameter continuity. J. Statist. Res. 47 193–169. BERNARDO, J.-M. (1979). Reference posterior distributions for FRASER,A.M.,FRASER,D.A.S.andSTAICU, A.-M. (2010). Bayesian inference. J. Roy. Statist. Soc. Ser. B 41 113–147. Second order ancillary: A differential view from continuity. MR0547240 Bernoulli 16 1208–1223. MR2759176 BRAZZALE,A.R.,DAV I S O N ,A.C.andREID, N. (2007). Applied FRASER,D.A.S.andREID, N. (1995). Ancillaries and third or- Asymptotics: Case Studies in Small-Sample Statistics. Cam- der significance. Util. Math. 47 33–53. MR1330888 bridge Series in Statistical and Probabilistic Mathematics 23. FRASER,D.A.S.andREID, N. (2002). Strong matching of fre- Cambridge Univ. Press, Cambridge. MR2342742 quentist and Bayesian parametric inference. J. Statist. Plann. CAKMAK,S.,FRASER,D.A.S.,MCDUNNOUGH,P.,REID,N. Inference 103 263–285. MR1896996 and YUAN, X. (1998). Likelihood centered asymptotic model exponential and location model versions. J. Statist. Plann. In- FRASER,D.A.S.,REID,N.,MARRAS,E.andYI, G. Y. (2010). ference 66 211–222. MR1614476 Default priors for Bayesian and frequentist inference. J. R. Stat. Soc Ser BStat Methodol 72 DANIELS, H. E. (1954). Saddlepoint approximations in statistics. . . . . 631–654. MR2758239 Ann. Math. Statist. 25 631–650. MR0066602 GHOSH, M. (2011). Objective priors: An introduction for frequen- DAWID,A.P.,STONE,M.andZIDEK, J. V. (1973). Marginal- tists. Statist. Sci. 26 187–202. MR2858380 ization paradoxes in Bayesian and structural inference. J. Roy. JEFFREYS, H. (1946). An invariant form for the prior probability in Statist. Soc. Ser. B 35 189–233. MR0365805 estimation problems. Proc. Roy. Soc. London, Ser. A. 186 453– EFRON, B. (2013). Bayes’ theorem in the 21st century. Science 340 461. MR0017504 1177–1178. MR3087705 JEFFREYS, H. (1961). Theory of Probability, 3rd ed. Clarendon FISHER, R. (1930). Inverse probability. Proc. Camb. Philos. Soc. Press, Oxford. MR0187257 26 528–535. LAPLACE, P. S. (1812). Théorie Analytique des Probabilités. FISHER, R. A. (1956). Statistical Methods and Scientific Inference. Courcier, Paris. Oliver and Boyd, Edinburgh. MCNUTT, M. (2014). Reproducibility. Science 343 229. FRASER, D. A. S. (2003). Likelihood for component parameters. MCNUTT, M. (2015). Editorial retraction. Science 348 1100. Biometrika 90 327–339. MR1986650 NEYMAN, J. (1937). Outline of a theory of statistical estimation FRASER, D. A. S. (2011). Is Bayes posterior just quick and dirty based on the classical theory of probability. Phil. Trans. Roy. confidence? Statist. Sci. 26 299–316. MR2918001 Soc. A 237 333–380. FRASER, D. A. S. (2013). Bayes’ confidence. Science 341 1452. REID,N.andFRASER, D. A. S. (2010). Mean loglikelihood FRASER, D. A. S. (2014). Why does statistics have two theo- and higher-order approximations. Biometrika 97 159–170. ries? In Past, Present and Future of Statistical Science (X. Lin, MR2594424 Statist. Soc. Ser. B 25 318–329. MR0173309 SAVAG E , L. J. (1972). The Foundations of Statistics, revised ed. Dover Publications, New York. MR0348870 YANG,R.andBERGER, J. O. (1996). A catalog of noninformative WELCH,B.L.andPEERS, H. W. (1963). On formulae for confi- priors. Institute of Statistics and Decision Sciences, Duke Univ., dence points based on integrals of weighted likelihoods. J. Roy. Durham, DC. Statistical Science 2016, Vol. 31, No. 4, 591–610 DOI: 10.1214/16-STS580 © Institute of Mathematical Statistics, 2016 A Review and Comparison of Age–Period–Cohort Models for Cancer Incidence Theresa R. Smith and Jon Wakefield

Abstract. Age–period–cohort models have been used to examine and forecast cancer incidence and mortality for over three decades. However, the fitting and interpretation of these models requires great care because of the well-known identifiability problem that exists; given any two of age, period, and cohort, the third is determined. In this paper, we review the identifiability problem and models that have been proposed for analysis, from both frequentist and Bayesian standpoints. A number of recent analyses that use age–period–cohort models are described and critiqued before data on cancer incidence in Washington State are analyzed with various models, including a new Bayesian approach based on an identifiable parameterization. Key words and phrases: Age–period–cohort models, identifiability, random walk priors.

REFERENCES (2015). The end of the decline in cervical cancer mortality in Spain: Trends across the period 1981–2012. BMC Cancer 15 AHMAD,A.S.,ORMISTON-SMITH,N.andSASIENI,P.D. 287. (2015). Trends in the lifetime risk of developing cancer in Great CLAYTON,D.andSCHIFFLERS, E. (1987a). Models for temporal Britain: Comparison of risk for those born from 1930 to 1960. variation in cancer rates. I: Age–period and age–cohort models. Br. J. Cancer 112 943–947. Stat. Med. 6 449–467. ANANTH,C.V.,KEYES,K.M.,HAMILTON,A.,GISSLER,M., CLAYTON,D.andSCHIFFLERS, E. (1987b). Models for temporal WU,C.,LIU,S.,LUQUE-FERNANDEZ,M.A.,SKJÆR- variation in cancer rates. II: Age–period–cohort models. Stat. VEN,R.,WILLIAMS,M.A.andTIKKANEN, M. (2014). An Med. 6 469–481. international contrast of rates of placental abruption: An age– ESCEDY,J.andHUNTER, D. (2008). The origin of cancer. In Text- period–cohort analysis. PLoS ONE 10 e0125246. book of Cancer Epidemiology, 2nd ed. (H. O. Adami, D. Hunter BERZUINI,C.andCLAYTON, D. (1994). Bayesian analysis of sur- and D. Trichopoulos, eds.). Oxford Univ. Press, Oxford. vival on multiple time scales. Stat. Med. 13 823–838. FERKINGSTAD,E.andRUE, H. (2015). Improving the INLA ap- BESAG,J.,GREEN,P.,HIGDON,D.andMENGERSEN, K. (1995). proach for approximate Bayesian inference for latent Gaussian Bayesian computation and stochastic systems. Statist. Sci. 10 3– models. Electron. J. Stat. 9 2706–2731. MR3433587 66. MR1349818 FIENBERG,S.E.andMASON, W. M. (1979). Identiﬁcation and BROWN,C.C.andKESSLER, L. G. (1988). Projections of lung estimation of age–period–cohort models in the analysis of dis- cancer mortality in the United States: 1985–2025. J. Natl. Can- crete archival data. Sociol. Method. 10 1–67. cer Inst. 80 43–51. FONG,Y.,RUE,H.andWAKEFIELD, J. C. (2010). Bayesian infer- CARSTENSEN, B. (2007). Age–period–cohort models for the Lexis ence for generalized linear mixed models. Biostatistics 11 397– diagram. Stat. Med. 26 3018–3045. MR2370989 412. CARSTENSEN,B.,PLUMMER,M.,LAARA,E.andHILLS,M. FROST, W. H. (1939). The age selection of mortality from tu- (2014). Epi: A package for statistical analysis in epidemiology. berculosis in successive decades. American Journal of Hygiene Available at http://CRAN.R-project.org/package=Epi, R pack- 30 91–96. Reprinted in American Journal of Epidemiology 141 age version 1.1.67. (1995) 4–9. CERVANTES-AMAT,M.,LÓPEZ-ABENTE,G.,ARAGONÉS,N., GHOSH,K.,TIWARI,R.C.,FEUER,E.J.,CRONIN,K.andJE- POLLÁN,M.,PASTOR-BARRIUSO,R.andPÉREZ-GÓMEZ,B. MAL, A. (2007). Predicting US cancer mortality counts using

Theresa R. Smith is Senior Research Associate, Lancaster Medical School, Lancaster University, Lancaster LA1 4YG, United Kingdom (e-mail: [email protected]). Jon Wakefield is Professor, Departments of Statistics and Biostatistics, University of Washington, Seattle, Washington 98195-4322, USA (e-mail: [email protected]). state space models. In Computational Methods in Biomedical MØLLER,B.,FEKJÆR,H.,HAKULINEN,T.,SIGVALDASON,H., Research (R. Khattree and D. N. Naik, eds.) 131–151. STORM,H.H.,TALBÄCK,M.andHALDORSEN, T. (2003). GREENBERG,B.G.,WRIGHT,J.J.andSHEPS, C. G. (1950). Prediction of cancer incidence in the Nordic countries: Empir- A technique for analyzing some factors affecting the incidence ical comparison of different approaches. Stat. Med. 22 2751– of syphilis. J. Amer. Statist. Assoc. 45 373–399. 2766. HEUER, C. (1997). Modeling of time trends and interactions in vi- NIELSEN, B. (2014). apc: A package for age–period–cohort tal rates using restricted regression splines. Biometrics 53 161– analysis. Available at http://CRAN.R-project.org/package=apc, 177. R package version 1.0. HO,M.-L.,HSIAO,Y.-H.,SU,S.-Y.,CHOU,M.-C.and NIELSEN,B.andNIELSEN, J. P. (2014). Identification and fore- LIAW, Y.-P. (2015). Mortality of breast cancer in Taiwan, casting in mortality models. Sci. World J. 2014 347043. 1971–2010: Temporal changes and an age–period–cohort anal- O’BRIEN, R. M. (2000). Age period cohort characteristic models. ysis. Journal of Obstetrics & Gynaecology 35 60–63. Soc. Sci. Res. 29 123–139. O’BRIEN, R. (2014). Age–Period–Cohort Models: Approaches HOLFORD, T. R. (1983). The estimation of age, period and cohort and Analyses with Aggregate Data. Chapman & Hall/CRC effects for vital rates. Biometrics 39 311–324. MR0714415 Press, London. HOLFORD, T. R. (1991). Understanding the effects of age, period, OSMOND,C.andGARDNER, M. J. (1982). Age, period and cohort and cohort on incidence and mortality rates. Annu. Rev. Public models applied to cancer mortality rates. Stat. Med. 1 245–259. Health 12 425–457. PAPOILA,A.L.,RIEBLER,A.,AMARAL-TURKMAN,A.,SÃO- HOLFORD, T. R. (2006). Approaches to fitting age–period– JOÃO,R.,RIBEIRO,C.,GERALDES,C.andMIRANDA,A. cohort models with unequal intervals. Stat. Med. 25 977–993. (2014). Stomach cancer incidence in Southern Portugal 1998– MR2225187 2006: A spatio-temporal analysis. Biom. J. 56 403–415. KEIDING, N. (1990). Statistical inference in the Lexis diagram. MR3256497 Philos. Trans. R. Soc. Lond. Ser. AMath. Phys. Eng. Sci. 332 RENSHAW,A.E.andHABERMAN, S. (2006). A cohort-based ex- 487–509. MR1084720 tension to the Lee–Carter model for mortality reduction factors. KEIDING, N. (2011). Age–period–cohort analysis in the 1870s: Di- Insurance Math. Econom. 38 556–570. agrams, stereograms, and the basic differential equation. Canad. RIEBLER,A.andHELD, L. (2010). The analysis of heteroge- J. Statist. 39 405–420. MR2842421 neous time trends in multivariate age–period–cohort models. KNORR-HELD,L.andRAINER, E. (2001). Projections of lung Biostatistics 11 57–69. cancer mortality in West Germany: A case study in Bayesian RIEBLER,A.andHELD, L. (2016). Projecting the future burden prediction. Biostatistics 2 109–129. of cancer: Bayesian age–period–cohort analysis with integrated KUANG,D.,NIELSEN,B.andNIELSEN, J. P. (2008a). Forecast- nested Laplace approximations. Biom. J. To appear. ing with the age–period–cohort model and the extended chain- RIEBLER,A.,HELD,L.andRUE, H. (2012). Estimation ladder model. Biometrika 95 987–991. MR2461225 and extrapolation of time trends in registry data—borrowing KUANG,D.,NIELSEN,B.andNIELSEN, J. P. (2008b). Identifi- strength from related populations. Ann. Appl. Stat. 6 304–333. cation of the age–period–cohort model and the extended chain- MR2951539 ladder model. Biometrika 95 979–986. MR2461224 RIEBLER,A.,HELD,L.,RUE,H.andBOPP, M. (2012). Gender- LEE,R.D.andCARTER, L. R. (1992). Modeling and forecasting specific differences and the impact of family integration on time US mortality. J. Amer. Statist. Assoc. 87 659–671. trends in age-stratified Swiss suicide rates. J. Roy. Statist. Soc. LEXIS, W. H. R. A. (1875). Einleitung in die Theorie der Ser. A 175 473–490. MR2905049 Bevölkerungsstatistik. KJ Trübner. ROBERTS,G.O.andTWEEDIE, R. L. (1996). Exponential convergence of Langevin distributions and their discrete approxi- LINDGREN,F.andRUE, H. (2008). On the second-order random walk model for irregular locations. Scand. J. Stat. 35 691–700. mations. Bernoulli 2 341–363. MR1440273 ROBERTSON,C.andBOYLE, P. (1986). Age, period and cohort MR2468870 models: The use of individual records. Stat. Med. 5 527–538. LOUIE,K.S.,MEHANNA,H.andSASIENI, P. (2015). Trends in RODGERS, W. L. (1982). Estimable functions of age, period, and head and neck cancers in England from 1995 to 2011 and pro- cohort effects. Am. Sociol. Rev. 47 774–787. jections up to 2025. Oral Oncology 51 341–348. ROSENBERG,P.S.andANDERSON, W. F. (2011). Age–period– LUNN,D.J.,THOMAS,A.,BEST,N.andSPIEGELHALTER,D. cohort models in cancer surveillance research: Ready for prime (2000). WinBUGS—a Bayesian modelling framework: Con- time? Cancer Epidemiol. Biomark. Prev. 20 1263–1268. cepts, structure, and extensibility. Stat. Comput. 10 325–337. ROSENBERG,P.S.,CHECK,D.P.andANDERSON, W. F. (2014). LUO, L. (2013). Assessing validity and application scope of the A web tool for age–period–cohort analysis of cancer incidence intrinsic estimator approach to the age–period–cohort problem. and mortality rates. Cancer Epidemiol. Biomark. Prev. 23 2296– Demography 50 1945–1967. 2302. MARTÍNEZ MIRANDA,M.D.,NIELSEN,B.andNIELSEN,J.P. RUE,H.andHELD, L. (2005). Gaussian Markov Random Fields: (2015). Inference and forecasting in the age–period–cohort Theory and Applications. Monographs on Statistics and Ap- model with unknown exposure with an application to mesothe- plied Probability 104. Chapman & Hall/CRC, Boca Raton, FL. lioma mortality. J. Roy. Statist. Soc. Ser. A 178 29–55. MR2130347 MR3291760 RUE,H.,MARTINO,S.andCHOPIN, N. (2009). Approximate MASON,K.O.,MASON,W.M.,WINSBOROUGH,H.H.and Bayesian inference for latent Gaussian models by using inte- POOLE, W. K. (1973). Some methodological issues in cohort grated nested Laplace approximations. J. R. Stat. Soc. Ser. B. analysis of archival data. Am. Sociol. Rev. 38 242–258. Stat. Methodol. 71 319–392. MR2649602 RUTHERFORD,M.J.,LAMBERT,P.C.andTHOMPSON,J.R. screening in the Nordic countries: Quantifying the effects on (2010). Age–period–cohort modeling. Stata J. 10 606–627. cervical cancer incidence. Br. J. Cancer 111 965–969. SASIENI, P. D. (2012). Age–period–cohort models in Stata. Stata VALERY,P.C.,LAVERSANNE,M.andBRAY, F. (2015). Bone J. 12 45–60. cancer incidence by morphological subtype: A global assess- SCHMID,V.J.andHELD, L. (2007). Bayesian age–period–cohort ment. Cancer Causes Control 26 1127–1139. modeling and prediction—BAMP. J. Stat. Softw. 21 1–15. YANG,Y.,FU,W.J.andLAND, K. C. (2004). A methodological SEOANE-MATO,D.,ARAGONÉS,N.,FERRERAS,E., comparison of age–period–cohort models: The intrinsic estima- GARCÍA-PÉREZ,J.,CERVANTES-AMAT,M.,FERNÁNDEZ- tor and conventional generalized linear models. Sociol. Method. NAVA R RO ,P.,PASTOR-BARRIUSO,R.andLÓPEZ- 34 75–110. ABENTE, G. (2014). Trends in oral cavity, pharyngeal, YANG,Y.andLAND, K. C. (2013). Age–Period–Cohort Analysis: oesophageal and gastric cancer mortality rates in Spain, New Models, Methods, and Empirical Applications. CRC Press, 1952–2006: An age–period–cohort analysis. BMC Cancer 14 Boca Raton, FL. MR3026791 254. ZHENG,R.,PENG,X.,ZENG,H.,ZHANG,S.,CHEN,T., SMITH,T.R.andWAKEFIELD, J. (2016). Supplement to “A re- WANG,H.andCHEN, W. (2015). Incidence, mortality and sur- view and comparison of age–period–cohort models for cancer vival of childhood cancer in China during 2000–2010 period: incidence.” DOI:10.1214/16-STS580SUPPA, DOI:10.1214/16- A population-based study. Cancer Letters 363 176–180. STS580SUPPB. ZHU,L.,PICKLE,L.W.,GHOSH,K.,NAISHADHAM,D., SPRINGETT, V. H. (1950). A comparative study of tuberculosis PORTIER,K.,CHEN,H.,KIM,H.,ZOU,J.,CUCINELLI,Z., mortality rates. J. Hyg. 48 361–395. KOHLER,B.,EDWARDS,B.K.,KING,J.,FEUER,E.J.and TZENG,I.S.andLEE, W. C. (2015). Forecasting hepatocellu- JEMAL, A. (2012). Predicting US- and state-level cancer counts lar carcinoma mortality in Taiwan using an age–period–cohort for the current calendar year. Cancer 118 1100–1109. model. Asia-Pacific Journal of Public Health 27 NP65–NP73. ZHU,L.,PICKLE,L.W.,ZOU,Z.andCUCINELLI, J. (2014). VACCARELLA,S.,FRANCESCHI,S.,ENGHOLM,G., Trends and patterns of childhood cancer incidence in the United LÖNNBERG,S.,KHAN,S.andBRAY, F. (2014). 50 years of States, 1995–2010. Stat. Interface 7 121–134. MR3197576 Statistical Science 2016, Vol. 31, No. 4, 611–623 DOI: 10.1214/16-STS587 © Institute of Mathematical Statistics, 2016 Multiple Change-Point Detection: A Selective Overview Yue S. Niu, Ning Hao and Heping Zhang

Abstract. Very long and noisy sequence data arise from biological sciences to social science including high throughput data in genomics and stock prices in econometrics. Often such data are collected in order to identify and understand shifts in trends, for example, from a bull market to a bear market in ﬁnance or from a normal number of chromosome copies to an excessive number of chromosome copies in genetics. Thus, identifying multiple change points in a long, possibly very long, sequence is an important problem. In this article, we review both classical and new multiple change-point detection strategies. Considering the long history and the extensive literature on the change-point detection, we provide an in-depth discussion on a normal mean change-point model from aspects of regression analysis, hypothesis testing, consistency and inference. In particular, we present a strategy to gather and aggregate local information for change-point detection that has become the cornerstone of several emerging methods because of its attractiveness in both computational and theoretical properties. Key words and phrases: Binary segmentation, consistency, multiple testing, normal mean change-point model, regression, screening and ranking algorithm.

REFERENCES CARLIN,B.P.,GELFAND,A.E.andSMITH, A. F. (1992). Hier- archical Bayesian analysis of changepoint problems. Appl. Stat. ARIAS-CASTRO,E.,DONOHO,D.L.andHUO, X. (2005). Near- 41 389–405. optimal detection of geometric objects by fast multiscale meth- CARLSTEIN,E.,MÜLLER,H.-G.andSIEGMUND,D.,eds. ods. IEEE Trans. Inform. Theory 51 2402–2425. MR2246369 (1994). Change-Point Problems. Institute of Mathematical BENJAMINI,Y.andHOCHBERG, Y. (1995). Controlling the false Statistics Lecture Notes—Monograph Series 23.IMS,Hayward, discovery rate: A practical and powerful approach to multiple CA. MR1477909 testing. J. Roy. Statist. Soc. Ser. B 57 289–300. MR1325392 CHEN,J.andGUPTA, A. K. (2000). Parametric Statistical Change BRAUN,J.V.,BRAUN,R.K.andMÜLLER, H.-G. (2000). Point Analysis. Birkhäuser, Inc., Boston, MA. MR1761850 Multiple changepoint ﬁtting via quasilikelihood, with applica- COBB, G. W. (1978). The problem of the Nile: Conditional tion to DNA sequence segmentation. Biometrika 87 301–314. solution to a changepoint problem. Biometrika 65 243–251. MR1782480 MR0513930 BRODSKY,E.andDARKHOVSKY, B. S. (1993). Nonparamet- CSÖRGO˝ ,M.andHORVÁTH, L. (1997). Limit Theorems in ric Methods in Change Point Problems. Kluwer, Dordrecht. Change-Point Analysis. Wiley Series in Probability and Statis- MR1228205 tics. Wiley, Chichester. MR2743035 BRODSKY,B.E.andDARKHOVSKY, B. S. (2000). Non- EFRON,B.,HASTIE,T.,JOHNSTONE,I.andTIBSHIRANI,R. parametric Statistical Diagnosis: Problems and Methods. Math- (2004). Least angle regression. Ann. Statist. 32 407–499. ematics and Its Applications 509. Kluwer Academic, Dordrecht. MR2060166 MR1862475 FRICK,K.,MUNK,A.andSIELING, H. (2014). Multiscale change

Yue S. Niu is Assistant Professor and Ning Hao is Assistant Professor, Department of Mathematics, University of Arizona, 617 N. Santa Rita Ave., Tucson, Arizona 85721, USA (e-mail: [email protected]; [email protected]). Heping Zhang is Susan Dwight Bliss Professor of Biostatistics, Yale School of Public Health, 60 College Street, New Haven, Connecticut 06520-8034, USA (e-mail: [email protected]). point inference. J. R. Stat. Soc. Ser. B. Stat. Methodol. 76 495– PEIN,F.,SIELING,H.andMUNK, A. (2015). Hetero- 580. MR3210728 geneuous change point inference. Preprint. Available at FRIEDMAN,J.,HASTIE,T.,HÖFLING,H.andTIBSHIRANI,R. arXiv:1505.04898. (2007). Pathwise coordinate optimization. Ann. Appl. Stat. 1 QIAN,J.andJIA, J. (2012). On pattern recovery of the fused 302–332. MR2415737 Lasso. Preprint. Available at arXiv:1211.5194. FRYZLEWICZ, P. (2014). Wild binary segmentation for mul- RINALDO, A. (2009). Properties and refinements of the fused tiple change-point detection. Ann. Statist. 42 2243–2281. lasso. Ann. Statist. 37 2922–2952. MR2541451 MR3269979 SCOTT,A.J.andKNOTT, M. (1974). A cluster analysis method for grouping means in the analysis of variance. Biometrics 30 HAO,N.,NIU,Y.S.andZHANG, H. (2013). Multiple change- point detection via a screening and ranking algorithm. Statist. 507–512. SEN,A.andSR I VA S TAVA , M. S. (1975). On tests for detecting Sinica 23 1553–1572. MR3222810 change in mean. Ann. Statist. 3 98–108. MR0362649 HARCHAOUI,Z.andLÉVY-LEDUC, C. (2010). Multiple change- SHIN,S.J.,WU,Y.andHAO, N. (2014). A backward procedure point estimation with a total variation penalty. J. Amer. Statist. for change-point detection with applications to copy number Assoc. 105 1480–1493. MR2796565 variation detection. Unpublished manuscript. HAWKINS, D. M. (1977). Testing a sequence of observations SIEGMUND, D. (1988). Confidence sets in change-point problems. for a shift in location. J. Amer. Statist. Assoc. 72 180–186. Int. Stat. Rev. 56 31–48. MR0963139 MR0451496 TIBSHIRANI,R.andWANG, P. (2008). Spatial smoothing and hot HINKLEY, D. V. (1970). Inference about the change-point in a se- spot detection for CGH data using the fused lasso. Biostat. 9 quence of random variables. Biometrika 57 1–17. MR0273727 18–29. HUANG,T.,WU,B.,LIZARDI,P.andZHAO, H. (2005). Detection TIBSHIRANI,R.,SAUNDERS,M.,ROSSET,S.,ZHU,J.and of DNA copy number alterations using penalized least squares KNIGHT, K. (2005). Sparsity and smoothness via the fused regression. Bioinform. 21 3811–3817. lasso. J. R. Stat. Soc. Ser. B. Stat. Methodol. 67 91–108. JACKSON,B.,SCARGLE,J.D.,BARNES,D.,ARABHI,S., MR2136641 ALT,A.,GIOUMOUSIS,P.,GWIN,E.,SANG- VENKATRAMAN,E.S.andOLSHEN, A. B. (2007). A faster circu- TRAKULCHAROEN,P.,TAN,L.andTSAI, T. T. (2005). lar binary segmentation algorithm for the analysis of array CGH An algorithm for optimal partitioning of data on an interval. data. Bioinform. 23 657–663. IEEE Signal Process. Lett. 12 105–108. VOSTRIKOVA, L. Y. (1981). Detecting “disorder” in multidimen- KILLICK,R.,FEARNHEAD,P.andECKLEY, I. A. (2012). Opti- sional random processes. Sov. Math., Dokl. 24 55–59. mal detection of changepoints with a linear computational cost. WORSLEY, K. J. (1986). Confidence regions and test for a change- J. Amer. Statist. Assoc. 107 1590–1598. MR3036418 point in a sequence of exponential family random variables. LEE, T.-S. (2010). Change-point problems: Bibliography and re- Biometrika 73 91–104. MR0836437 view. J. Stat. Theory Pract. 4 643–662. MR2758751 YAO, Y.-C. (1988). Estimating the number of change-points via Schwarz’ criterion. Statist. Probab. Lett. 6 181–189. LEVIN,B.andKLINE, J. (1985). The cusum test of homogeneity MR0919373 with an application in spontaneous abortion epidemiology. Stat. YAO, Q. W. (1993). Tests for change-points with epidemic alterna- Med. 4 469–488. tives. Biometrika 80 179–191. MR1225223 NIU,Y.S.andZHANG, H. (2012). The screening and ranking al- YAO,Y.-C.andAU, S. T. (1989). Least-squares estimation of a gorithm to detect DNA copy number variations. Ann. Appl. Stat. step function. Sankhya, Ser. A 51 370–381. MR1175613 6 1306–1326. MR3012531 YIN, Y. Q. (1988). Detection of the number, locations and mag- OLSHEN,A.B.,VENKATRAMAN,E.S.,LUCITO,R.and nitudes of jumps. Commun. Stat., Stoch. Models 4 445–455. WIGLER, M. (2004). Circular binary segmentation for the anal- MR0971600 ysis of array-based DNA copy number data. Biostat. 5 557–572. ZHANG,N.R.andSIEGMUND, D. O. (2007). A modified Bayes PAGE, E. S. (1954). Continuous inspection schemes. Biometrika information criterion with applications to the analysis of com- 41 100–115. MR0088850 parative genomic hybridization data. Biometrics 63 22–32. PAGE, E. S. (1955). A test for a change in a parameter occurring at MR2345571 an unknown point. Biometrika 42 523–527. MR0072412 ZHANG,Z.,LANGE,K.,OPHOFF,R.andSABATTI, C. (2010). PAGE, E. S. (1957). On problems in which a change in a parameter Reconstructing DNA copy number by penalized estimation and occurs at an unknown point. Biometrika 44 248–252. imputation. Ann. Appl. Stat. 4 1749–1773. MR2829935 Statistical Science 2016, Vol. 31, No. 4, 624–636 DOI: 10.1214/16-STS574 © Institute of Mathematical Statistics, 2016 A Conversation with Jeff Wu Hugh A. Chipman and V. Roshan Joseph

Abstract. Chien-Fu Jeff Wu was born January 15, 1949, in Taiwan. He earned a B.Sc. in Mathematics from National Taiwan University in 1971, and a Ph.D. in Statistics from the University of California, Berkeley in 1976. He has been a faculty member at the University of Wisconsin, Madison (1977– 1988), the University of Waterloo (1988–1993), the University of Michigan (1995–2003; department chair 1995–8) and currently is the Coca-Cola Chair in Engineering Statistics and Professor in the H. Milton Stewart School of In- dustrial and Systems Engineering at the Georgia Institute of Technology. He is known for his work on the convergence of the EM algorithm, resampling methods, nonlinear least squares, sensitivity testing and industrial statistics, including design of experiments, robust parameter design and computer experiments, and has been credited for coining the term “data science” as early as 1997. Jeff has received several awards, including the COPSS Presidents’ Award (1987), the Shewhart Medal (2008), the R. A. Fisher Lectureship (2011) and the Deming Lecturer Award (2012). He is an elected member of Academia Sinica (2000) and the National Academy of Engineering (2004), and has received many other awards and honors including an honorary doctorate from the University of Waterloo. Jeff has supervised 45 Ph.D. students to date, many of whom are active researchers in the statistical sciences. He has published more than 170 peer- reviewed articles and two books. He was the second Editor of Statistica Sinica (1993–96). Jeff married Susan Chang in 1979, and they have two chil- dren, Emily and Justin. This conversation took place in Atlanta, Georgia, on April 21, 2015. Key words and phrases: Industrial statistics, data science, experimental design, computer experiment, EM algorithm, resampling methods.

REFERENCES HAMADA,M.S.andWU, C. F. J. (1992). Analysis of designed experiments with complex aliasing. J. Qual. Technol. 24 130– BOX,G.E.P.,HUNTER,W.G.andHUNTER, J. S. (1978). Statis- 137. tics for Experimenters: An Introduction to Design, Data Analy- sis, and Model Building. Wiley, New York. MR0483116 HECKMAN, J. L. (1999). Causal parameters and policy analysis in economics: A twentieth century retrospective. National Bureau COCHRAN,W.G.andCOX, G. M. (1957). Experimental Designs, 2nd ed. Wiley, New York. MR0085682 of Economic Research, No. 7333. DEMPSTER,A.P.,LAIRD,N.M.andRUBIN, D. B. (1977). Max- IMBENS,G.W.andRUBIN, D. B. (2015). Causal Inference— imum likelihood from incomplete data via the EM algorithm. J. for Statistics, Social, and Biomedical Sciences: An Introduction. R. Stat. Soc. Ser. B. Stat. Methodol. 39 1–38. MR0501537 Cambridge Univ. Press, New York. MR3309951 FERGUSON, T. S. (1967). Mathematical Statistics: A Decision JOSEPH,V.R.,GUL,E.andBA, S. (2015). Maximum projec- Theoretic Approach. Probability and Mathematical Statistics 1. tion designs for computer experiments. Biometrika 102 371– Academic Press, New York. MR0215390 380. MR3371010

Hugh A. Chipman is Professor, Department of Mathematics and Statistics, Acadia University, Wolfville, Nova Scotia, B4P 2R6, Canada (e-mail: [email protected]). V. Roshan Joseph is Professor, Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332, USA (e-mail: [email protected]). KIEFER,J.andWOLFOWITZ, J. (1960). The equivalence of two of large numbers. Bull. Inst. Math. Acad. Sin.(N.S.) 1 121–124. extremum problems. Canad. J. Math. 12 363–366. MR0117842 MR0322935 PEARL, J. (2009). Causality: Models, Reasoning, and Inference, WU, C.-F. J. (1983). On the convergence properties of the EM al- 2nd ed. Cambridge Univ. Press, Cambridge. MR2548166 gorithm. Ann. Statist. 11 95–103. MR0684867 QIAN, P. Z. G. (2009). Nested Latin hypercube designs. WU,C.F.J.andHAMADA, M. S. (2009). Experiments: Planning, Biometrika 96 957–970. MR2767281 Analysis, and Optimization, 2nd ed. Wiley, Hoboken, NJ. (First QIAN, P. Z. G. (2012). Sliced Latin hypercube designs. J. Amer. Edition, 2000). MR2583259 Statist. Assoc. 107 393–399. MR2949368 ZANGWILL, W. I. (1969). Nonlinear Programming: A Uniﬁed Ap- WU, C. F. (1973). A note on the convergence rate of the strong law proach. Prentice-Hall, Englewood Cliffs, NJ. MR0359816 Statistical Science 2016, Vol. 31, No. 4, 637–647 DOI: 10.1214/16-STS579 © Institute of Mathematical Statistics, 2016 A Conversation with Samad Hedayat Ryan Martin, John Stufken and Min Yang

Abstract. A. Samad Hedayat was born on July 11, 1937, in Jahrom, Iran. He ﬁnished his undergraduate education in bioengineering with honors from the University of Tehran in 1962 and came to the U.S. to study statistics at Cor- nell, completing his Ph.D. in 1969. Just a few years later, in 1974, Samad accepted a full professor position at the University of Illinois at Chicago Circle—now called University of Illinois at Chicago (UIC)—and was named UIC Distinguished Professor in 2003. He was an early leader in the Depart- ment of Mathematics, Statistics and Computer Science and he remains a driv- ing force to this day. Samad has also made substantial contributions in terms of research and service to the ﬁeld, as evidenced by his numerous honors: he is an elected member of the International Statistical Institute, a fellow of the Institute of Mathematical Statistics and the American Statistical Associ- ation and an honorary member of the Iranian Mathematical Society, among others. This conversation, which was conducted in September 2015 and May 2016, touches on Professor Hedayat’s career, and the past, present and future of statistics. In keeping with one of his great passions, it also offers an abundance of advice for students and junior faculty. Key words and phrases: Design of experiments, survey sampling, biostatistics, statistical research, statistical consulting, advising students.

REFERENCES HEDAYAT,A.andSEIDEN, E. (1970). F -square and orthogonal F -squares design: A generalization of Latin square and or- BOSE,R.C.,SHRIKHANDE,S.S.andPARKER, E. T. (1960). thogonal Latin squares design. Ann. Math. Stat. 41 2035–2044. Further results on the construction of mutually orthogonal Latin MR0267702 squares and the falsity of Euler’s conjecture. Canad. J. Math. 12 HEDAYAT,A.andSEIDEN, E. (1974). On the theory and applica- 189–203. MR0122729 tion of sum composition of Latin squares and orthogonal Latin EULER, L. (1782). Recherches sur une nouvelle espèce de quar- squares. Paciﬁc J. Math. 54 85–113. MR0373925 rés magiques. Zeeuwsch Genootschap der Wetenschappen 9 85– HEDAYAT,A.S.andSINHA, B. K. (1991). Design and Inference 239. in Finite Population Sampling. Wiley, New York. MR1253978 FEDERER, W. T. (1955). Experimental Design: Theory and Appli- HEDAYAT,A.S.andSINHA, B. K. (2003). On a sampling design cation. Macmillan, New York. for estimation of negligible accident rates involving electronic HEDAYAT,A.S.,IZENMAN,A.J.andZHANG, W. G. (1996). toys. Amer. Statist. 57 249–252. Corrigendum: Ibid 59 (2005) Random sampling for the forensic study of controlled sub- 280. MR2016259 stances (with discussion). In Proceedings of the Physical and HEDAYAT,A.S.,SLOANE,N.J.A.andSTUFKEN, J. (1999). Or- Engineering Sciences Section 12–23. Amer. Statisc. Assoc., thogonal Arrays: Theory and Applications. Springer, New York. Alexandria, VA. With a foreword by C. R. Rao. MR1693498 HEDAYAT,A.S.andMAJUMDAR, D. (1985). Families of A- HEDAYAT,S.andSTUFKEN, J. (2009). Comment on “What optimal block designs for comparing test treatments with a con- is statistics?” [MR2750071]. Amer. Statist. 63 115–116. trol. Ann. Statist. 13 757–767. MR0790570 MR2759685

Ryan Martin is Associate Professor, Department of Statistics, North Carolina State University, 2311 Stinson Drive, Campus Box 8203, Raleigh, North Carolina 27695, USA (e-mail: [email protected]). John Stufken is Charles Wexler Professor of Statistics, School of Mathematical and Statistical Sciences, Arizona State University, P.O. Box 871804, Tempe, Arizona 85287-1804, USA (e-mail: [email protected]). Min Yang is Professor, Department of Mathematics, Statistics, and Computer Science, University of Illinois at Chicago, 322 Science and Engineering Ofﬁces (M/C 249), 851 S. Morgan Street, Chicago, Illinois 60607-7045, USA (e-mail: [email protected]).