Imprecise Dirichlet process with application to the hypothesis test on the probability that X Y ≤ Alessio Benavolia, Francesca Mangilia, Fabrizio Ruggerib, Marco Zaffalona a IPG IDSIA, Manno, Switzerland b CNR IMATI, Milano, Italy Abstract The Dirichlet process (DP) is one of the most popular Bayesian nonparametric models. An open problem with the DP is how to choose its infinite-dimensional parameter (base measure) in case of lack of prior information. In this work we present the Imprecise DP (IDP)—a prior near-ignorance DP-based model that does not require any choice of this probability measure. It consists of a class of DPs obtained by letting the normalized base measure of the DP vary in the set of all probability measures. We discuss the tight connections of this approach with Bayesian robustness and in particular prior near-ignorance modeling via sets of probabilities. We use this model to perform a Bayesian hypothesis test on the probability P(X Y ). We study the theoretical properties of the IDP test (e.g., asymptotic consistency), and compare≤ it with the frequentist Mann-Whitney-Wilcoxonrank test that is commonlyemployedas a test on P(X Y ). In particular we will show that our method is more robust, in the sense that it is able to isolate≤ instances in which the aforementioned test is virtually guessing at random. Keywords: Bayesian nonparametric test, Imprecise Dirichlet Process, Wilcoxon rank sum. 1. Introduction The Dirichlet process (DP) is one of the most popular Bayesian nonparametric models. It was introduced by Ferguson (1973) as a prior over probability distributions. In his seminal paper, Ferguson showed that the DP leads to tractable posterior inferencesand can be used for Bayesian analysis of several nonparametric models, such as the estimation of a distribution function, of a mean, of quantiles, of a variance, etc. He also considered the estimation of P(X Y ) assigning independentDirichlet process priors to the distribution functions of X and Y . The Mann-Whitney≤ arXiv:1402.2755v2 [math.ST] 20 Feb 2014 statistic naturally arises in this case. Susarla and Van Ryzin (1976) and Blum and Susarla (1977) extended the results of Ferguson on estimation of the distribution function in case of right cen- sored data obtaining a Bayesian version of the Kaplan-Meier estimator. Dalal and Phadia (1983) considered the problem of estimating a measure of dependence for a bivariate distribution. The Bayes estimate is computed using a two-dimension Dirichlet prior and Kendall’s tau is seen to appear naturally. A review of other similar applications of the DP can be found in (Phadia, 2013). The beauty of the DP is that most of these results are in closed form and that it provides a Bayesian justification of the classic nonparametric estimators. In spite of all these nice properties and of the promising initial outcomes, it is a matter of fact that such a research has not resulted in the development of DP-based Bayesian nonparametric procedures for hypothesis testing. For Preprint submitted to Elsevier June 4, 2018 2 instance, the most used statistical packages for DP-based Bayesian nonparametric modeling, “DPpackage” (Jara et al., 2011) and “Bayesm” (Rossi and McCulloch, 2010), include proce- dures for density estimation, clustering and regression, but do not include any Bayesian version of the Wilcoxon rank sum, Wilcoxon sign test or other classic nonparametric tests. It is arguable that this absence may be related to the unsettled question of how to choose the prior “parameters” of the DP in case of lack of prior information. It is well known in fact that a DP is completely characterized by its prior “parameters”: the prior strength (or precision), which is a positive scalar number, and the normalized base measure. How should we choose these prior “parameters” in case of lack of prior information? The only solution to this problem that has been proposed so far, first by Ferguson (1973) and then by Rubin (1981) under the name of Bayesian Bootstrap (BB), is the limiting DP obtained when the prior strength goes to zero. But the BB model has faced quite some controversy, since it is not actually noninformative and moreover it assigns zero posterior probability to any set that does not include the observations. We will discuss these two points with more details in Section 3. In this paper we present an alternative viewpoint to the problem of choosing the prior base measure of the DP in case of lack of prior information that overcomes the above drawbacks of the BB. The model we present generalizes to nonparametrics earlier ideas developed in Bayesian parametric robustness, see Berger (1994) and Berger et al. (2000) for a review. Here lack of prior information is expressed in terms of a family T consisting of all prior probability measures that are compatible with the available prior information. Inferences are then carried out by consider- ing the whole family T . In case almost no prior information is available on the parameters of interest, T should be as large as possible in order to describe this state of prior ignorance. The natural candidate for T to represent complete ignorance is the set of all probability measures. However, it turns out that the posterior inferences obtained from this set are vacuous (Walley, 1991, Sec. 7.3.7), i.e., the posterior set coincides with the prior set. This means that that there is no learning from data. Therefore, the vacuous prior model is not a statistically useful way to model our prior ignorance. There is then a compromise to be made. Pericchi and Walley (1991) and Walley (1991) suggest, as an alternative, the use of an almost vacuous model which they call “near-ignorance” or “imprecise” model. This is a model that behaves a priori as a vacuous model for some basic inferences (e.g., prior mean, prior credible regions) but it always provides non-vacuous posterior inferences. While Bayesian robust models have already been extended to the nonparametric setting (Ruggeri, 2010), that has not been the case for near-ignorance models.1 The main aim of this paper is to derive a prior near-ignorance DP, called Imprecise DP (IDP). This is the class T of all DPs obtained by fixing the prior strength of the DP and letting the normalized base measure vary in the set of all probability measures. We will show that the IDP behaves a priori as a vacuous model for all predictive inferences. This, together with the fact that it is a nonparametric model, allows us to start a statistical analysis with very weak assumptions about the problem of interest. However, contrarily to a full vacuous model, we will show that the IDP can learn from data. Moreover, we will employ the IDP to develop a new Bayesian nonparametric hypothesis test on the probability that X Y; we will call this test IDP rank-sum test, due to its similarity with the Mann-Whitney-Wilcoxon≤ (MWW) rank-sum test. This hypothesis test is widely ap- plied; for instance, if X and Y are health status measures in a clinical trial, P(X Y ) is, roughly ≤ 1A nonparametric model that uses lower and upper bounds for probabilities to quantify uncertainty has been proposed by Augustin and Coolen (2004); Coolen and Augustin (2009). This model is a purely predictive model, based on post- data assumptions, and, thus, it cannot be used straightforwardly to perform hypothesis tests. 3 speaking, the probability that the treatment represented by Y is better (not worse) than the treat- ment represented by X. A Bayesian nonparametric near-ignorance model presents several ad- vantages with respect to a traditional approach to hypothesis testing. First of all, the Bayesian approach allows us to formulate the hypothesis test as a decision problem. This means that we can verify the evidence in favor of the null hypothesis and not only rejecting it, as well as take decisions that minimize the expected loss. Second, because of the nonparametric near-ignorance prior, the IDP rank-sum test allows us to start the hypothesis test with very weak prior assump- tions, much in the direction of letting data speak for themselves. From a computational point of view, we will show that posterior inferences from the IDP can be derived by computing lower and upper bounds of expectations w.r.t. the class of DPs T and that, for some inference, these lower and upper bounds can be computed in closed-form (e.g., mean and variance of P(X Y)). When no closed form expression exists, these bounds can be computed by a simple Monte≤ Carlo sampling from two Dirichlet distributions. This means that we do not need to use stick breaking or other sampling approaches specific for DP. This computational advantage comes for free from prior near-ignorance. In our view, the IDP rank-sum test appears to be a natural way to complete the work of Ferguson (1973), who first showed the connection between the expectation of P(X Y) w.r.t. the DP and the Mann-Whitney statistic: it develops a Bayesian nonparametric near-ignorance-≤ prior test for the probability that X Y , which is computationally efficient and that, also for this reason, provides an effective practical≤ alternative to the MWW test. Note that, although the IDP test shares several similarities with a standard Bayesian ap- proach, at the same time it embodies a significant change of paradigm when it comes to take decisions. In fact the IDP rank-sum test has the advantage of producing an indeterminate out- come when the decision is prior-dependent. In other words, the IDP test suspends the judgment (which can be translated as “I do not know whether Y is better than X”) when the option that minimizes the expected loss changes depending on the DP base measure we focus on.