Statistica Sinica 18(2008), 1185-1200 BAYESIAN HYPOTHESIS TESTS USING NONPARAMETRIC STATISTICS Ying Yuan and Valen E. Johnson University of Texas and M.D. Anderson Cancer Center Abstract: Traditionally, the application of Bayesian testing procedures to classi- cal nonparametric settings has been restricted by difficulties associated with prior specification, prohibitively expensive computation, and the absence of sampling densities for data. To overcome these difficulties, we model the sampling distri- butions of nonparametric test statistics—rather than the sampling distributions of original data—to obtain the Bayes factors required for Bayesian hypothesis tests. We apply this methodology to construct Bayes factors from a wide class of non- parametric test statistics having limiting normal distributions and illustrate these methods with data. Finally, we consider the extension of our methodology to non- parametric test statistics having limiting χ2 distributions. Key words and phrases: Bayes factor, Kruskal-Wallis test, Logrank test, Mann- Whitney-Wilcoxon test, nonparametric hypothesis test, Wilcoxon signed rank test. 1. Introduction In parametric settings, the use of Bayesian methodology for conducting hy- pothesis tests has been limited by two factors. First, the calculation of Bayes factors often involves the evaluation of high-dimensional integrals. This can be a prohibitively expensive undertaking for non-statisticians, both from a numerical and conceptual perspective. Second, Bayes factors require the specification of informative prior densities on parameters appearing in the parametric statistical models that comprise each hypothesis. And unlike Bayesian estimation proce- dures, tests based on Bayes factors retain their sensitivity to prior assumptions even when sample sizes become large. Prior specification is therefore an impor- tant task and one which can be difficult in models containing many parameters. In nonparametric hypothesis testing, a third difficulty arises. Namely, sam- pling distributions for data are not specified. Without sampling distributions for data, Bayesian hypothesis tests cannot be defined. The goal of this article is to overcome these obstacles to Bayesian testing by extending methodology proposed in Johnson (2005) to the classical non- parametric setting. We accomplish this by using results from the asymptotic 1186 YINGYUANANDVALENE.JOHNSON theory of U-statistics and linear rank statistics (e.g., Serfling (1980)) to define al- ternative distributions for test statistics that take the form of Pitman translation alternatives (e.g., Randles and Wolfe (1979)). In so doing, we obtain sampling distributions for non-parametric statistics under alternative models that contain only two unknown parameters: a scale parameter and an asymptotic test-efficacy parameter. Because the distribution of the test statistics under the null hypothesis is known, by specifying a sampling distribution for nonparametric test statistics under a class of alternative models we are able to eliminate two of the obstacles to Bayesian hypothesis testing. That is, by modeling the distribution of test statistics directly we obtain the sampling distributions required for the definition of Bayes factors. We also obtain closed-form expressions for the resulting Bayes factors, which means that numerical evaluation of their values is unnecessary. The third obstacle to the use of Bayes factors—the specification of subjective prior densities—is also considerably simplified. Within our framework, alterna- tive models contain only two scalar parameters. One, the test-efficacy parameter, is determined by the choice of the test statistic. In principle this leaves only a scale parameter for which a prior density must be specified. Methods for handling this parameter are discussed below. To begin, it is worthwhile to review the definition of Bayes factors for the case of nested models. Suppose then that data x arise from a sampling density p(X θ, φ) with unknown parameter vectors θ and φ, and suppose we wish to test | the null hypothesis H : θ = θ0 against the alternative hypothesis H : θ = θ0 0 1 6 where θ0 is assumed known. If p0(x) and p1(x) represent the marginal densities of x under H0 and H1, then the Bayes factor between H0 and H1 can be expressed p (x) p(x θ0, φ)p(φ)dφ B = 0 = | , 01 p (x) p(x θ, φ)p(θ, φ)dθ dφ 1 R | where p(φ) and p(θ, φ) are the priorR densities on unknown parameters under H0 and H1, respectively. Furthermore, if p(H0) and p(H1) denote the prior probabilities assigned to H0 and H1, then the posterior odds of H0 versus H1 is obtained from B01 according to p(H x) p(H ) 0| = 0 B . p(H x) p(H ) 01 1| 1 Thus, Bayes factors represent the weight of evidence contained in data in support of each hypothesis. When deciding between two simple hypotheses, the Bayes factor is simply the likelihood ratio; in more complicated settings it can be re- garded as an integrated likelihood ratio. Further discussion of Bayes factors can be found in, for example, Kass and Raftery (1995). BAYESIAN HYPOTHESIS TESTS USING NONPARAMETRIC STATISTICS 1187 Our motivation for defining Bayes factors based on nonparametric test statis- tics is to avoid the many pitfalls inherent to the use of p values in formal test procedures. For a discussion of these issues in the classical setting, interested readers may consult (among many other articles) Berger and Delampady (1987), Berger and Sellke (1987), and Goodman (1999a,b). Similar issues arise in the use of Bayesian p values for testing model adequacy as described in, for example, Gelman, Meng and Stern (1996) who base their definition of Bayesian p values on posterior-predictive distributions. However, because posterior predictive p values also represent tail-area probabilities, they too are subject to many of the pitfalls inherent to classical p values. As we demonstrate in the sequel, our methodology allows us to transform nonparametric test statistics to an appropriate—and interpretable—probability scale, rather than to what is essentially an uncalibrated and comparatively un- interpretable p-value scale. Our approach is based on the observation that, although the sampling den- sity p(x θ) of the data X may not be specified in nonparametric settings, the | distribution of the test statistic T (X) is often known under both null and alter- native hypotheses, at least asymptotically. To make this notion more precise, we assume for the remainder of this article that the sampling density of the test statistic T (X) can be expressed as p(T (x) θ), where the parameter θ may be | either a scalar or vector-valued parameter. Under the null hypothesis H0, we assume that θ = θ0 for a known value θ0. Under the alternative hypothesis H1, we assume that the sampling distribution of T is obtained by averaging over a prior density p(θ) defined on the domain of θ. When these assumptions hold, the Bayes factor based on t = T (X) can be defined as p(t θ ) BF (t)= | 0 . 01 p(t θ)p(θ)dθ | For suitable choices of p(θ), we find thatR Bayes factors based on nonparametric test statistics can often be expressed in simple form. From such expressions, it is possible to obtain an upper bound on the weight of evidence against the null hypothesis. Such bounds are often useful and may serve to illustrate the maximum extent to which data provide evidence against the null hypothesis. Their use also eliminates much of the subjectivity associated with the definition of Bayes factors. 2. Theory Many nonparametric test statistics have limiting distributions that are either normal or χ2. While our methods can sometimes be applied in finite sample 1188 YINGYUANANDVALENE.JOHNSON settings, it is generally more straightforward to specify alternative distributions in the large sample setting. For this reason, we restrict attention to this case and begin with nonparametric statistics that have limiting normal distributions. The asymptotic normality of a variety of nonparametric test statistics has been established by the theory of U-statistics and linear rank statistics (e.g., Serfling (1980)). These results are widely used in practice to approximate the exact sampling distributions of nonparametric test statistics, which often do not have closed forms and have to be computed numerically. The class of nonparametric test statistics with limiting normal distribu- tions includes many commonly used nonparametric statistics. Among these are the sign test and Wilcoxon signed rank test for one-sample location problems, the Mann-Whitney-Wilcoxon test for two-sample location problems, the Ansari- Bradley test and Mood test for scale problems, Kendall’s tau and Spearman test for testing independence, the Theil test for slope parameters in regression problems, the Mantel test (or logrank test), and the Hollander-Proschan test of exponentiality in survival analysis. In order to describe how statistics from these tests can be used to define Bayes factors, let Tk, k = 1, 2 . ., denote a sequence of nonparametric test statistics based on nk observations, and suppose that nk as k . Consider the → ∞ → ∞ test of the null hypothesis H0 : θ = θ0 versus the local alternative H1(nk) : θk = θ0 +∆/√nk, where ∆ is a bounded constant. This form of the alternative hypothesis is often called the Pitman translation alternative (e.g., Randles and Wolfe (1979)). Our attention focuses on the asymptotic distribution of the standardized value of Tk, ∗ Tk µk(θ0) Tk = − , σk(θ0) where µk and σk are the mean and standard deviation of Tk, respectively. Under ∗ H0, we assume that Tk has a limiting standard normal distribution. Under ∗ H1(nk), the asymptotic distribution of Tk is given in the following lemma. The proof follows Noether (1955) and appears in the Appendix. Lemma 1. Assume H1(nk) and L (A1) [Tk µk(θk)]/σk(θk) N(0, 1); − p −→ (A2) σk(θk)/σk(θ ) 1; 0 −→ (A3) µk(θ) is differentiable at θ0; BAYESIAN HYPOTHESIS TESTS USING NONPARAMETRIC STATISTICS 1189 ′ p (A4) µk(θ0)/√nkσk(θ0) C where C is a constant.