Stein's Method for Discrete Gibbs Measures
Total Page:16
File Type:pdf, Size:1020Kb
The Annals of Applied Probability 2008, Vol. 18, No. 4, 1588–1618 DOI: 10.1214/07-AAP0498 c Institute of Mathematical Statistics, 2008 STEIN’S METHOD FOR DISCRETE GIBBS MEASURES1 By Peter Eichelsbacher and Gesine Reinert Ruhr-Universit¨at Bochum and University of Oxford Stein’s method provides a way of bounding the distance of a prob- ability distribution to a target distribution µ. Here we develop Stein’s method for the class of discrete Gibbs measures with a density eV , where V is the energy function. Using size bias couplings, we treat an example of Gibbs convergence for strongly correlated random vari- ables due to Chayes and Klein [Helv. Phys. Acta 67 (1994) 30–42]. We obtain estimates of the approximation to a grand-canonical Gibbs ensemble. As side results, we slightly improve on the Barbour, Holst and Janson [Poisson Approximation (1992)] bounds for Poisson ap- proximation to the sum of independent indicators, and in the case of the geometric distribution we derive better nonuniform Stein bounds than Brown and Xia [Ann. Probab. 29 (2001) 1373–1403]. 0. Introduction. Stein [17] introduced an elegant method for proving convergence of random variables toward a standard normal variable. Bar- bour [2, 3] and G¨otze [10] developed a dynamical point of view of Stein’s method using time-reversible Markov processes. If µ is the stationary dis- tribution of a homogeneous Markov process with generator , then X µ if and only if E g(X) = 0 for all functions g in the domainA ( ) of∼ the operator . ForA any random variable W and for any suitable functionD A f, to assess theA distance Ef(W ) f dµ we first find a solution g of the equation | − | g(Rx)= f(x) f dµ. A − Z If g is in the domain ( ) of , then we obtain D A A arXiv:0808.2877v1 [math.PR] 21 Aug 2008 (0.1) Ef(W ) f dµ = E g(W ) . − | A | Z Received April 2006; revised October 2007. 1Supported in part by the DAAD Foundation and the British Council, ARC Project, Contract 313-ARC-00/27749 and by the RiP program at Oberwolfach, Germany. AMS 2000 subject classifications. Primary 60E05; secondary 60F05, 60E15, 82B05. Key words and phrases. Stein’s method, Gibbs measures, birth and death processes, size bias coupling. This is an electronic reprint of the original article published by the Institute of Mathematical Statistics in The Annals of Applied Probability, 2008, Vol. 18, No. 4, 1588–1618. This reprint differs from the original in pagination and typographic detail. 1 2 P. EICHELSBACHER AND G. REINERT Bounding the right-hand side of (0.1) for a sufficiently large class of func- tions f leads to bounds on distances between the distributions. Here we will mainly focus on the total variation distance, where indicator functions are the test functions to consider. Such distributional bounds are not only useful for limits, but they are also of particular interest, for example, when the distance to the target distri- bution is not negligible. Such relatively large distances occur, for example, when only few observations are available, or when there is considerable de- pendence in the data slowing down the convergence. The bound on the dis- tance can then be taken into account explicitly, for example when deriving confidence intervals. In Section 1 we introduce Stein’s method using the generator approach for discrete Gibbs measures, which are probability measures on N0 with proba- bility weights µ(k) proportional to exp(V (k)) for some function V : N0 R. The discrete Gibbs measures include the classical distributions Poisson,→ bi- nomial, geometric, negative binomial, hypergeometric and the discrete uni- form, to name but a few. One can construct simple birth–death processes, which are time-reversible, and which have a discrete Gibbs measure as its equilibrium measure. In the context of spatial Gibbs measures this connec- tion was introduced by Preston [15]. In Section 2 we not only recall bounds for the increments of the solution of the Stein equation from [5], but we also derive bounds on the solution itself, in terms of potential function of the Gibbs measure; see Lemmas 2.1, 2.4 and 2.5. The bounds, which to our knowledge are new, are illustrated for the Poisson, the binomial and the geometric distribution. For nonnegative random variables, the size bias coupling is a very useful approach to disentangle dependence. Its formulation and its application to assess the distance to Gibbs measures are described in Section 3. We compare the distributions by comparing their respective generators, an idea also used in [11], while paying special attention to the case that the domains of the two generators are not identical. The size bias coupling then naturally leads to Theorem 3.5 and Corollary 3.8. Section 4 applies these theoretical results to assess the distance to a Gibbs distribution for the law of a sum of possibly strongly correlated random variables. The main results are Theorem 4.2 and Proposition 4.4, where we give general bounds for the total variation distance between the distribution of certain sums of strongly correlated random variables and discrete Gibbs distributions of a grand-canonical form; see [16], Section 1.2.3. In particular, Theorem 4.2 gives a bound on the rate of convergence for the qualitative results in [6] by bounding the rate of convergence. Considering two examples with nontrivial interaction, we obtain bounds to limiting nonclassical Gibbs distributions. Our bound on the approxima- tion error is phrased in terms of the particle number and the average density of the particles. STEIN’S METHOD FOR DISCRETE GIBBS MEASURES 3 Summarizing, the main advantage of our considerations is the application of Stein’s method to models with interaction described by Gibbs measures. When applying our bounds on the solution of the Stein equation to the Poisson distribution and the geometric distribution, surprisingly we obtain improved bounds for these well-studied distributions. Thus our investigation of discrete Gibbs measures serves also as a vehicle for obtaining results for classical discrete distributions. The results presented here will also provide a foundation for introducing Stein’s method for spatial Gibbs measures and Gibbs point processes, in forthcoming work. 1. Gibbs measures and birth–death processes. 1.1. Birth–death processes. A birth–death process X(t),t R is a Markov process on the state space 0, 1,...,N , where N N { ,∈ characterized} { } ∈ 0 ∪ {∞} by (nonnegative) birth rates bj, j 0, 1,...,N and (nonnegative) death rates d , j 1,...,N and{ has a∈ generator { }} { j ∈ { }} (1.1) ( h)(j)= b (h(j + 1) h(j)) d (h(j) h(j 1)) A j − − j − − with j 0, 1,...,N . ∈ { } It is well known that for any given probability distribution µ on N0 one can construct a birth–death process which has this distribution as its station- ary distribution. For N = , recurrence of the birth–death process X( ) is equivalent to d1···dn ∞= . The process X( ) is ergodic if and only· if n≥1 b1···bn ∞ · the process is recurrent and c :=1+ b0···bn−1 < . For N < , irre- P n≥1 d1···dn ducibility and hence ergodicity holds if b > 0, j = 0,∞1,...,N 1,∞b = 0 P j − N and dj > 0, j = 1,...,N. In either case the stationary distribution of the ergodic process is given by µ(0) = 1/c and b b (1.2) µ(n)= µ(0) 0 · · · n−1 . d d 1 · · · n For any given probability distribution µ on N0 these recursive formulas give the corresponding class of birth–death processes which have µ as the sta- tionary distribution. For the choice of a unit per capita death rate dj = j one simply obtains that µ(j + 1) (1.3) b = (j + 1), j µ(j) for j N 1. Here and throughout, if N = , then by j N 1 we mean j = 0≤, 1,....− The choice of these rates corresponds∞ to the≤ case− where the detailed balance condition (1.4) µ(j)b = µ(j + 1)d , j = 0, 1,...,N 1, j j+1 − 4 P. EICHELSBACHER AND G. REINERT holds; see, for example, [1] and [12]. We will apply these well-known facts to the discrete Gibbs measure introduced in the next subsection. Now might be a good time to note that not all probability distributions on N0 are given in closed form expressions; notable exceptions occur, for example, in compound Poisson distributions. 1.2. Gibbs measures as stationary distributions of birth–death processes. Gibbs measures can be viewed as stationary measures of birth–death pro- cesses, as follows. We start with a discrete Gibbs measure µ; for convenience we assume that µ has support supp(µ)= 0,...,N , where N N0 , so that µ is given by { } ∈ ∪ {∞} 1 ωk (1.5) µ(k)= eV (k) , k = 0, 1,...,N, k! Z N R N ωk for some function V : 0 . Here = k=0 exp(V (k)) k! , and ω> 0 is fixed. We assume that →exists; isZ known as the partition function in P models of statistical mechanics.Z WeZ set V (k)= for k > N. In terms of statistical mechanics, µ is a grand-canonical ensemble,−∞ ω is the activity and V is the potential energy; see [16], Chapter 1.2. The class of discrete Gibbs measures in (1.5) is equivalent to the class of all discrete probability distributions on N0 by the following simple identifi- cation: For a given probability distribution (µ(k))k∈N0 we have (1.6) V (k) = log µ(k) + log k! + log k log ω, k = 0, 1,...,N, Z− V (0) with V (0) = log µ(0) + log .