Journal of Machine Learning Research 8 (2007) 1769-1797 Submitted 7/06; Revised 1/07; Published 8/07 Characterizing the Function Space for Bayesian Kernel Models Natesh S. Pillai
[email protected] Qiang Wu
[email protected] Department of Statistical Science Duke University Durham, NC 27708, USA Feng Liang
[email protected] Department of Statistics University of Illinois at Urbana-Champaign Urbana-Champaign, IL 61820, USA Sayan Mukherjee
[email protected] Department of Statistical Science Institute for Genome Sciences & Policy Duke University Durham, NC 27708, USA Robert L. Wolpert
[email protected] Department of Statistical Science Professor of the Environment and Earth Sciences Duke University Durham, NC 27708, USA Editor: Zoubin Ghahramani Abstract Kernel methods have been very popular in the machine learning literature in the last ten years, mainly in the context of Tikhonov regularization algorithms. In this paper we study a coherent Bayesian kernel model based on an integral operator defined as the convolution of a kernel with a signed measure. Priors on the random signed measures correspond to prior distributions on the functions mapped by the integral operator. We study several classes of signed measures and their image mapped by the integral operator. In particular, we identify a general class of measures whose image is dense in the reproducing kernel Hilbert space (RKHS) induced by the kernel. A conse- quence of this result is a function theoretic foundation for using non-parametric prior specifications in Bayesian modeling, such as Gaussian process and Dirichlet process prior distributions. We dis- cuss the construction of priors on spaces of signed measures using Gaussian and Levy´ processes, with the Dirichlet processes being a special case the latter.