IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 17, NO. 2, FEBRUARY 1995 21 1 APPENDIXI1 A Bayesian Segmentation Methodology E(YJ =Yi for Parametric Image Models E(X?) =xi‘+d Steven M. LaValle and Seth A. Hutchinson E(Y,z) =y?+d E(Xf) = xf + 6x?d + 304 = yf 6y?d 3d Abstract - Region-based image segmentation methods require some E(Yf) + + criterion for determining when to merge regions. This paper presents a E(X,ZY?) = $y,Z + (x?+y?)d + d novel approach by introducing a Bayesian probability of homogeneity in E(X,ZY;) = xi’y; +yid a general statistical context. Our approach does not require parameter estimation and is therefore particularly beneficial for cases in which es- timation-based methods are most prone to error: when little information 2 +kv2: is contained in some of the regions and, therefore, parameter estimates are unreliable. We apply this formulation to three distinct parametric E(AN)=M+(l +k)d model families that have been used in past segmentation schemes: im- 1N plicit polynomial surfaces, parametric polynomial surfaces, and Gaus- E(BN) = M2 (6x: +6y: +2k(x? +y;))02 +(3+3k2 +2k)04 +-x,N 1=1 sian Markov random fields. We present results on a variety of real range and intensity images. Index Terms - Statistical image segmentation, Bayesian methods, likelihoods, Bayes factor, range images, Markov random field, texture From the last equation we see that E(BN)- (E(AN))2is positive. Sec- segmentation. ond, we can continue the calculations for E(&): I. INTRODUCTION E(BN)- 4dS~= E2(A~) - 2( 1 + P)04 The problem of image segmentation, partitioning an image into a set of homogeneous regions, is fundamental in computer vision. Ap- But AN +,,,?(AN) and therefore proaches to the segmentation problem can be grouped into region-based methods in which image subsets are grouped together when they share 2(1 + k2)Z2- 4SNZ+ (BN -Ai) + 0 some property (e.g., [26]); edge-based methods, in which dissimilarity between regions is used to partition the image (e.g., [9]); and combined where Z = d.Solving the quadratric in d and taking the positive region- and edge-based methods (e.g., [22]). In this paper, we present a square root as d must be positive gives the result. new, Bayesian region-based approach to Segmentation. A standard approach to region-based segmentation is to character- ize region homogeneity using parameterized models. With this ap- proach, two regions are considered to be homogeneous if they can be ,? +kv=M: explained by a single instance of the model, i.e., if they have a com- mon parameter value. For example, in range image applications, ob- ject surfaces are often modeled as being piecewise algebraic (e.g., [30]). The parameters of such a surface are the coefficients of the cor- responding polynomial. Two regions are homogeneous and, thus, Continuing as in the last case we get the final result. should be merged if they belong to a single polynomial surface (i.e., if the coefficients for their corresponding polynomials are the same). REFERENCES In practice, a region’s parameters cannot be observed directly but can only be inferred from the observed data and knowledge of the [I] R. Duda and P. Hart, Pattern Classification and Scene Analysis, Wiley, imaging process. In statistical approaches, this inference is made us- 1973. ing Bayes’s rule and the conditional density, p(ykl uk), which ex- H. Imai, K. Kato, and P. Yamamoto, “A linear time algorithm for linear [2] presses the probability that certain data (or statistics derived from the 11 approximation of points,” Algorithmica, vol. 4, pp. 77-96, 1989. data), Yk, will be observed, given that region k has the parameter [3] C.H. Kummel, “Reduction of observed equations which contain more than one observed quantity,” The Analyst, vol. 6, pp. 97-105, 1979. .. value N. In typical statistical region merging algorithms (e.g., [27]) [4] A. Madansky, “The fitting of straight lines when both variables are point estimates in the parameter space are obtained for different re- subject to error,” Amer. Statistical Assoc. J., vol. 54, pp. 173-205, gions and merging decisions are based on the similarity of these es- 1959. timates. Often the maximum a posteriori (MAP) estimate is used, [5] P.A.P. Moran, “Estimating structural and functional relationships,” J. of which is obtained by maximizing p(ykl uk). Multivariate Analysis, vol. I, pp. 232-255, 1971. An inherent limitation of nearly all estimation-based segmentation [6] F.P. Preparata and I.M. Shamos, Comptational Geometry, New York: methods reported to date is that they do not explicitly represent the Springer-Verlag, 1985. uncertainty in the estimated parameter values and, therefore, are [7] A. Stein and M. Werman, “Finding the repeated median regression line,” Third Symp. on Discrete Algorirhms, 1992. prone to error when parameter estimates are poor (one notable ex- [8] A. Stein and M. Werman, “Robust statistics in shape fitting,” CVPR, ception to this is the work of Szeliski [29] in which both optimal es- pp. 540-546, Champaign, Ill. 1992. [9] A. Wald, “Fitting of straight lines if both variables are subject to error,” Manuscript received January 13, 1993; revised November 24, 1993. Annals qfMuth. Statistics, vol. 1I, pp. 284-300, 1940. Recommended for acceptance by Editor-in-Chief A.K. Jain. [IO] M. Werman, A.Y. Wu, and R.A. Melter, “Recognition and characteri- The authors are with the Beckman Institute for Advanced Science and Tech- zation of digitized curves,” Pattern Recognition Letters, vol. 5, pp. nology and the Dept. of Electrical and Computer Engineering, University of Illi- 207-213, 1987. nois, 405 N. Mathews Ave., Uhana, IL61801; e-mail [email protected]. IEEECS Log Number P95055. 0 162-8828/95$04 ..OO0 1995 IEEE 212 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 11, NO. 2, FEBRUARY 1995 timates and the variance in the estimates are computed). To overcome will be called adjacent if there exists some D[il,jl] E RI and D[iz,jz] this problem, we present a Bayesian probability of homogeneity that E R2 that are neighbors. directly exploits all of the information contained in the statistical im- It is often profitable to begin with some initial partition of the im- age models, as opposed to computing point parameter estimates. age into small regions, and to construct new segmentations by com- The probability of homogeneity is based on the ability to formu- bining these regions. This is a standard approach taken in the region late a prior probability density on the parameter space, and assess merging paradigm. For instance, Sabata et al initially generate an im- homogeneity by taking the expectation of the data likelihood over a age of sels, which corresponds to regions that have near-constant posterior parameter space. This type of expectation was also used by differential properties [26], and Silverman and Cooper begin with an Cohen and Fan to formulate a data likelihood for segmentation ap- initial grid of small regions [27]. We denote the initial set of regions plied to the Gaussian Markov random field model [SI. In their work, as R, which represents a partition of D. segmentations are defined by a space of pixel labelings and, through window-based iterative optimization, a segmentation is determined TABLE I that maximizes the data likelihood. By considering the region-based THE KEY COMPONENTS IN OUR GENERALSTATISTICAL FRAMEWORK probability of homogeneity, we introduce a different decomposition Parameter space: and prior on the space ofsegmentations. A random vector, uk, which could, for instance, represent Our probability of homogeneity can also be considered a function as a space of polynomial surfaces. of the Bayes factor from recent statistical literature [l], [15], [23], [28] Observation space: which has been developed for statistical decision making such model as A random vector, Yk, which represents the data or selection. A detailed description of our model and the derivation of the functions of the data x E Rk . Bayesian probability of homogeneity are given in Section 11. In addition to providing an explicit accounting of the uncertainty Degradation model: associated with a segmentation (which could feasibly be used in A conditional density p(ykluk) which models noise and higher level vision processes, such as recognition), our method ex- uncertainty. tends in a straightforward way to allow application of multiple, inde- Prior model: pendent image models. Furthermore, our framework does not require An inital parameter space density. the specification of arbitrary parameters (e.g., threshold values), since context dependent quantities can be statistically estimated. For each Rk E R we associate the following: a parameter space, an We have applied our Bayesian probability of homogeneity to seg- observation space, a degradation model, and a prior model (see Ta- mentation problems using three popular model families: implicit ble I). The parameter space directly captures the notion of homogeneity: polynomial surfaces, in Section 111; parametric (explicit) polynomial every region has a parameter value (a point in the parameter space) as- surfaces, in Section IV; and Gaussian Markov random fields for tex- sociated with it, which is unknown to the observer. The observation ture segmentation, in Section V. In Section VI1 we present experi- space defines statistics that are functions of the image elements, and that mental results from each of the model families. These results were contain information about the region’s parameter value. We could use obtained using the algorithm described in Section VI. the image data directly for the observation or we could choose some Further, we have developed special numerical computation methods function (possibly a sufficient statistic, depending on the application) for directly computing the probability of homogeneity using the that increases the efficiency of the Bayesian computations.