Unsup ervised Progressive Parsing of Poisson Fields Using Minimum Description Length Criteria y Rob ert D. Nowak Mario A. T. Figueiredo Dept. of Electr. and Comp. Eng. Instituto de Telecomunicac~oes, and Rice University Instituto Sup erior Tecnico Houston, TX, U.S.A. 1049-001 Lisb oa, Portugal Abstract This paper describes novel methods for estimating piecewise homogeneous Poisson elds based on mini- mum description length (MDL) criteria. By adopt- ing a coding-theoretic approach, our methods are able to adapt to the the observed eld in an unsupervised (a) (b) (c) manner. We present a parsing scheme basedon xed multiscale trees (binary, for 1D, quad, for 2D) and an Figure 1: Illustration of the problem addressed in this adaptive recursive partioning algorithm, both guided pap er: (a) piecewise-constantintensity function (Pois- by MDL criteria. Experiments show that the recursive son intensities 0.05, 0.2, and 0.4); (b) scatter-plot of scheme outperforms the xedtree approaches. observed photon events; (c) intensity eld parsing by MDL-based recursive algorithm. 1 Intro duction MDL criteria without recourse to asymptotic approx- Consider a realization from a spatial Poisson p oint imations. In contrast, most applications of MDL in- pro cess whose underlying intensity function is (or can volve Gaussian statistics (which are real-valued) and be approximated as) piecewise constant (as exempli- require asymptotic arguments. Hence, our application ed in Figure 1(a)). This typ e of data arises in photon- of MDL here is esp ecially simple and well motivated. limited imaging, particle and astronomical physics, This work is a co ding-theoretic alternative to related computer trac network analysis, and many other ap- Bayesian estimation schemes [2, 3,4]. plications involving counting statistics. In particular, photon-limited images are formed by detecting and 2 Mininum Description Length counting individual photon events (e.g., in gamma-ray The MDL principle addresses the following ques- astronomyornuclear medicine). tion: given a set of generation mo dels, which b est From an observed realization (see Figure 1(b)), our explains the observed data? To get a handle on goal is to parse,orsegment, the observation space into the notion of \b est," Rissanen employed the follow- regions of (roughly) homogeneous intensity (Figure ing gedankenexperiment. Supp ose that we wish to 1(c)); i.e., regions of space in which the distribution of transmit the observed data x to a hyp othetical re- points is well mo deled by a spatial Poisson distribution ceiver. Given a (probabilistic) generation mo del for with constantintensity. In this pap er, we prop ose two the data, say p(xj ), the Shannon-optimal co de length new metho ds for unsup ervised progressive parsing of is log p(xj ). Of course, the receiver would also need Poisson elds based on Rissanen's minimum descrip- to know the mo del parameters to deco de the trans- tion length (MDL) principle [1]. One of the interesting mission; then, if is a priori unknown, we also need asp ects of our development is that, since the Poisson to estimate it, co de it, and transmit it. Now, consider K data (counts) are integer-valued, we are able to derive a set of K comp eting mo del classes fp (xj )g . In i i i=1 each class i, the \b est" mo del is the one that gives the Supp orted by the National Science Foundation, grantno. minimum co de length, MIP{9701692. y Supp orted bytheFoundation for Science and Technology b = arg min f log p (xj )g = arg max p (xj ); i i i i i (Portugal), grant Praxis-XXI T.I.T.-1580. i i this is simply the maximum likelihood (ML) estimate counts x and x , resp ectively. So, j;k j +1;2k within mo del class i. But if the class is a priori p(x j x ; ) = B i(x j x ; ) j +1;2k j;k j;k j +1;2k j;k j;k unknown, the \b est" overall mo del is the one that x x leads to the minimum description length: the sum of j;k j +1;2k x j +1;2k +1 = (1 ) : (2) j;k j;k x log p (xj ) with the length of the co de for itself. j +1;2k i i i The fundamental asp ect of MDL is that it p erforms Now consider two available mo del classes. Mo del mo del selection (which the ML criterion alone do es Class 0 assumes a homogeneous Poisson pro cess; then, not) by p enalizing more complex mo del classes (those = and consequently, since = j +1;2k j +1;2k +1 j;k requiring longer parameter co de lengths). MDL crite- 1 . Alternatively, + ,wehave = j +1;2k j +1;2k +1 j;k 2 ria have b een successfully used in several image anal- in Mo del Class 1, is a free parameter. j;k ysis/pro cessing problems (see references in [5]). Hence, wehavetwo p ossible description lengths (re- The delicate issue in applying MDL is in how to call that x is already known by the receiver). j;k enco de the parameter ; appropriate parameter co de i Mo del Class 0: Since =1=2, it do esn't require lengths are usually based on asymptotic approxima- j;k enco ding; the description length is then simply tions; e.g., the well known (1=2) log N ,whereN is the amount of data, is an asymptotic co de length [1]. In L = log B i(x j x ; 1=2) (3) 0 j +1;2k j;k this pap er, we are able to avoid asymptotic approxi- x j;k mations and obtain exact co de lengths. = log + x log 2 j;k x j +1;2k 3 MDL for Poisson Data Mo del Class 1: In this case, the rst step consists in We intro duce next two progressive (multiscale or estimating, co ding, and transmitting . Its ML j;k multiresolution) approaches to co ding Poisson data. x j +1;2k estimate is b = . Because x was al- j;k j;k x These approaches use MDL mo del class selection cri- j;k ready transmitted, it suces to enco de and trans- teria as the basic building blo ck. mit x ;sincex 2f0; 1;:::;x g, this re- j +1;2k j +1;2k j;k 3.1 Binary Multiscale Tree quires log(x + 1) bits. Surprisingly,we nd that j;k Assume that the observed data is a (1D) sequence while enco ding the ML estimate of the parame- N 1 J of counts, fx g , whose length N =2 . Let x k J;k ter, wehave enco ded the data x itself, and k =0 j +1;2k J x , for k = 0;:::;2 1; for j = J 1;:::;0, let us k so no additional co ding is needed. The resulting de ne the multiscale analysis of the data according to: description length is simply j 1 x x + x ; k =0;:::;2 1: (1) j;k j +1;2k j +1;2k +1 : (4) L =log(x +1)= log 1 j;k x +1 j;k The fx g are Haar scaling co ecients; for more de- j;k We transmit the data according to Mo del Class 0 tails on multiscale analyses of Poisson data, see [2, 3]. if L <L , and using Mo del Class 1, otherwise (what 0 1 Now supp ose we wish to transmit the observed wecho ose when L = L is irrelevant). We then de ne 0 1 data fx g. Adopting a predictive co ding approach, J;k the optimal MDL parsing of the data according to the op erating in scale, from coarse to ne, we naturally same criterion, applied at each scale and lo cation. If start by transmitting the total count x ;thiscanbe 0;0 x j +1;2k L <L ,weput b =1=2; otherwise, b = . 0 1 j;k j;k x co ded using Elias' technique for arbitrarily large in- j;k The underlying piece-wise constantintensity(scale tegers [1]. We then progressively transmit the data J ) is reconstructed/estimated from the total counts (in a coarse-to- ne fashion) by next sending x , then 1;0 x and the sequence of splitting prop ortion esti- 0;0 x ;x , and so on. Note that we only need to send 2;0 2;2 b mates fb g. Set x . Next, with j = 1, the scaling co ecients with even k indices; the cor- j;k 0;0 0;0 b b b b resp onding o dd-indexed data are deduced from them = b and = (1 b ). Rep eat this 1;0 0;0 0;0 1;1 0;0 0;0 and the previously transmitted coarser data (e.g., b re nement pro cess for j =2;:::;J, to obtain f g. J;k x = x x ). At each stage, our progressive 1;1 0;0 1;0 Finally,itisworth p ointing out that the same cri- transmission scheme takes advantage of the coarser terion to cho ose between Mo del Classes 0 and 1can data already sent. Formally, we are interested in be obtained under a Bayesian mo del selection per- the conditional probability p(x jx ). It is well j +1;2k j;k sp ective. Let y denote a sample of a binomial ran- known (see [2, 3]) that this probability is binomial dom variable (same as x ) with probabilityfunc- j +1;2k j +1;2k with parameters x and , where tion B i(y j n; ) and consider the problem of decid- j;k j;k j;k j;k ing between two hyp otheses: H : = 1=2, or H : and are the intensities underlying the Poisson 0 1 j +1;2k 6= 1=2 (otherwise totally unknown).