J Math Imaging Vis (2009) 33: 39–65 DOI 10.1007/s10851-008-0104-3 Probabilistic Models for Shapes as Continuous Curves Jeong-Gyoo Kim · J. Alison Noble · J. Michael Brady Published online: 23 July 2008 © Springer Science+Business Media, LLC 2008 Abstract We develop new shape models by defining a stan- 1 Introduction dard shape from which we can explain shape deformation and variability. Currently, planar shapes are modelled using Our work contributes to shape analysis, particularly in med- a function space, which is applied to data extracted from ical image analysis, by defining a standard representation images. We regard a shape as a continuous curve and identi- of shape. Specifically we develop new mathematical mod- fied on the Wiener measure space whereas previous methods els of planar shapes. Our aim is to develop a standard rep- have primarily used sparse sets of landmarks expressed in a resentation of anatomical or biological shapes that can be Euclidean space. The average of a sample set of shapes is used to explain shape variations, whether in normal subjects defined using measurable functions which treat the Wiener or abnormalities due, for example, to disease. In addition, measure as varying Gaussians. Various types of invariance we propose a quasi-score that measures global deformation of our formulation of an average are examined in regard to and provides a generic way to compare statistical methods practical applications of it. The average is examined with of shape analysis. relation to a Fréchet mean in order to establish its valid- As D’Arcy Thompson pointed in [50], there is an impor- ity. In contrast to a Fréchet mean, however, the average al- tant relationship between shape of a biological structure and ways exists and is unique in the Wiener space. We show that its function. To understand the identity, structure and func- the average lies within the range of deformations present in tion of anatomical objects non-invasively, one must study the sample set. In addition, a measurement, which we call a their shape as it appears in images. One of the principal quasi-score, is defined in order to evaluate “averages” com- puted by different shape methods, and to measure the over- problems in medical image analysis is the description of all deformation in a sample set of shapes. We show that the shapes in a way that represents biological variability. This is average defined within our model has the least spread com- often termed shape variation or deformation. Evidently, de- pared with methods based on eigenstructure. We also derive formation has to be explained in terms of correspondences a model to compactly express shape variation which com- between shapes. prises the average generated from our model. Some exam- There has been considerable progress in shape analysis ples of average shape and deformation are presented using during the last two decades. Shape has often been defined by well-known datasets and we compare our model to previous a set of landmarks, significant points on a shape, and which work. are usually selected manually. For this reason, sparse set of landmarks tend to be used. We define a shape space as an Keywords Shape space · Average over a function space · underlying set of shape representations. The representation Wiener measure space · Fréchet mean chosen for a shape space is important, because manipula- tions of shape are determined in large part by how shape in- formation is represented. Several authors have provided pre- · · J.-G. Kim ( ) J.A. Noble J.M. Brady cise definitions of shape and shape space. These include: dif- Dept. Engineering Science, Oxford University, Oxford OX1 3PJ, UK ferential manifolds [4, 12, 13, 20, 27, 33], eigenstructures, e-mail: [email protected] [6, 7], and function spaces [3, 22]. 40 J Math Imaging Vis (2009) 33: 39–65 Kendall has defined shape as “what is left when the dif- ferential manifold of the descriptors. They use eigenstruc- ferences which can be attributed to translations, rotations, ture on the differential manifolds to capture shape variabil- and dilations have been quotiented out” [20]. Kendall’s ap- ity. On the other hand, Klassen et al. [22] built another type proach, using a Procrustean metric, has been a cornerstone of differential manifold on L2. In their work, a single para- for a great deal of shape related research, and adopted in meter, such as a direction function or the curvature of con- many applications, both theoretically and practically. tours, is described on Hilbert manifold. They used a mean in In Kendall’s approach, shapes characterised by land- terms of Karcher [18], in fact a Frechét mean [14]. In their marks form a Riemannian manifold with the Procrustean shape space, analytic expressions for geodesics are not pre- metric; a shape is represented by a point on a sphere. The sented so that their model is difficult to be practical. This Procrustes Analysis that Kendall [20] employed on a Rie- is a common limitation from which differential geometric mannian manifold has been used for Euclidean space in nu- approaches suffer. merous methods. On the other hand, Bookstein [4] consid- Sparr [42–44] formulated both a shape representation and ers triangular shapes (characterised by three points) which a basis for the shape space for a finite number of points, form a differential manifold, a sphere but with a differ- a polyhedron. The shape space is designed to be invariant ent metric from Kendall’s; the Poincaré plane. Both shape with respect to affine transformations, so is named the Affine spaces have been developed and statistics of the result- Shape. Berthilsson and Åström [3] extended Sparr’s idea for ing spaces have been investigated by many researchers, shapes represented by finitely many points to shapes repre- sometimes separately [24, 26, 27], sometimes comparatively sented by continuous curves. [10, 28, 29, 41]. Pennec et al. [33, 34] regards shapes as We contend that methods using sparse sets of landmarks a combination of a feature (such as a point or curve) and are suitable only for shapes that can be characterised by a a transformation (rigid-body). Both the feature set and the small number of landmarks. Some geometric shapes have transformation set constitute differential manifolds, respec- evident landmarks that are sufficient to characterise them. tively, with invariant metrics. Very recently and indepen- For example, the planar shape of 3 pyramids in an aerial dently of our own work, Pennec [32] studies statistics on view in Fig. 1(left) is perfectly captured by the four land- Riemannian manifolds, focusing on Fréchet means. marks shown on each of them. However, this approach is of Cootes et al. [6, 7] use generalised Procrustes analysis questionable relevance for anatomical shapes, such as that [16] and proposed a shape model in terms of an eigenspace showninFig.1 (right), which tend to have very few distinc- by using Principal Component Analysis (PCA). In this ap- tive landmarks. It is precisely this kind of shape that is the proach, a shape is represented by the eigenvectors of the co- focus of this paper. variance matrix of the locations of landmarks, with weights There are widely recognised fundamental problems in as parameters. The weights express shape variations un- methods that depend on sparse sets of landmarks. First, lo- der the assumption that these are independent and follow cating landmarks on images is not only time-consuming; a Gaussian distribution. This approach results in a huge re- but also, especially for noisy medical images, often requires duction of the dimensionality of their shape space. For this expert knowledge. Second, landmarks are not only used to reason, the method is computationally efficient and easy to characterise a particular shape; but are also used to match apply and test. It has been widely applied, especially in med- corresponding points over the whole sample set of shapes. In ical image analysis [2, 8, 9, 53]. methods that use a sparse set of landmarks, mis-represented Fletcher et al. [12, 13] represented shapes in terms of shapes can lead to insufficient or false analysis. For this rea- medial axis descriptions and developed a model on a dif- son, recent work has tended to use denser sets of landmarks. Fig. 1 Landmarking annotated by ∗: planar shapes of 3 pyramids (left) and a femoral head (right) J Math Imaging Vis (2009) 33: 39–65 41 Fig. 2 Femurs: (left) each femoral shape consists of 31 landmarks; (right) every 5 point are marked by + and their average by ∗; the average of others by · However, the fundamental problems persist, since the same istence and uniqueness are not guaranteed. There has been techniques are applied. Also, to overcome the first of these a study of its existence [18] under quite restrictive assump- difficulties, there have been attempts to automate landmark- tions and its uniqueness, with even more assumptions [26]; ing [8, 9, 23, 49] but to date these techniques are limited in but both are limited to a narrow set of a structures in Rie- accuracy and the correspondence problem remains. mannian manifolds. Due to the lack of existence and unique- Methods of shape analysis increasingly tend to be devel- ness of a Fréchet mean, one needs more than one term for oped within a statistical framework. In such methods, land- a mean and has to distinguish them as in [28] (a mean of marks are labelled points (locations) as shown in Fig. 1. shapes, the shape of a mean) and [33] (a mathematical mean, After being filtered by Procrustes analysis, corresponding empirical mean). landmarks are grouped according to a set of labels. That is, a set of corresponding landmarks across all shapes of a 1.1 The proposed model sample set is formed for each label.