COMPUTER VISION, GRAPHICS, AND IMAGE PROCESSING 35, 283-305 (1986)

Introduction to

JEAN SERRA

E. N. S. M. de , Paris, Received October 6,1983; revised March 20,1986

1. BACKGROUND As we saw in the foreword, there are several ways of approaching the description of phenomena which spread in space, and which exhibit a certain spatial structure. One such approach is to consider them as objects, i.e., as of their space of definition. The method which derives from this point of view is. called mathematical morphology [l, 21. In order to define mathematical morphology, we first require some background definitions. Consider an arbitrary space (or ) E. The “objects” of this :space are the subsets X c E; therefore, the family that we have to handle theoretically is the set 0 (E) of all the subsets X of E. The set p(E) is incomparably less arbitrary than E itself; indeed it is constructured to be a Boolean algebra [3], that is: (i) p(E) is a complete , i.e., is provided with a partial-ordering relation, called inclusion, and denoted by “ c .” Moreover every (finite or not) family of members Xi E p(E) has a least upper bound (their C/Xi) and a greatest lower bound (their intersection f7 X,) which both belong to p(E); (ii) The lattice p(E) is distributiue, i.e., xu(Ynz)=(XuY)n(xuZ) VX,Y,ZE b(E) and is complemented, i.e., there exist a greatest set (E itself) and a smallest set 0 (the empty set) such that every X E p(E) possesses a Xc defined by the relationships: XUXC=E and xn xc= 0. Given two sets X and Y E p(E), the notion of their set diference X/Y derives from the intersection and the complement, as follows:

X/Y=Xn YC;

X/Y is the part of X which does not belong to Y (see Fig. lc). In brief, given two sets B and X belonging to p(E), we may have (Fig. 1): (a) B is included in X (notation: B c X) (p) B hits X (notation: B fi X) meaning B n X # 0 (y) B misses X (notation B c Xc) meaning B n X = $I and these relationships will apply when B is replaced by every family {B;). Finally, the structure of a Boolean algebra provides the general framework on which we shall perform morphological treatments. Mathematical morphology is the application of lattice theory to spatial structures. 283 0734-189X/86 $3.00 Copyright $’ 1986 by Academic Press, lnc All rights of reproduction in any form resrrwd 284

FIG. 1. (a) B, hits X (B, R X); E, misses X (B2 C Xc); B, is included in X (B, C X); (b) complement X’ of set X; (c) difference X/Y of the two sets X and Y.

These definitions may seem rather abstract and far away from the practical applications. As a matter of fact, we shall pursue the discussion by concentrating mainly upon the R” with dimension n = 1,2, or 3, or its digital version Z” in terms of grids of points. Pedagogically speaking, the Euclidean case is the first to be considered, since it corresponds to the physical world in which we live; on the other hand, many images are reasonably binary, at least in a first approximation (such as microstructures (Fig. 10) biological cells, (Fig. 11) seg- mented zones in remote sensing, etc.). However, to restrict the approach to the Euclidean sets would mean ignoring other important domains to which the method applies. Let us briefly quote, for example:

-functions of W” considered as sets of R x Wn-‘, via their umbrae (see [25]); -planar graphs, such as the partition of a map into counties [4]; -products of spaces, when several images are defined at each pixel. The conditional operations introduced in Sections 5 and 6 below illustrate this mor- phology. They result in context dependent transformations; -vector spaces, such as propagations, when a range of directions is of each pixel [5]; -topological spaces which are necessary for introducing probabilities, and for treating the questions of robustness [l]; -etc. All in all, these various extensions indicate that we have to keep in mind two different levels of generality: the Euclidean (or digital) binary case, which is useful and gives a good intuition of what is going on, and the Boolean algebra associated with the general framework of the method.

2. MORPHOLOGICAL TRANSFORMATIONS

2.a. Hit-or-Miss Transformation, , The final goal of computer vision is often to segment images into objects and textures in accordance with the judgment of the human eye. But this is not always the case. It may happen that slight nuances escape human perception although they INTRODUCTION 2X5

FIG. 2. (a) Cat brain cortex after hypoxia. The human eye does not detect the slightest difference with the normal state. Nevertheless, by investigating one thousand neurons by morphological openings, significant differences are exhibited [6]. (b) Sandstone rock from the oil reservoir of Hassi-Messaoud (Sahara): how does one predict the permeability of this material from its texture?

are significant (see Fig. 2a). Moreover, the purpose of an image treatment may not be to match vision, but to estimate some physio-chemical properties. In the case of Fig. 2b, the Navier-Stokes equation, which governs the physical process, corre- sponds to nothing intuitive. Therefore, we do not look for a computerized substitute for human vision, but for a coherent framework for describing spatial organization. To this end, we define the structure of an object by the set of the relationships existing between the various parts of the object. We will study the structure experimentally by trying each of the possible relationships in turn, and examining whether or not it is satisfied. Of course, such knowledge will greatly depend on the choice made for the system of relationships considered possible, and this a priori choice determines the relative worth of the resulting concept of structure. We saw that the media under study may have no recognizable pattern. This lack of meaning (for us) leads to the position of probing them systematically, by starting from the simplest relations that one can imagine. From this comes the idea of a . With each point x of the space E in which we work, we associate a set B(x) called a structuring element. (Note that B(x) may vary from one place to another). We can modify every set x E $I( E) by some B(x) in several ways. The most important ones are as follows:

dilation of X: (x: B(x) fl X} (1)

erosionof X: {x: B(x) C X}. (2)

The dilation of X by B(x) is the set of all the points x such that B(x) hits X. The erosion of X by B(x) is the set of all the points x such that B(x) is included in X. Starting from two structuring elements B’(x) and B2(x) we also define the 286 JEAN SERRA hit-or-miss transformation (in brief HMT) as being the difference of X eroded by B’(x) and X dilated by B2(x).

HMT of X: (erosion of X by B’)/(dilation of X by B2).

Dilation and erosion turn out to be particular cases of HMTs. The class of transformations generated by the (possibly infinite) unions, products, and complementations of HMTs constitutes, by definition, the morphological truns- formations over p(E). Of course, these operations are not the only pieces of information; in particular, they can be combined with measures on p(E) (e.g., the area or the volume of sets in the Euclidean space) or with other types of operations (convolution, for example).

2. b. Basic Properties First of all, a general comment: all of the morphological transformations are non-reversible, (except, in each particular case, for some subclasses of sets such as the invariant ones). In fact, the idea of restoring the images is quite irrelevant here; on the contrary, our philosophy will consist of stating that the images under study exhibit too much information, and that the goal of any morphological treatment is to manage the loss of information through the successive transformations. In order to do this, we must play with a few underlying general properties which are the key to any morphological analysis. We now present the four most important ones. In what follows, $ is the generic symbol of a morphological transformation.

(i) Increasing: 4 is increasing when it preserves inclusion, i.e., when xc y=, 44x) = J/(y) ‘dx,YE b(E). (4)

For example, the transformation “replace X by its boundary JX” is not increasing. (ii) Anti-extensiuity: r/ is anti-extensive when it shrinks X, i.e., when J/(x)= x Vx E b(E). (5)

(iii) : # is idempotent when the result q(X) remains unchanged if we reapply the transformation, i.e., when

vwxN = W) vx E b(E). (6)

(iv) Homotopy: Here the set E is considered as a topological space. However, for the sake of simplicity, we limit ourselves to the bounded sets of the plane. With each bounded set K, associate its homotopy tree whose trunk corresponds to the background K, (i.e., the infinite connected component of Kc), the first branches corresponding to the connected components K1 of K adjacent to K, and the second branches to the pores K, of K adjacent to K,, etc. (see Fig. 3). INTRODUCTION 287

FIG. 3. Two homotopic figures and their tree.

A transformation is homotopic when it preserves the homotopic tree of K. Note that homotopy is more severe than connectivity: a disk and a ring are both connected sets, but they are not homotopic. These four properties are the corner stones for a classification of the morphologi- cal criteria. The reader must always keep them in mind if he wants to master the method. However, they are far from being the only ones; one may also wish to treat figure and background symmetrically, to perform isotropic analyses, to preserve connectivity, to be monotonic at each pixel, etc. Table 1 lists the status of the four properties for the basic Euclidean morphologi- cal transformations and a few other geometrical transformations (displacements, convex hulls, etc.) introduced in this paper and accompanying papers in this issue.

TABLE 1 Euclidean Morphological Criteria and Their Basic Properties --__-. Properties Extensive (or anti- Preserve Criteria Increasing extensive) Idempotent homotopy _I- - -~.-~ Complementation, Hit-or-Miss, No NO No No boundary Erosion and dilation when the Yes No No No origin is not contained in the structuring element, median filtering Thinning and thickening No Yes No No Erosion and dilation when the Yes Yes No No origin is contained in the struc- turing element Boundary of the boundary No No Yes No Projection, morphological filtering Yes No Yes No Sequential thinning, &ix, skeleton, No Yes Yes No cond. bisector watershed, ulti- mate erosion , , convex hull, size Yes Yes Yes No distribution, thresholding, umbra Displacement, similarity, al&ity, Yes No No Yes symmetry Homotopic and sequential thin- No Yes Yes Yes ning and thickening 288 JEAN SERRA

2.~. Morphological Filters As an example of a type of morphological transformation, we introduce the notion of a morphological jilter. In signal processing in one or several dimensions, one usually calls a any Euclidean operation which is linear, translation invariant, and continuous. According to a well-known result [8], every filter turns out to be the convolution product f *QJof a signal f by a (generalized) function ‘p. More in practice than theoretically, a fourth property also holds: the filters are often implicitly equivalenced to band-pass filters, i.e., to processes which are applied non-iteratively. A high frequency signal cut off above 400 kHz by a first filter should theoretically not be modified by a second filter identical to the first. This fourth filter property corresponds exactly to what we just defined by the property of idempotence (Eq. (6)). A transformation 4 is linear when +(f + f’) = 4(f) + (p(f’). Such a condition is apparent when the structure of the physical phenomenon is itself linear. In case of human perception of sounds, the ear adds the (log of) signals stemming from various sources. In image analysis, when a photograph is blurred due to bad focusing or motion of the camera, the blurred image is the sum of a series of unblurred images. The transformations necessary to restore the blurred image must be linear, as was the distortion process. However, although many signals combine additively, visual signals obey a very different law of composition. The physical world around us is not transparent. It is generally made up of opaque objects and the nearest object occludes the ones located behind it. Therefore, the first prerequisite a morphological filter (or M-filter) 4 must fulfill is to preserve the relations of inclusion which may exist between every pair of objects, i.e., to be increasing instead of being linear (the two notions are incompatible). Unlike convolution, which often leaves the information content of an image unchanged, the increasing operations always reduce it. To block this simplifying effect, we introduce a second condition, and demand the property of idempotence from M-filters. In the general case of a space E, there is no other requirement; in the Euclidean cases we add continuity and translation invariance, just as for the linear filters. It is remarkable that, based upon only these two conditions of increasing and idempotence a very powerful theory of filtering can be constructed. Some of its features differ from those of the convolution products. For example, M-filters may commute with anamorphoses (the log of a filter is the filter of the log), they may specifically modify the positive contrasts in a gray tone image while ignoring the negative contrasts, they may preserve the vertical walls (very high gradients) in a grey tone image while smoothing the rest, they may remove some specific features, and leave unchanged the rest of the image, etc. As an illustration of M-filtering, consider this analogy of the procedure of “cleaning” an image. My hands are dirty, I wash them, they become clean. If I wash them once more, they remain unchanged since they are already clean: the operation is idempotent. Moreover, soap has suppressed some dust or spots, but nothing is added: the process is anti-extensive. Finally, if I wash my left hand, the limited result I obtain is part of the complete cleaning process: the operation is increasing. Hence, every transformation satisfying these three properties provides a cleaning technique (note the lack of homotopy requirement). Transformations of this type INTRODUCTION 289 are openings, and of course, they turn out to be a particular kind of M-filtering. The anti-extensivity that we have introduced will, in addition, be extremely useful when dealing with grey tone functions. It guarantees that the difference “initial image minus filtered one” is always positive, hence is a grey tone image. Both Sternberg’s rolling ball algorithm and Meyer’s top hat algorithm use this property (see [25,26]). However, the M-filters need not be anti-extensive (or extensive). An example is the iterative M-filter given in Stemberg’s paper [25]. Other M-filter types, satisfying special properties such as self duality, can be found in [9, lo., and II]. 2.d. Duality with Respect to Complementation Morphological transformations go by pairs; as soon as we define a #, we also introduce, by the same act, another transformation #*, called the dual of #. Indeed, the operation 1c,which transforms the set X into the set q(X) also works on the background Xc, since Xc becomes [q(X)]“. This latter operation is the dual #* of # with respect to the complementation, and we have by definition, It/*(x) = bwcr. (1)

In particular # may depend on a given set (structuring element) or on a scalar parameter, and X may represent several sets (i.e., a set in a product space). Similarly, the notion of duality is valid for a logical relation R. For example, the dual form of the relation “B included in X,” with B a given parameter, is “B hits X ” (the complement of B included in Xc). The duality is less a property of 11,than a general and direct consequence of the decision to work on sets. The practitioner has to grow into the habit of systematically thinking in terms of the dual form of the processes he carries out. Further, experience shows that such an attitude is rather unnatural. Although Minkowski studied a set operation close to dilation, the notion of an erosion is practically absent from his work. In processing, thinnings have been much more commonly used and studied than thickenings. Several recent attempts at generalized homotopy to grey tone function are unsatis- factory due to the lack of duality. If the workers in digital treatment had really understood the meaning of duality, they would not have put such a strong emphasis on the use of the square grid, which leads to contradictions (see Sect. 7.c below), etc. How are the four key properties modified by duality? Obviously, anti-extensivity, idempotence, and homotopy become extensivity, idempoteace, and homotopy, respectively. But if # is increasing, so is #* (and not decreasing). Appendix 1 contains a table of the notions corresponding to each other by duality. The majority of them have not been introduced yet; they will appear in the various parts of this issue.

2.e. The Program of Mathematical Morphology By itself, the HMT possesses practically no interesting properties except for one, namely that of being the logical ancestor of all the other transformations involved in mathematical morphology. Figure 4 illustrates this point. The two branches of thinnings and erosions put the emphasis on the anti-extensivity and the increasing properties, respectively, although each property is not specific to the branch. The third branch (parameters) deals with questions of integral and of stereol- ogy, which we do not develop any further here. 290 JEAN SERRA

FIG. 4. Classes of criteria and algorithms in mathematical morphology (the dual operations, e.g., “dilation” for the erosion, are not indicated).

The set of the items indicated in Fig. 4 has to be considered as one input of a matrix, the other input corresponding to methodological questions such as:

-Which spaces are worth considering (Euclidean, various grids, graphs . . . )? -What is a criterion when applied to grey tone functions? -For which type of sets is it defined? interesting? robust? -What are its digital versions? isotropic versions? probabilistic versions? -What are its stereological implications? -What about its local and global versions and the associated sampling problems? -What are its particular properties when used on subclasses of sets exhibiting simpler shapes (i.e., convex sets, balls, etc.)?

The theoretical program of mathematical morphology is not only to extend the tree of Fig. 4, but also to fill up the spaces in the matrix, i.e., to study each methodological question with regard to each criterion. The practical problem is of course to produce efficient programs for image analysis, which are based on combinations of the criteria. In this issue, we limit ourselves to a condensed presentation of some criteria, a few examples, and brief comments on some methodological aspects. The random sets have already been presented in [27] and so are deliberately ignored this time. INTRODUCTION 291

b

‘

FIG. 5. (a) Homothetics XX of set X: (b) sets X and Z and their translates X, and Z ,2: (c) transposed set X of X.

3. EUCLIDEAN MORPHOLOGY 3.a. Euclidean Space From now on, we specifically take for the space E the Euclidean space and/or its digital versions, and continue by exploring the two major branches of the tree represented in Fig. 4, associated with the increasing (Sects,, 4 and 5) and the (anti-)extensive (Sect. 6) operations, respectively. The last section (Sect. 7) will be devoted to the matching of grids versus continuous spaces. Our Euclidean space is exceptional, for it remains unchanged under the oper- ations of translation, scaling, and symmetric reversal, and because it is dense in points and in sets. Its extremely rich structure will enable us to make the morpho- logical transformations suitable to handle it more precisely. But before this, let us introduce some notation. Given a set X, its translate by vector h, its homothetics after magnifying by the scalar h and its transposition (i.e., its symmetrical set with respect to the origin) are written X,, XX, and 2, respectively (Fig. 5). Symbolically,

X,=(x:x-hEX};XX= ( x :;EX)

k= {x: -XEX}. (8)

Remark. In the notation X,, h represents a vector in 2D or in 3D and not a coordinate, although A, in XX, is a scalar. 3. b. Euclidean Morphological Principles Intuitive reasoning can be specious; one might guess that “quantitative” is synonymous with “numerical.” Hence, to obtain a quantitative description of a set, it should be sufficient to extract from it sufficiently representative parameters (area, perimeter, number of objects.. . ). In fact the quantization here is a 2-s&p process. Let us take an example from physical sciences. In sieving analysis, when we want to know the size distribution of, say, the result of the milling of rocks, we generate a sequence of subsets of the milled product, i.e., for each sieve size, the is the corresponding oversize (that which is left after sieving); then we weigh the oversizes. There are an infinity of such examples we could present; they always proceed in two 292 JEAN SERRA steps: geometrical transformations and then measurements. Anyone who wants to quantify morphology correctly must take into account the existence of these two distinct steps. Here we concentrate only upon the first step (the second one is more classical [12, 131). How can suitable transformations be defined and selected in every particular case study? First of all, they cannot be completely arbitrary and must satisfy at least three general prerequisites that we shall call principles. Consider a transformation #, i.e., a criterion for changing a set X into a new set #(X): (i) 1c, may depend on the origin 0, and hence be denoted +“, #h being the transformation which applies to the point x the criterion formerly defined at point x - h. For example, take for r/~” the intersection of X with a given measuring mask z:

+O=xnz (where qh = X n Z,).

In this particular case, we see from Fig. la that it is the same (up to a translation) to shift the set X by vector h, as it is to shift Z by -h. More precisely, X, n Z = (x n z-h)+h.

We shall take the same property, written for the general transformation #, as the first euclidean morphological principle

$“txh) = [+-hl h’

Here we say that qh( X) is translation compatible. When # does not depend on the origin, i.e., is translation invariant, relation (9) becomes 4(X,) = [$( X)lh, which simply means that the transformation commutes with the translation. (ii) Transformation $ may depend on a scalar factor, say J/x. Here is a biological example. Consider a population of cells (set X) and take for qx(X) all those cells whose areas exceed 100 p2 (here X = 100 p2). Obviously such a # does not depend on the magnification under which we are working. We have

wo = WlWV (10) which states in formal terms, that $,, is compatible with the magn@cations. We shall systematically admit only those transformations Gx for which condition (10) holds (a counterexample: \clx extracts all the cells for which “area = X . perimeter”). When $J does not depend on any scalar h, i.e., is invariant under changes of scales, then relation (10) becomes X$(X) = $( XX), i.e., J, commutes with the magnifica- tions (e.g., a skeleton). (iii) The third prerequisite concerns only the objects that we cannot insert in a measuring mask, say Z, such as a TV frame. When we look at a connected medium (porous medium, bone, dendritic structure . . .) the edges of the mask Z limit the object under study. Under those conditions, we can only consider those transforma- tions # for which the resulting \c,(X), taken in a bounded mask Z’, comes from a bounded zone Z in which we know X. In symbolic terms, Vbounded Z’,gbounded Z: [#(Xn Z)] n Z’=+(X) n Z’. 01) INTRODUCTION 293

A transformation or an algorithm satisfying (11) is said to be local; if not, it is global. Instead of “local,” the expressions “piecewise” or “regional” are sometimes used, with slight differences. Remarks. (i) Such principles must not be considered as dogmas, but rather as ways for selecting “good” transformations 1+5among a maze of possibilities. (ii) The first two principles reflect the fact that we are working in Euclidean spaces. In this sense, the third principle is more general and applies when the working space is unbounded. Note that it also introduces a certain type of robustness. (iii) Principles 1 and 3 are immediately transposable to grids of points. Principle 2 may only be an approximation, the quality of which is estimated by fitting with the underlying Euclidean space (see Sect. 7). For example, increasing digital disks or balls cannot be strictly similar.

4. EROSION AND DERIVATED NOTIONS The results presented in this section are valid in both Euclidean space R’ and grid Z “, in n dimensions (n = 1,2,3,. . . , or more).

4. u. Dejhition Consider the particular but important case of the HMT, wh!ere B*(x) = +, that we have called an erosion (I!!$. (2)). The first principle of Euclidean morphology (Eq. (9)) implies that all the B(x) involved in (2) are identical, up to a translation. More precisely, if B = B, denotes the structuring element associated ,with the origin, then the eroded set Y turns out to be the locus of the points x such the translate Bx of B,, by x is included in set X:

Y= (x: B,cX}.

This transformation looks like the classical Minkowski substr,action X 8 B of set X by set B used in integral geometry [14] and defined as the (possibly infinite) intersection:

XeB= nX,,. hGB

Indeed, when for each y E B,, the point x + y lies in X iB x belongs to the translate X _ ). of X, i.e.,

y={x;B,cX}= n X-,= n x,.=Xeil (12) Y~BO -.YE&

where 5 = U { -y } is the transposed set of B, i.e., the symmetrical set of B with YCB respect to the origin. Consider now the dual operation, which describes what happens to the complementary set Xc (i.e., the “pores”) when eroding the set X (the “grams”). We shah call this operation dilution, and denote it by the symbol @ . According to (7) we have XQB=(XeB)‘. (13) 294 JEAN SERRA

FIG. 6. Two different B’s extract different features when eroding the same X (the arrow indicates the location of the origin in the structuring element).

Just as with erosion, dilation has a local interpretation, since the complement of the proposition “Bx is included in the pores Xc,” is the proposition “Bx hits the grains X.” Thus the dilate X @ fi is the locus of the centers of the B, which hit the set X (see Fig. 6):

xeaii= {x: B,f-lX# 0) = {x: BJjX}. 04

By applying (12) and (13) to the pores Xc, we find an analytical expression of the dilation of X by B:

From the last equality of (15) we recognize the , classical in integral geometry [14]. A brief example will clearly show the difference between X @ k, and X $ B. Take for X as well as for B the same equilateral triangle of side a. Then X @ B is the equilateral triangle of side 2a, although X @ B turns out to be a regular hexagon of side a (Fig. 7). In practice, a large majority of the structuring elements are symmetrical, i.e., B = ii, and the hats may just be ignored.

FIG. 7. Difference between the dilation by B and by B. INTRODUCTION 2%

4.b. Algebraic Properties of the Erosion

(i) Basic properties. Erosion is not idempotent, nor does it preserve homotopy, When the origin 0 belongs to B, it is anti-extensive. In fact its major property is to be increasing and we have XeBcX’eB QB, XcX’- ( X@BcX’@B QB, and BcB’==X~BIX@B QX.

(Note that the erosion is decreasing with respect to B.) ThLe logical link between erosion and the increasing property turns out to be extremely strong, since the theory shows that every transformation which is increasing and translation invariant is necessarily a union of erosions, or equivalently an intersection of dilations [l]. There- fore, erosion appears as the prototype from which all the increasing morphological mappings, such as the M-filters, derive and can be built. (ii) Distributivity. Dilation distributes the union. We can write XCQBUB')=(X@B)U(X@B'I and by duality

The technological implications of the three relations are important. From the first two, we can dilate, or erode, X by taking B piece by piece and then combining the intermediate results by unions or intersections, respectively. The third formula is more basic again and solves the problem of the local knowledge (take Z for the measuring mask, then Z 8 B is the zone in which X 8 B, and therefore X 8 B, are known without error). (iii) Zterativity. We have

(Xe B) 8 B’ = Xe (B @ B’) (X@ B) 8 B’= XCB (B@ B’).

These two basic relations allow us to easily decompose squares, cubes, hexagons, and rhombododecanedrons into series of two, three, and four segments, respectively PI- 4. c. Example Wood anatomy is structured by cells that form a skeleton of vertical rigid walls surrounding cavities that are different in dimension and shape. At a ten micron scale, we see the fibers (the smallest cells), lined up in strings. They have a direction parallel to the rays (the long slim cells); this direction is the one which radiates from the trunk center (Fig. 8). Both these series of cells induce a strong anisotropy in the wood structure. At a hundred micron scale, the vessels appear round and isolated, 296 JEAN SERRA

FIG. 8. (a) Transverse section of Aucoumea ( x 23, i.e., normal to the trunk, with vessels structure, fibers, and rays: (b) variograms of Aucoumea.

big tubes conducting the sap. Then they are more or less regrouped in small clusters, and, at a millimetric scale, we can see their density slowly change. At the same scale we can also observe slow variations of darkness in the fibers; these variations, called “tree-rings,” are due to a regionalized flattening of the fibers and a thickening of their walls. The wood anatomist asks the morphologist for an overall description of the texture, in order to correlate it with some physical properties of the material (anisotropy of shrinkage during drying). Here, only the vessels are structured as isolated particles (in sections), the others, the rays and the rings, the walls and the fiber strings, look much like a continuous medium. Thus we need an investigation: (i) independent from the notion of connected particles, (ii) involving a directional character, which is required by the structural anisotropy, (iii) able to scan the various scales from the fiber strings to the year rings. The simplest structuring element able to satisfy these three requirements is a pair of points. The measurement associated with this is either the covariance or the variogram. INTRODUCTION 297

The stationarity of the material under study X (in white on Fig. 8a) suggests a probabilistic approach. For B, take the pair of points (x, x -C h) which are the ends of the vector h (]h], a). The area percentage of X 8 B is just the probability C(h). called covariance, that both x and x + h lie in X: C(h) = Prob{x E X,x + h E X}. (161 We may use this covariance, but it treats foreground and background in an asymmetrical way. On the other hand, for statistical reasons (21, the estimates of the covariance are not independent from the sizes of the measuring fields, and are shifted upwards when the latter increase. We shall overcome these two troubles by taking the variogram y(h), i.e., the probability that one of the two points of B, falls in X. and the other in Xc: y(h)=:Prob{x~X,x+h~X’ or ~EX~,.X+~EX} i.e.,

y(h) = Prob{x E X, x + h E X’} since y(h) = y(-h). (17) Obviously, we derive from (16) and (17) that

Y(h) = w - C(h) and, in particular, y(o) = 0. The experimental variograms have been computed in 20 fields similar to Fig. 8a, for both directions u = 0 (tangential) and (Y = r/2 (radial). The two average curves are plotted in Fig. 8b. They reveal four kinds of features: (i) The slope of the tangent at the origin is equal to the number of intercepts of X per unit of test line in the considered direction (Y. Practically, the average of the two slopes for (Y = 0 and (Y = m/2, multiplied by ~7. a (a = spacing of the digital grid) equals the perimeter of X per unit area. (ii) A strong growth from 0 to 15~, followed by an oscillation called “hole- effect,” which corresponds to a pseudo-periodicity of the fibers. The abscissa of the minimum gives the average distance between fibers in the described direction. The difference of ordinates between minimum and maximum designates the degree of regularity in the periodicity of distances between the fibers. (iii) The variograms do not reach a sill, but keep on growing slowly. This phenomenon proves the existence of macrostructures due here to the vessels. Similarly as in (i), the slope of this growth, multiplied by ?ra, equals the perimeter of the vessels, per unit area. (iv) When comparing the two variograms, one may notice that they are nearly identical, up to an additive constant d. This loss d of the ordinate in the radial direction, called “zonal anisotropy,” simply means that when a couple of points B fall in a ray, the two points are likely to stay in the cavities, even for high values of h. If p is the area proportion of the rays, we have d = p(1 - p). Finally, without having to segment the various components of the texture, we can give a lot of information about them, just by measuring parameters on the variograms [ 151. FIG. 9. (a) Thin section of a human lung. The presence of a vessel disturbs the measurement of the alveolae perimeter. (b) An opening of the background (i.e., closing at the alveolae) allows a correct computation. (Photographs taken on the monitor of a LEITZ T.A.S.)

5. OPENINGS AND CLOSINGS

5. a. Dejinition Having eroded X by B, it is not possible, in general, to recover the initial set by dilating X 8 B by B. It turns out that the new set actually filters out a subset of X which is extremely rich in morphological and size properties. We define X,, the opening of X with respect to B by

X,=(Xeii)@B 08) to which corresponds by duality the closing XB of X by B

XB = (XCB ii) 8 B = [(Xc)&

These two notions, introduced by Matheron [7] are among the most powerful of mathematical morphology. Cl~ly, a point z E, X, iff & jj,

X,cXcXB (anti-) extensivity XcX’-XBcX+ndXEcX’Bx increasing (20) (X,), = X, and(XB)B = XB idempotence.

In algebra, the operation “opening” is axiomatically defined by the above three properties (20) of the morphological opening (thus explaining the origin of its INTRODUCTION 299 name). Here again, the theory shows that every set transformation # is an algebrair opening and invariant under translations @it is a union of openings I) = U XB,, where the Bi’s, their translates and their unions generate the class of sets invariant under $ [l]. In other words, all the anti-extensive M-filters (cleaning procedures) derive from the morphological opening.

5.b. Openings and Size Distribution Recall the example of sieving introduced in Section 3b, and consider the oversize #&( X) of an initial set of particles, say X, for a sieve of mesh X. Obviously,

(9 $h( X) is a subset of X, (ii) if Y c X is a subpopulation of X. we have #x(Y) c J/h(X); (21) (iii) if we iterate the sieving, we do not modify the result: $h[#X( X)] = #h(X).

Hence, the three properties (21) are satisfied, and qx is an algebraic opening. Indeed, the sieving $x possesses another property, which is :more demanding than the simple idempotence. Take a second sieve, with a mesh p, and re-sieve the oversize Gx, by using the p-sieve. If p I X the oversize +L,, is unchanged; if p > X, then $h becomes the same as if we had directly sieved the initial set X by the p-sieve, i.e., #h becomes #p. Symbolically,

(iv) (22)

Following Matheron [l] we shall call size distribution every family of set transfor- mations depending upon a positive parameter h > 0, and satisfying the three properties (i), (ii), and (iv) above. As a matter of fact, it is easy to check that each time somebody speaks of size distribution, in the common language, as well as in physics, these three axioms are underlying. Now, if we add the three euclidean principles to the three sizing axioms, we restrict the range of the sizing processes. More precisely, we find Iclx(X) = U XXR,, where the Bi’s, are compact convex sets (eventually in an infinite number) [l]. In other words, more heuristically,

openings + convex * Size distribution in Iw n struct . elements

Note that we have carried out this analysis without using the notion of connectiv- ity. We can perfectly well speak of size distribution for a connected medium. For example, in two dimensions, reduce the collection of the Bi’s to the unique unit ball B (then #x(X) = X,,) and consider the two micrographs of Fig. 10. Perform the successive openings X,, Xzs, X3B, etc. on the white phases and plot, versus h, the area proportion which vanishes between X,, and XCx+ijB. This quantity is neces- sarily positive and the total integral of the curve equals one; it is certainly a density function. A size has been given to each point x of X (the whites), namely the radius of the largest disk containing x and included in X. In the case of Fig. 14a, we see 300 JEAN SERRA

b

d

FIG. 10. (a) Ferrites (in black) in a phenomenon of phase transformations; (b) and (d) density function of opening performed on the whites of Figs. (a) and (c); (c) sinter (porous medium in black). three maxima, i.e., three scales of ferrite packings: the narrow stripes between aligned crystals, the clusters of a few ferrites, and large empty zones. In the second case, the uniformity of the growth is translated by a uni-modal curve. An interesting feature is the flat beginning of the function. According to (22) this means that the whites are already physical& open by the sintering process: 2~ measures this effect.

S.C. Conditional Dilations and Erosions Despite their names, these transformations can be interpreted in terms of alge- braic openings. We now consider two sets X and Y, the latter being called the marker of the former. In the case of Fig. (II), Y is included in X, but it is not necessary. We can easily generalize the basic morphological operations, and make them hold on the product space p(W”) x p(R”) (or p(Z”) x p(Z"), by introducing a pair of structuring elements B, and C, acting on X and Y, respectively. Then the cross HMT is written (Y $ B) n (X 8 C).

FIG, 11. (a) Set X and its marker Y (Y = nuclei (in black), X = cytoplasms (in grey); (b) extraction of the cytoplasms containing a nucleus, which is convex and large enough. INTRODUCTION 301

The most useful case is when B is a ball and C the origin. The corresponding HMT is called conditional dilation of Y in X and denoted by

Y@B;X=(Y@B)nX. (23) By iterating the operation, we generate an increasing sequence whose (possibly unbounded) limit is written Y 83 {B}, X:

Ye {B};X= -.-[[[(Y@B)nX] @B] nx] @B ... . (24)

This sequential algorithm serves when X = Ui Xi is the union of disjoint bounded particles X, which remain disjoint after dilation by a small .ball B. Then the limit (Y 83 { B } ; X) is made of all the Xi’s which hit Y. In the example of Fig. (ll), we used it for extracting all the portions of cytoplasm from a cytological smear which exhibit a nucleus. But the same algorithm may be used for other purposes [2]: -if Y stands for the edges of the measuring field, it separates the particles strictly included in the field from those which hit the edges. -if Y stands for those particles which did not disappear after a given erosion. it filters out the smallest ones, etc. In every case, the transformation X + Y @ { B } ; X, where: Y is a fixed parame- ter, is an algebraic opening in X. Unlike the pure morphological openings, this one is related to the connectivity of the Xi?. In the digital case, B stands for the elementary square (or octagon, or hexagon), and the operation has a Euclidean interpretation if X belongs to the regular model (Sect. 7).

6. THINNING!5 AND THICKENINGS The thinnings and their connections with digital skeletons have been extensively studied [16,17]. In what follows, our purpose is only to insert these notions in the range of the morphological concepts and not to survey the question.

6.a. Dejinitions and Properties We now change the branch on the tree of Fig. 5; instead of insisting on the increasing property as we did with erosions, we will completely Iignore it and put the emphasis on the extensivity. Given the structuring element 2” = {T’, T2} define the thinning, XOT, by the set difference between X and its hit-or-miss transform X 0 T, [2] (see Fig. 12): XOT=X/X@T=Xn[(Xef')n(Xce2'2)]c. (25) The corresponding dual operation is called thickening X0 T of X; we have, by definition, (X0 T)C = X”OT* whereT* = (T2, T’}. (26) By construction, the thickenings are extensive and the thinnings anti-extensive, More precisely, if T’ c T means T" c T' and Tt2 c T2, we.may write T'c T- XOT'c XOTc Xc XOTc XOT'. The more comprehensive the structuring element is, the less it removes points of X. In particular in E2, with a hexagonal grid, if T' and T2 belong to the unit 302 JEAN SERRA Tz ...... k ...... ‘.. ,’...... -J-.._. ._.._.., ...... TI X X@T XOT

FIG. 12. Examples of a hit-or-miss transform and of a thinning; when T’ contains the origin, we get X= XOT. hexagon H, the thinning may suppress only points of the boundary X/X 0 H (Fig. 12). Sequential thinnings. We will provide the thinnings with the property of idempo- tence by introducing sequences of thinnings. Let q = { ql, q’} be a sequence of pairs of compact sets. If q is infinite, the sequential limit

is idempotent. We shall denote it by X0{ T;}. In particular the standard sequence, where Ti+, derives from q by a rotation of 60”, is denoted by {T), without the subscript i. Conditional thickening. Exactly as with the conditional dilation (23), we can thicken Y within a limiting set X, the operation is then called conditional thickening (of Y with respect to X) and is denoted by

YOT; X = (YOT) n X. (28) By duality we obtain the conditional thinning YOT; X = (YOT) U X. The associated sequential forms derive immediately

YO{rr;:}; X= -.- ([ a-. ([(YOT,) n X]oT2) n X . ..I. (29) Homotopic thinnings. We shall limit ourselves to regular planar sets of Theorem 2. Hence, we can work on the digital graphs which are homotopically equivalent (hexagonal grid with unit hexagon H). Moreover, if we demand that (T’ U T 2, c H, a condition consistent with the “Golay alphabet” [18], then the only structuring elements which preserve the homotopy are

Therefore the standard sequence { L } (as well as { M }, or { D }) defines a sequential thinning (resp. thickening) X0 { L } (resp. X0 { L }) which is anti-exten- sive, idempotent, and homotopic (resp. extensive, idempotent, and homotopic). 6. b. Example We borrow this example from a study of dendritic structures [2,19]. This time the image understanding is much easier than the feature extraction. We want to measure the number and the length distribution of the defect lines shown in the dendritic FIG. 13. (a) Transverse section of lamellae; (b) extraction of its defect lines structure of Fig. 13a. The passage from Fig. 13a to Fig. 14b is not trivial since we want to extract a feature whose main property is a disappearance of a feature rather than an appearance. The steps of this automatic extraction arl:: given in Fig. 14. In spite of its sophistication, the procedures of prunings and skizing (steps (c) and (e). respectively) make it particularly robust; indeed it has been used routinely on several hundred fields such as Fig. 14a, without trouble.

7. CONCLUSION: CONCEPTS AND MODES OF OPERATION There are many ways for performing morphological operatio.ns. For example, one can use cellular logic [21], where each pixel is compared to its immediate neighbors. This technique was implemented in many devices, such as the Leitz Texture Analyser, the Colter Diff 3, or the ERIM cytocomputer, among others. For binary

FIG. 14. Steps involved in defect line detection (the defect lines X, = X,0 ( L} are those visualized in Fig. 17b): (a) X = initial set, in black (n = mean width of X); (b) thinning (of set X: XI = X0{ L) (note that the triple points correspond to the flexure zones); (c) pruning X,: X: = X,0( E’), { E’ } = n cycles of rotations of E = 0 0 , X, = X2 U ([U,( X2 0 E,)] $ (H); X,); (d) extraction i 0 0 1 of the end points: Y = Ug_ 1[ X, 0 E(j)] (end points), X, = [ X,/( Y 69 N)] U Y (disconnection of the end points): (e) skiz of Xi, i.e., X5 = X,0 ( M* ) O{ E* }; (f) particle extraction (the small final dilation will conriect the particles): X6 = (Y d {h }; X5) Gd H. 304 JEAN SERRA morphology, another technology consists of using a convolver, giving weights 2O 2l ,2”-l to the n pixels of the convolution function and in thresholding the relul; i221. Does this mean that mathematical morphology is equivalent to convolution or to cellular logic transforms ? Yes, if you consider that knowing letters (Y to o is equivalent to speaking Greek. If you do not think so, then the present paper can provide you some information. We have tried to show that organization of the elementary transforms is not random and submits to logical prerequisites. These requirements introduce meaning into the operations (e.g., I want my transform to be increasing, idempotent, and self-dual). They act at various levels, just as words, sentences, and paragraphs in language [23,24]. From this point of view, it would be a mistake to consider morphological filtering equivalent to convolution, whereas they are logically incompatible, and it would be detrimental to those who want to master the method. If convolvers and cellular logics may implement morphological treatments, it is simply because euerything can be processed on an image as soon as one can shift pixels, compare the values at two pixels, add them, and subtract them. This said, there exist excellent devices for morphological treatments which are not based at all on cellular logics (Stemberg’s Genesis 2000, 1983; or even this old dinosaur of Serra’s first Texture Analyser, 1965). Finally, if I had to advise a neophyte in mathematical morphology, I should propose to him to clearly distinguish between concepts and modes of operation, i.e., to answer separately the questions “what treatment?” and “how to perform it.” He may acquire a knowledge about the first point by reading the major books on the method and, above all, dissecting the published cases of study. Concerning the second point, he can find currently several systems which provide libraries of morphological primitives. Then the actual question is less to judge the choice of capable operators, than to estimate the Relevance, the completeness, and the speed of the whole system.

APPENDIX Notions Which Correspond to Each Other by Duality Initial notions Dual notions Set X, its caps, isthmuses, islands, boundary Set X, its gulfs, channels and lakes, boundary X, Y arguments Intersection X fl Y Union XU Y (Y parameter Set difference X/Y B C X (B parameter) B fi X (i.e., “B hits A”) [lJi E ,&I’ (1, finite or not integ:r) Equals n i E, ( Xi)’ (Morgan’s theorem) Erosion X 8 B, opening (X 0 B) 8 B (B par.) Dilation X 0 B, closing (X 0 g) 0 B Hit-or-miss x 0 T, (T = (Tt, Tr) par.) Hit-or-miss X 0 T* with T* = (T,, Ti) Thinning X0 T (T par.) Thickening XOT* Cond. thickening (resp. dilation) XOT, Y Cond. thinning (resp. erosion) XOT*; Y ( X, Y arguments, T = (Ti, T2 ) parameter) Closed set; space P of all closed sets (R”) Open set; space p of all open sets (R”) xi + x;Gii(X,); xi t x Xi+ X,Lim(X); &LX Upper semi-continuity: YZii\k( 4) C *k(X) Lower semi-continuity: Lim \k( X,) 3 q(X) Homotopic sets (or functions) Homotopic sets (or func=) area A, (X); perimeter I/, (X) 1 -A,(r); &(X’) Specific connectivity number NA (X) - N,(X’), respectively euclidean Equal i convexity number NT (A) G(p) INTRODUCTION 305

APPENDIX-Continued _.. Initial notions Dual notions

Function f(x); picture p(x) (0 5 p -< m) Function -f(x); pictm-e m - p(x) Sup( f, g ) ( f, g functions or pictures) I@/, g) - ( f 9 g) (functions) Eiymls -f 8 g m - (f 9 g) (pictures) Equals(m -f, 8 g maximum, summits minimum, sinks For functions divides, watersheds and pictures Become channels, hills, respectively i ridges, saddles i ruts, saddles Va%wms: u,dh); y,(h); y,(h) Equal yxc(h); v-f(h); y,,-,(h), rap. Boundary a X Equals boundary a XL‘

Note added in proof. Since the submission of this paper in December, 1982, mathematical mor- phology and its applications have considerably evolved. For a more current introduction see “Morpho- logical Optics,” Journal of Microscopy, in press.

REFERENCES 1. G. Matheron. Random Sets and Integral Geometry, Wiley, New York, 19728. 2. J. Serra, Image Analysis and Mathematical Morphology, Academic Press, New York/London, 1982. 3. G. James and R. James, Mathematical Dictionary, Van Nostrand Reinhold, New York, 1976. 4. F. Meyer, internal report, EMP 5. Ch. Lantuejoul and F. Maisonneuve, Geodesic methods in quantitative image analysis, internal report EMP, 1982. 6. 0. Hunziker, U. Schultz, J. Serra and Ch. Walliser, Morphometric analysis of neurons in different depths of the cat’s brain cortex after hypoxia, NBS Spec. Publ. (U.S.) 431, 1976, 203-206. 7. G. Matheron, Elements pour une thborie des milieux poreux, Masson, Paris, 1967. 8. W. F. Donoghue, Distribution and Fourier trans/orms, Academic Press, New York/London, 1969. 9. G. Matheron, Les applications idempotentes, internal report, EMP. 1982. 10. F. Maisonneuve, Ordinaux transfinis et sur (ou sous) potentes, internal report EMP, 1982. 11. J. Serra, Quelques semi-groupes de filtrages morphologiques, internal report EMP, 1982. 12. L. A. Santalo. Integral Geometry and Geometric Probability, Addison-Wesley, Reading, Mass., 1916. 13. E. R. Weibel, Practical Metho& for Biological Morphometry, Vol. I. Stereological Methods, Academic Press, New York/London, 1980. 14. H. Hadwiger. Vorslesungen iiber Inhalt, Oberjhiche und Isoperimetrie, Springer Berlin, 1957. 15. J. Serra, A. Mariaux, and 0. Peray, The texture analyser measurements and physical anisotropy of tropical woods, NBS Spec. Publ. (U.S.) 431, 1976, 257-260. 16. A. Rosenfeld and A. C. Kak, Digital Picture Processing, Academic Press, New York/London, 1976. 17. S. Levialdi, Parallel pattern processing, IEEE Trans. System Man. Cybern. SMC-1, 1971, 292-296. 18. M. J. E. Golay, Hexagonal parallel pattern transformation. IEEE Trans. Comput. C-18, 1969. 733-740. 19. Ch. Lantuejoul, Skeletonization in quantitative metallography, in Issues of Digital Image Proc. (R. M. Haralick and J. C. Simon, Eds.), Sijthoff & Noordhoff, Winchester, Mass., 1980. 20. A. Rosenfeld, Connectivity in digital pictures, J. Assoc. Comput. Mach. 17, No. 1, 1970. 146-160. 21. K. Preston, Multicomputers and Image Processing, Academic Press, London/New York, 1982. 22. K. Preston, Feature extraction by Golay hexagonal pattern transforms, IEEE Trans. Comput. C-20. 1971, 1007-1014. 23. J. Serra, Structure synaxiques en morphologie mathtmatique, in Premier Colloque Image GRETSI- CESTA, Biarritz, May 1984. 24. J. Serra, (Ed.). Advances in Mathematical Morpholohy, Academic Press, New York/London, in press. 25. S. Stemberg, Comput. Vision Graphics Image Process. 35, 1986. 26. F. Meyer, Comput. Vision Graphics Image Process. 35, 1986. 27. J. Serra, The Boolean model and random sets, Comput. Graphics Image Process. 12. 1980, 99-126.