In IEEE Seventh International Conference on Computer Vision, Volume 2, pages 1039-1045, September 1999 Corner Detection in Textured Color Images Mark A. Ruzon Carlo Tomasi Computer Science Department Stanford University Stanford, CA 94305 Abstract [4]showed that edges found by rst-derivative op era- tors tend to \round o " corners. Without using the Corner models in the literature have laggedbehind image itself, it is imp ossible to distinguish true cor- edge models with respect to color and shading. We use ners from curved b oundaries. Therefore, we opt for a botharegion model, based on distributions of pixel col- direct approach. ors, and an edge model, which removes false positives, At a conceptual level, corner and edge detection to perform corner detection on color images whose re- algorithms b oth compute the degree to whichtwo ad- gions contain texture. We show results on a variety jacent regions are dissimilar. Corners do not bisect an of natural images at di erent scales that highlight the op erator's supp ort, however, and the resulting asym- problems that occur when boundaries between regions metry must b e accounted for. Also, corners are p oint have curvature. features, so only one resp onse to the same part of the 1 Intro duction image can b e accepted. Detecting junctions, though, requires accepting multiple resp onses of the corner de- Corners and junctions (multiple corners at the same tector in the same or nearly the same image lo cation. image lo cation) are crucial for high-level vision tasks b ecause they represent o cclusions useful to stereo and Our approach uses b oth a region mo del, from which motion algorithms, and they provide shap e informa- we create a set of corner candidates, and an edge tion for ob ject recognition. They are arguably at least mo del, which decides whether to accept or reject a as imp ortant as edges, yet currentedgemodelsinvoke candidate. We mo del a corner as twoadjacent regions fewer assumptions and are more robust than current that di er in their color distributions. The resulting corner mo dels. op erator generalizes edge detection to asymmetric re- gions with multiple colors p er region. Multiple colors Sp eci cally, edge mo dels prop osed in the literature are represented by a set of p oint masses in a color are sup erior to existing corner mo dels with resp ect to space. The distance b etween two such sets is found color and shading. Many color edge detectors have using the Earth Mover's Distance, which measures the b een prop osed ([1], [5], and [11] form a representative minimum amountof\work" required to transform one sample), but corner detectors have b een con ned to set into the other in that space. greyscale images. Certain algorithms (e.g. [8]) app ear to b e easily extendable to color images. The e ects The advantage of this mo del is that we can detect of shading on the direction of the image gradientwere corners (and edges, see [10]) in textured regions where mo deled byWang and Binford [12] to create an edge other detectors cannot. Two textures mayhavethe detector insensitive to shading, while most corner de- same \mean color," for example, even though they tectors assume that regions are of constantintensity have no color in common. Furthermore, the texture (Alvarez and Morales [2] assumed level sets). need not b e homogeneous as long as its colors are suf- ciently di erent from its neighb ors. Many corner detectors start with an edge map rather than an image (e.g. [6] and [7]), whichwould An edge mo del is also necessary, b ecause corners app ear to mitigate such e ects. However, wear- cannot exist indep endently of edges. Our mo del, in- gue against using these indirect metho ds for tworea- spired by Deriche and Giraudon's, presumes that at sons: (1) using the output of an algorithm whose goal the twoendpoints of the corner, there is a strong is something other than corner detection causes un- edge resp onse in the same direction. Furthermore, the known biases and errors to propagate into the corner edge resp onse b etween the two endp oints of the corner detector, and (2) the analysis of Deriche and Giraudon should b e weaker than the corner resp onse. Wecan A B R (x,y) α SO S θ I partial normalized (a) (b) A B A B S 1 0 S 1 0 I I Figure 1: Parts of the region mo del. (a) Illustra- S 1 6 S 1=7 6=7 O O tion of op erator parameters. (b) The pixel weighting EMD: 0 EMD: 6=7 function, a surface of revolution of half of a Gaussian derivative function. Figure 2: The normalized EMD can detect corners that the partial EMD cannot. compare corner candidates with this mo del to exclude most false p ositives in the op erator's resp onse. If mul- equally breaks the circle into wedges, allowing ecient tiple candidates all resp ond well to the same corner, up dating of the signatures. Weuse15 wedges. we group them and cho ose the \b est" candidate. We represent colors in the CIE-Lab color space [13], The next two sections explain the region and edge in which short, Euclidean distances are p erceptually mo dels, resp ectively, after whichwe present the results accurate. Toaccount for the fact that long distances and our conclusions. are not, we use a normalized measure that saturates: 2 The Region Mo del d =1 exp(E = ); ij ij In this section we develop a mo del of two adjacent where E is the Euclidean distance b etween color i regions and the p erceptual distance b etween them. In ij and color j ,and =14:0 is a constant determining Section 2.1 we summarize the representation of a re- the steepness of the function. gion as a color signature; details are in [10]. Section 2.2 The distance b etween two color signatures is found tackles the problem of asymmetry b etween the two re- using the Earth Mover's Distance (EMD) [9]. The gions, and Section 2.3 explains how initial corner can- EMD measures the minimum amountofphysical work didates are selected. needed to move the masses of one signature into cor- 2.1 Color Signatures resp ondence with the other. In our formulation, the A color signature is a set of p oint masses that rep- EMD lies in [0, 1] since the maximum amountofmass resents one of the two regions. There are ve param- that can b e moved and the maximum distance it can eters that determine which pixels will b elong to each move are b oth 1. region: (x; y ), the lo cation of the center of the win- 2.2 Partial EMD vs. Normalized EMD dow; 2 [0; 360), the orientation of the corner (de- After creating two color signatures, S inside the I ned as the angle formed by the p ositive x-axis and corner and S outside it, we can use the EMD to O the \clo ckwise" side of the corner); 2 (0; 180], the measure the similaritybetween the two regions. An angle subtended by the corner; and R, the scale pa- imp ortant issue in this computation that is not present rameter (Figure 1(a)). Because it is natural to con- when using this mo del for edge detection is that S O sider a corner as a wedge, the window is a circle of always has more mass than S . I radius R. We normalize S to have a mass of 1, regardless I Vector quantization applied to the circle deter- of the value of , to preserve the same output range. mines the numb er and lo cation of the p oint masses. There are twoways to normalize S :wecanusethe O Each pixel contributes a weight dep endent onlyonits same constant and nd the EMD b etween signatures distance to the center. The p olar function f (r )= of unequal mass (\partial" EMD), or we can assign 2 r 2 2 cr e S a mass of 1 also (\normalized" EMD). , where c is a normalizing constantand = O Eachtyp e of EMD has di erent advantages. In R=3, is the p ositive half of a 1-D Gaussian deriva- Figure 2 the normalized EMD detects a corner that the tive function revolved around the y -axis (Figure 1(b)). partial EMD do es not. A 45 corner consists entirely Isotropy simpli es computations over all combinations of color A, while the oustide region has amounts of of and b ecause the mass that each pixel contributes colors A and B (a p erceptual distance of 1 from A)in remains constant. Sampling the ranges of and or the corners that we do detect, B in natural images. F it is b est to describ e them as accurately as p ossible. SO Finding Corner Candidates 2.3 y measur- The pro cess of corner detection b egins b SI ver all circular windows and for all ing the EMD o A combinations of and . The result is a list of three- 30 partial normalized dimensional tensors, one for eachvalue of . Corner A B A B candidates are maximum values over x, y , and that S 1 0 S 1 0 I I are ab ove a threshold. Parab olic interp olation over S 0.8 10.2 S 0:07 0:93 O O gives the actual strength and orientation of a candi- EMD: 0.2 EMD: 0.93 date.