Corner Detection with Covariance Propagation

Qiang Ji and Robert M. Haralick Intelligent Systems Laboratory Department of Electrical Engineering, FT-10 University of Washington Seattle, WA 98 195

Abstract proposed for detecting comer points including maximum curvature, deflection angle, maximum deviatior,, and total This paper presents a statistical approach for detect- fitting errors [1][5][6][11]. A major problem with existing ing corners from chain encoded digital arcs. An arc point approaches is that the employed criterion is not tied to a sta- is declared as a corner if the estimated parameters of the tistical analysis, therefore rendering existing methods sus- two fitted lines of the two arc segments immediately to the ceptible to noise. To overcome this, we present a statistical right and left of the arc point are statistically sign9cantly approach for corner detection. Here, the corner criterion different. The corner detection algorithm consists of two is treated as a random variable and is subject to perturba- steps: corner detection and optimization. While corner de- tion. Given an arc segment, a comer is defined to be a point tection involves statistically identifying the most likely cor- where two underlying line segments meet and form a ver- ner points along an arc sequence, corner optimization deals tex, whose included angle is statistically larger than an angle with improving the locational errors of the detected corners. threshold. The comer detection procedure involves sliding The major contributions of this research include devel- a context window of specified length over the arc sequence, oping a method for analytically estimating the covariance performing a least square line fitting to the arc points lo- matrix of the fitted line parameters and developing a hy- cated immediately to the left and right of the center of the pothesis test statistic to statistically test the difference be- window, estimating the parameters of the fitted lines, and tween the parameters of two fitted lines. Performance eval- their covariance matrices, and finally performing a hypoth- uation study showed that the algorithm is robust and ac- esis test to test the statistical difference between the param- curate for complex images. It has an average misdetection eters of the two fitted lines. If the difference is significant, rate of 2.5% and false alarm rate of 2.2% for the complex the arc point located in the center of the window is declared RADIUS images. This paper discusses the theory and per- as a comer point. Finally, a comer optimization procedure formance characterization of the proposed corner detector. is performed to improve the locational errors of the detected comers. The major contributions of this research include developing a method for analytically estimating the covariance 1. Introduction matrix of the fitted line parameters and developing a hypothesis test statistic to statistically test the difference be- Comers have long been important two dimensional fea- tween the parameters of two fitted lines. This paper is ar- ranged as follows. In section 2, we state the problem and tures for computer vision research. They have been used ex- tensively for matching, pattern recognition, and data com- present the associated noise and comer models. Section 3 discusses in detail the theoretical aspects of the corner de- pression. Various algorithms have been developed for detector. The tecting comers. Comer detection algorithms can be roughly performance evaluation of the comer detector is covered in sections 4 and The paper ends in section 6 grouped into two categories: one is based on the detection 5. directly from the underlying grayscale images; the other is with a discussion and summary of the proposed approach. based on the digital arcs, resulting from edge detection and linking. Various techniques have been developed for comer 2. Problem statement detection from digital arc sequences. The basis for these techniques is to identify the locations of the endpoints of A comer point represents a discontinuity in the curva- each maximal line segment. Different criteria have been ture of a curve. The location of the discontinuity can be

1063-6919/97 $10.00 0 1997 IEEE 362 approximated by the intersection of two straight lines that underlie the arc segments to the right and left of the corner point, Perturbation to the points on the ideal underlying H~ : e12< eo H~ : e122 eo (3) lines gives rise to the observed arc segments. This section is where 812 repfesents the population mean of random vari- concerned with the definition of the perturbation model and able 612. the comer model. The hypothesis testing identifies the most likely comer point along an arc sequence. Specifically, Jiven a signifi- 2.1. Perturbation model cant level a,the P-value of each observed 42is computed and compared with a. If the P-value 5 a,the null hypoth- Given an observed sequence of ordered points from a esis is rejected and the arc point being considered (&, &) line arc segment, S = { ($n, gn)l, = 1,.. . ,N}, where N is a comer point. Figure 1 illustratively shows the comer is the number of points on the arc segment, the perturba- model just described. In the section to follow, we describe tion model assumes that (On, dn) result from random per- the theoretical derivations that lead to the solution to the turbations to the ideal points (xn, gn), n = 1,. . . ,N, con- above hypothesis testing problem. strained to be on the line xncos8+ynsin8-p=0, n=l,..., N; I, rMovingwindow where 8 and p are the parameters of the underlying line that \ gives rise to the observed arc segment. It is further assumed that the random perturbations are independently and iden- tically Gaussian distributed in the direction perpendicular to the underlying line. Analytically, the perturbation model can be expressed as follows:

digiia arc where n = 1,. . . ,N and tnare independently and identi- Figure 1. An example of the corner model cally distributed as N(0, a2). 2.2. Corner model 3. Theory for the proposed approach

For a piecewise linear approximation of a curve, comer In this section, we detail the theoretical aspects of the points are the end points of each line segment. Thus, an end developed algorithm. Specifically, we describe least-square point is a comer point if the underlying two line segments line fitting, covariance propagation, and hypothesis testing. immediately to the right and left of the point meet and form a vertex, whose included angle is statistically larger than a 3.1. Least-square line fitting given angle threshold. A comer is defined as follows. Given an observed sequence of ordered points from an To estimate the line parameters for each arc segment, we arc segment, S = { ( ) In = 1,. . . ,N} and a point perform a least square line fitting to the arc points. The least-square fitting can be formulated as follows: (&, &) along the arc segment, the arc point divides the Assume phts (Pn,in),n = I.." lie on an arc seg- arc segment S into two sub-segments SI and SZ , where ment S, resulting from perturbation of ideal points (xn, Vn) locating on the line 2, cos t9 + yn sin 8 - p = 0. Perturba- tion to each point follow the perturbation model in equation IC + 1,. . . ,N}. Let 81 and 62 be the estimated orientations (1). of the two lines that fit to SIand S2,and i12be the included To estimate the best fitting line parameters b and j using angle between the lines, is then defined as the least square method, we need to minimize the sum of squared residual errors:

Given an included angle threshold eo, the comer detection problem may be formulated as a hypothesis testing problem as follows: (4)

363 where 5 is called design matrix, 6 the parameter matrix, Then based on [21, CAe, the covariance matrix of the esti- and S = atfithe scatter matrix. fi and 0 are defined as mated line parameters 6 can be computed from:

where& = cos8andp = sin8

As a result, we need to minimize DtSh subject to b2 + where & and $& are evaluated at ideal line parameter 8 j2= 1. Introducing the Lagrange multiplier A, the func- and points X, and CAX represents the input perturbation. tion to be minimized can be expressed as From eq(9), we can easily obtain E2 = && - A($& - 1) (5) where C is referred to as constraint matrix and is defined as c=(i ; g) and,

Taking partial derivatives of e2 w.r.t. 6 and A, and setting them to zeros yields the simultaneousequations ag2Nx2 $&-A& = 0 (6) ax @& = 1 (7) This system is readily solved by considering generalized eigenvectors of eq(6). 2N x2

3.2. Covariance propagation For the given perturbation model in equation (l),the input covariance matrix CAX is given by The random perturbation on ideal X=(zn, points yn), d ... 0 0 n = 1, ...,N,lyingonlinesncosO+ynsinO-p = 0, 0 d ... 0 yields observed arc points 2 = (Zn,fjn), n = 1,. . . , N. The use of 2 for estimating line parameter 0 = (3 yields 6 = ( ) , a least square estimate of 0.ne per- where turbation accompanying 2 induces a correspondingpertur- cos2 e sin 8 cos e bation on 6. In this section, we will analytically estimate d= ( sinecos8 sin20 A@,the perturbation of Q, expressed in its covariance ma- ) trix CA@,in terms of the covariance matrix cAX of 2. Define Based on the covariance propagation theory [21, the scaler criterion function F that needs to be minimized can be defined as N and ~(6,2)= C(2,cosB + insin8 - j12 n=l Define g as follows:

364 After algebraic operations and simplifications,we obtain me distribution of the test statistic under null hypothesis is a non-central Chi-squared with two degrees of freedom.

TNX; where the noncentrality parameter h is

Geometrically, k can be interpreted as the signed distance between a point (x,y) and the point on the line closest to the origin. As a result, Si represents the spread of points along Given the test statistic and its distribution, a h,pificant level the lie. A larger Si, i.e., points with larger spread along of a = 0.01 was selected to perform the test. If the p-value the line yields better fit as indicated with smaller covariance of a test is larp than c,the null hypothesis is accepted, i.e., matrix trace. In addition, is the mean position of the no comer exists between SIand SZ.On the other hand, if points along the line. It acts like a moment arm. A larger the pvalue of the test is less than a,the null hypothesis is pk, i.e., a longer moment ann, can induce more variance to rejected and the vertex formed by arcs S1 and Sa is declared the estimated b. Further investigation of eq( 10) reveals that a comer. (T; is invariant to coordinate translation and rotation while 6; is variant to coordinate translations that change pk. 3.4. Corner optimization The way in which we have derived the covariance matrix CA~requires that the matrices mandvbe The set of comer points detected in a digital arc are only optimal locally but not globally. It is not globally optimal known. But X and 0 are not observed. 2 and 6 are ob- because not all the points on the arc are used to detect the served instead. So, to obtain an estimated covariancematrix comer points. This may result in high locational errors. To kae, we substitute X and 6 for X and 0 in eq(l0). reduce the location errors with the detected comers, we perform a comer optimization. The comer optimization, based 3.3. Hypothesis testing on Pavlidis's discrete optimization method [5], iteratively shifts the detected comer points to produce a better approx- With covariance matrices computed, we can proceed to imation of the arc sequence. While the iterative optimiza- develop a test statistic to decide statistically whether the an- tion procedure is guaranteed to terminate with improved lo- gular parameters of the two fitted lines differ by a threshold cation errors, it however.may terminate at a local minimum 80. Given two arc segments Sl and S2,a least-square line rather than at the global minimum. fitting is performed to fit a line to SIand a line to SZ using the method described in section 3.1, thus resulting in esti- 4. Performance evaluation mated line orientation parameters 81 and 82. From equation (lo), we obtain e:, and e;,, the estimated variances of 61 Thii section discusses results from a series of experi- and 62. The hypothesis testing can then be formulated as: ments aimed at characterizing the performance of the pro- H, : e12 < eo H~ : e12 2 e, (1 1) posed algorithm using images from the RADIUS database. A total of 80 model board images are used. Each image where 0, (ranging from 0 to 90 degree) is a user supplied represents an outdoor scene, containing primarily building angular threshold, and 642 is the population mean of random structures. The input to the comer detector are sequences of variable &,which is defined as arc segments resulting from an edge detection and linking operation. The groundtruth points for these images are ob- 612 = 181 - 021 tained by manually annotating the aerial images to delineate since the edges of the buildings and other structures in the image [lo]. The criteria used for the evaluation are misdetection (MD) and false alarm (FA) rates. Figure 2 plots the average false alarm rate and misdetection rate versus the context window length for all images. Thus, a likelihood ratio test statistic can be designed as follows: It shows as e::window length increases, the false alarm rate decreases quadratically while the misdetection rate increases quadratically. A larger context window yields more data for more accurate statistical analysis, which leads to

365 ~

a decrease in false alarm rate. On the other hand, the as- length and included angle thresholds derived from previous sumption of only one comer present in the context win- experiments, i.e., optimal window length of 30 pixels and dow leads to an increase in misdetection as window size optimal included angle threshold of 5 degree. The results increases. However, this increase only becomes apparent show that the comer detector has an average misdetection after the window size exceeds certain threshold. rate of about 2.5% and false alarm rate of about 2.2% re- Figure 3 gives the average MD and FA rates versus the spectively. included angle threshold. It shows as the included angle increases, the false alarm rate tends to decrease. This is be- 5. Performmce comparison cause we are looking for building comers, which often have large included angles. Increasing the included angles filters This section describes results of evaluating the perfor- out comers with small included angles, therefore reducing mance of our comer detector against that of Lowe's al- the false alarm rate. On the other hand, increasing the in- gorithm [4] using both synthetic data and RADIUS data. cluded angle may increase the misdetection rate. Figure 3 The criterion used for the evaluation include both visual also shows that while a small increase in the angle threshold inspection, and false alarm and misdetection rates when leads to marginal improvement in false it however alarm, groundtruth data are available. Lowe's algorithm has been could lead to a dramatic increase in misdetection. widely cited and was found superior to most comer detection algorithms available [8] 171 [3].

...... 5.1. Synthetic curves ...... Here we evaluate the performance of the two algorithms 010mm~)~mmm using the synthetic curves for polygonal approximation. Synthetic curves were generated by sampling the original model curve consisting of piecewise linear line segments and by perturbing each sampled pixel with iid Gaussian noise with mean 0 and variance cr2. The reconstructed test curves cgasist of +rturbed sampled points. Figure 4 io m 30 U) M e, m ea cmn*lrgh shows two synthetic curves adapted from those of Rosin [7] and Teh [9] respectively. Figure 5 shows the results from Figure 2. Misdetection and false alarm Lowe's algorithm (a and b) and our algorithm (c and d). Vi- rates versus context window length, 80 = 30 sually, both algorithms performed equally well on the two curves; but our algorithm outperformed Lowe's algorithm for both curves. Lowe's algorithm tends to detect local irregularity like small bumps or dips as comers, therefore yielding a higher false alarm rate......

Oo io P D U) n e, m BO ea

...... )._...... :_...... I...... (a) (b) .. f io m 30 U) 50 e, M a, so -rg. Figure 4. Synthetic test curves adapted from Rosin (a) and Teh (b). Figure 3. Misdetection and false alarm rates versus included angle threshold, with window length being 25 pixels 5.2. RADIUS data

We also study the average performance of the comer de- A comparison was also carried out on 4 different RA- tector for all 80 RADIUS images using the context window DIUS images. The goal here is to find the building comers

366 form a vertex. The arc point closest to the vertex point is declared as a comer if the angular orientations of the two lines that form the vertex are statistically significantly different. Performance evaluation study showed that the algorithm is robust and accurate for complex images. It has an average misdetection rate of 2.5% and false alarm rate of 2.2% for the complex RADIUS images. The study also revealed that our algorithm has consistently outperformed Lowe's tech- nique on both synthetic and real data, with much lower false alarm rate. A major factor that contributes to the low false alarm rate of our algorithm is that we take the perturbation on the estimaN line parameters into consideration, allow- ing us to treat the included angle as a random variable and statistically test its range. This represents a better model for comer detection than existing techniques, where the corner criteria like curvatures or tangent angles are treated as Figure 5. Results of polygonal approximation to the scalars rather than random variables, ignoring any pertur- curves shown in figure 4 using Lowe's algorithm (a) and (b); bation they may be subject to and therefore leading to high and algorithm (c) and (d), where window length=50 and our false alarm rate for noisy images. angular threshold=25. Polygons are obtained by connecting the detected comers. References from the edge images of the buildings. Groundtruth build- [ 11 J. G. Dunham. Optimal uniform piecewise linear approxi- ing comers were obtained via the annotation procedure as mation of planar curves. IEEE Trans. on Pattern Analysis and Machine Intelligence, 657-75, 1986. described above. The criterion is therefore the misdetec- [2] R. M. Haralick. Propagating covariance in computer vision. tion and false alarm rates. Tables 1 and 2 show the per- Proc. of 12th IAPR, pages 493-498, 1994. formance of the two algorithms for each of the four images. [3] T. Kadonaga and K. Abe. Comparison of methods for de- As expected, while both algorithms are comparablein terms tecting corner points from digital curves. Int. Workshop on of misdetection rates, our algorithm has much lower false Graphics Recognition, 1995. alarm rates for all four images. Lowe's algorithm tends to [4] D. G.Love. Three dimensional object recognition from sin- make 2 to 7 times as many false alarm mistakes. This again gle two dhensionai images. Artifkiul Intelligence, 31:355- demonstrates the superiority of our algorithm. 395,1987. [5] T. Pavlidis. Waveform segmentation through functional ap- I Table 1. Performance of Lowe's aleorithm I proximation. IEEE Trm on Computers, 22(7):689,1973. " [6] T. Pavlidis S. L. Horowitz. Segmentation plane Images FA MD and of I 1 curves. IEEE Transactwns on Computers, 23(8):860, 1974. 1 I 10.6 I 1.3 [7] P. L. Rosin and G. A. W. West. Nonparametric segmentation 2 9.2 1.6 of curves into various representations. IEEE Trans. on Pat-

3 9.3 I 0.7 tern Analysis andMachine Intelligence. 17(12):1140-1153, 4 [ 10.1 I 3.8 j 1995. [SI R. L. Rosin. Techniques for assessing polygonal approxi- mations of curves. 7th British Machine Kswn CO$, Edin- Table 2. Performance of our algorithm burgh, 1996, 1996. [9] C. H. Teh and R. T. Chin. On the detection of dominant points in digital curves. IEEE Trans. on Pattern Analysis 5.4 1.8 and Machine Intelligence, 11:859-872, 1989. 3 1 1.3 1.8 [lo] K. Thomton and et al. Groundtruthing the radius model- 4 I 4.4 2.0 board imagery. IUE Workshop, 1995. [ll] J. A. Ventura and J. M. Chen. Segmentation of two-dimensional curve contours. Pattem Recognition, 6. Discussion and Conclusions 25( 10):11?3-'1140, 1992.

In this paper, we have presented a statistical approach for detecting comers on digital arc sequences. A comer is defined to be an arc point where two line segments meet and

367