in Pro c. IEEE International Conference on Pattern Recognition, Brisbane, Australia, 1998 Multi-feature Hierarchical Template Matching Using Distance Transforms D.M. Gavrila Daimler-Benz AG, Research and Technology Wilhelm Runge St. 11 89081 Ulm, Germany [email protected] ture representation. Matching pro ceeds by cor- Abstract relating the template against the DT image; the We describe a multi-feature hierarchical algo- correlation value is a measure of similarityinim- rithm to eciently match N objects (templates) age space. with an image using distancetransforms (DTs). Previous work on DT-based matching [1] [2] The matching is under translation, but it can [7] [3] [11] [5] [10] [6] has dealt with the case cover more general transformations by generating of matching one template against an image, al- the various transformed templates explicitly. The lowing certain geometrical transformations (e.g. novel part of the algorithm is that, in addition to translation, rotation, ane). Here we consider acoarse-to- ne search over the translation pa- a more general case of matching N templates rameters, the N templates aregrouped o -line with an image under translation. Matching of into a template hierarchy based on their similar- one template under more general transformations ity. This way, multiple templates can be matched can b e seen as a sp ecial case when all the trans- simultaneously at the coarse levels of the search, formed templates are generated explicitly. In ad- resulting in various speed-up factors. Further- dition to a coarse-to- ne searchover the trans- more, in matching, features are distinguishedby lation parameters, the N templates are group ed type and separate DT's arecomputed for each o -line into a template hierarchy based on their type (e.g. basedonedge orientations). These similarity. Multiple templates can b e matched concepts are il lustrated in the application of traf- simultaneously at the coarse levels of the search, c sign detection. resulting in various sp eed-up factors. The outline of the pap er is as follows. Section 1 Intro duction 2 reviews previous work on distance transforms, distance measures and matching strategies. Sec- Matching is a central problem in pattern recog- tion 3 discusses the prop osed extensions to the nition and computer vision. A common applica- DT matching scheme, whichinvolve the use of tion is ob ject detection and tracking. The vari- multiple features and an ecent match strategy ous matching metho ds that have b een prop osed by means of a template hierarchy. Section 4 lists can b e distinguished bywhattyp e of features are exp eriments in the application of trac sign de- used [12]. At the one end there are pixel-based tection. Finally,we conclude in Section 5. metho ds, which t mo dels directly to ( ltered) image pixels. At the other end there are sym- b olic matching metho ds which op erate on a few 2 Previous Work high-level features (e.g. parts of ob jects and their relations) and apply graph matching metho ds to 2.1 Distance Transforms establish corresp ondence. A distance transform (DT) converts a binary im- In this pap er, we consider metho ds for im- age, which consists of feature and non-feature age matching using distance transforms (DTs). pixels, into an image where each pixel value de- Matching using DTs involves intermediate-level notes the distance to the nearest feature pixel. features [2] which are extracted lo cally at various DTs approximate global distances by propagat- image lo cations, e.g. edge p oints. A DT converts ing lo cal distances at image pixels. Particular the binary image, which consists of feature and DT algorithms dep end on a variety of factors. non-feature pixels, into a DT image where each One factor is whether they result in a Euclidean pixel denotes the distance to the nearest feature distance metric or not (EDTs vs. WDT) [8] [13]. pixel. Similarly, the ob ject of interest is repre- Figure 1 illustrates a EDT. WDTs de ne vari- sented by a binary template using the same fea- ximations of the \true" Euclidean dis- ous appro Raw Image tance measure. One such approximation is the chamfer-2-3 metric [1] [2] [13], used in our exp er- feature extraction iments. Another factor is how the distances are Feature Feature 4.23.6 2.82.22.0 2.2 2.8 3.6 4.2 Image Template 3.6 2.8 2.2 1.4 1.0 1.4 2.2 2.8 3.6 (binary) (binary) 2.8 2.2 1.4 1.0 0.0 1.0 1.4 2.2 2.8 2.2 1.4 1.0 0.0 1.0 0.0 1.0 1.4 2.2 DT DT 1.4 1.0 0.0 1.0 1.4 1.0 0.0 1.0 1.4 correlation 1.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0 1.0 DT DT 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 Image Template 1.41.0 1.0 1.0 1.0 1.0 1.0 1.0 1.4 2.22.0 2.0 2.0 2.0 2.0 2.0 2.0 2.2 Figure 2: Matching using a DT Figure 1: A binary pattern and its Euclidean Dis- tance Transform distances of the template features to the near- est features in the image. The lower these dis- propagated over the image, whether in a raster tances are, the b etter the matchbetween image scan or a contour scan fashion. Most algorithms and template at this lo cation. There are a num- use a raster scan fashion where the propagation of ber of matching measures that can b e de ned on distances is in a manner indep endent of the fea- the distance distribution. One p ossibilityisto ture lo cations in the image, with a mask of xed use the average distance to the nearest feature. size and shap e. Contour scan algorithms prop- This is the chamfer distance. agate the distances from the feature lo cations. X 1 Some DT approaches also weigh the distances D (T; I) d (t) (1) chamf er I jT j from features by their salience, where salientfea- t2T tures (e.g. edge strength, length, curvature) re- where jT j denotes the numb er of features in T sult in comparably lower "distance" values [10]. and d (t) denotes the distance b etween feature t Finally, there are sequential and parallel DT al- I in T and the closest feature in I . The cham- gorithms [4]. fer distance consists thus of a correlation b e- tween T and the distance image of I ,followed 2.2 Match Measures and Strategies by a division. Other more robust measures re- duce the e ect of missing features (i.e. due to Matching with DT is illustrated schematically in o cclusion or segmentation errors) by using the Figure 2. It involves two binary images, a seg- average truncated distance or the f -th quantile mented template T and a segmented image I , value (the Hausdor distance) [7] [11]. In ap- whichwe'll call "feature template" and "feature plications, a template is considered matched at image". The "on" pixels denote the presence of lo cations where the distance measure D (T; I)is a feature and the "o " pixels the absence of a below a user-supplied threshold feature in these binary images. What the actual features are, do es not matter for the matching D (T; I) < (2) metho d. Typically, one uses edge- and corner- p oints. The feature template is given o -line for a particular application, and the feature image Figure 3 illustrates the matching scheme of is derived from the image of interest by feature Figure 2 for the typical case of edge features. Fig- extraction. ure 3a-b shows an example image and template. Figure 3c-d shows the edge detection and DT Matching T and I involves computing the dis- transformation of the edge image. The distances tance transform of the feature image I . The in the DT image are intensity-co ded; lighter col- template T is transformed (e.g. translated, ro- ors denote increasing distance values. tated and scaled) and p ositioned over the result- ing DT image of I ; the matching measure D (T; I) The advantage of matching a template (Figure is determined by the pixel values of the DT im- 3b) with the DT image (Figure 3d) rather than age which lie under the "on" pixels of the tem- with the edge image (Figure 3c) is that the re- plate. These pixel values form a distribution of sulting similarity measure will b e more smo oth computation of DT image: serial vs. parallel, salience weighing match measures: Euclidean vs. robust measures, directed vs. undirected measures matching N templates: none global search algorithms: exhaustive vs. (a) (b) hierarchical (in transformation space, in im- age resolution) 3 Extensions 3.1 Multiple Feature-Typ es: Edge Orientation (c) (d) So far, no distinction has b een made regarding the typ e of features. All features would app ear Figure 3: (a) original image (b) template (c) edge in one feature image (or template), and subse- image (d) DT image quently, in one DT image. If there are several fea- ture typ es, and one considers the match of a tem- plate at a particular lo cation of the DT image, as a function of the template transformation pa- it is p ossible that the DT image entries re ect rameters.