Low-Level Vision Tutorial 1

Low-level vision tutorial 1

complexity and sophistication – so much so that  Feature detection and sensitivity the problems of low-level vision are often felt to  Image filters and morphology be unimportant and are forgotten. Yet it remains  Robustness of object location the case that information that is lost at low level  Validity and accuracy in shape analysis Low-level vision – a tutorial is never regained, while distortions that are  Scale and affine invariance. introduced at low level can cause undue trouble at higher levels [1]. Furthermore, image Professor Roy Davies In what follows, references are given for the acquisition is equally important. Thus, simple topics falling under each of these categories, and measures to arrange suitable lighting can help to for other topics that could not be covered in make the input images easier and less ambiguous Overview and further reading depth in the tutorial. to interpret, and can result in much greater

reliability and accuracy in applications such as 3. Feature detection and sensitivity automated inspection [1]. Nevertheless, in

applications such as surveillance, algorithms General [1]. Abstract should be made as robust as possible so that the Edge detection [2, 3, 4].

vagaries of ambient illumination are rendered Line segment detection [5, 6, 7]. This tutorial aims to help those with some relatively unimportant. Corner and interest point detection [1, 8, 9]. experience of vision to obtain a more in-depth General feature mask design [10, 11]. understanding of the problems of low-level 2. Low-level vision The value of thresholding [1, 12, 13]. vision. As it is not possible to cover everything in the space of 90 minutes, a carefully chosen This tutorial is concerned with low-level vision, 4. Image filters and morphology series of topics is presented. This section of the and aims to indicate how some of its problems notes is a commentary to accompany the slides and limitations can be solved. This is a huge General [1]. in the presentation. subject covering image acquisition, noise Effect of applying rank-order filters [1, 14].

suppression, segmentation, colour and texture Mathematical morphology [1, 15]. 1. The nature of vision analysis, shape analysis, object recognition, and

many other topics. Hence it is not possible to do 5. Robustness of object location Vision is a complex process and has long been justice to it in the space of one and a half hours. modelled, rather loosely, as a cascade of low, It is therefore assumed that participants have a General [1, 16]. intermediate and high-level stages following reasonable concept of the subject area, and the Centroidal profiles v. Hough transforms [1, 17– image acquisition. However, this model is tutorial therefore focusses on a number of topics 19]. somewhat impoverished in that it ignores the that have been chosen because they involve Fast object location by selective scanning [1, 20]. possibility of downward as well as upward flow important issues. The topics fall into five broad Robust statistics [1, 21, 22]. of information which can help with interpretation. categories: The RANSAC approach [23]. In addition, there is need for considerable

6. Validity and accuracy in shape analysis Much of this work employs the interest point developing algorithms to cope with the specific methods of Harris and Stephens [9], and is idiosyncrasies of their data to make the most of it Shape analysis [1, 24]. underpinned by careful in-depth experimental in all the ways outlined above. Object labelling [1]. investigations and comparisons [37, 41, 42]. Distance transforms and skeletons [1, 25, 26]. Finally, there are signs that the tide may be 9. Further reading Boundary distance measures [27, 28]. turning in other directions, specifically the Symmetry detection [29, 30]. design of feature detectors that concentrate on Participants may wish to further explore the especially robust forms of scale invariance – as material covered in this tutorial. The author’s 7. Scale and affine invariance in the case of the ‘Speeded Up Robust Features’ book [1] covers many of the topics in fair depth; (SURF) approach [43, 44]. See also the fuller particular attention is drawn to the following At this point it will be useful to consider a topic review article [45]. chapters: that has been developing for just over a decade, starting with papers [31, 32]. It arose largely 8. Concluding remarks  Chapter 3, for median and mode filters, because of difficulties in wide baseline stereo including edge shifts; work, and with tracking object features over The topics presented in this tutorial necessarily  Chapter 5, for edge detection; many video frames – because features change give an incomplete picture because of the short  Chapters 6 and 7, for shape analysis – their character over time and correspondences span of time available. Nevertheless, they especially thinning and centroidal profiles; are easily lost. To overcome this problem it was provide interesting lessons, and demonstrate  Chapter 10, for Hough-based methods of necessary first to eliminate the relatively simple important factors – specifically, the need for circular object location; problem of features changing in size, thereby sensitivity, accuracy, robustness, reliability and  Appendix on robust statistics. necessitating ‘scale invariance’ (it being implicit validity. Further factors enter into the equation that translation and rotation invariance have when practical systems are being designed – The book also contains a considerable amount of already been dealt with). Later, improvements speed and cost of real-time hardware being detailed information on 3D interpretation, became necessary to cope with ‘affine amongst the most relevant. invariants, texture analysis, automated inspection invariance’ (which also covers shear and skew and practical issues. invariance). Lindeberg’s pioneering theory [31] Overall, it is clear that low-level vision is an was soon followed by Lowe’s work [32, 33] on essential ingredient of the vision hierarchy and Acknowledgements the ‘Scale Invariant Feature Transform’ (SIFT). that its problems are fundamental to the whole of This was followed by affine invariant methods vision. Although the focus of attention in the The author would like to credit the following [34–37]. In parallel with these developments, subject may have moved on to more modern sources for permission to reproduce material work was proceeding on maximally stable topics, it is definitely not the case that this side from his earlier publications: extremal regions [38] and other extremal of the subject is played out and that everything is methods [39, 40] (the latter concept has also now known about it. While data sets change and  Elsevier for permission to reprint text and been followed in a different context [13]). specifications for low-level algorithms remain figures from [1, 2, 20, 26]; incomplete, workers will have to go on  IET for permission to reprint text and figures

from [5, 7, 8, 11, 13, 14, 16, 28]; Conf. Publication no. 465, pp. 225–229 Image and signal processing with  IFS Publications Ltd. for permission to 8. Davies, E.R. (2005) Using an edge-based mathematical morphology. IEE Electronics reprint figures from [18]. model of the Plessey operator to determine and Commun. Journal 10, no. 3, 117–128 localisation properties, Proc. IEE Int. 16. Davies, E.R. (2000) Low-level vision References Conference on Visual Information requirements. Electron. Commun. Eng. J. 12, Engineering (VIE 2005), University of no. 5, 197–210 1. Davies, E.R. (2005) Machine Vision: Theory, Strathclyde, Glasgow (4–6 April), pp. 385– 17. Hough, P.V.C. (1962) Method and means Algorithms, Practicalities, Morgan 391 for recognising complex patterns. US Patent Kaufmann (3rd edition) 9. Harris, C. and Stephens, M. (1988) A 3069654 2. Davies, E.R. (1986) Constraints on the combined corner and edge detector. Proc. 18. Davies, E.R. (1984) Design of cost-effective design of template masks for edge detection. Alvey Vision Conf., Manchester University, systems for the inspection of certain Pattern Recogn. Lett. 4, no. 2, 111–120 UK, pp. 147–151 foodproducts during manufacture. In Pugh, 3. Canny, J. (1986) A computational approach 10. Davies, E.R. (1992) Optimal template masks A. (ed.), Proc. 4th Conf. on Robot Vision to edge detection. IEEE Trans. Pattern Anal. for detecting signals with varying and Sensory Controls, London (9–11 Oct.), Mach. Intell. 8, 679–698 background level. Signal Process. 29, no. 2, pp. 437–446 4. Petrou, M. and Kittler, J. (1988) On the 183–189 19. Ballard, D.H. (1981) Generalizing the optimal edge detector. Proc. 4th Alvey 11. Davies, E.R. (1999) Designing optimal Hough transform to detect arbitrary shapes. Vision Conf., Manchester (31 Aug.–2 Sept.), image feature detection masks: equal area Pattern Recogn. 13, 111–122 pp. 191–196 rule. Electronics Lett. 35, no. 6, 463–465 20. Davies, E.R. (1987) A high speed algorithm 5. Davies, E.R. (1997) Designing efficient line 12. Hannah, I., Patel, D. and Davies, E.R. (1995) for circular object location. Pattern Recogn. segment detectors with high orientation The use of variance and entropic Lett. 6, no. 5, 323–333 accuracy. Proc. 6th IEE Int. Conf. on Image thresholding methods for image 21. Meer, P., Mintz, D., Rosenfeld, A. and Kim, Processing and its Applications, Dublin (14– segmentation. Pattern Recogn. 28, no. 8, D.Y. (1991) Robust regression methods for 17 July), IEE Conf. Publication no. 443, pp. 1135–1143 computer vision: a review. Int. J. Comput. 636–640 13. Davies, E.R. (2008) Stable bi-level and Vision 6, 59–70 6. Davies, E.R. (1997) Vectorial strategy for multi-level thresholding of images using a 22. Kim, D.Y., Kim, J.J., Meer, P., Mintz, D. designing line segment detectors with high new global transformation. IET Computer and Rosenfeld, A. (1989) Robust computer orientation accuracy. Electronics Lett. 33, no. Vision, 2, no. 2, Special Issue on Visual vision: a least median of squares based 21, 1775–1777 Information Engineering, Ed. Valestin, S., approach. Proc. DARPA Image 7. Davies, E.R., Bateman, M., Mason, D.R., pp. 60–74 Understanding Workshop, Palo Alto, CA Chambers, J. and Ridgway, C. (1999) 14. Davies, E.R. (1999) Image distortions (23–26 May), pp. 1117–1134 Detecting insects and other dark line features produced by mean, median and mode filters. 23. Fischler, M.A. and Bolles, R.C. (1981) using isotropic masks. Proc. 7th IEE Int. IEE Proc. – Vision Image and Signal Random sample consensus: A paradigm for Conf. on Image Processing and its Processing 146, no. 5, 279–285 model fitting with applications to image Applications, Manchester (13–15 July), IEE 15. Bangham, J.A. and Marshall, S. (1998) analysis and automated cartography. Comm.

ACM 24, no. 6, 381–395 Wide baseline stereo matching based on performance evaluation of local descriptors. 24. Hu, M.K. (1962) Visual pattern recognition local, affinely invariant regions. Proc. IEEE Trans. Pattern Anal. Mach. Intell. 27, by moment invariants. IRE Trans. Inf. British Machine Vision Conf. (BMVC), no. 10, 1615–1630 Theory 8, 179–187 Bristol University, UK, pp. 412–422 43. Bay, H., Tuytelaars, T. and Van Gool, L. 25. Rosenfeld, A. and Pfaltz, J.L. (1968) 35. Mikolajczyk, K. and Schmid, C. (2002) An (2006) SURF: speeded up robust features. Distance functions on digital pictures. affine invariant interest point detector. Proc. Proc. 9th European Conf. on Computer Pattern Recogn. 1, 33–61 European Conf. on Computer Vision Vision (ECCV), Springer LNCS volume 26. Davies, E.R. and Plummer, A.P.N. (1981) (ECCV), Copenhagen, Denmark, pp. 128– 3951, part 1, pp. 404–417 Thinning algorithms: a critique and a new 142 44. Bay, H., Ess, A., Tuytelaars, T., Van methodology. Pattern Recogn. 14, 53–63 36. Mikolajczyk, K. and Schmid, C. (2004) Gool, L. (2008) Speeded-up robust features 27. Koplowitz, J. and Bruckstein, A.M. (1989) Scale and affine invariant interest point (SURF). Computer Vision Image Design of perimeter estimators for digitized detectors. Int. J. Computer Vision 60, no. 1, Understanding 110, no. 3, 346–359 planar shapes. IEEE Trans. Pattern Anal. 63–86 45. Tuytelaars, T. and Mikolajczyk, K. (2008) Mach. Intell. 11, 611–622 37. Mikolajczyk, K., Tuytelaars, T., Schmid, C., Local invariant feature detectors: a survey. 28. Davies, E.R. (1991) Insight into operation of Zisserman, A., Matas, J., Schaffalitzky, F., Foundations and Trends in Computer Kulpa boundary distance measure. Kadir, T. and Van Gool, L. (2005) A Graphics and Vision 3, no. 3, 177–280 Electronics Lett. 27, no. 13, 1178–1180 comparison of affine region detectors. Int. J. 29. Kuehnle, A. (1991) Symmetry-based Computer Vision 65, 43–72 recognition of vehicle rears. Pattern Recogn. 38. Matas, J., Chum, O., Urban, M. and Pajdla, Lett. 12, 249–258 T. (2002) Robust wide baseline stereo from 30. Cho, M. and Lee, K.M. (2009) Bilateral maximally stable extremal regions. Proc. symmetry detection via symmetry-growing. British Machine Vision Conf. (BMVC), Proc. British Machine Vision Conf. (BMVC), Cardiff University, UK, pp. 384–393 paper 286, pp. 1–11 39. Kadir, T. and Brady, M. (2001) Scale, 31. Lindeberg, T. (1998) Feature detection with saliency and image description. Int. J. automatic scale selection. Int. J. Computer Computer Vision 45, no. 2, 83–105 Vision 30, no. 2, 79–116 40. Kadir, T., Brady, M. and Zisserman, A. 32. Lowe, D.G. (1999) Object recognition from (2004) An affine invariant method for local scale-invariant features. Proc. 7th Int. selecting salient regions in images. Proc. 8th Conf. on Computer Vision (ICCV), Corfu, European Conf. on Computer Vision Greece, pp. 1150–1157 (ECCV), pp. 345–457 33. Lowe, D. (2004) Distinctive image features 41. Schmid, C., Mohr, R. and C. Bauckhage from scale-invariant keypoints. Int. J. (2000) Evaluation of interest point detectors. Computer Vision 60, 91–110 Int. J. Computer Vision 37, no. 2, 151–172 34. Tuytelaars, T. and Van Gool, L. (2000) 42. Mikolajczyk, K. and Schmid, C. (2005) A