Improving the Effectiveness of Local Feature Detection
Total Page:16
File Type:pdf, Size:1020Kb
Improving the Effectiveness of Local Feature Detection Shoaib Ehsan A thesis submitted for the degree of Doctor of Philosophy at the School of Computer Science and Electronic Engineering University of Essex May 2012 Abstract The last few years have seen the emergence of sophisticated computer vision systems that target complex real-world problems. Although several factors have contributed to this success, the ability to solve the image correspondence problem by utilizing local image features in the presence of various image transformations has made a major contribution. The trend of solving the image correspondence problem by a three-stage system that comprises detection, description, and matching is prevalent in most vision systems today. This thesis concentrates on improving the local feature detection part of this three-stage pipeline, generally targeting the image correspondence problem. The thesis presents offline and online performance metrics that reflect real- world performance for local feature detectors and shows how they can be utilized for building more effective vision systems, confirming in a statistically meaningful way that these metrics work. It then shows how knowledge of individual feature detectors’ functions allows them to be combined and made into an integral part of a robust vision system. Several improvements to feature detectors’ performance in terms of matching accuracy and speed of execution are presented. Finally, the thesis demonstrates how resource-efficient architectures can be designed for local feature detection methods, especially for embedded vision applications. i ii Copyright © 2012 Shoaib Ehsan All Rights Reserved iii iv To Abbu, Ammi, Maria & Guria v vi Acknowledgements First of all, I would like to thank ALLAH Almighty for giving me the strength and determination to complete this work. The last four years have been miraculous for me due to the mercy of ALLAH Almighty; I don’t have the words to express my gratitude. Without any doubt, this thesis will be incomplete without highlighting the great supervision and support of Prof. Klaus McDonald-Maier and Dr. Adrian Clark. To be honest, this thesis is a product of our team work. Not only I learnt the art of research and academic writing from them, Klaus and Adrian are hugely responsible for transforming my outlook as a researcher and always motivated me to achieve the highest research standards. Thanks to them for being so patient with me during those countless enlightening discussions. I could not have accomplished so much without their unlimited and untiring support. I must also appreciate the contribution of Adrian in capturing all those large image databases. Many thanks to Dr. David Hunter for being on my supervisory board and for all the encouragement. The guidance I received from Dr. Tom Foulsham towards the end of my research really deserves high appreciation. I am also thankful to Prof. Roy Davies for being very compassionate and supportive to me. I will forever remember Prof. Atta Badii as a person who was always there for me whenever I needed help. I am really grateful to Dr. Andrew Hopkins for being a great inspiration for me throughout my PhD years. I also wish to acknowledge the great support of Sheineez (FunTech) during the early stages of this work. I really appreciate the never ending support of my parents. Whatever I am today is due to their determined efforts. I cannot pay them back for all their love, care and prayers. I feel obliged to my grandparents. I also wish to thank my parents-in-law. My wife Maria deserves a special mention as she has backed me through all tough times. I definitely owe her a lot. Aila, my vii beloved daughter, has been a source of joy and comfort all along. The smiles of my dearly loved son Abdullah have always helped me to relax. I would also like to acknowledge the support of my sisters, Nadia, Rabia and Ayesha. Thanks to Naveed bhai, Rizwan bhai, Usman bhai, Jahangir bhai and their families. I am also grateful to Ikram uncle, Najma aunty and Rasheeda aunty. I do not have words to express gratitude for Farooq bhai, Aqsa bhabhi and Naima. They helped me and my family in every possible way and made our stay here really wonderful. I will always remember the good times that we spent together. Big thanks to all my colleagues, especially Nadia Kanwal, Erkan Bostanci, Yasir Qadri, Philip Cheung and Yevgeniya Kovalchuk. I am really obliged to Bayar Menzat and Dragos Stanciu. Thanks is also due for Marisa Bostock, Ramona Bacon, Jayne Bates, Simon Moore, Alfonso Torrejon, Maria Ntovrou, Alex Schillings, David Himsworth, Cristian Juganaru, Atanas Kuzmanov and David Snow. My stay in the UK would not have been as pleasurable as it was, had it not been for family-friends like Waqar Nabi, Tahseen Waqar, Dr. Sajid Baloch, Fowad Murtaza, Saima Fowad, Dr. Zahid Waheed, Ammara Zahid, Yasir Qadri, Nadia Qadri, Nadia Kanwal, Tahseen Muzaffar, Erkan Bostanci, Betul Bostanci, Mamoona Asghar, Zain-ul-Abdin Khuhro, Farhat Memon, Tahir Qadri and Ayesha Tahir. My long-time colleagues and friends also deserve credit: Junaid Afzal, Umar Hamid, Naveed-ur-Rehman, Arslan Khan, Shakaiba Majeed, Hamid Jabbar, Saima Siddiqi, Imran Siddiqi, Omer Zaman, Ahsen Javed, Sammar Javed and Sahar Javaid. Thanks for all your help and support over the years. Special thanks to the University of Essex and EPSRC for providing the financial support for this work. I am grateful to the University of Essex, BMVA and ICIAR committee for awarding conference travel grants. viii Contents 1 Introduction ........................................................................... 1 1.1 The Renaissance of Local Feature Detection .........................................2 1.2 Challenges .................................................................................................4 1.3 Thesis Contributions ................................................................................7 1.4 Thesis Structure ........................................................................................9 1.5 List of Publication ...................................................................................12 2 Local Invariant Feature Detection: A Review .............. 15 2.1 Introduction .............................................................................................16 2.2 Local Invariant Features........................................................................16 2.3 A Very Brief History of Local Feature Detection .................................18 2.4 State-of-the-art Local Invariant Feature Detectors ............................19 2.4.1 Scale Invariant Feature Transform (SIFT) .....................................19 2.4.2 Speeded-Up Robust Features (SURF) .............................................20 2.4.3 Harris-Laplace/Affine ........................................................................21 2.4.4 Hessian-Laplace/Affine .....................................................................21 2.4.5 Edge-Based Regions (EBR) ...............................................................22 2.4.6 Intensity-Based Regions (IBR) .........................................................22 2.4.7 Maximally Stable Extremal Regions (MSER).................................22 2.4.8 Salient Regions ..................................................................................23 2.4.9 Scale Invariant Feature Operator (SFOP) ......................................23 2.5 Hardware-Based Local Feature Detection ...........................................24 2.6 Summary..................................................................................................28 3 Improved Repeatability Measures .................................. 29 3.1 Introduction .............................................................................................30 3.2 Related Work ...........................................................................................32 3.3 Improved Repeatability Measures ........................................................34 3.3.1 An Overview of the Repeatability Metric ........................................35 3.3.2 Limitations of Repeatability .............................................................35 3.3.3 Proposed Measure 1...........................................................................38 ix 3.3.4 Proposed Measure 2 .......................................................................... 39 3.3.5 Qualitative Results ........................................................................... 39 3.3.6 Verification of Improved Measures using Pearson’s Correlation Coefficient .......................................................................................... 41 3.4 Evaluation of State-of-the-art Detectors .............................................. 47 3.4.1 Results under Various Transformations using Proposed Measure 1 .......................................................................................... 48 3.4.2 Results under Various Transformations using Proposed Measure 2 ........................................................................................... 56 3.5 Summary ................................................................................................. 60 4 Repeatability: A Systems Design Perspective ............... 61 4.1 Introduction............................................................................................. 62 4.2 Proposed Framework ............................................................................. 65 4.2.1 Component 1 .....................................................................................