Efficient Implementations of Machine Vision Algorithms Using a Dynamically Typed Programming Language
Total Page:16
File Type:pdf, Size:1020Kb
Efficient Implementations of Machine Vision Algorithms using a Dynamically Typed Programming Language Jan Wedekind A thesis submitted in partial fulfilment of the requirements of Sheffield Hallam University for the degree of Doctor of Philosophy February 2012 Abstract Current machine vision systems (or at least their performance critical parts) are predo- minantly implemented using statically typed programming languages such as C, C++, or Java. Statically typed languages however are unsuitable for development and maintenance of large scale systems. When choosing a programming language, dynamically typed languages are usually not considered due to their lack of support for high-performance array operations. This thesis presents efficient implementations of machine vision algorithms with the (dynam- ically typed) Ruby programming language. The Ruby programming language was used, because it has the best support for meta-programming among the currently popular pro- gramming languages. Although the Ruby programming language was used, the approach presented in this thesis could be applied to any programming language which has equal or stronger support for meta-programming (e.g. Racket (former PLT Scheme)). A Ruby library for performingI /O and array operations was developed as part of this thesis. It is demonstrated how the library facilitates concise implementations of machine vision algorithms commonly used in industrial automation. That is, this thesis is about a different way of implementing machine vision systems. The work could be applied to prototype and in some cases implement machine vision systems in industrial automation and robotics. The development of real-time machine vision software is facilitated as follows 1. A just-in-time compiler is used to achieve real-time performance. It is demonstrated that the Ruby syntax is sufficient to integrate the just-in-time compiler transparently. 2. VariousI /O devices are integrated for seamless acquisition, display, and storage of video and audio data. In combination these two developments preserve the expressiveness of the Ruby program- ming language while providing good run-time performance of the resulting implementa- tion. To validate this approach, the performance of different operations is compared with the performance of equivalent C/C++ programs. i Publications Refereed Journal Articles • M. Boissenin, J. Wedekind, A. N. Selvan, B. P. Amavasai, F. Caparrelli, and J. R. Travis. Computer vision methods for optical microscopes. Image and Vision Computing, 25(7):1107–16, 07/01 2007 (Boissenin et al., 2007) • A. J. Lockwood, J. Wedekind, R. S. Gay, M. S. Bobji, B. P. Amavasai, M. Howarth, G. Mobus,¨ and B. J. Inkson. Advanced transmission electron microscope triboprobe with automated closed-loop nanopositioning. Measurement Science and Technol- ogy, 21(7):075901, 2010 (Lockwood et al., 2010) Refereed Conference Publications • Jan Wedekind, Manuel Boissenin, Balasundram P. Amavasai, Fabio Caparrelli, and Jon R. Travis. Object Recognition and Real-Time Tracking in Microscope Imaging. Proceedings of the 2006 Irish Machine Vision and Image Processing Conference (IMVIP 2006), pages 164–171, Dublin City University, 2006. (Wedekind et al., 2006) • J. Wedekind, B. P. Amavasai, and K. Dutton. Steerable filters generated with the hypercomplex dual-tree wavelet transform. In 2007 IEEE International Confer- ence on Signal Processing and Communications, pages 1291–4, Piscataway, NJ, USA, 24–27 Nov. 2007 2008. Mater. & Eng. Res. Inst., Sheffield Hallam Univ., Sheffield, UK, IEEE (Wedekind et al.,a) • J. Wedekind, B. P. Amavasai, K. Dutton, and M. Boissenin. A machine vision extension for the Ruby programming language. In 2008 International Conference on Information and Automation (ICIA), pages 991–6, Piscataway, NJ, USA, 20– 23 June 2008 2008. Microsyst. & Machine Vision Lab., Sheffield Hallam Univ., Sheffield, UK, IEEE (Wedekind et al.,b) Formal Presentations • Jan Wedekind. Real-time Computer Vision with Ruby. O’Reilly Open Source Con- vention (OSCON), Portland, Oregon, USA, July 23rd 2008 (Wedekind, 2008) ii • Jan Wedekind. Computer Vision Using Ruby and libJIT. (RubyConf), San Fran- cisco, California, USA, Nov. 19th 2009 (Wedekind, 2009) • Jan Wedekind, Jacques Penders, Hussein Abdul-Rahman, Martin Howarth, Ken Dutton, and Aiden Lockwood. Implementing Machine Vision Systems with a Dy- namically Typed Language. 25th European Conference on Object-Oriented Pro- gramming, Lancaster, United Kingdom, July 28th 2011 (Wedekind et al., 2011) Published Software Packages • malloc1 • multiarray2 • hornetseye-alsa3 • hornetseye-dc13944 • hornetseye-ffmpeg5 • hornetseye-fftw36 • hornetseye-frame7 • hornetseye-kinect8 • hornetseye-linalg9 • hornetseye-narray10 • hornetseye-opencv11 • hornetseye-openexr12 • hornetseye-qt413 • hornetseye-rmagick14 1http://github.com/wedesoft/malloc/ 2http://github.com/wedesoft/multiarray/ 3http://github.com/wedesoft/hornetseye-alsa/ 4http://github.com/wedesoft/hornetseye-dc1394/ 5http://github.com/wedesoft/hornetseye-ffmpeg/ 6http://github.com/wedesoft/hornetseye-fftw3/ 7http://github.com/wedesoft/hornetseye-frame/ 8http://github.com/wedesoft/hornetseye-kinect/ 9http://github.com/wedesoft/hornetseye-linalg/ 10http://github.com/wedesoft/hornetseye-narray/ 11http://github.com/wedesoft/hornetseye-opencv/ 12http://github.com/wedesoft/hornetseye-openexr/ 13http://github.com/wedesoft/hornetseye-qt4/ 14http://github.com/wedesoft/hornetseye-rmagick/ iii • hornetseye-v4l15 • hornetseye-v4l216 • hornetseye-xorg17 15http://github.com/wedesoft/hornetseye-v4l/ 16http://github.com/wedesoft/hornetseye-v4l2/ 17http://github.com/wedesoft/hornetseye-xorg/ iv Acknowledgements First I would like to thank Bala Amavasai for his supervision, support, and his unshakable optimism. He developed a large part of the Mimas C++ computer vision library and organised the Nanorobotics grant. Without him I would not have been able to do this work. I am also very indebted to Jon Travis who has been a valuable source of help and advice when coming to the UK and while working at university. I would also like to thank Ken Dutton, Jacques Penders, and Martin Howarth for continuing supervision of the PhD, for their advice and support and for giving me room to do research work. I would also like to thank Arul Nirai Selvan, Manuel Boissenin, Kim Chuan Lim, Kang Song Tan, Amir Othman, Stephen, Shuja Ahmed and others for being good col- leagues and for creating a friendly working environment. In particular I would like to express my gratitude to Georgios Chliveros for his advice and moral support. Thanks to Julien Faucher who introduced 3D camera calibration to the research group. A special thanks to Koichi Sasada for his research visit and for the many interesting and motivating discussions. Thanks to Aiden Lockwood, Jing Jing Wang, Ralph Gay, Xiaojing Xu, Zineb Saghi, Gunter¨ Mobus,¨ and Beverly Inkson for their valuable help in applying the work to trans- mission electron microscopy in context of the Nanorobotics project. Finally I would like to thank my parents who sacrificed a lot so that I can achieve the best in my life. Without their support I would not have made it this far. A seven year part-time PhD is a long time to work and make friends. My apologies but there is just not enough room to mention you all. The work presented in this thesis was partially funded by the EPSRC Nanorobotics18 project. I also received a student bursary of the Materials and Engineering Research Institute. 18http://gow.epsrc.ac.uk/ViewGrant.aspx?GrantRef=GR/S85696/01 v Declaration Sheffield Hallam University Materials and Engineering Research Institute Mobile Machines and Vision Laboratory The undersigned hereby certify that they have read and recommend to the Faculty of Arts, Computing, Engineering and Sciences for acceptance a thesis entitled “Efficient Implementations of Machine Vision Algo- rithms using a Dynamically Typed Programming Language” by Jan Wedekind in partial fulfilment of the requirements for the degree of Doc- tor of Philosophy. Date: February 2012 Director of Studies: Dr. Martin Howarth Research Supervisor: Dr. Jacques Penders Research Supervisor: Dr. Jon Travis Research Advisor: Dr. Ken Dutton Research Advisor: Dr. Balasundram Amavasai vi Disclaimer Sheffield Hallam University Author: Jan Wedekind Title: Efficient Implementations of Machine Vision Algorithms using a Dynamically Typed Programming Language Department: Materials and Engineering Research Institute Degree: PhD Year: 2012 Permission is herewith granted to Sheffield Hallam University to circulate and to have copied for non-commercial purposes, at its discretion, the above title upon the request of individuals or institutions. THE AUTHOR ATTESTS THAT PERMISSION HAS BEEN OBTAINED FOR THE USE OF ANY COPYRIGHTED MATERIAL APPEARING IN THIS THESIS (OTHER THAN BRIEF EXCERPTS REQUIRING ONLY PROPER ACKNOWLEDGEMENT IN SCHOLARLY WRITING) AND THAT ALL SUCH USE IS CLEARLY ACKNOWLEDGED. Signature of Author vii Contents Contents viii Symbols xiii Acronyms xv List of Figures xviii List of Tables xxii Listings xxiii 1 Introduction1 1.1 Interpreted Languages...........................1 1.2 Dynamically Typed Languages.......................3 1.3 Contributions of this Thesis.........................6 1.4 Thesis Outline................................7 2 State of the Art9 2.1 Object Localisation.............................9 2.2 Existing FOSS for Machine Vision..................... 10 2.2.1 Statically Typed Libraries..................... 12 2.2.2 Statically Typed Extensions...................