Efficient implementations of machine vision algorithms using a dynamically typed programming language WEDEKIND, Jan Available from the Sheffield Hallam University Research Archive (SHURA) at: http://shura.shu.ac.uk/6633/ A Sheffield Hallam University thesis This thesis is protected by copyright which belongs to the author. The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the author. When referring to this work, full bibliographic details including the author, title, awarding institution and date of the thesis must be given. Please visit http://shura.shu.ac.uk/6633/ and http://shura.shu.ac.uk/information.html for further details about copyright and re-use permissions. Efficient implementations of machine vision algorithms using a dynamically typed programming language WEDEKIND, Jan Available from Sheffield Hallam University Research Archive (SHURA) at: http://shura.shu.ac.uk/6633/ This document is the author deposited version. You are advised to consult the publisher's version if you wish to cite from it. Published version WEDEKIND, Jan (2012). Efficient implementations of machine vision algorithms using a dynamically typed programming language. Doctoral, Sheffield Hallam University. Repository use policy Copyright © and Moral Rights for the papers on this site are retained by the individual authors and/or other copyright owners. Users may download and/or print one copy of any article(s) in SHURA to facilitate their private study or for non- commercial research. You may not engage in further distribution of the material or use it for any profit-making activities or any commercial gain. Sheffield Hallam University Research Archive http://shura.shu.ac.uk Efficient Implementations of Machine Vision Algorithms using a Dynamically Typed Programming Language Jan Wedekind A thesis submitted in partial fulfilment of the requirements of Sheffield Hallam University for the degree of Doctor of Philosophy February 2012 Abstract Current machine vision systems (or at least their performance critical parts) are predo- minantly implemented using statically typed programming languages such as C, C++, or Java. Statically typed languages however are unsuitable for development and maintenance of large scale systems. When choosing a programming language, dynamically typed languages are usually not considered due to their lack of support for high-performance array operations. This thesis presents efficient implementations of machine vision algorithms with the (dynam- ically typed) Ruby programming language. The Ruby programming language was used, because it has the best support for meta-programming among the currently popular pro- gramming languages. Although the Ruby programming language was used, the approach presented in this thesis could be applied to any programming language which has equal or stronger support for meta-programming (e.g. Racket (former PLT Scheme)). A Ruby library for performingI /O and array operations was developed as part of this thesis. It is demonstrated how the library facilitates concise implementations of machine vision algorithms commonly used in industrial automation. That is, this thesis is about a different way of implementing machine vision systems. The work could be applied to prototype and in some cases implement machine vision systems in industrial automation and robotics. The development of real-time machine vision software is facilitated as follows 1. A just-in-time compiler is used to achieve real-time performance. It is demonstrated that the Ruby syntax is sufficient to integrate the just-in-time compiler transparently. 2. VariousI /O devices are integrated for seamless acquisition, display, and storage of video and audio data. In combination these two developments preserve the expressiveness of the Ruby program- ming language while providing good run-time performance of the resulting implementa- tion. To validate this approach, the performance of different operations is compared with the performance of equivalent C/C++ programs. i Publications Refereed Journal Articles • M. Boissenin, J. Wedekind, A. N. Selvan, B. P. Amavasai, F. Caparrelli, and J. R. Travis. Computer vision methods for optical microscopes. Image and Vision Computing, 25(7):1107–16, 07/01 2007 (Boissenin et al., 2007) • A. J. Lockwood, J. Wedekind, R. S. Gay, M. S. Bobji, B. P. Amavasai, M. Howarth, G. Mobus,¨ and B. J. Inkson. Advanced transmission electron microscope triboprobe with automated closed-loop nanopositioning. Measurement Science and Technol- ogy, 21(7):075901, 2010 (Lockwood et al., 2010) Refereed Conference Publications • Jan Wedekind, Manuel Boissenin, Balasundram P. Amavasai, Fabio Caparrelli, and Jon R. Travis. Object Recognition and Real-Time Tracking in Microscope Imaging. Proceedings of the 2006 Irish Machine Vision and Image Processing Conference (IMVIP 2006), pages 164–171, Dublin City University, 2006. (Wedekind et al., 2006) • J. Wedekind, B. P. Amavasai, and K. Dutton. Steerable filters generated with the hypercomplex dual-tree wavelet transform. In 2007 IEEE International Confer- ence on Signal Processing and Communications, pages 1291–4, Piscataway, NJ, USA, 24–27 Nov. 2007 2008. Mater. & Eng. Res. Inst., Sheffield Hallam Univ., Sheffield, UK, IEEE (Wedekind et al.,a) • J. Wedekind, B. P. Amavasai, K. Dutton, and M. Boissenin. A machine vision extension for the Ruby programming language. In 2008 International Conference on Information and Automation (ICIA), pages 991–6, Piscataway, NJ, USA, 20– 23 June 2008 2008. Microsyst. & Machine Vision Lab., Sheffield Hallam Univ., Sheffield, UK, IEEE (Wedekind et al.,b) Formal Presentations • Jan Wedekind. Real-time Computer Vision with Ruby. O’Reilly Open Source Con- vention (OSCON), Portland, Oregon, USA, July 23rd 2008 (Wedekind, 2008) ii • Jan Wedekind. Computer Vision Using Ruby and libJIT. (RubyConf), San Fran- cisco, California, USA, Nov. 19th 2009 (Wedekind, 2009) • Jan Wedekind, Jacques Penders, Hussein Abdul-Rahman, Martin Howarth, Ken Dutton, and Aiden Lockwood. Implementing Machine Vision Systems with a Dy- namically Typed Language. 25th European Conference on Object-Oriented Pro- gramming, Lancaster, United Kingdom, July 28th 2011 (Wedekind et al., 2011) Published Software Packages • malloc1 • multiarray2 • hornetseye-alsa3 • hornetseye-dc13944 • hornetseye-ffmpeg5 • hornetseye-fftw36 • hornetseye-frame7 • hornetseye-kinect8 • hornetseye-linalg9 • hornetseye-narray10 • hornetseye-opencv11 • hornetseye-openexr12 • hornetseye-qt413 • hornetseye-rmagick14 1http://github.com/wedesoft/malloc/ 2http://github.com/wedesoft/multiarray/ 3http://github.com/wedesoft/hornetseye-alsa/ 4http://github.com/wedesoft/hornetseye-dc1394/ 5http://github.com/wedesoft/hornetseye-ffmpeg/ 6http://github.com/wedesoft/hornetseye-fftw3/ 7http://github.com/wedesoft/hornetseye-frame/ 8http://github.com/wedesoft/hornetseye-kinect/ 9http://github.com/wedesoft/hornetseye-linalg/ 10http://github.com/wedesoft/hornetseye-narray/ 11http://github.com/wedesoft/hornetseye-opencv/ 12http://github.com/wedesoft/hornetseye-openexr/ 13http://github.com/wedesoft/hornetseye-qt4/ 14http://github.com/wedesoft/hornetseye-rmagick/ iii • hornetseye-v4l15 • hornetseye-v4l216 • hornetseye-xorg17 15http://github.com/wedesoft/hornetseye-v4l/ 16http://github.com/wedesoft/hornetseye-v4l2/ 17http://github.com/wedesoft/hornetseye-xorg/ iv Acknowledgements First I would like to thank Bala Amavasai for his supervision, support, and his unshakable optimism. He developed a large part of the Mimas C++ computer vision library and organised the Nanorobotics grant. Without him I would not have been able to do this work. I am also very indebted to Jon Travis who has been a valuable source of help and advice when coming to the UK and while working at university. I would also like to thank Ken Dutton, Jacques Penders, and Martin Howarth for continuing supervision of the PhD, for their advice and support and for giving me room to do research work. I would also like to thank Arul Nirai Selvan, Manuel Boissenin, Kim Chuan Lim, Kang Song Tan, Amir Othman, Stephen, Shuja Ahmed and others for being good col- leagues and for creating a friendly working environment. In particular I would like to express my gratitude to Georgios Chliveros for his advice and moral support. Thanks to Julien Faucher who introduced 3D camera calibration to the research group. A special thanks to Koichi Sasada for his research visit and for the many interesting and motivating discussions. Thanks to Aiden Lockwood, Jing Jing Wang, Ralph Gay, Xiaojing Xu, Zineb Saghi, Gunter¨ Mobus,¨ and Beverly Inkson for their valuable help in applying the work to trans- mission electron microscopy in context of the Nanorobotics project. Finally I would like to thank my parents who sacrificed a lot so that I can achieve the best in my life. Without their support I would not have made it this far. A seven year part-time PhD is a long time to work and make friends. My apologies but there is just not enough room to mention you all. The work presented in this thesis was partially funded by the EPSRC Nanorobotics18 project. I also received a student bursary of the Materials and Engineering Research Institute. 18http://gow.epsrc.ac.uk/ViewGrant.aspx?GrantRef=GR/S85696/01 v Declaration Sheffield Hallam University Materials and Engineering Research Institute Mobile Machines and Vision Laboratory The undersigned hereby certify that they have read and recommend to the Faculty of Arts, Computing, Engineering and Sciences for acceptance