Development of Computer Vision Algorithms Using J2me For
Total Page：16
File Type：pdf, Size：1020Kb
DEVELOPMENT OF COMPUTER VISION ALGORITHMS USING J2ME FOR MOBILE PHONE APPLICATIONS A thesis submitted in partial fulfilment of the requirements for the Degree of Master of Science in the University of Canterbury by Jian Gu SUPERVISOR: Dr. R Mukundan Co-SUPERVISOR: Prof. Mark Billinghurst (HIT Lab NZ) Department of Computer Science and Software Engineering University of Canterbury November 2008 ABSTRACT DEVELOPMENT OF COMPUTER VISION ALGORITHMS USING J2ME FOR MOBILE PHONE APPLICATIONS Jian Gu This thesis describes research on the use of Java to develop cross-platform computer vision applications for mobile phones with integrated cameras. The particular area of research that we are interested in is Mobile Augmented Reality (AR). Currently there is no computer vision library which can be used for mobile Augmented Reality using the J2ME platform. This thesis introduces the structure of our J2ME computer vision library and describes the implementation of algorithms in our library. We also present several sample applications on J2ME enabled mobile phones and report on experiments conducted to evaluate the compatibility, portability and efficiency of the implemented algorithms. i ACKNOWLEDGMENTS This thesis would not have been possible without the help of many people. I want to thank foremost my supervisor and co-supervisor Dr R Mukundan and Prof. Mark Billinghurst for their constant support and guidance during the development of this work. They suggested new ideas for me to explore and have helped me appreciate the principles of scientific work. I also would like to thank Anders Henrysson and Charles Han for generously allowing me to use their existing applications in my experiments. Many thanks to Daniel Wagner and Michael Rohs who provided the source code of STBtracker and Visual code for me to study. I also want to thank all the members of HITLabNZ, which offers an extremely supportive environment for people to carry out research and to explore new research directions. Also, thanks to Adam Chang who had provided some new ideas for evaluation and helping in the checking grammar mistakes in this thesis. ii TABLE OF CONTENTS ABSTRACT ........................................................................................................................ i ACKNOWLEDGEDMENTS........................................................................................... ii TABLE OF CONTENTS ................................................................................................ iii LIST OF TABLES .......................................................................................................... vii LIST OF FIGURES ......................................................................................................... ix LIST OF ACRONYMS ................................................................................................... xi 1 Introduction ............................................................................................................... 1 1.1 Motivation for Research ..................................................................................... 1 1.2 Computer Vision Applications on Mobile Phones ............................................. 2 1.3 Why use J2ME ? ................................................................................................. 4 1.4 Organization of Thesis ........................................................................................ 5 2 Background Study and Related Work .................................................................... 7 2.1 Existing Non-J2ME Computer Vision Applications .......................................... 7 2.1.1 General image processing ............................................................................... 7 2.1.2 2D Real Time Image Processing ................................................................... 11 2.1.3 3D Camera Pose Determination .................................................................... 15 2.2 Existing Computer Vision Libraries ................................................................. 19 2.3 Existing J2ME Computer Vision Applications ................................................ 21 3 Developing Computer Vision Application for Mobile Phones Using Java ........ 25 3.1 Using Java for Mobile Phone Applications ...................................................... 25 iii 3.1.1 Developing a J2ME Application ................................................................... 28 3.1.2 Using Java 2D and 3D Graphics and Multimedia Library............................ 32 3.2 Structure of Our Computer Vision Library ....................................................... 34 3.3 The Criteria for Choosing the Algorithms ........................................................ 35 4 Implementation ....................................................................................................... 37 4.1 Using the Camera in J2ME ............................................................................... 38 4.2 Threshold .......................................................................................................... 40 4.3 Contour Detection ............................................................................................. 43 4.4 Rectangle Fitting and Labelling ........................................................................ 44 4.5 Second order moments ...................................................................................... 45 4.6 Homogeneous Transformation ......................................................................... 47 4.7 Pose Estimation Algorithm ............................................................................... 51 4.8 Rendering 3D Objects ....................................................................................... 52 4.9 Edge Detection .................................................................................................. 54 4.10 Motion Estimation ............................................................................................ 59 5 Applications and Experiments ............................................................................... 63 5.1 Test the Portability of the J2ME Application. .................................................. 63 5.1.1 Simple Video Camera Program Testing ....................................................... 63 5.1.2 3D Java Graphics Capability Testing ........................................................... 66 5.1.3 Our Application 1 (Combining Camera and 3D object) ............................... 67 5.2 Testing Our AR Application ............................................................................. 71 5.2.1 Testing Our AR Application on Different Phones ........................................ 71 5.2.2 Testing the Efficiency of Each AR Algorithm.............................................. 73 5.2.3 Comparing the J2ME AR application with a Symbian C++ AR application ..................................................................................................... 76 5.2.4 Test Pose Estimation Accuracy .................................................................... 77 iv 5.3 Motion Estimation Application ........................................................................ 79 5.4 Edge Detection Application .............................................................................. 81 5.4.1 Compare the Performance of Sobel and Canny Algorithms ......................... 81 5.4.2 Comparing J2ME edge detection with a Symbian C++ application. ............ 82 5.5 Problems ........................................................................................................... 84 5.5.1 Long Processing Time .................................................................................. 84 5.5.2 Camera Calibration Decreases the Portability of Our Application ............... 86 6 Conclusions .............................................................................................................. 89 References ........................................................................................................................ 91 Appendix A. Adaptive Threshold Method (J2ME) ................................................ 97 Appendix B. Second Order Moments (J2ME) ........................................................ 99 Appendix C. Pose Estimation Algorithm (J2ME) ................................................ 102 v LIST OF TABLES Table 2-1: Mobile phone computer vision applications ........................................................18 Table 4-1: The key part of the code for using camera in J2ME ............................................39 Table 4-2: The key part code for producing gray scaled image ............................................41 Table 4-3: The key part code for rendering 3D object ...........................................................53 Table 5-1: Results from the test applications described in Sections 5.1.1, Sections 5.1.2 and Section 5.1.3 ........................................................................................................65 Table 5-2: Performance of the experimental AR application ................................................73 Table 5-3: Summary of performance of each of algorithms which runs in different phones 74 Table 5-4: shows the experimental results for the average search points and FPS comparisons for the 25 frames of our mobile camera. ...............................................80 Table 5-5: Time required for motion estimation in different phones.