Eindhoven University of Technology MASTER Bokode based fiducial augmented reality system Salunkhe, H.L. Award date: 2011 Link to publication Disclaimer This document contains a student thesis (bachelor's or master's), as authored by a student at Eindhoven University of Technology. Student theses are made available in the TU/e repository upon obtaining the required degree. The grade received is not published on the document as presented in the repository. The required complexity or quality of research of student theses may vary by program, and the required minimum study period may vary in duration. General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain Department of Mathematics and Computer Science Den Dolech 2, 5612 AZ Eindhoven P.O. Box 513, 5600 MB Eindhoven The Netherlands Author Hrishikesh Salunkhe Bokode Based Fiducial Augmented Reality Date System August 25, 2011 Author: Hrishikesh Salunkhe Supervisors: Prof. Dr. Gerard de Haan (TU/e) Ir. Frits de Bruijn(Philips Research) Ing. Harold Schmeitz(Philips Research) Where innovation starts Acknowledgments I have worked with a great number of people whose contribution in assorted ways to the research and the making of the thesis deserve special mention. It is a pleasure to convey my gratitude to them all in my humble acknowledgements. First and foremost I offer my sincerest gratitude to my three supervisors, Prof. Dr. Gerard de Haan, Frits de Bruijn and Harold Schmeitz, who have supported me throughout my thesis with patience and knowledge while allowing me the room to work in my own way. I attribute the level of my Masters degree to their encouragement and effort and without them this thesis would not have been completed. My university supervisor, Dr. Gerard de Haan has helped me to evolve from a mere student into a com- petitive, ambitious and research oriented master student. His systematic supervision has lead into a productive, research oriented and timely completion of my thesis. I could never have embarked and started all of this without his prior teachings in video and image processing and thus opened up exciting and challenging areas to me. His advice and guidance through the timely meetings and discussion has led me to manage my entire graduation process efficiently and seamlessly. I gratefully thank my two company supervisors Frits de Bruijn and Harold Schmeitz for giving me extraor- dinary experiences through out the work. Above all and the most needed, they provided me encouragement and support in various ways. Their truly scientist intuition has made them as a constant oasis of ideas and passions in science, which exceptionally inspire and enrich my growth as a student, a researcher and a scientist want to be. Their involvement and guidance has triggered and nourished my intellectual maturity that I will benefit from, for a long time to come. I am indebted to them more than they know. I gratefully thank Kees van Berkel, my defence committee member, for his constructive comments on this thesis. I am thankful that in the midst of all his activity, he accepted to be a member of the defence committee. I am grateful to him in every possible way and hope to keep up our collaboration in the future as a PhD candidate. I am much indebted to Mukul Rocque and Sachin Bhardwaj for their valuable suggestions and advice in discussion and ideas and, furthermore, using their precious times to read this thesis and giving critical comments about it. Words fail me to express my appreciation to my parents. My family members deserve special mention for their inseparable support and prayers. Their dedication, love and persistent confidence in me, has taken the load off my shoulder and without their support nothing would have been possible. Finally, I would like to thank everybody who was important and helpful to the successful realization of my thesis, as well as expressing my apology that I could not mention personally one by one. Abstract Augmented Reality (AR) is one of the latest developments in the field of human-computer interaction (HCI) that employs the marker based video tracking. Generally, an AR system makes use of 2D fiducial markers that are mounted on a specific object, to compute the pose of the object. AR systems primarily rely on the fiducial marker tracking and its shape approximation to estimate the object pose to facilitate an accurate registration of a virtual world. This conventional method suffers from several drawbacks such as lesser accuracy and small distance range of detectability. A novel Bokode based approach, introduced by the Massachusetts Institute of Technology (MIT), intends to eliminate these drawbacks. The Bokode is a tiny device, that consists of a pattern and a lens-let. The Bokode pattern is made up of fiducial markers, that encode the Bokode pose. It uses the principle of the Bokeh effect, along with the lens-let, to magnify the Bokode pattern that appears sharp on the camera sensor when the camera is configured in out of focus mode. The research at MIT, shows that the Bokode based pose estimation method enables to detect the tag from larger distances. The thesis work carried out at Philips Research, Eindhoven analyses the Bokode technique and its mech- anism in detail. It also derives and analyses relations among various critical Bokode based parameters that influence the performance and precision of the system. Furthermore, the entire Bokode based fiducial aug- mented reality system is designed and implemented using several designed Bokode patterns in Matlab. Various experiments are conducted in order to determine the distance range, accuracy and robustness of the system. For the current implementation, the Bokode can be detected from the distance of 2:12meters with the worst-case error of 6 ◦ in viewing (orientation) angles. The Bokode can be used in pose estimation applications that demand an accurate pose estimation with the detection from large distances. Keywords - Video Tracking, Augmented Reality, Fiducial Marker, Pose Estimation, Bokode. Contents 1 Introduction 1 1.1 Background.............................................1 1.2 Related Work............................................2 1.2.1 Fiducial Markers......................................2 1.2.2 Augmented Reality.....................................3 1.2.3 Bokode...........................................5 1.3 Problem Description........................................6 1.4 Report Organization.........................................6 2 Theory and Analysis 7 2.1 Bokode...............................................7 2.1.1 Bokeh Effect........................................7 2.1.2 Focusing at infinity.....................................8 2.2 Pose Estimation...........................................9 2.2.1 Conventional Method....................................9 2.2.2 Bokode Based Method................................... 10 2.2.2.1 Spherical Coordinate system.......................... 10 2.2.2.2 Bokode Pattern Encoding............................ 11 2.2.2.3 Bokode Decoding................................ 11 2.2.2.4 Angular Decoding................................ 11 2.2.2.5 Distance Computation.............................. 13 2.3 Bokode Pattern Selection...................................... 14 2.3.1 QR Codes.......................................... 14 2.3.2 Data Matrix......................................... 15 2.3.3 Binary De Bruijn Codes.................................. 16 2.3.4 Manchester encoded Binary De Bruijn Codes....................... 18 2.3.5 Comparison......................................... 20 2.4 Bokode Parameters......................................... 20 2.4.1 Resolution limit....................................... 21 2.4.2 Minimal Optical Window Size............................... 22 2.4.3 Redundancy......................................... 23 2.4.4 Module Size........................................ 23 2.4.5 Minimal Marker Size.................................... 24 2.4.6 Magnification........................................ 24 i 3 Implementation 26 3.1 Pattern Generation......................................... 27 3.1.1 Specification........................................ 27 3.1.2 Generation......................................... 27 3.1.2.1 ZXing Library.................................. 27 3.1.2.2 Bokode patterns................................. 28 3.2 Bokode Realization......................................... 29 3.2.1 Setup............................................ 29 3.2.2 Analysis.......................................... 30 3.3 Bokode Pattern Detection and Decoding.............................. 31 3.3.1 Pattern Detection...................................... 31 3.3.1.1 QR Code Detector................................ 31 3.3.1.2 Data Matrix Detector.............................. 32 3.3.1.3 Binary De Bruijn Detector............................ 32 3.3.1.4 Manchester encoded Binary De Bruijn Detector................ 37 3.3.2 Pattern Decoding...................................... 37 3.3.2.1 QR Code Decoder................................ 37 3.3.2.2 Data Matrix Decoder.............................