LOCAL POINTS OF INTEREST USING AUGMENTED REALITY _______________ A Thesis Presented to the Faculty of San Diego State University _______________ In Partial Fulfillment of the Requirements for the Degree Master of Science in Computer Science _______________ by Sudeshna Mukherjee Spring 2013 iii Copyright © 2013 by Sudeshna Mukherjee All Rights Reserved iv DEDICATION I dedicate this thesis work to my parents and teachers. The constant encouragement of my parents has given me direction in life. The guidance and support of my teachers has helped me in my educational journey of enlightenment. v ABSTRACT OF THE THESIS Local Points of Interest Using Augmented Reality by Sudeshna Mukherjee Master of Science in Computer Science San Diego State University, 2013 As the worldwide smartphone market continues to expand, by the third quarter of 2012 Android had a 75% share of the global smartphone market according to the research firm IDC. One reason that smartphones are so popular is because of the wide range of smartphone applications that they provide these applications are also so useful because they can integrate intimately with our personal lives. Augmented reality is a new technology which integrates the virtual world of a smart phone with the real world of nearby locations as seen by the user. This thesis implements a number of augmented reality features by blending both visual, map-based and non-map- based elements like live projection of nearby landmark on camera preview. The main screen of the proposed application gives the address of the current location and provides two buttons. The first button is for a map view and gives the representation of the current location on the map, with options. The second button on the main screen is for showing a camera preview. When this button is clicked a camera preview is shown, and as we hold the phone vertically, it projects various nearby landmarks as icons on the phone screen. Again there is a menu bar with useful options. vi TABLE OF CONTENTS PAGE ABSTRACT ...............................................................................................................................v LIST OF TABLES ................................................................................................................. viii LIST OF FIGURES ................................................................................................................. ix ACKNOWLEDGEMENTS ..................................................................................................... xi CHAPTER 1 INTRODUCTION .........................................................................................................1 1.2 Motivation ..........................................................................................................1 1.3 Thesis Organization ...........................................................................................2 2 USE OF AUGMENTED REALITY FOR MOBILE PHONE ......................................3 2.1 Introduction to Augmented Reality ...................................................................3 2.2 Hardware Components of a Phone Used for Mobile Augmented Reality ................................................................................................................3 2.3 Popular Mobile Augment Reality Applications .................................................3 2.4 Introduction to Unity Software Developent Kit .................................................5 2.4.1 Features of Unity Software Development Kit ..........................................5 2.4.2 A Sample Augmented Reality Application Developed Using Unity .........................................................................................................6 3 TECHNOLOGY DETAILS ...........................................................................................8 3.1 Android Architecture .........................................................................................8 3.2 Android Application Life Cycle ........................................................................9 3.3 Composition of Android Application ................................................................9 4 ANDROID APPLICATION PROGRAMMING INTERFACE DETAILS ................12 4.1 Location Services .............................................................................................12 4.1.1 Location Manager ...................................................................................12 4.1.2 Location Class .........................................................................................12 4.1.3 Geocoder Class .......................................................................................12 4.1.4 Address Class ..........................................................................................13 4.2 Google Maps Android API ..............................................................................13 vii 4.2.1 Google Maps Android API Map Types ..................................................14 4.2.2 Obtaining Google Map API Key ............................................................14 4.3 Data Source ......................................................................................................15 4.4 Android Package ORG.JSON ..........................................................................15 4.5 Android Camera API .......................................................................................15 4.6 Usage of Device Sensors .................................................................................16 4.6.1 Android Sensor API Methods .................................................................17 4.6.2 Pitch Roll Azimuth for Android Phone ..................................................19 5 IMPLEMENTATION DETAILS ................................................................................22 5.1 Permissions to Access Device Hardware and Other Services .........................22 5.2 Main Activity and Map View Details ..............................................................22 5.3 Usage of Google Map API Key in XML File ..................................................25 5.4 Obtaining Information About Landmarks Using JSONOBJECT ....................25 5.5 Plotting Icons for Nearby Landmark on Camera Preview ...............................26 5.5.1 Relationship between the Plotted Points and Radius ..............................30 5.5.2 Usage of Thread Pool for Dynamic Update of Phone Screen ................33 6 EXECUTION RESULTS ON ANDROID PHONE ....................................................35 6.1 Android Logging System .................................................................................36 6.2 Procedure for Recording Execution Times ......................................................38 6.3 Conclusion .......................................................................................................40 7 CONCLUSIONS AND OBSTACLES ........................................................................41 8 FUTURE SCOPE AND IMPROVEMENT.................................................................42 BIBLIOGRAPHY ....................................................................................................................43 APPENDIX SETTING OF DEVELOPMENT AND TEST ENVIRONMENT ....................................46 viii LIST OF TABLES PAGE Table 4.1. Table Describing Function Names and Their Descriptions ....................................13 Table 4.2. Table Describing the Parameters for the getRotationMatrix Method .....................19 Table 4.3. Table Describing the Parameters for remapCoordinateSystem Method ................20 Table 6.1. Table Describing Execution Times at Different Radius for Samsung Galaxy S3 .....................................................................................................................35 Table 6.2. Table Describing Execution Times at Different Radii for HTC Thunderbolt ........36 Table 6.3. Table Showing Summarized Data of Execution Times at Different Radius for HTC Thunderbolt and Samsung Galaxy S3 (From Table 6.1 and 6.2) ..................37 Table 6.4. Table Describing Phone Configurations Used for Running the Application .........38 ix LIST OF FIGURES PAGE Figure 2.1. Snapshot of translated text.......................................................................................4 Figure 2.2. Snapshot of Google Sky map application. ..............................................................4 Figure 2.3. User interface of Unity. Snapshot taken after downloading, installing and launching the trial version of the unity IDE. ..................................................................5 Figure 2.4. Snapshot of menu items of Unity. ...........................................................................6 Figure 2.5. Augmentation of GMCS building on map using Unity IDE. ..................................7 Figure 2.6. Snapshot of phone screen after augmentation. ........................................................7 Figure 3.1. Android architecture. ...............................................................................................8 Figure 3.2. Android application lifecycle. ...............................................................................10 Figure 4.1. Flowchart describing the order of call to various camera API methods needed to use the phone camera. ..................................................................................17 Figure 4.2. Default orientation of phone. .................................................................................18