A Mobile Augmented Reality Application for Visualizing and Interacting with Online Data
Total Page:16
File Type:pdf, Size:1020Kb
A mobile Augmented Reality application for visualizing and interacting with online data Author Nurane Kuqi Supervisor Aris Alissandrakis Co-supervisor Nico Reski Examiner Ilir Jusufi Exam date 30th May 2017 Subject Social Media and Web Technologies Level Master Course code 5ME11E i “Computing is not about computers anymore. It is about living.” Nicholas Negroponte (MIT Media Lab founder) ii Abstract Augmented Reality (AR) technology has recently received a lot of attention for hav- ing lots of potential for uses in a variety of contexts. From an interaction perspective, an important factor is not only the usability of the applications, but also the engage- ment that these allow. Of particular interest is the possibility of allowing users to interact with data from online sources via an Augmented Reality interface. The main focus of this master thesis is to create a mobile AR application that allows visualization and interaction with online data. For this purpose a prototype was designed and implemented using the Unity game engine with C# and the Vuforia SDK on the front-end (mobile app) and NodeJS with MongoDB on the back-end (database server). The prototype was tested, and also deployed in a two week user study to evaluate a) how the system allowed the participants to explore available data, and b) how the use of the system affected the participants daily behaviour. The results indicate that the prototype was well received by the users. They enjoyed the experience not only due to the functionality of the system, but also given the opportunity for better involvement and completion of the given task (scheduling their lunch at the university campus restaurant) successfully. The outcomes of the thesis are a proposed architecture for a mobile AR system that allows users to interact in an engaging manner with online data, the developed pro- totype, and the initial testing and evaluation of the prototype in a specific use case. It is hoped that this work constructively contributes to the overall Augmented Real- ity field, especially regarding the development of mobile Augmented Reality appli- cations. Keywords: Augmented Reality, mobile Augmented Reality, AR Cube, Vuforia, Unity 3D, NodeJS, MongoDB. iii Acknowledgements In order to successfully complete the Master Studies, many components have given support financially, spiritually and academically. First, I want to thank the Swedish Institute for the scholarship provision during these two years of studies. SI is the first institute that made possible a dream, to accomplish the master studies abroad. Throughout studies I have always thanked SI for this great opportunity to advance my knowledge in the IT field. I wish to express my sincere gratitude to the Professor Marcelo Milrad for providing me the opportunity to work as a student researcher within the Media Technology Department. I would like to express my deepest appreciation to Dr. Aris Alissandrakis. Our first academic collaboration was throughout the student research work that I had within the Media Technology department. Based on the good working experience, I de- cided to chose him as a supervisor for my thesis. He has not only been a great mentor, but he has also supported me a lot to advance my skills and knowledge. In addition, as part of the academic level support, I want to thank Nico Reski, for his sincere support and technical consultancy. Nico has not only been a co-supervisor but also a great friend. I feel lucky for the possibility to have spent this journey with professional mentors that helped me advance through their consultancy, pro- fessional conduct and fruitful discussions. My family has been one of the key motivations for pursuing the studies abroad. Their support in everything and heart warming care has boosted my motivation to successfully complete the studies. My fiancé has played an important role on the whole journey. He has been the greatest support of mine during all this time. He has supported me in every step, with advices, technical support and has been with me in the bad and in the good. His unconditional love and help has motivated me to successfully pass the difficult battles throughout this time. I also want to thank all the friends that helped me through this masters. Their sup- port and fika time together helped me become a better person in life. Moreover, participants of the user study played a crucial role in the validation of the prototype. Thank you for the readiness to participate and add value to the study. In order to bridge the gaps throughout the phase of literature review and technical implementation I have consulted a lot of online resources from very scientific to less formal communities of people. I appreciate the work of each of the people that contribute to the shared knowledge online. Thank you all good people! iv Contents Abstract ii Acknowledgements iii Contents iv List of Figures vi List of Tables vii List of Abbreviations viii 1 Introduction1 1.1 Background..................................1 1.2 Motivation...................................2 1.3 Research Question and Hypotheses.....................3 1.4 Thesis Outline.................................4 2 Related Work5 2.1 Literature Review...............................5 2.2 The Previous AR Cube Prototype......................7 3 Methodology9 3.1 System Development............................. 10 3.2 User Study................................... 10 3.2.1 Testing................................. 11 3.2.2 Data Collection............................ 11 4 Prototype 12 4.1 Prototype Design............................... 12 4.2 Server and Database Implementation.................... 14 4.2.1 Query Data From MongoDB and Google Calendar API..... 14 4.2.2 Activity Diagram........................... 16 4.3 User scenario................................. 16 5 User study 19 5.1 User Study Setting.............................. 19 5.1.1 Testing................................. 20 5.1.2 Data collection............................ 20 5.2 Summary.................................... 21 6 Results and Analysis 22 6.1 Testing..................................... 22 6.2 Weekly Diaries................................. 22 v 6.3 System Logs.................................. 23 6.4 Interview.................................... 25 7 Discussion and Conclusion 27 7.1 Research Question.............................. 27 7.2 Hypotheses.................................. 28 7.3 Limitations................................... 28 7.4 Lessons Learned................................ 29 7.5 Future Work.................................. 30 A Test Protocol 31 B Participant Initial Questionnaire 35 C Participant Weekly Diaries 38 D System Development 42 D.1 Unity 3D.................................... 42 D.2 Vuforia SDK.................................. 42 D.3 NodeJS..................................... 43 D.3.1 Single Threaded Approach..................... 43 D.3.2 JSON.................................. 44 D.4 MongoDB................................... 45 D.4.1 Database API............................. 45 Bibliography 46 vi List of Figures 1.1 Hype Cycle for Emerging Technologies...................3 2.1 The previous AR Cube prototype......................7 3.1 Iterative/Incremental software development approach of the research study...................................... 10 4.1 The AR prototype cube interface...................... 12 4.2 Mock-up of the menu opportunities of the implemented prototype (left), and example of the menu options layout based on the interface of the prototype................................ 13 4.3 Examples of all menu options......................... 13 4.4 System architecture of the implemented prototype............ 14 4.5 Server and database implementation of the prototype........... 15 4.6 Activity diagram................................ 18 5.1 Example of the weekly menu......................... 19 6.1 Participants’ average activity from the system logs............ 25 vii List of Tables 6.1 Dates that the participants ate at the restaurant, based on their weekly diaries...................................... 22 6.2 How often participants used the app, based on their weekly diaries... 23 6.3 System logs of participants’ overall activity................. 24 6.4 System logs of participants’ choices and cancellations........... 24 6.5 Participants’ overall activity from the system logs............. 24 6.6 Menu options chosen by the participants during the study........ 25 6.7 Matches and mismatches between system logs and participants diaries. 26 viii List of Abbreviations API Application Programming Interface AR Augmented Reality CLI Command Line Interface CRUD Create Read Update Delete CS Computer Science ECMA European Computer Manufacturer’s Association GQM Goal Question Metric HCI Human Computer Interaction HDM Head Mounted Display HTTP HyperText Transfer Protocol IDE Integrated Development Environment IoT Internet of Things IT Information Technology JSON JavaScript Object Notation MARS Mobile Augmented Reality Systems PEAR Public Engagement Augmented Reality POI Point Of Interest REST REpresentational State Transfer SDK Software Development Kit SSCI Social Science Citation Index UI User Interface URL Uniform Resource Locator UX User Experience VR Virtual Reality WWW World Wide Web ix To my parents Mr. Xhevad Kuqi and Mrs. Saide Kuqi, To my siblings Fatlinda Kuqi and Elida Kuqi and To my life partner Sazan Lushi This work is a sign of my love to you! 1 Chapter 1 Introduction 1.1 Background According to (Wu et al., 2013),“[Augmented Reality]