EXAMENSARBETE INOM TEKNIK, GRUNDNIVÅ, 15 HP STOCKHOLM, SVERIGE 2018 An exploratory research of ARCore's feature detection ANNA EKLIND LOVE STARK KTH SKOLAN FÖR ELEKTROTEKNIK OCH DATAVETENSKAP Abstract Augmented reality has been on the rise for some time now and begun making its way onto the mobile market for both IOS and Android. In 2017 Apple released ARKit for IOS which is a software development kit for developing augmented reality applications. To counter this, Google released their own variant called ARCore on the 1st of march 2018. ARCore is also a software development kit for developing augmented reality applications but made for the Android, Unity and Unreal platforms instead. Since ARCore is released recently it is still unknown what particular limitations may exist for it. The purpose of this paper is give an indication to companies and developers about ARCore's potential limitations. The goal with this paper and work is to map how well ARCore works during different circumstances, and in particular, how its feature detection works and behaves. A quantitative research was done with the usage of the case study method. Various tests were performed with a modified test-application supplied by Google. The tests included test- ing how ARCore's feature detection, the process that analyzes the environment presented to the application. This which enables the user of an application to place a virtual object on the physical environment. The tests were done to see how ARCore works during different light levels, different types of surfaces, different angles and the difference between having the de- vice stationary or moving. From the testing that were done some conclusions could be drawn about the light levels, surfaces and differences between a moving and stationary device. More research and testing following these principles need to be done to draw even more conclusions of the system and its limitations. How these should be done is presented and discussed. Keywords| ARCore; augmented reality; Android; feature detection; markerless tracking; Google 2 Abstract F¨orst¨arktverklighet (augmented reality) har stigit under en tid och b¨orjatta sig in p˚a mobilmarknaden f¨orb˚adeIOS och Android. Ar˚ 2017 sl¨appteApple ARKit f¨orIOS vilket ¨ar en utvecklingsplattform f¨oratt utveckla applikationer inom f¨orst¨arktverklighet. Som svar p˚a detta s˚asl¨appteGoogle sin egna utvecklingsplattform vid namn ARCore, som sl¨apptesden 1 mars 2018. ARCore ¨arocks˚aen utvecklingsplattform f¨orutvecklandet av applikationer inom f¨orst¨arktverklighet men ist¨allet inom Android, Unity och Unreal. Sedan ARCore sl¨apptes nyligen ¨ardet fortfarande ok¨ant vilka s¨arskildabegr¨ansningarsom kan finnas f¨ordet. Syftet med denna avhandling ¨aratt ge f¨oretagoch utvecklare en indikation om ARCores potentiella begr¨ansningar. M˚aletmed denna avhandling och arbete ¨aratt kartl¨aggahur v¨alARCore fungerar under olika omst¨andigheter,och i synnerhet hur dess struktursdetektor fungerar och beter sig. En kvantitativ forskning gjordes med anv¨andningav fallstudie metoden. Olika tester utf¨ordes med en modifierad test-applikation fr˚anGoogle. Testerna inkluderade testning av hur AR- Cores struktursdetektor, processen som analyserar milj¨onrunt om sig, fungerar. Denna teknik m¨ojligg¨oratt anv¨andarenav en applikation kan placera ett virtuellt objekt p˚aden fysiska milj¨on.Testen innebar att se hur ARCore arbetar under olika ljusniv˚aer,olika typer av ytor, olika vinklar och skillnaden mellan att ha enheten station¨areller r¨orp˚asig. Fr˚antestningen som gjordes kan man dra n˚agraslutsatser om ljusniv˚aer,ytor och skillnader mellan en r¨orlig och station¨arenhet. Mer forskning och testning enligt dessa principer m˚asteg¨orasf¨oratt dra ¨annu mer slutsatser av systemet och dess begr¨ansningar.Hur dessa ska g¨oraspresenteras och diskuteras. Keywords| ARCore; f¨orst¨arktverklighet; Android; struktursdetektor; mark¨orl¨ossp˚arning;Google 3 Acknowledgements We would like to thank Slagkryssaren whom were the stakeholders of the project and which enabled us to implement it and carry out the work with the thesis. We would especially like to thank our supervisor at Slagkryssaren, Oskar Henriksson, whom provided a lot of guidance and support in the development work. We would also like to thank our supervisor at KTH, Firdose Saeik, whom gave us guidance throughout the thesis. Finally, we would like to thank our examiner at KTH, Konrad Tollmar, for making this work possible. 4 Contents 1 Introduction 7 1.1 Background.............................................7 1.2 Problem...............................................7 1.3 Purpose...............................................8 1.4 Goal.................................................8 1.4.1 Social benefits, Ethics and Sustainability........................8 1.5 Methodology............................................8 1.6 Stakeholder.............................................9 1.7 Delimitations............................................9 1.8 Disposition.............................................9 2 Background theory 11 2.1 Augmented Reality......................................... 11 2.1.1 Trackers and Feature Descriptors............................. 11 2.2 ARCore............................................... 11 2.2.1 Motion tracking...................................... 12 2.2.2 Environmental understanding............................... 12 2.2.3 Light estimation...................................... 12 2.2.4 User interaction...................................... 12 2.2.5 Oriented Points....................................... 12 2.2.6 Anchors and Trackables.................................. 12 2.3 Related work............................................ 13 3 Development/Models/Methods 14 3.1 Empirical research......................................... 14 3.1.1 Case study......................................... 14 3.2 Software development....................................... 14 3.2.1 Scrum............................................ 15 3.3 ANOVA............................................... 15 3.3.1 T-test............................................ 16 4 Evaluation criteria 17 4.1 Light intensity........................................... 17 4.2 Surface................................................ 18 4.3 Angle................................................ 18 4.4 Motion................................................ 18 4.5 Performance............................................. 18 4.6 Battery............................................... 18 4.7 Non-horizontal surface....................................... 19 4.8 Data collection........................................... 19 4.8.1 Detected points....................................... 19 4.8.2 Size of plane........................................ 19 4.8.3 Time............................................. 19 4.8.4 Analysis of the collected data............................... 19 5 Implementation 20 5.1 Literature study.......................................... 20 5.2 Case study............................................. 20 5.2.1 Definition of objectives and planning........................... 20 5.2.2 Data collection procedure................................. 20 5.2.3 Execution of data collection procedure.......................... 20 5.2.4 Analysis of collected data................................. 25 5.2.5 Presentation and conclusion of outcome......................... 25 5.3 Software development....................................... 25 5.3.1 Application......................................... 26 5.3.2 Added features....................................... 26 6 Result 27 6.1 Comparing light levels....................................... 27 6.2 Comparing angles.......................................... 29 6.3 Comparing surfaces......................................... 32 5 7 Data analysis 37 7.1 ANOVA test............................................ 37 7.2 Post-hoc test............................................ 38 8 Conclusions 42 8.1 Ethical aspect............................................ 42 8.2 Choice of methods......................................... 42 8.3 Analysis of results......................................... 43 8.4 Overall conclusions......................................... 45 8.5 Delimitations............................................ 46 8.6 Future work............................................. 46 Appendices 51 A Appendix 51 B Appendix 51 6 1 Introduction ARCore is a new software development kit released by Google with the intention of making it easier to create augmented reality applications. Applications developed through ARCore allows the user to interact with the environment, the technology overlays virtual contents onto the physical reality which we perceive with eyes, ears and other sensory organs. Augmented Reality has already been implemented within different sectors and categories such as industry, healthcare, education, marketing and entertainment, however the success of Augmented Reality depends on social acceptance and whether the technology is sufficiently user-friendly or not [1,2]. 1.1 Background Augmented reality is the concept of combining a direct or indirect view of the real-world with virtual content, where the elements that are shown will be enhanced in some way or another. This enhancement is usually done via some graphical addition to the real-world projection shown. Unlike virtual reality which projects a completely virtual world to the user, augmented reality only alters the current perception of the world [3]. There are