Technical University of Crete School of Electrical and Computer Engineering Interactive Dynamic Time Lapse Video for Immersive and Personalized Tourism Applications Author: Theodoros Geronymakis A thesis submitted in partial fulfillment of the requirements for the diploma of Electrical & Computer Engineer. Committee: Emeritus Professor Stavros Christodoulakis (Supervisor) Associate Professor Katerina Mania Associate Professor Antonios Deligiannakis 2 Acknowledgments First of all, I would like to express my gratitude to my supervisor, Stavros Christodoulakis, for the patient guidance, encouragement and advice he has provided throughout my thesis. I would also like to thank Prof. Katerina Mania and Prof. Antonios Deligiannakis for serving on my thesis committee, as well as the members of the staff of the MUSIC laboratory for their help and support. Furthermore I would like to express my deepest gratitude to my parents, my uncle and especially my brother for their continuous support, endless encouragement, and confidence in me. Finally, I want to thank my friends for all the good times we had together during our studies. 3 “All's well that ends well.” ― William Shakespeare 4 Abstract We present a system that can be used to generate web based interactive dynamic and personalized video based navigations through a city or geographic area. The dynamic video presentations are generated from a database of panoramic images for the area, and/or from StreetView panoramic images. User preferences and interactions can change what and how it is presented during the navigation and they give a sense of personalization but also freedom of navigation, flexibility and immersion to the final user. 5 Contents 1 Introduction 16 1.1 Background ................................................................................................................................. 16 1.2 Thesis Layout ............................................................................................................................... 17 2 Related Work & State of The Art 18 2.1 Introduction ................................................................................................................................ 18 2.2 Geographic Context Understanding with Mobile Applications in Culture and Tourism ............ 18 2.3 Paneek: Virtual Tour Software .................................................................................................... 19 2.3.1 Navigation ........................................................................................................................... 19 2.3.2 Hotspots .............................................................................................................................. 21 2.3.3 Differences with our application ......................................................................................... 21 2.4 Audio Tours with Mobile Device GPS Navigation ....................................................................... 23 2.4.1 PocketGuide Audio Travel Guide: ....................................................................................... 23 2.4.2 Google Trips: ....................................................................................................................... 24 2.4.3 Audio augmented reality: a prototype automated tour guide ........................................... 25 2.4.4 Detour - Immersive Audio Tours: ........................................................................................ 25 2.4.5 Field Trip.............................................................................................................................. 26 2.4.6 Izi.Travel .............................................................................................................................. 27 2.5 Augmented Reality Tourism Applications: Viewranger, Google Goggles, AR City...................... 31 2.5.1 Google Goggles ................................................................................................................... 31 2.5.2 Viewranger .......................................................................................................................... 32 2.5.3 Terrain Understanding from Pictures and Augmented Reality........................................... 33 2.5.4 Collaborative Augmented Reality for Outdoor Navigation and Information Browsing ...... 33 2.5.5 AR City: Augmented Reality Maps and Navigation (Beta) .................................................. 34 2.5.6 Differences with our application ......................................................................................... 35 2.6 Route Simulators with StreetView Integration ........................................................................... 36 2.6.1 Google Maps StreetView Player ......................................................................................... 36 2.6.2 Gaiagi Driving Simulator ...................................................................................................... 37 2.6.3 Hyperlapse .......................................................................................................................... 38 2.7 Technologies & Implementation Architecture ............................................................................ 38 2.7.1 HTML5 ................................................................................................................................. 39 2.7.2 JavaScript & Ajax ................................................................................................................. 39 6 2.7.3 Servlets ................................................................................................................................ 40 2.7.4 Java Server Pages (JSP) ........................................................................................................ 41 2.7.5 Google Maps JavaScript API ................................................................................................ 41 2.7.6 Photo Sphere Viewer .......................................................................................................... 42 2.7.7 Materialize CSS and JavaScript Library ............................................................................... 43 2.7.8 Programming Tools ............................................................................................................. 44 2.7.9 Google Camera .................................................................................................................... 45 3 Functional Overview & Contributions 47 3.1 Introduction ................................................................................................................................ 47 3.2 Functionality Overview and Contributions ................................................................................. 47 3.3 Information and Services Model ................................................................................................. 51 3.4 Motivation and Information Details ........................................................................................... 56 4 Database Design 67 4.1 Introduction ................................................................................................................................ 67 4.2 Entities and Relationships of the Database ................................................................................ 71 4.3 Functions of the Database .......................................................................................................... 76 4.4 Relational Schema ....................................................................................................................... 78 5 Application Functionality and Algorithms 82 5.1 Introduction ................................................................................................................................ 82 5.2 Construction of the Map Object ................................................................................................. 82 5.3 Construction of the Route Object ............................................................................................... 83 5.4 Construction of Objects of Interest ............................................................................................ 85 5.4.1 Single Object of Interest...................................................................................................... 86 5.4.2 Polygon Object of Interest: ................................................................................................. 87 5.5 Construction of Object Presentation Sections ............................................................................ 89 5.6 Construction of Object Presentation Methods .......................................................................... 91 5.7 Construction of UserCaptured Panoramic Pictures Route Section............................................. 92 5.8 Construction of Superimposed Images on top of UserCaptured Panoramic Pictures ................ 97 5.8.1 Retrieval of StreetView Panoramic Picture Binary Data ..................................................... 99 5.8.2 Placement of Superimposed Images on top of the Existing StreetView Panoramic Pictures 103 5.9 Placement of Superimposed Images on UserCaptured Panoramic Pictures ............................ 109 7 5.10 Database Connection ................................................................................................................ 112 5.11 Automatic Camera Orientation and Tuning .............................................................................