POLITECNICO DI TORINO COLLEGIO DI INGEGNERIA INFORMATICA, DEL CINEMA E MECCATRONICA Master Degree in Computer Engineering Master Degree Thesis Immersive Analysis of Hierarchical Semantic Datasets Supervisors Candidate Prof. Valentina Gatteschi Salvatore Calogero Di Cara Prof. Fabrizio Lamberti matricola: 251608 December 2019 I hereby declare that the contents and organisation of this thesis constitute my own original work and does not compromise in any way the rights of third parties, including those relating to the security of personal data. ................................... Salvatore Calogero Di Cara Turin, October 10, 2019 This work is subject to the Creative Commons Licence Sommario I recenti avanzamenti tecnologici nell’ambito della Realtà Virtuale (RV) hanno reso degli innovativi ed efficienti visori sufficientemente economici dapoter essere alla portata di un ampio pubblico. Oltre alle classiche applicazioni in RV, dedicate soprattutto all’intrattenimento e alla simulazione, gli studi in un nuovo campo di ricerca, detto Immersive Analytics, hanno evidenziato come le moderne tecnologie di visualizzazione e interazione abbiano un grande po- tenziale per creare degli utili strumenti di analisi dei dati. In quest’epoca, in cui la quantità di dati disponibili continua a crescere esponenzialmente, può essere di fondamentale importanza riuscire a estrarre informazioni utili da grandi collezioni di documenti. Inoltre, un’applicazione che rappresenti que- ste informazioni in maniera stimolante potrebbe suscitare l’interesse non solo degli esperti di dominio e degli analisti, ma anche degli utenti generici. Il pre- sente lavoro di tesi si occupa di discutere la progettazione e lo sviluppo di una piattaforma che, sfruttando le moderne tecnologie di RV, fornisca un ambien- te immersivo per la visualizzazione, l’analisi e la condivisione di dati relativi a dei documenti. In particolare, l’informazione è stata rappresentata mediante termini raggruppati in categorie semantiche. L’obiettivo è di fornire agli uten- ti un utile strumento che permetta loro di accrescere la conoscenza relativa a un certo tema e di individuarne facilmente gli argomenti principali. Le carat- teristiche fondamentali dell’applicazione realizzata includono un’accessibilità distribuita basata sul web, portabilità su diversi dispositivi, interazioni natura- li con l’ambiente virtuale, meccanismi di collaborazione asincrona e menu che permettono di modificare la categorizzazione rappresentata. Il progetto è stato condotto in collaborazione con il gruppo di ricerca DEI dell’Università Carlos III di Madrid. I membri del gruppo, in virtù della loro esperienza accumulata, hanno supervisionato il lavoro e fornito i requisiti necessari alla sua realiz- zazione. Inoltre, sono stati condotti dei test di valutazione della piattaforma, stimandone l’usabilità e mettendo in luce i vantaggi derivanti dall’utilizzarla in un ambiente immersivo, come quello offerto da un visore per la RV. Iri- sultati di tali test verranno discussi nella parte finale di questo documento. Abstract The recent improvements in Virtual Reality (VR) technologies made novel and efficient headsets cheap enough to be affordable for the wide public. Inaddi- tion to the classic VR applications, mainly with entertainment and simulation purposes, studies in the emerging research field of Immersive Analytics have highlighted the great potential of modern visualization and interaction tech- niques to create engaging data analysis tools. In this era, where the amount of available data keeps growing exponentially, it can be crucial to extract use- ful knowledge from large collections of documents. Furthermore, an appli- cation to depict such information in a stimulating way can arouse enthusi- asm not only in domain experts and data analysts but also in casual users. The present thesis work discusses the design and development of a platform which, exploiting modern VR technologies, provides an immersive environ- ment for the visualization, analysis, and sharing of document-related data. In particular, information has been represented as terms grouped into semantic categories. The goal is to provide users with an useful instrument that allows them to increase their knowledge on a certain subject and easily detect main topics. The key features of the realized application include a distributed web- based accessibility, portability on several devices, natural interactions with the virtual elements, asynchronous collaboration mechanisms, and menu in- terfaces to edit the presented categorization. The project has been conducted in collaboration with the DEI research team at University Carlos III of Madrid. The members of the team, thanks to their previous experience, supervised the work and supplied the requirements for its realization. An evaluation of the platform has also been conducted, assessing its usability and pointing out the benefits brought by using it in an immersive environment, such astheone offered by a VR headset. The results of this evaluation will be discussed inthe final part of the present document. Acknowledgements I would like to acknowledge Prof. Paloma Díaz and PhD. Teresa Onorati from University Carlos III of Madrid, who supported the development of this research. 3 Contents List of Tables 8 List of Figures 9 1 Introduction 11 1.1 Virtual Reality ........................... 11 1.1.1 Immersion and presence ................. 12 1.1.2 VR technologies overview ................ 13 1.2 Immersive Analytics ........................ 15 1.2.1 Historical Context .................... 15 1.2.2 What is Immersive Analytics? .............. 17 1.3 Motivations and goals ....................... 18 1.4 Chapters overview ......................... 19 2 State of the Art 21 2.1 Literature overview ........................ 21 2.1.1 A virtual reality visualization tool for neuron tracing ........................... 22 2.1.2 Visualization of vector field by virtual reality ..... 23 2.1.3 An immersive visualization study on molecules manipulation ....................... 23 2.1.4 Immersive and collaborative data visualization using virtual reality platforms .............. 25 2.1.5 Immersive visualization of abstract information: an evaluation on dimensionally-reduced data scatterplots ........................ 26 2.1.6 Building immersive data visualizations for the web .. 27 4 2.1.7 Immersive collaborative analysis of network connectivity: Cave-style or head-mounted display? .. 28 2.1.8 Maps and globes in virtual reality ............ 29 2.1.9 Origin-destination flow maps in immersive environments ....................... 30 2.1.10 FiberClay: sculpting three dimensional trajectories to reveal structural insights ................. 31 2.1.11 ImAxes: immersive axes as embodied affordances for interactive multivariate data visualisation ....... 32 2.1.12 3D visualization of astronomy data cubes using immersive displays .................... 33 2.1.13 A study of layout, rendering, and interaction methods for immersive graph visualization ...... 33 2.1.14 VRMiner: a tool for multimedia database mining with virtual reality ....................... 34 2.1.15 The hologram in my hand: how effective is interactive exploration of 3D visualizations in immersive tangible Augmented Reality? ........ 35 2.1.16 Multilingual semantic cyberspace of scientific papers based on WebVR technology .......... 37 2.1.17 Development of emergency drills system for petrochemical plants based on WebVR ......... 37 2.1.18 Exploring data in virtual reality: comparisons with 2D data visualizations .................... 38 2.1.19 The data in your hands: exploring novel interaction techniques and data visualization approaches for immersive data analytics ........ 39 2.2 Related Works ........................... 40 2.2.1 A taxonomy generation tool for semantic visual analysis of large corpus of documents ......... 40 2.2.2 Designing an immersive visualization for semantic datasets ..................... 41 2.3 Considerations ........................... 42 3 Requirements 51 4 Technologies 57 4.1 Software .............................. 57 5 4.1.1 Node.js ........................... 57 4.1.2 A-Frame .......................... 59 4.1.3 D3.js ............................ 63 4.1.4 WebSpeech API ...................... 63 4.1.5 Hosting .......................... 64 4.1.6 Atom ............................ 64 4.2 Hardware .............................. 64 4.2.1 Oculus Rift ........................ 65 5 Development 67 5.1 Server-side ............................. 67 5.1.1 Providing application ................... 67 5.1.2 File managing ....................... 68 5.1.3 Vocal commands handling ................ 68 5.2 Client-side ............................. 69 5.2.1 Taxonomy file format .................. 72 5.2.2 Scene design ....................... 73 5.2.3 Interaction ......................... 78 5.2.4 Navigation ......................... 82 5.2.5 Term details ........................ 85 5.2.6 Bubble menu ....................... 87 5.2.7 Support to collaboration ................. 91 6 Results 95 6.1 Evaluation ............................. 95 6.2 Experimental settings ....................... 97 6.2.1 Apparatus ......................... 97 6.2.2 Procedure ......................... 97 6.2.3 Case Study ........................ 98 6.2.4 Questionnaire ....................... 98 6.3 Participants ............................ 99 6.4 Results and findings ........................ 99 6.4.1 Simulator sickness .................... 100 6.4.2 SUS score ........................