VISUALIZATION FOR EXPLORATORY ANALYSIS OF SPATIO-TEMPORAL DATA by HASAN SERDAR ADALI Submitted to the Graduate School of Engineering and Natural Sciences in partial fulfillment of the requirements for the degree of Master of Science Sabancı University August 2012 © HASAN SERDAR ADALI 2012 All Rights Reserved VISUALIZATION FOR EXPLORATORY ANALYSIS OF SPATIO-TEMPORAL DATA Hasan Serdar Adalı EECS, M.Sc. Thesis, 2012 Thesis Advisor: Assist.Prof.Dr.Selim Balcısoy Keywords: Heat Map, Data Visualization, Spatio-Temporal Data, Visual Analytics Abstract Analysis of spatio-temporal data has become critical with the emerge of ubiquitous location sensor technologies and applications keeping track of such data. Especially with the widespread availability of low cost GPS devices, it is possible to record data about the location of people and objects at a large scale. Data visualization plays a key role in the successful analysis of these kind of data. Due to the complex nature of this analysis process, current approaches and analytical tools fail to help spatio-temporal thinking and they are not effective when solving large range of problems. In this work, we propose an interactive visualization tool to support human analyst understand user behaviors by analyzing location patterns and anomalies in massive collections of spatio-temporal data. The tool that we developed within this work combines a geovisualization framework with 3D visualizations and histograms. Tool's effectiveness in exploratory analysis is tested by trend analysis and anomaly detection in a real mobile service dataset with almost 1.5 million rows. UZAM-ZAMANSAL VERILER_ IN_ KES¸IFSEL_ ANALIZ_ I_ IC¸_ IN_ GORSELLES¸T¨ IRME_ Hasan Serdar Adalı EECS, Y¨uksekLisans Tezi, 2012 Tez Danı¸smanı:Yrd. Do¸c.Selim Balcısoy Anahtar Kelimeler: Isı haritası, Veri g¨orselle¸stirmesi,Uzam-zamansal veriler, G¨orselAnalitik Ozet¨ Konum sens¨or¨uteknolojilerinin geli¸smesive bu sens¨orlerden elde edilen verilerin kullanımının yaygınla¸smasıile birlikte, uzam-zamansal veri adı verilen, hem konum hem de zaman bilgisi i¸cerenveri setlerinin analizi ¸cokdaha kritik bir hale gelmi¸stir. Ozellikle¨ ucuza maledilebilen GPS sens¨orl¨ucihazlarının kolay eri¸silebilirli˘gisayesinde artık b¨uy¨uk¸captakiinsan topluluklarının ve objelerin pozisyonlarını kayıt altında tutabilmek kolayla¸smı¸sve analiz amacıyla depolanan bu uzam-zamansal verilerin miktarını ¸coky¨uksekboyutlara ula¸stırmı¸stır.Veri g¨orselle¸stirmesi,depolanan bu verilerin etkili analizi i¸cingereken yardımı sa˘glamadakilit role sahiptir. Ancak uzam-zamansal verinin analiz s¨urecinin karma¸sıkyapısından dolayı, g¨un¨um¨uzdekig¨orselle¸stirmeyakla¸sımlarıve ara¸cları ile istenilen d¨uzeydehızlı bir analiz yapmak, her durumda m¨umk¨unolmamaktadır. Bu ¸calı¸smada,¸cokb¨uy¨ukkapasitedeki uzam-zamansal verilerin analizine katkıda bulunabilmek ve verinin i¸cerdi˘gipatern ve anormalliklerin tespiti amacıyla, ¸ce¸sitli g¨orselle¸stirmetekniklerinin bir arada bulunduran interaktif bir ara¸csunmaktayız. Co˘grafig¨orselle¸stirme,histogram ve ¨u¸cboyutlu teknikler i¸cerenbu aracın efektifli˘gini¨ol¸cmekamacıyla yaptı˘gımız¸calı¸smalarda,T¨urkiye'de hala kullanılmakta olan bir mobil servis uygulamasının verilerinden faydalandık. Acknowledgements First and foremost, I wish to express my sincere gratitude to my supervisor Asst. Prof. Dr. Selim Balcısoy for his guidance and advices. He inspired and motivated me to work in this project. Without his encouragment, I would have never managed to finish my work. I am honored to have Berrin Yanıko˘glu,Cemal Yılmaz, Erkay Sava¸sand Burin Bozkaya as members of my thesis committee. I am grateful for their valuable review and comments on the thesis. I also would like to thank all my lab colleagues in Computer Graphics Laboratory, especially Dr. Ekrem Serin, for his great efforts and cooperation in all the studies we made together. I would like to thank Ceren Kayalar, Seluk Smengen, Murat elik Cansoy and Candemir D˘gerfor their great support and friendship. Last but not least, thanks to Mustafa Tolga Eren who made this project much better with his great vision and expertise. Finally, I wish to thank my family for always loving and supporting me all the way. i TABLE OF CONTENTS Acknowledgements i List of Figures iv 1 INTRODUCTION 1 1.1 Overview . .1 1.2 Thesis Outline . .2 2 RELATED WORK 4 2.1 Visualization . .4 2.1.1 Overview . .4 2.1.2 Time-Series Data Visualization . .5 2.1.3 Geovisualization . .6 2.1.4 Spatio-temporal Data Visualization . .8 2.2 Analysis of Spatio-temporal Data . 10 2.2.1 Visual Analytics . 10 2.2.2 Applications and Tools . 10 3 GEOVISUALIZATION FRAMEWORK USING HEAT MAPS 13 3.1 Input Data Characteristics . 14 3.2 Dot Density Maps . 14 3.2.1 Coordinate Transformation . 15 3.2.2 Interaction . 16 3.3 Spatial Clustering . 17 3.4 Intensity Map . 19 3.4.1 Vector Grids for Binning Data . 20 3.4.2 Radial Gradient Blending . 21 3.5 Colorization . 22 ii 4 VISUALIZATION OF CHANGE OVER TIME IN GEOGRAPHIC DATA 25 4.1 Heat Map Raster Animation . 25 4.2 Heat Map of Change . 26 4.3 Overlaying Maps . 27 4.3.1 Average of Geographic Data . 28 4.3.2 Bump Mapping . 29 4.3.3 Contour Map . 31 4.4 HeatCube . 31 5 SPATIO-TEMPORAL DATA ANALYSIS TOOL 34 5.1 Design . 34 5.1.1 Temporal Component . 35 5.1.2 Geographic Component . 36 5.2 Exploratory Data Analysis Tasks . 36 5.2.1 Identify . 37 5.2.2 Compare . 37 6 RESULTS 42 6.1 Anomaly Detection Scenerio . 42 6.2 Trend Analysis Scenerio . 43 7 CONCLUSION AND FUTURE WORK 46 7.1 Conclusion . 46 7.2 Future work . 46 iii List of Figures 2.1 Police assignments data that shows the number of deployed units for intervals that are five minutes long. Each block represents one day. Inside the blocks, the hours are shown in rows. Each row has one pixel for every five-minute-interval[1]. .6 2.2 A good example of using color to visualize the data density[2] . .7 2.3 In this visualization[3], the origins and the destinations of the flows are displayed in two separate maps, and the changes over time of the flow magnitudes are represented in a separate heatmap view in the middle. .9 2.4 Spatio-temporal visualization of vessel trajectories[4] . .9 2.5 The sense making loop for Visual Analytics [5] . 11 2.6 Tools for analysis of spatio-temporal data . 12 3.1 Overview of Heat Map System . 13 3.2 Dot density map on OpenStreetMap . 17 3.3 Spatial Cluster Map of daily data . 18 3.4 Comparison between intensity calculation methods. 19 3.5 Intensity calculation with hexagonal binning . 21 3.6 Intensity map calculation with radial gradient . 22 3.7 Colorization . 24 4.1 Heat map representation of change in data. Red means increase in the areas while blue inditaces negative change. 27 4.2 Geographic data average with grid method . 29 4.3 Heat map representation of average data from Istanbul . 29 4.4 Normal map representation of geographic data . 30 4.5 Overlaying Bump map with Heat Map . 31 iv 4.6 Overlaying isolines on heat maps . 32 4.7 An example view from our interactive HeatCube visualization . 33 5.1 Main structure of our tool . 34 5.2 Our tool for spatio-temporal data analysis. 36 5.3 Trend analysis routine using our tool . 40 6.1 Temporal anomaly detection . 43 6.2 Spatial anomaly detection . 44 6.3 Trend Analysis . 45 7.1 Isosurface experiments with data . 47 v 1 INTRODUCTION 1.1 Overview \Exploratory data analysis is detective work{numerical detective work{or counting detective work{or graphical detective work. A detective investigating a crime needs both tools and understanding. If he has no fingerprint powder, he will fail to find fingerprints on most surfaces. If he does not understand where the criminal is likely to have put his fingers, he will will not look in the right places. Equally, the analyst of data needs both tools and understanding (p 1: Tukey (1977))" When John Tukey[6] came up with the idea of visually exploring data sets to discover their main characteristics, graphical representation of data had become crucial for data analysis. Nowadays, as the amount of data that we produce gets massive, data visualization field plays more important role in analyzing massive data sets to derive information from them. Considering the type of data and the purpose of analysis, appropriate use of visualization techniques aids users to understand their massive data without using a statistical model or having formulated a hypothesis. In this thesis, we propose an interactive geographic data visualization framework which contains several visualization techniques to help users detect patterns and anomalies in their spatio-temporal data. Over the past few years, visualization community have worked in closely related problems with the cartographic and geographic information system(GIS) commu- nities. The crossdisciplinary connection between these fields facilitates the visual display and interactive exploration of geospatial data and the information derived from it. Analysis of this geographic information in time and space is becoming a very important subject with the increasing use of location data. Recently, re- searchers are looking for approaches to deal with the complexities of the current 1 data and problems. In their extensive work, Andrienko and Andrienko[7] emphasize the need of visualization techniques and analytical tools that will support spatio- temporal thinking and contribute to solving a large range of problems. However, due to the sophisticated nature of spatio-temporal data analysis, current visualiza- tion techniques and analytical tools[8] are not fully effective and they need to be improved. To address issues in spatio-temporal data analysis, we experimented several different visualization techniques and proposed an interactive tool to support ex- ploratory analysis of spatio-temporal data.