Performance & Scalability of a Spatial Database in a GIS-Web Service Environment Mark Olthof Arnhem, 2007 Thesis for a Ir. degree in Computer Science Faculty of EEMCS, chair Databases, University of Twente, The Netherlands Performance & Scalability of a Spatial Database in a GIS-Web Service Environment Thesis Project Description Mark Olthof (S0009202) Author Faculty of EEMCS, University of Twente Chair Databases LogicaCMG Arnhem Company Meander 901, 6801 HA Arnhem The Netherlands Department PSAR & GEO-ICT (Nieuwegein) Project Title SABRE: Optimization of Oracle Spatial database design Period 1 February 2007 – 18 November 2007 Maurice van Keulen (1st supervisor) Harold van Heerde (2nd supervisor) Gradutation Raynni Jourdain (LogicaCMG, Project leader WT) Commitee Olaf Lem (LogicaCMG, Project leader SABRE) To Lien I Abstract LogicaCMG developed SABRE, a spatial business rules server. Its purpose is to process queries for predefined business rules based on a given location. SABRE was rapidly developed as a project for 'master- class' employees. Although a functional demo was created, SABRE remained a simple prototype not ready for commercial purposes. LogicaCMG's goal is to redesign SABRE for commercial purposes, such as for instance tracking and tracing. This project focusses on the optimization of the database part, which will be the foundation of SABRE. If SABRE will be commercially deployed we can expect massive usage, and very high performance requirements. Therefore it's essential to develop SABRE from the ground up in such a way that maximum performance is achieved. The SABRE architecture consists basically of a Service Provider (SP), Web service (WS) and a Database (DB). The SP requests certain information from the DB through the WS. The aim of this project is to develop a working version of the redesigned SABRE application fit for demonstration purposes. The focus will be on the DB part, therefore the WS will support only one service (e.g. AREA-event). The objective of this assignment is to study the performance and scalability of the DB. The two most important scalability aspects are: how does the DB cope when the amount of requests increases and how does the DB cope when the amount of data increases. A scalable database design has been created of which a prototype has been implemented. The prototype was used for testing the performance and scalability of SABRE. To improve the performance and scalability three optimizations have been used, SQL Tuning, Materialized Views and Range Partitioning. The Materialized View optimization showed the best result with a 60% performance improvement. As a result of the optimization the SABRE performed well for four out of the five used datasets. The largest dataset was too large for the database to handle in terms of response times. However, since the tests have pointed out that SABRE is scalable, addressing the issue of the largest dataset should only be a matter of adding resources. When the required resources have been added the SABRE application will meet its requirements for commercial exploitation. Therefore it is expected to hear more from SABRE in the near future. Thesis Performance & Scalability of a Spatial Database in a GIS-Web Service Environment Mark Olthof - 1 - II Preface This thesis describes my research done for an Ir. Degree in Computer Science. The graduation project took nearly nine months, of which six took place at LogicaCMG Arnhem. The other three months I worked at home, mostly busy writing this thesis. During this project I gained a better understanding of databases and performance and scalability aspects every single day. A lot of my time went into the design and realisation of a prototype. When the prototype had finished, the time came to test my prototype which also took a considerable amount of time. The last part of my project, writing this thesis, took a large amount of time as well, way more than I expected. Although sometimes the outcome of the performance and scalability tests drove me insane because of the unexpected results which caused mind boggling theorizing, I can look back at a interesting nine months. I would like to thank my LogicaCMG supervisors Raynni Jourdain and Olaf Lem for their devoted support during my stay at LogicaCMG Arnhem. I would especially like to thank Olaf for his insights and the great discussions we have had on the subject matter. I would like to thank my UT supervisors Maurice van Keulen and Harold van Heerde for their hints and tips they gave me which resulted in a more profound research project. Especially during the writing of my thesis their support was crucial. Last but certainly not least I would like to thank my girlfriend, Caroline, for always supporting me during these nine months, especially when I needed it the most. Mark Olthof Nijmegen, 18 November 2007 Thesis Performance & Scalability of a Spatial Database in a GIS-Web Service Environment Mark Olthof - 2 - Table of Contents I Abstract............................................................................................................................................... 1 II Preface............................................................................................................................................... 2 1 Introduction......................................................................................................................................... 6 1.1 Motivation................................................................................................................................................6 1.2 Problem Description................................................................................................................................ 7 1.3 Project Context.........................................................................................................................................8 1.4 Approach..................................................................................................................................................8 1.5 Project Goal............................................................................................................................................. 9 1.6 Thesis Structure....................................................................................................................................... 9 2 Related Work..................................................................................................................................... 10 2.1 GIS......................................................................................................................................................... 10 2.1.1 GIS Databases.............................................................................................................................................. 11 2.2 DB Techniques.......................................................................................................................................11 2.3 Benchmarking........................................................................................................................................ 11 2.4 Definitions..............................................................................................................................................12 2.5 Chapter Summary.................................................................................................................................. 13 3 Design SABRE Application.............................................................................................................. 14 3.1 Requirements SABRE........................................................................................................................... 14 3.2 Original version..................................................................................................................................... 16 3.3 SABRE Architecture..............................................................................................................................16 3.3.1 XML Component.......................................................................................................................................... 17 3.3.2 Web Service Component............................................................................................................................... 18 3.3.3 Database Component................................................................................................................................... 18 3.4 Detailed SABRE Design........................................................................................................................20 3.5 SABRE Data Model...............................................................................................................................25 3.6 Design Choices...................................................................................................................................... 28 3.6.1 Division into three separate databases......................................................................................................... 28 3.6.2 Data Model.................................................................................................................................................. 28 3.7 Chapter Summary.................................................................................................................................. 29 4 SABRE Prototype.............................................................................................................................