OPEN CLOUD BASED DISTRIBUTED GEO-ICT SERVICES A Thesis submitted to Gujarat Technological University for the Award of Doctor of Philosophy In Computer Engineering by JHUMMARWALA ABDUL TAIYAB ABUZAR Enrollment Number: 129990907004 Under the Supervision of Dr M.B. Potdar GUJARAT TECHNOLOGICAL UNIVERSITY AHMEDABAD September – 2018 © Jhummarwala Abdul Taiyab Abuzar DECLARATION I declare that the thesis entitled “Open Cloud based Distributed Geo-ICT Services ” submitted by me for the degree of Doctor of Philosophy in Computer Engineering is the record of research work carried out by me during the period from January 2013 to September 2018 under the supervision of Dr. M. B. Potdar, Project Director, Bhaskaracharya Institute for Space Applications and Geo-informatics (BISAG), Gandhinagar, and this has not formed the basis for the award of any degree, diploma, associate ship, fellowship, titles in this or any other University or other institution of higher learning. I further declare that the material obtained from other sources has been duly acknowledged in the thesis. I shall be solely responsible for any plagiarism or other irregularities, if noticed in the thesis. Signature of Research Scholar Date: 29 th September, 2018 Name of Research Scholar: Jhummarwala Abdul Taiyab Abuzar i CERTIFICATE I certify that the work incorporated in the thesis “Open Cloud based distributed Geo-ICT services ” submitted by Jhummarwala Abdul Taiyab Abuzar was carried out by the candidate under my guidance. To the best of my knowledge: (i) the candidate has not submitted the same research work to any other institution for any degree, diploma, Associate ship, Fellowship or other similar titles (ii) the thesis submitted is a record of original research work done by the Research Scholar during the period of study under my supervision, and (iii) the thesis represents independent research work on the part of the Research Scholar. Signa ture of Supervisor: …………………………………. Date: 29 th September, 2018 Name of Supervisor: Dr. M. B. Potdar, Project Director, Bhaskaracharya Institute for Space Applications and Geo-informatics Place: Gandhinagar ii Originality Report Certificate It is certified that PhD Thesis titled “Open Cloud based Distributed Geo-ICT Services ” by Jhummarwala Abdul Taiyab Abuzar has been examined by us. We undertake the following: a. Thesis has significant new work as compared to already published or are under consideration to be published elsewhere. No sentence, equation, diagram, table, paragraph or section has been copied verbatim from previous work unless it is placed under quotation marks and duly referenced. b. The work presented is original and own work of the author (i.e. there is no plagiarism). No ideas, processes, results or words of others have been presented as Authors own work. c. There is no fabrication of data or results which have been compiled / analysed. d. There is no falsification by manipulating research materials, equipment or processes or changing or omitting data or results such that the research is not accurately represented in the research record. e. The thesis has been checked using Turnitin Plagiarism Software (copy of originality report attached) and found within limits as per GTU Plagiarism Policy and instructions issued from time to time (i.e. permitted similarity index <= 25 %). Signature of Research Scholar Date: 29 th September, 2018 Name of Research Scholar: Jhummarwala Abdul Taiyab Abuzar Place: Gandhinagar Signature of Supervisor Date: 29 th September, 2018 Name of Supervisor: Dr. M. B. Potdar, Project Director Bhaskaracharya Institute for Space Applications and Geo-Informatics (BISAG) Place: Gandhinagar iii iv v vi vii viii PhD THESIS Non-Exclusive License to GUJARAT TECHNOLOGICAL UNIVERSITY In consideration of being a PhD Research Scholar at GTU and in the interests of the facilitation of research at GTU and elsewhere, I, Jhummarwala Abdul Taiyab Abuzar , having Enrollment No. 129990907004 hereby grant a non-exclusive, royalty free and perpetual licence to GTU on the following terms: a. GTU is permitted to archive, reproduce and distribute my thesis, in whole or in part, and / or my abstract, in whole or in part (referred to collectively as the “Work”) anywhere in the world, for non-commercial purposes, in all forms of media; b. GTU is permitted to authorize, sub-lease, sub-contract or procure any of the acts mentioned in paragraph (a); c. GTU is authorized to submit the Work at any National / International Library, under the authority of their “Thesis Non -Exclusive Licence”; d. The University Copyright Notice © shall appear on all copies made under the authority of this license; e. I undertake to submit my thesis, through my University, to any Library and Archives. Any abstract submitted with the thesis will be considered to form part of the thesis. f. I represent that my thesis is my original work, does not infringe any rights of others, including privacy rights, and that I have the right to make the grant conferred by this nonexclusive license. g. If third party copyrighted material was included in my thesis for which, under the terms of the Copyright Act, written permission from the copyright owners is required, I have obtained such permission from the copyright owners to do the acts mentioned in paragraph (a) above for the full term of copyright protection. ix h. I retain copyright ownership and moral rights in my thesis, and may deal with the copyright in my thesis, in any way consistent with rights granted by me to my University in this non- exclusive license. i. I further promise to inform any person to whom I may hereafter assign or license my copyright in my thesis of the rights granted by me to my University in this non-exclusive license. j. I am aware of and agree to accept the conditions and regulations of PhD including all policy matters related to authorship and plagiarism. Signature of Research Scholar Date: 29 th September, 2018 Name of Research Scholar: Jhummarwala Abdul Taiyab Abuzar Place: Gandhinagar Signature of Supervisor Date: 29 th September, 2018 Name of Supervisor: Dr. M. B. Potdar, Project Director, Bhaskaracharya Institute for Space Applications and Geo-informatics (BISAG) Place: Gandhinagar x Thesis Approval Form The viva-voce of the PhD Thesis submitted by Jhummarwala Abdul Taiyab Abuzar, (Enrollment No. 129990907004 ) entitled “Open Cloud based Distributed Geo-ICT Services ” was conducted on …………………….…………, at Gujarat Technological University. (Please tick any one of the following option) We recommend that he be awarded the PhD degree. We recommend that the viva-voce be re-conducted after incorporating the following suggestions: The performance of the candidate was unsatisfactory. We recommend that he should not be awarded the Ph. D. Degree. ………………………..…………………….. …………...……………………..………… Name and Signature of Supervisor with Seal 1) External Examiner 1 Name and Signature ………………………..………………..…… ………………………………..………… 2) External Examiner 2 Name and Signature 3) External Examiner 3 Name and Signature xi Dedicated to My family xii ABSTRACT A Geographic Information System (GIS) consists of a collection of applications which operate upon geographic data and are utilized for planning purposes. Geospatial data is collected by Earth Observation Satellites which is used across domains of weather forecasting, oceanography, forestry, climate, rural and urban planning, disaster management, etc and increases daily in volume by hundreds of terabytes. Storing such large volumes of data is indeed a challenge but processing such volumes and deriving useful information which is required for planning purposes and accurate prediction required for decision making forms one of the most important parts of the challenge. This processing of large volumes of data requires application of new technologies and distributed architectures to make possible to extract value from them. Geospatial data which is mostly collected in raster formats, apart from the global positioning system (GPS), is transformed to a more usable vector format after application of image processing techniques (which may include manual editing, etc). The derived vector data can be stored by the specialized software in consideration in form one of the various 100’s of formats devised to store vector data. Shapefiles (.shp) and XML (eXtensible Markup Language) format forms the most prominent vector data formats. Crowd sourced, publicly available and license free dataset available from OpenStreetMap project (OSM) represents vector features in XML and is stored in OSM XML format. For the year 2016, this OSM data amounts to > 800 GB and consists of more than 3.5 billion unique vector features. This is just one representation of massiveness of geo-spatial data. The analysis required to be performed on such huge datasets require large amounts of storage, compute and memory and is not possible with a single computer using traditional geographic information systems. The advancement and ease of managing IaaS (Infrastructure as a Service) resources in Cloud Computing for utilizing remote infrastructure for distributed storage, distributed processing and advanced network capabilities compels their development and application for use with these large amounts of geographical data. Our focus is on processing of large amount of geospatial vector data which is available in form of Shapefiles (a collection of xiii .prj, .shp, .shx, .dbf and other shapefile component files). The available shapefile dataset consists of more than 330,000 shapefiles which