DOCSLIB.ORG

Developing a Framework for Geographic Question Answering Systems

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Developing a Framework for Geographic Question Answering Systems Developing a Framework for Geographic Question Answering Systems Using GIS, Natural Language Processing, Machine Learning, and Ontologies DISSERTATION Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School of The Ohio State University By Wei Chen Graduate Program in Geography The Ohio State University 2014 Dissertation Committee: Dr. Eric Fosler-Lussier (committee member) Dr. Rajiv Ramnath (committee member) Dr. Daniel Sui (committee member) Dr. Ningchuan Xiao (committee chair) Copyrighted by Wei Chen 2014 Abstract Geographic question answering (QA) systems can be used to help make geographic knowledge accessible by directly giving answers to natural language questions. In this dissertation, a geographic question answering (GeoQA) framework is proposed by incorporating techniques from natural language processing, machine learning, ontological reasoning and geographic information system (GIS). We demonstrate that GIS functions provide valuable rule-based knowledge, which may not be available elsewhere, for answering geographic questions. Ontologies of space are developed to interpret the meaning of linguistic spatial terms which are later mapped to components of a query in a GIS; these ontologies are shown to be indispensable during each step of question analysis. A customized classifier based on dynamic programming and a voting algorithm is also developed to classify questions into answerable categories. To prepare a set of geographic questions, we conducted a human survey and generalized four categories that have the most questions for experiments. These categories were later used to train a classifier to classify new questions. Classified natural language questions are converted to spatial SQLs to retrieve data from relational databases. Consequently, our demo system is able to give exact answers to four categories of geographic questions within an average time of two seconds. The system has been evaluated using classical ii machine learning-based measures and achieved an overall accuracy of 90% on test data. Results show that spatial ontologies and GIS are critical for extending the capabilities of a GeoQA system. Spatial reasoning of GIS makes it a powerful analytical engine to answer geographic questions through spatial data modeling and analysis. iii Dedication To Ying Dong (Susie), my wife To Jianxi Chen, Donglin Wei, my parents To Tiejing Dong, Liqun Wang, my parents in law To Baoyuan Chen, my deceased grandpa iv Acknowledgments I would like to acknowledge the efforts of all committee members Dr. Daniel Sui, Dr. Eric Fosler-Lussier and Dr. Rajiv Ramnath for their significant contribution to the completion of this dissertation. My Ph.D. advisor Dr. Ningchuan Xiao was very helpful throughout the entire process of my Ph.D. study. He helped me with academic, financial, and spiritual support, especially during the last two semesters of my masters in geography and the final year of my Ph.D.. Many thanks also extend to my friends from both computer science and geography department at the Ohio State University, Dr. Shanshan Cai, Dr. Meng Guo, Dr. Shiguo Jiang, Dr. Michael Webb as well as Ph.D. candidates Bo Zhao, Xiaolin Zhu, Xiang Chen, Lili Wang, Xining Yang, Zhe Xu, Nan Deng, Grey Evenson, Sam Kay, Nicholas Crane, Scott Stuckman, James Baginski, and Hyeseon Jeong for their friendship and support for my six years’ academic life at Ohio State. My gratitude also go to pastor Chris Kauffman, Mr. Steve Will and Ms. Kelly Will, Mr. Emerson Wu and Ms. Ivy Wu, Mr. Dennis Shimer and Ms. Dori Shimer for their help with my life in Columbus OH. v Table of Contents Abstract ............................................................................................................................... ii Dedication .......................................................................................................................... iv Acknowledgments............................................................................................................... v Table of Contents ............................................................................................................... vi List of Tables ..................................................................................................................... xi List of Figures .................................................................................................................. xiii Chapter 1: Introduction ....................................................................................................... 1 1.1 Question answering (QA) ..................................................................................... 1 1.1.1 Advantages of QA systems ................................................................................. 1 1.1.2 Open-domain QA systems .................................................................................. 3 1.1.3 Closed-domain QA systems ............................................................................... 3 1.1.3 Challenges of QA systems .................................................................................. 4 1.2 Geographic question answering (GeoQA) ................................................................ 6 1.2.1 Challenges of GeoQA ......................................................................................... 6 1.2.3 Major problems in GeoQA ................................................................................. 9 vi 1.3 Previous approaches to geographic QA .................................................................. 11 1.3.1 Annotation-based approach .............................................................................. 11 1.3.2 IBM Watson: the deep QA approach ............................................................... 13 1.4 Purpose of This Study ............................................................................................. 14 1.5 An Overview of This Thesis ................................................................................... 17 Chapter 2 Underpinning Technologies ............................................................................. 18 2.1 Natural language processing ................................................................................... 18 2.1.1 Structure matching ............................................................................................ 18 2.1.2 NLP for information retrieval ........................................................................... 22 2.2 Machine learning ..................................................................................................... 27 2.3 Ontology .................................................................................................................. 28 2.3.1 Ontology and knowledge base .......................................................................... 28 2.3.2 Types of human knowledge .............................................................................. 29 2.3.4 Ontological representation of knowledge ......................................................... 30 2.3.5 Spatial ontologies ............................................................................................. 31 2.4 Geographic information systems ............................................................................. 32 Chapter 3 Framework and System Components ............................................................... 34 3.1 GIS as The Foundation of Knowledge-based QA Systems .................................... 34 3.2 An Example of a Geographic Question Used in This Research ............................. 35 vii 3.3 Potential applications .............................................................................................. 36 3.3.1 Internet search Google, Bing and Yahoo: ........................................................ 36 3.2.2 Local business search via Amazon, eBay, Craigslist, PayPal .......................... 37 3.2.3 Local search on social network sites such as Facebook and Twitter ................ 39 3.4 The GeoQA Framework and System Components ................................................. 40 3.4.1 Natural Language Processing Component ....................................................... 41 3.4.2 Machine Learning Components........................................................................ 54 3.4.3 Ontological Reasoning Component .................................................................. 62 3.4.5 GIS Components .............................................................................................. 79 Chapter 4 Data .................................................................................................................. 82 4.1 Daily Geography Practice Categories .................................................................... 84 4.2 Survey Questions..................................................................................................... 85 4.2.1 Survey Process .................................................................................................. 85 4.2.2 Survey Statistics ............................................................................................... 87 4.2.3 Validation of Survey Answers .......................................................................... 88 4.2.4 Observations about the Survey ......................................................................... 89 Chapter 5 Corpus Analysis Results ..................................................................................
Load more

Recommended publications
  • IR2 Trees for Spatial Database Keyword Search NSF Award
    IR2 Trees for Spatial Database Keyword Search NSF Award: CNS-0220562 Mil: Infrastructure for Research and Training in Database Management for Web-based Geospatial Data Visualization with Applications to Aviation PI: Naphtali Rishe Florida International University School of Computing and Information Sciences Co-PI: Boonserm Wongsaroj Florida Memorial University Department of Computer Sciences and Mathematics ABSTRACT Our Mil-provided infrastructure and research support has enabled us to perform basic research in keyword search. Many applications require finding objects closest to a specified location that contains a set of keywords. For example, online yellow pages allow users to - specify an address and a set of keywords. In return, the user obtains a list of businesses whose description contains these keywords, ordered by their distance from the specified address. The problems of nearest neighbor search on spatial data and keyword search on text data have been extensively studied separately. However, to the best of our knowledge there is no efficient method to answer spatial keyword queries, that is, queries that specify both a location and a set of keywords. In this work, we present an efficient method to answer top-k spatial keyword queries. To do so, we introduce an indexing structure called IR2-Tree (Information Retrieval R-Tree) which combines an R-Tree with superimposed text signatures. We present algorithms that construct and maintain an IR2-Tree, and use it to answer top-k spatial keyword queries. Our algorithms are experimentally and analytically compared to current methods and are shown to have superior performance and excellent scalability. 1. INTRODUCTION An increasing number of applications require the efficient execution of nearest neighbor queries constrained by the properties of the spatial objects.
    [Show full text]
  • Spatial Databases and Spatial Indexing Techniques
    Spatial Databases and Spatial Indexing Techniques Timos Sellis Computer Science Division Department of Electrical and Computer Engineering National Technical University of Athens Zografou 15773, GREECE Tel: +30-1-772-1601 FAX: +30-1-772-1659 e-mail: [email protected] Spatial Database Systems Timos Sellis Spatial Databases and Spatial Indexing Techniques Timos Sellis National Technical University of Athens e-mail: [email protected] Aalborg, June 1998 Outline • Data Models • Algebra • Query Languages • Data Structures • Query Processing and Optimization • System Architecture • Open Research Issues Spatial Database Systems 1 1 Spatial Database Systems Timos Sellis Introduction : Spatial Database Management Systems (SDBMS) QUESTION “What is a Spatial Database Management System ?” ANSWER • SDBMS is a DBMS • It offers spatial data types in its data model and query language • support of spatial relationships / properties / operations • It supports spatial data types in its implementation • efficient indexing and retrieval • support of spatial selection / join Spatial Database Systems 2 Applications of SDBMS Traditional GIS applications • Socio-Economic applications • Urban planning • Route optimization, market analysis • Environmental applications • Fire or Pollution Monitoring • Administrative applications • Public networks administration • Vehicle navigation Spatial Database Systems 3 2 Spatial Database Systems Timos Sellis Applications of SDBMS (cont'd) Novel applications • Image and Multimedia databases • shape configuration
    [Show full text]
  • Compressed Spatial Hierarchical Bitmap (Cshb) Indexes for Efficiently Processing Spatial Range Query Workloads
    Compressed Spatial Hierarchical Bitmap (cSHB) Indexes for Efficiently Processing Spatial Range Query Workloads ∗ Parth Nagarkar K. Selçuk Candan Aneesha Bhat Arizona State University Arizona State University Arizona State University Tempe, AZ 85287-8809, USA Tempe, AZ 85287-8809, USA Tempe, AZ 85287-8809, USA [email protected] [email protected] [email protected] ABSTRACT terms of their coordinates in 2D space. Queries in this 2D In most spatial data management applications, objects are space are then processed using multidimensional/spatial in- represented in terms of their coordinates in a 2-dimensional dex structures that help quick access to the data [28]. space and search queries in this space are processed using spatial index structures. On the other hand, bitmap-based 1.1 Spatial Data Structures indexing, especially thanks to the compression opportuni- The key principle behind most indexing mechanisms is to ties bitmaps provide, has been shown to be highly effective ensure that data objects closer to each other in the data for query processing workloads including selection and ag- space are also closer to each other on the storage medium. gregation operations. In this paper, we show that bitmap- In the case of 1D data, this task is relatively easy as the based indexing can also be highly effective for managing total order implicit in the 1D space helps sorting the ob- spatial data sets. More specifically, we propose a novel com- jects so that they can be stored in a way that satisfies the pressed spatial hierarchical bitmap (cSHB) index structure above principle. When the space in which the objects are to support spatial range queries.
    [Show full text]
  • Building a Spatial Database in Postgresql
    Building a Spatial Database in PostgreSQL David Blasby Refractions Research [email protected] http://postgis.refractions.net Introduction • PostGIS is a spatial extension for PostgreSQL • PostGIS aims to be an “OpenGIS Simple Features for SQL” compliant spatial database • I am the principal developer Topics of Discussion • Spatial data and spatial databases • Adding spatial extensions to PostgreSQL • OpenGIS and standards Why PostGIS? • There aren’t any good open source spatial databases available • commercial ones are very expensive • Aren’t any open source spatial functions • extremely difficult to properly code • building block for any spatial project • Enable information to be organized, visualized, and analyzed like never before What is a Spatial Database? Database that: • Stores spatial objects • Manipulates spatial objects just like other objects in the database What is Spatial data? • Data which describes either location or shape e.g.House or Fire Hydrant location Roads, Rivers, Pipelines, Power lines Forests, Parks, Municipalities, Lakes What is Spatial data? • In the abstract, reductionist view of the computer, these entities are represented as Points, Lines, and Polygons. Roads are represented as Lines Mail Boxes are represented as Points Topic Three Land Use Classifications are represented as Polygons Topic Three Combination of all the previous data Spatial Relationships • Not just interested in location, also interested in “Relationships” between objects that are very hard to model outside the spatial domain. • The most
    [Show full text]
  • Introduction to Postgis
    Introduction to PostGIS Tutorial ID: IGET_WEBGIS_002 This tutorial has been developed by BVIEER as part of the IGET web portal intended to provide easy access to geospatial education. This tutorial is released under the Creative Commons license. Your support will help our team to improve the content and to continue to offer high quality geospatial educational resources. For suggestions and feedback please visit www.iget.in. IGET_WEBGIS_002 Introduction to PostGIS Introduction to PostGIS Objective In this tutorial we will learn, how to register a new server, creating a spatial database in PostGIS, importing data to the spatial database, spatial indexing and some important PostGIS special functions. Software: OpenGeo Suite 3.0 Level: Beginner Time required: 3 Hour Software: OpenGeo Suite 3.0 Level: Beginner Prerequisites and Geospatial Skills Basic computer skills IGET_WEBGIS_001 should be completed Basic knowledge of SQL is excepted Readings 1. Introduction to the OpenGeo Suite, Chapter 2: PostGIS, pp. 21 – 38. http://presentations.opengeo.org/2012_FOSSGIS/suiteintro.pdf 2. Chapter 13. PostGIS Special Functions Index, http://suite.opengeo.org/docs/postgis/postgis/html/PostGIS_Special_Functions_Index.html Tutorial Data: The tutorial data of this exercise may be downloaded from this link: 2 IGET_WEBGIS_002 Introduction to PostGIS Introduction PostGIS is an open source spatial database extension that turns PostgreSQL database system into a spatial database. It adds support for geographic objects allowing location queries, analytical functions for raster and vector data, raster map algebra, spatial re-projection, network topology, Geodetic measurements and much more. In this tutorial we will learn how to register a server and creation of a spatial database, and importing the shapefiles into it, for this purpose we are using the shapefiles digitized during the tutorial IGET_GIS_005: Digitization of Toposheet using Quantum GIS.
    [Show full text]
  • Tutorial for Course of Spatial Database
    Tutorial for Course of Spatial Database Spring 2013 Rui Zhu School of Architecture and the Built Environment Royal Institute of Technology-KTH Stockholm, Sweden April 2013 Lab 1 Introduction to RDBMS and SQL pp. 02 Lab 2 Introduction to Relational Database and Modeling pp. 16 Lab 3 SQL and Data Indexing Techniques in Relational Database pp. 18 Lab 4 Spatial Data Modeling with SDBMS (1) Conceptual and Logical Modeling pp. 21 Lab 5 Spatial Data Modeling with SDBMS(2) Implementation in SDB pp. 25 Lab 6 Spatial Queries and Indices pp. 30 Help File Install pgAdmin and Shapefile Import/Export Plugin pp. 36 1 AG2425 Spatial Databases Period 4, 2013 Spring Rui Zhu and Gyözö Gidófalvi Last Updated: March 18, 2013 Lab 1: Introduction to RDBMS and SQL Due March 27th, 2013 1. Background This introductory lab is designed for students that are new to databases and have no or limited knowledge about databases and SQL. The instructions first introduce some basic relational databases concepts and SQL, then show a number of tools that allow users to interact with PostgreSQL/PostGIS databases (pgAdmin: administration and management tool administration; QuantumGIS). Lastly, the instructions point to a short and simple SQL tutorial using an online PostgreSQL/PostGIS database and ask the students to write a few simple queries against this database. 2. Tutorial 2.1. What is SQL? As it is stated in Wikipedia, SQL (Structured Query Language) is a special-purpose programming language designed for managing data held in a relational database management system (RDBMS). SQL is a simple and unified language, which offers many features to make it as a powerfully diverse that people can work with a secure way.
    [Show full text]
  • Analyzing the Performance of Nosql Vs. SQL Databases for Spatial And
    Free and Open Source Software for Geospatial (FOSS4G) Conference Proceedings Volume 17 Boston, USA Article 4 2017 Analyzing the performance of NoSQL vs. SQL databases for Spatial and Aggregate queries Sarthak Agarwal International Institute of Information Technology Hyderabad Gachibowli, Hyderabad, India KS Rajan International Institute of Information Technology Hyderabad Gachibowli, Hyderabad, India Follow this and additional works at: https://scholarworks.umass.edu/foss4g Part of the Data Storage Systems Commons Recommended Citation Agarwal, Sarthak and Rajan, KS (2017) "Analyzing the performance of NoSQL vs. SQL databases for Spatial and Aggregate queries," Free and Open Source Software for Geospatial (FOSS4G) Conference Proceedings: Vol. 17 , Article 4. DOI: https://doi.org/10.7275/R5736P26 Available at: https://scholarworks.umass.edu/foss4g/vol17/iss1/4 This Paper is brought to you for free and open access by ScholarWorks@UMass Amherst. It has been accepted for inclusion in Free and Open Source Software for Geospatial (FOSS4G) Conference Proceedings by an authorized editor of ScholarWorks@UMass Amherst. For more information, please contact [email protected]. Analyzing the performance of NoSQL vs. SQL databases for Spatial and Aggregate queries Sarthak Agarwala,∗, KS Rajana aInternational Institute of Information Technology Hyderabad Gachibowli, Hyderabad, India Abstract: Relational databases have been around for a long time and spatial databases have exploited this feature for close to two decades. The recent past has seen the development of NoSQL non-relational databases, which are now being adopted for spatial object storage and handling, too. While SQL databases face scalability and agility challenges and fail to take the advantage of the cheap memory and processing power available these days, NoSQL databases can handle the rise in the data storage and frequency at which it is accessed and processed - which are essential features needed in geospatial scenarios, which do not deal with a fixed schema(geometry) and fixed data size.
    [Show full text]
  • Oracle Spatial
    Oracle Spatial An Oracle Technical White Paper May 2002 Oracle Spatial Abstract...............................................................................................................1 1.0 Introduction...............................................................................................1 2.0 Spatial Data and ORDBMS.....................................................................3 2.1 Object-Relational Database Management Systems (ORDBMS)...3 2.2 Challenges of Spatial Databases.........................................................3 2.2.1 Geometry .......................................................................................3 2.2.2 Distribution of Objects in Space................................................4 2.2.3 Temporal Changes........................................................................4 2.2.4 Data Volume .................................................................................4 2.3 Requirements of a Spatial Database System.....................................4 2.3.1 Classification of Space..................................................................5 2.3.2 Data Model ....................................................................................5 2.3.3 Query Language ............................................................................5 2.3.4 Spatial Query Processing .............................................................6 2.3.5 Spatial Indexing.............................................................................6 3.0 Oracle Spatial.............................................................................................7
    [Show full text]
  • Towards Integrating Heterogeneous Data: a Spatial DBMS Solution from a CRC-LCL Project in Australia †
    International Journal of Geo-Information Article Towards Integrating Heterogeneous Data: A Spatial DBMS Solution from a CRC-LCL Project in Australia † Wei Li * , Sisi Zlatanova , Abdoulaye A. Diakite , Mitko Aleksandrov and Jinjin Yan Faculty of the Built Environment, The University of New South Wales, Kensington 2052, Australia; [email protected] (S.Z.); [email protected] (A.A.D.); [email protected] (M.A.); [email protected] (J.Y.) * Correspondence: [email protected] † This Paper Is an Extended Version of Our Paper Published in GI4SDG 2019. Received: 30 December 2019; Accepted: 19 January 2020; Published: 21 January 2020 Abstract: Over recent decades, more and more cities worldwide have created semantic 3D city models of their built environments based on standards across multiple domains. 3D city models, which are often employed for a large range of tasks, go far beyond pure visualization. Due to different spatial scale requirements for planning and managing various built environments, integration of Geographic Information Systems (GIS) and Building Information Modeling (BIM) has emerged in recent years. Focus is now shifting to Precinct Information Modeling (PIM) which is in a more general sense to built-environment modeling. As scales change so do options to perform information modeling for different applications. How to implement data interoperability across these digital representations, therefore, becomes an emerging challenge. Moreover, with the growth of multi-source heterogeneous data consisting of semantic and varying 2D/3D spatial representations, data management becomes feasible for facilitating the development and deployment of PIM applications. How to use heterogeneous data in an integrating manner to further express PIM is an open and comprehensive topic.
    [Show full text]
  • Spatial Join Techniques
    Spatial Join Techniques EDWIN H. JACOX and HANAN SAMET Computer Science Department Center for Automation Research Institute for Advanced Computer Studies University of Maryland College Park, Maryland 20742 [email protected] and [email protected] A variety of techniques for performing a spatial join are reviewed. Instead of just summarizing the literature and presenting each technique in its entirety, distinct components of the different techniques are described and each technique is decomposed into an overall framework for per- forming a spatial join. A typical spatial join technique consists of the following components: partitioning the data, performing internal memory spatial joins on subsets of the data, and check- ing if the full polygons intersect. Each technique is decomposed into these components and each component is addressed in a separate section so as to compare and contrast the similar aspects of each technique. The goal of this survey is to describe algorithms within each component in detail, comparing and contrasting competing methods, thereby enabling further analysis and experimen- tation with each component and allowing for the best algorithms for a particular situation to be built piecemeal, or, even better, enabling an optimizer to choose which algorithms to use. Categories and Subject Descriptors: H.2.4 [Systems]: Query processing; H.2.8 [Database Ap- plications]: Spatial databases and GIS General Terms: Algorithms,Design Additional Key Words and Phrases: external memory algorithms, plane-sweep, spatial join 1. INTRODUCTION This article presents an in-depth survey and analysis of spatial joins. A large body of diverse literature exists on the topic of spatial joins.
    [Show full text]
  • Characteristic of Spatial Data and the Design of Data Model
    Characteristic of Spatial Data and the Design of Data Model Tiejun CUI （Institute for Surveying and Mapping of PLA Information Engineering University, Zhengzhou 450052） Abstract In this paper, the integration of spatial data , its characteristic for analyse , its spatial location and attribute relation are discussed in detail. The problem , division of spatial relation caused by delamination management of map element is settled successfully. Also ,this paper brings forward the way to connect spatial relation of multi maps and give the spatial data model based on spatial relation of map data. Key words Spatial data , Spatial relation , Spatial data model, Spatial data database 1 Introduction If you want to design the spatial data model ,you should know the spatial relation of all kinds of things in realistic world ,such as plants and farmlands beside the road ,residential area connecting with the road ,and etc. We get the information from maps by thinking of images .With the development of computer on graphics and data disposal ,scholars of graphics begin to ponder how to use this technique to manage and reflect the distributing ,combination ,relation of nature and society phenomenal and their space-time development and movement .This gradually forms a new subject --geographic information system(GIS).The core of GIS is to study how to describe scientifically and factually and simulate geography entity or phenomena ,its interrelation and distributing character in computer storage medium . Primeval system only abstract all geographic elements into points ,edges and faces ,and this can not satisfy practical needs .We must do more to study their spatial relation to develop the prospect of application .
    [Show full text]
  • Database Concepts
    Database Concepts presented by: Tim Haithcoat University of Missouri Columbia IntroductionIntroduction Very early attempts to build GIS began from scratch, using limited tools like operating systems & compilers More recently, GIS have been built around existing database management systems (DBMS) – purchase or lease of the DBMS is a major part of the system’s software cost – the DBMS handles many functions which would otherwise have to be programmed into the GIS Any DBMS makes assumptions about the data which it handles – to make effective use of a DBMS it is necessary to fit those assumptions – certain types of DBMS are more suitable for GIS than others because their assumptions fit spatial data better 2 TwoTwo waysways toto useuse DBMSDBMS withinwithin aa GIS:GIS: Total DBMS solution – all data are accessed through the DBMS, so must fit the assumptions imposed by the DBMS designer Mixed solution – some data (usually attribute tables and relationships) are accessed through the DBMS because they fit the model well – some data (usually locational) are accessed directly because they do not fit the DBMS model 3 GISGIS asas aa DatabaseDatabase ProblemProblem Some areas of application, notable facilities management: – deal with very large volumes of data – often have a DBMS solution installed before the GIS is considered The GIS adds geographical access to existing methods of search and query Such systems require very fast response to a limited number of queries, little analysis In these areas it is often said that GIS is a “database problem” rather than an algorithm, analysis, data input or data display problem 4 DefinitionDefinition A database is a collection of non-redundant data which can be shared by different application systems – stresses the importance of multiple applications, data sharing – the spatial database becomes a common resource for an agency Implies separation of physical storage from use of the data by an application program, i.e.
    [Show full text]