DOCSLIB.ORG

Sparc: Cross-Domain Semantic Parsing in Context

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Sparc: Cross-Domain Semantic Parsing in Context SParC: Cross-Domain Semantic Parsing in Context Tao Yuy Rui Zhangy Michihiro Yasunagay Yi Chern Tany Xi Victoria Lin{ Suyi Liy Heyang Ery Irene Liy Bo Pangy Tao Cheny Emily Jiy Shreya Dixity David Proctory Sungrok Shimy Jonathan Krafty Vincent Zhangy Caiming Xiong{ Richard Socher{ Dragomir Radevy yDepartment of Computer Science, Yale University {Salesforce Research ftao.yu, r.zhang, michihiro.yasunaga, [email protected] fxilin, cxiong, [email protected] Abstract D1 : Database about student dormitory containing 5 tables. We present SParC, a dataset for cross-domain C1 : Find the first and last names of the students who are living S Par C in the dorms that have a TV Lounge as an amenity. emantic sing in ontext. It consists of 4,298 coherent question sequences (12k+ indi- Q : How many dorms have a TV Lounge? 1 ​ vidual questions annotated with SQL queries), S : SELECT COUNT(*) FROM dorm AS T1 JOIN has_amenity 1 ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ AS T2 ON T1.dormid = T2.dormid JOIN dorm_amenity obtained from controlled user interactions ​ ​ ​ ​ ​ ​ AS T3 ON T2.amenid = T3.amenid WHERE ​ ​ ​ ​ ​ with 200 complex databases over 138 do- T3.amenity_name = 'TV Lounge' Q : What is the total capacity of these dorms? mains. We provide an in-depth analysis of 2 ​ S : SELECT SUM(T1.student_capacity) FROM dorm AS T1 SParC and show that it introduces new chal- 2 ​ ​ ​ ​ ​ ​ ​ ​ JOIN has_amenity AS T2 ON T1.dormid = T2.dormid ​ ​ ​ ​ ​ ​ lenges compared to existing datasets. SParC JOIN dorm_amenity AS T3 ON T2.amenid = T3.amenid ​ ​ ​ ​ ​ ​ WHERE T3.amenity_name = 'TV Lounge' (1) demonstrates complex contextual depen- ​ ​ Q : How many students are living there? dencies, (2) has greater semantic diversity, and 3 ​ S : SELECT COUNT(*) FROM student AS T1 JOIN lives_in (3) requires generalization to new domains 3 ​ ​ ​ ​ ​ ​ ​ ​ ​ AS T2 ON T1.stuid = T2.stuid WHERE T2.dormid IN ​ ​ ​ ​ ​ ​ ​ due to its cross-domain nature and the un- (SELECT T3.dormid FROM has_amenity AS T3 JOIN ​ ​ ​ ​ ​ ​ ​ dorm_amenity AS T4 ON T3.amenid = T4.amenid WHERE seen databases at test time. We experiment ​ ​ ​ ​ ​ T4.amenity_name = 'TV Lounge') with two state-of-the-art text-to-SQL mod- Q : Please show their first and last names. 4 ​ els adapted to the context-dependent, cross- S : SELECT T1.fname, T1.lname FROM student AS T1 JOIN 4 ​ ​ ​ ​ ​ ​ ​ lives_in AS T2 ON T1.stuid = T2.stuid WHERE domain setup. The best model obtains an ​ ​ ​ ​ ​ ​ T2.dormid IN (SELECT T3.dormid FROM has_amenity ​ ​ ​ ​ ​ ​ ​ exact set match accuracy of 20.2% over all AS T3 JOIN dorm_amenity AS T4 ON T3.amenid = ​ ​ ​ ​ ​ ​ ​ ​ T4.amenid WHERE T4.amenity_name = 'TV Lounge') questions and less than 10% over all inter- ​ ​ ​ ​ ​ action sequences, indicating that the cross- -------------------------------------- D : Database about shipping company containing 13 tables domain setting and the contextual phenomena 2 ​ C : Find the names of the first 5 customers. of the dataset present significant challenges 2 ​ for future research. The dataset, baselines, Q1 : What is the customer id of the most recent customer? and leaderboard are released at https:// S : SELECT customer_id FROM customers ORDER BY 1 ​ ​ ​ ​ ​ ​ ​ ​ ​ date_became_customer DESC LIMIT 1 yale-lily.github.io/sparc. ​ ​ ​ ​ ​ Q2 : What is their name? S : SELECT customer_name FROM customers ORDER BY 1 Introduction 2 ​ ​ ​ ​ ​ ​ ​ ​ ​ date_became_customer DESC LIMIT 1 ​ ​ ​ ​ ​ Q : How about for the first 5 customers? Querying a relational database is often challeng- 3 S : SELECT customer_name FROM customers ORDER BY 3 ​ ​ ​ ​ ​ ​ ​ ​ ​ ing and a natural language interface has long been date_became_customer LIMIT 5 ​ ​ ​ ​ ​ regarded by many as the most powerful database interface (Popescu et al., 2003; Bertomeu et al., Figure 1: Two question sequences from the SParC 2006; Li and Jagadish, 2014). The problem of dataset. Questions (Qi) in each sequence query a mapping a natural language utterance into exe- database (Di), obtaining information sufficient to com- cutable SQL queries (text-to-SQL) has attracted plete the interaction goal (C ). Each question is anno- i increasing attention from the semantic parsing S tated with a corresponding SQL query ( i). SQL token community by virtue of a continuous effort of sequences from the interaction context are underlined. dataset creation (Zelle and Mooney, 1996; Iyyer et al., 2017; Zhong et al., 2017; Finegan-Dollak While most of these work focus on precisely et al., 2018; Yu et al., 2018a) and the modeling mapping stand-alone utterances to SQL queries, innovation that follows it (Xu et al., 2017; Wang generating SQL queries in a context-dependent et al., 2018; Yu et al., 2018b; Shi et al., 2018). scenario (Miller et al., 1996; Zettlemoyer and Collins, 2009; Suhr et al., 2018) has been studied tain information that answers the Spider question. less often. The most prominent context-dependent At the same time, the students are encouraged to text-to-SQL benchmark is ATIS1, which is set in come up with related questions which do not di- the flight-booking domain and contains only one rectly contribute to the Spider question so as to database (Hemphill et al., 1990; Dahl et al., 1994). increase data diversity. The questions were subse- In a real-world setting, users tend to ask a se- quently translated to complex SQL queries by the quence of thematically related questions to learn same student. Similar to Spider, the SQL Queries about a particular topic or to achieve a complex in SParC cover complex syntactic structures and goal. Previous studies have shown that by al- most common SQL keywords. lowing questions to be constructed sequentially, We split the dataset such that a database appears users can explore the data in a more flexible man- in only one of the train, development and test sets. ner, which reduces their cognitive burden (Hale, We provide detailed data analysis to show the rich- 2006; Levy, 2008; Frank, 2013; Iyyer et al., 2017) ness of SParC in terms of semantics, contextual and increases their involvement when interacting phenomena and thematic relations (x 4). We also with the system. The phrasing of such questions experiment with two competitive baseline models depends heavily on the interaction history (Kato to assess the difficulty of SParC (x 5). The best et al., 2004; Chai and Jin, 2004; Bertomeu et al., model achieves only 20.2% exact set matching ac- 2006). The users may explicitly refer to or omit curacy3 on all questions, and demonstrates a de- previously mentioned entities and constraints, and crease in exact set matching accuracy from 38.6% may introduce refinements, additions or substitu- for questions in turn 1 to 1.1% for questions in tions to what has already been said (Figure1). turns 4 and higher (x 6). This suggests that there is This requires a practical text-to-SQL system to ef- plenty of room for advancement in modeling and fectively process context information to synthesize learning on the SParC dataset. the correct SQL logic. To enable modeling advances in context- 2 Related Work dependent semantic parsing, we introduce SParC Context-independent semantic parsing Early (cross-domain Semantic Parsing in Context), studies in semantic parsing (Zettlemoyer and an expert-labeled dataset which contains 4,298 Collins, 2005; Artzi and Zettlemoyer, 2013; Be- coherent question sequences (12k+ questions rant and Liang, 2014; Li and Jagadish, 2014; Pa- paired with SQL queries) querying 200 complex supat and Liang, 2015; Dong and Lapata, 2016; databases in 138 different domains. The dataset is Iyer et al., 2017) were based on small and single- 2 built on top of Spider , the largest cross-domain domain datasets such as ATIS (Hemphill et al., context-independent text-to-SQL dataset available 1990; Dahl et al., 1994) and GeoQuery (Zelle and in the field (Yu et al., 2018c). The large num- Mooney, 1996). Recently, an increasing number ber of domains provide rich contextual phenom- of neural approaches (Zhong et al., 2017; Xu et al., ena and thematic relations between the questions, 2017; Yu et al., 2018a; Dong and Lapata, 2018; Yu which general-purpose natural language interfaces et al., 2018b) have started to use large and cross- to databases have to address. In addition, it en- domain text-to-SQL datasets such as WikiSQL ables us to test the generalization of the trained (Zhong et al., 2017) and Spider (Yu et al., 2018c). systems to unseen databases and domains. Most of them focus on converting stand-alone nat- We asked 15 college students with SQL expe- ural language questions to executable queries. Ta- rience to come up with question sequences over ble1 compares SParC with other semantic parsing the Spider databases (x 3). Questions in the orig- datasets. inal Spider dataset were used as guidance to the students for constructing meaningful interactions: Context-dependent semantic parsing with each sequence is based on a question in Spider and SQL labels Only a few datasets have been the student has to ask inter-related questions to ob- constructed for the purpose of mapping context- dependent questions to structured queries. 1A subset of ATIS is also frequently used in context- independent semantic parsing research (Zettlemoyer and 3Exact string match ignores ordering discrepancies of Collins, 2007; Dong and Lapata, 2016). SQL components whose order does not matter. Exact set 2The data is available at https://yale-lily. matching is able to consider ordering issues in SQL evalu- github.io/spider. ation. See more evaluation details in section 6.1. Dataset Context Resource Annotation Cross-domain SParC X database SQL X ATIS (Hemphill et al., 1990; Dahl et al., 1994) X database SQL 7 Spider (Yu et al., 2018c) 7 database SQL X WikiSQL (Zhong et al., 2017) 7 table SQL X GeoQuery (Zelle and Mooney, 1996) 7 database SQL 7 SequentialQA (Iyyer et al., 2017) X table denotation X SCONE (Long et al., 2016) X environment denotation 7 Table 1: Comparison of SParC with existing semantic parsing datasets.
Load more

Recommended publications
  • Semantic Parsing for Single-Relation Question Answering
    Semantic Parsing for Single-Relation Question Answering Wen-tau Yih Xiaodong He Christopher Meek Microsoft Research Redmond, WA 98052, USA scottyih,xiaohe,meek @microsoft.com { } Abstract the same question. That is to say that the problem of mapping from a question to a particular relation We develop a semantic parsing framework and entity in the KB is non-trivial. based on semantic similarity for open do- In this paper, we propose a semantic parsing main question answering (QA). We focus framework tailored to single-relation questions. on single-relation questions and decom- At the core of our approach is a novel semantic pose each question into an entity men- similarity model using convolutional neural net- tion and a relation pattern. Using convo- works. Leveraging the question paraphrase data lutional neural network models, we mea- mined from the WikiAnswers corpus by Fader et sure the similarity of entity mentions with al. (2013), we train two semantic similarity mod- entities in the knowledge base (KB) and els: one links a mention from the question to an the similarity of relation patterns and re- entity in the KB and the other maps a relation pat- lations in the KB. We score relational tern to a relation. The answer to the question can triples in the KB using these measures thus be derived by finding the relation–entity triple and select the top scoring relational triple r(e1, e2) in the KB and returning the entity not to answer the question. When evaluated mentioned in the question. By using a general se- on an open-domain QA task, our method mantic similarity model to match patterns and re- achieves higher precision across different lations, as well as mentions and entities, our sys- recall points compared to the previous ap- tem outperforms the existing rule learning system, proach, and can improve F1 by 7 points.
    [Show full text]
  • Semantic Parsing with Semi-Supervised Sequential Autoencoders
    Semantic Parsing with Semi-Supervised Sequential Autoencoders Toma´sˇ Kociskˇ y´†‡ Gabor´ Melis† Edward Grefenstette† Chris Dyer† Wang Ling† Phil Blunsom†‡ Karl Moritz Hermann† †Google DeepMind ‡University of Oxford tkocisky,melisgl,etg,cdyer,lingwang,pblunsom,kmh @google.com { } Abstract This is common in situations where we encode nat- ural language into a logical form governed by some We present a novel semi-supervised approach grammar or database. for sequence transduction and apply it to se- mantic parsing. The unsupervised component While such an autoencoder could in principle is based on a generative model in which latent be constructed by stacking two sequence transduc- sentences generate the unpaired logical forms. ers, modelling the latent variable as a series of dis- We apply this method to a number of semantic crete symbols drawn from multinomial distributions parsing tasks focusing on domains with lim- ited access to labelled training data and ex- creates serious computational challenges, as it re- tend those datasets with synthetically gener- quires marginalising over the space of latent se- ated logical forms. quences Σx∗. To avoid this intractable marginalisa- tion, we introduce a novel differentiable alternative for draws from a softmax which can be used with 1 Introduction the reparametrisation trick of Kingma and Welling (2014). Rather than drawing a discrete symbol in Neural approaches, in particular attention-based Σx from a softmax, we draw a distribution over sym- sequence-to-sequence models, have shown great bols from a logistic-normal distribution at each time promise and obtained state-of-the-art performance step. These serve as continuous relaxations of dis- for sequence transduction tasks including machine crete samples, providing a differentiable estimator translation (Bahdanau et al., 2015), syntactic con- of the expected reconstruction log likelihood.
    [Show full text]
  • SQL Generation from Natural Language: a Sequence-To- Sequence Model Powered by the Transformers Architecture and Association Rules
    Journal of Computer Science Original Research Paper SQL Generation from Natural Language: A Sequence-to- Sequence Model Powered by the Transformers Architecture and Association Rules 1,2Youssef Mellah, 1Abdelkader Rhouati, 1El Hassane Ettifouri, 2Toumi Bouchentouf and 2Mohammed Ghaouth Belkasmi 1NovyLab Research, Novelis, Paris, France 2Mohammed First University Oujda, National School of Applied Sciences, LaRSA/SmartICT Lab, Oujda, Morocco Article history Abstract: Using Natural Language (NL) to interacting with relational Received: 12-03-2021 databases allows users from any background to easily query and analyze Revised: 21-04-2021 large amounts of data. This requires a system that understands user Accepted: 28-04-2021 questions and automatically converts them into structured query language such as SQL. The best performing Text-to-SQL systems use supervised Corresponding Author: Youssef Mellah learning (usually formulated as a classification problem) by approaching NovyLab Research, Novelis, this task as a sketch-based slot-filling problem, or by first converting Paris, France questions into an Intermediate Logical Form (ILF) then convert it to the And corresponding SQL query. However, non-supervised modeling that LARSA Laboratory, ENSAO, directly converts questions to SQL queries has proven more difficult. In Mohammed First University, this sense, we propose an approach to directly translate NL questions into Oujda, Morocco SQL statements. In this study, we present a Sequence-to-Sequence Email: [email protected] (Seq2Seq) parsing model for the NL to SQL task, powered by the Transformers Architecture exploring the two Language Models (LM): Text-To-Text Transfer Transformer (T5) and the Multilingual pre-trained Text-To-Text Transformer (mT5). Besides, we adopt the transformation- based learning algorithm to update the aggregation predictions based on association rules.
    [Show full text]
  • Semantic Parsing with Neural Hybrid Trees
    Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17) Semantic Parsing with Neural Hybrid Trees Raymond Hendy Susanto, Wei Lu Singapore University of Technology and Design 8 Somapah Road, Singapore 487372 {raymond susanto, luwei}@sutd.edu.sg Abstract QUERY : answer(STATE) We propose a neural graphical model for parsing natural lan- STATE : exclude(STATE, STATE) guage sentences into their logical representations. The graph- ical model is based on hybrid tree structures that jointly rep- : state( ) : next to( ) resent both sentences and semantics. Learning and decoding STATE all STATE STATE are done using efficient dynamic programming algorithms. The model is trained under a discriminative setting, which STATE : stateid(STATENAME) allows us to incorporate a rich set of features. Hybrid tree structures have shown to achieve state-of-the-art results on STATENAME :(tx ) standard semantic parsing datasets. In this work, we propose a novel model that incorporates a rich, nonlinear featurization Which states do not border Texas ? by a feedforward neural network. The error signals are com- puted with respect to the conditional random fields (CRFs) objective using an inside-outside algorithm, which are then Figure 1: An example tree-structured semantic representa- backpropagated to the neural network. We demonstrate that tion and its corresponding natural language sentence. by combining the strengths of the exact global inference in the hybrid tree models and the power of neural networks to extract high level features, our model is able to achieve new explicit labelings between words and meaning representa- state-of-the-art results on standard benchmark datasets across tion tokens are not available in the training data.
    [Show full text]
  • Learning Logical Representations from Natural Languages with Weak Supervision and Back-Translation
    Learning Logical Representations from Natural Languages with Weak Supervision and Back-Translation Kaylin Hagopiany∗ Qing Wangy Georgia Institute of Technology IBM Research Atlanta, GA, USA Yorktown Heights, NY, USA [email protected] [email protected] Tengfei Ma Yupeng Gao Lingfei Wu IBM Research AI IBM Research AI IBM Research AI Yorktown Heights, NY, USA Yorktown Heights, NY, USA Yorktown Heights, NY, USA [email protected] [email protected] [email protected] Abstract Semantic parsing with strong supervision requires a large parallel corpus for train- ing, whose production may be impractical due to the excessive human labour needed for annotation. Recently, weakly supervised approaches based on reinforce- ment learning have been studied, wherein only denotations, but not the logical forms, are available. However, reinforcement learning approaches based on, e.g., policy gradient training, suffer reward sparsity and progress slowly. In this work, we explore enriching the policy gradient framework with a back-translation mecha- nism that boosts the reward signal. In this vein, we jointly optimize the expected reward of obtaining the correct denotation and the likelihood of reconstructing the natural language utterance. Initial experimental results suggest the proposed method slightly outperforms a state-of-the-art semantic parser trained with policy gradient. Introduction Semantic parsing is the task of converting natural language utterances to logical representations that a machine can understand and execute as problem solving steps. There has recently been a surge of interest in developing machine learning methods for semantic parsing, owing to the creation of large- scale text corpora. There has also been a proliferation of logical representations used in this setting, including logical forms [28], domain specific languages [26], python code [24], SQL queries [31], and bash commands [20].
    [Show full text]
  • A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing
    Don’t Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing Subendhu Rongali∗ Luca Soldaini University of Massachusetts Amherst Amazon Alexa Search Amherst, MA, USA Manhattan Beach, CA, USA [email protected] [email protected] Emilio Monti Wael Hamza Amazon Alexa Amazon Alexa AI Cambridge, UK New York, NY, USA [email protected] [email protected] ABSTRACT ArtistName Virtual assistants such as Amazon Alexa, Apple Siri, and Google Play the song don't stop believin by Journey Assistant often rely on a semantic parsing component to understand PlaySongIntent SongName which action(s) to execute for an utterance spoken by its users. Traditionally, rule-based or statistical slot-filling systems have been Intent: PlaySongIntent used to parse “simple” queries; that is, queries that contain a single Slots: SongName(don't stop believin), action and can be decomposed into a set of non-overlapping entities. ArtistName(Journey) More recently, shift-reduce parsers have been proposed to process more complex utterances. These methods, while powerful, impose specific limitations on the type of queries that can be parsed; namely, Figure 1: Semantic parsing of a “simple” query. Simple they require a query to be representable as a parse tree. queries define single action (intent) and can be decomposed In this work, we propose a unified architecture based on Se- into a set of non-overlapping entities (slots). quence to Sequence models and Pointer Generator Network to handle both simple and complex queries. Unlike other works, our 1 INTRODUCTION approach does not impose any restriction on the semantic parse Adoption of intelligent voice assistants such as Amazon Alexa, schema.
    [Show full text]
  • Neural Semantic Parsing Graham Neubig
    CS11-747 Neural Networks for NLP Neural Semantic Parsing Graham Neubig Site https://phontron.com/class/nn4nlp2018/ Tree Structures of Syntax • Dependency: focus on relations between words ROOT I saw a girl with a telescope • Phrase structure: focus on the structure of the sentence S VP PP NP NP PRP VBD DT NN IN DT NN I saw a girl with a telescope Representations of Semantics • Syntax only gives us the sentence structure • We would like to know what the sentence really means • Specifically, in an grounded and operationalizable way, so a machine can • Answer questions • Follow commands • etc. Meaning Representations • Special-purpose representations: designed for a specific task • General-purpose representations: designed to be useful for just about anything • Shallow representations: designed to only capture part of the meaning (for expediency) Parsing to Special-purpose Meaning Representations Example Special-purpose Representations • A database query language for sentence understanding • A robot command and control language • Source code in a language such as Python (?) Example Query Tasks • Geoquery: Parsing to Prolog queries over small database (Zelle and Mooney 1996) • Free917: Parsing to Freebase query language (Cai and Yates 2013) • Many others: WebQuestions, WikiTables, etc. Example Command and Control Tasks • Robocup: Robot command and control (Wong and Mooney 2006) • If this then that: Commands to smartphone interfaces (Quirk et al. 2015) Example Code Generation Tasks • Hearthstone cards (Ling et al. 2015) • Django commands (Oda
    [Show full text]
  • Practical Semantic Parsing for Spoken Language Understanding
    Practical Semantic Parsing for Spoken Language Understanding Marco Damonte∗ Rahul Goel Tagyoung Chung University of Edinburgh Amazon Alexa AI [email protected] fgoerahul,[email protected] Abstract the intent’s slots. This architecture is popular due to its simplicity and robustness. On the other hand, Executable semantic parsing is the task of con- verting natural language utterances into logi- Q&A, which need systems to produce more com- cal forms that can be directly used as queries plex structures such as trees and graphs, requires a to get a response. We build a transfer learn- more comprehensive understanding of human lan- ing framework for executable semantic pars- guage. ing. We show that the framework is effec- One possible system that can handle such a task tive for Question Answering (Q&A) as well as is an executable semantic parser (Liang, 2013; for Spoken Language Understanding (SLU). Kate et al., 2005). Given a user utterance, an exe- We further investigate the case where a parser on a new domain can be learned by exploit- cutable semantic parser can generate tree or graph ing data on other domains, either via multi- structures that represent logical forms that can be task learning between the target domain and used to query a knowledge base or database. In an auxiliary domain or via pre-training on the this work, we propose executable semantic pars- auxiliary domain and fine-tuning on the target ing as a common framework for both uses cases domain. With either flavor of transfer learn- by framing SLU as executable semantic parsing ing, we are able to improve performance on that unifies the two use cases.
    [Show full text]
  • Arxiv:2009.14116V1 [Cs.CL] 29 Sep 2020
    A Survey on Semantic Parsing from the Perspective of Compositionality Pawan Kumar Srikanta Bedathur Indian Institute of Technology Delhi, Indian Institute of Technology Delhi, New Delhi, Hauzkhas-110016 New Delhi, Hauzkhas-110016 [email protected] [email protected] Abstract compositionality: evident by the choice of the for- mal language and the construction mechanism and Different from previous surveys in seman- lexical variation: grounding the words/phrases to tic parsing (Kamath and Das, 2018) and appropriate KB entity/relation. knowledge base question answering(KBQA) (Chakraborty et al., 2019; Zhu et al., 2019; Language Compositionality: According to Hoffner¨ et al., 2017) we try to takes a dif- Frege’s principle of compositionality: “the ferent perspective on the study of semantic meaning of a whole is a function of the meaning parsing. Specifically, we will focus on (a) meaning composition from syntactical struc- of the parts. The study of formal semantics ture (Partee, 1975), and (b) the ability of for natural language to appropriately represent semantic parsers to handle lexical variation sentence meaning has a long history (Szab, 2017). given the context of a knowledge base (KB). Most semantic formalism follow the tradition of In the following section after an introduction Montague’s grammar (Partee, 1975) i.e. there is of the field of semantic parsing and its uses a one-to-one correspondence between syntax and in KBQA, we will describe meaning represen- semantics e.g. CCG (Zettlemoyer and Collins, tation using grammar formalism CCG (Steed- 2005). We will not be delving into the concept of man, 1996).
    [Show full text]
  • Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training
    Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training Peng Shi1*, Patrick Ng2, Zhiguo Wang2, Henghui Zhu2, Alexander Hanbo Li2, Jun Wang2, Cicero Nogueira dos Santos2, Bing Xiang2 1 University of Waterloo, 2 AWS AI Labs [email protected], fpatricng,zhiguow,henghui,hanboli,juwanga,cicnog,[email protected] Abstract Pain Point 1: Fail to match and detect the column mentions. Utterance: Which professionals live in a city containing the Most recently, there has been significant interest in learn- substring ’West’? List his or her role, street, city and state. ing contextual representations for various NLP tasks, by Prediction: SELECT role code, street, state leveraging large scale text corpora to train large neural lan- FROM Professionals WHERE city LIKE ’%West%’ guage models with self-supervised learning objectives, such Error: Missing column city in SELECT clause. as Masked Language Model (MLM). However, based on a pi- lot study, we observe three issues of existing general-purpose Pain Point 2: Fail to infer columns based on cell values. language models when they are applied to text-to-SQL se- Utterance: Give the average life expectancy for countries in mantic parsers: fail to detect column mentions in the utter- Africa which are republics? ances, fail to infer column mentions from cell values, and fail Prediction: SELECT Avg(LifeExpectancy) FROM to compose complex SQL queries. To mitigate these issues, country WHERE Continent = ’Africa’ we present a model pre-training framework, Generation- Error: Missing GovernmentForm = ’Republic’. Augmented Pre-training (GAP), that jointly learns repre- Pain Point 3 sentations of natural language utterances and table schemas : Fail to compose complex target SQL.
    [Show full text]
  • On the Use of Parsing for Named Entity Recognition
    applied sciences Review On the Use of Parsing for Named Entity Recognition Miguel A. Alonso * , Carlos Gómez-Rodríguez and Jesús Vilares Grupo LyS, Departamento de Ciencias da Computación e Tecnoloxías da Información, Universidade da Coruña and CITIC, 15071 A Coruña, Spain; [email protected] (C.G.-R.); [email protected] (J.V.) * Correspondence: [email protected] Abstract: Parsing is a core natural language processing technique that can be used to obtain the structure underlying sentences in human languages. Named entity recognition (NER) is the task of identifying the entities that appear in a text. NER is a challenging natural language processing task that is essential to extract knowledge from texts in multiple domains, ranging from financial to medical. It is intuitive that the structure of a text can be helpful to determine whether or not a certain portion of it is an entity and if so, to establish its concrete limits. However, parsing has been a relatively little-used technique in NER systems, since most of them have chosen to consider shallow approaches to deal with text. In this work, we study the characteristics of NER, a task that is far from being solved despite its long history; we analyze the latest advances in parsing that make its use advisable in NER settings; we review the different approaches to NER that make use of syntactic information; and we propose a new way of using parsing in NER based on casting parsing itself as a sequence labeling task. Keywords: natural language processing; named entity recognition; parsing; sequence labeling Citation: Alonso, M.A; Gómez- 1.
    [Show full text]
  • Neural Semantic Parsing for Syntax-Aware Code Generation
    Neural Semantic Parsing for Syntax-Aware Code Generation Masterarbeit zur Erlangung des Master-Grades Master of Science im Studiengang Tech- nische Informatik an der Fakultät für Informations-, Medien- und Elektrotechnik der Technischen Hochschule Köln vorgelegt von: Artur Baranowski Matrikel-Nr.: 11130536 Adresse: Heidestraße 144 51147 Köln Erstprüfer: Prof. Dr. Hubert Randerath Zweitprüfer: Prof. Dr. Nico Hochgeschwender Köln, 23. September 2020 Abstract I Abstract The task of mapping natural language expressions to logical forms is referred to as semantic parsing. The syntax of logical forms that are based on programming or query languages, such as Python or SQL, is defined by a formal grammar. In this thesis, we present an efficient neural semantic parser that exploits the underlying grammar of logical forms to enforce well-formed expressions. We use an encoder-decoder model for sequence prediction. Syntactically valid programs are guaranteed by means of a bottom- up shift-reduce parser, that keeps track of the set of viable tokens at each decoding step. We show that the proposed model outperforms the standard encoder-decoder model across datasets and is competitive with comparable grammar-guided semantic parsing approaches. Keywords: semantic parsing, machine learning, natural language processing, code generation, formal grammars, parsing theory Statutory Declaration II Statutory Declaration I herewith declare that I have composed the present thesis myself and without use of any other than the cited sources and aids. Sentences or parts of sentences quoted literally are marked as such; other references with regard to the statement and scope are indicated by full details of the publications concerned. The thesis in the same or similar form has not been submitted to any examination body and has not been published.
    [Show full text]