DOCSLIB.ORG

Benchmarking Meaning Representations in Neural Semantic Parsing

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Benchmarking Meaning Representations in Neural Semantic Parsing Benchmarking Meaning Representations in Neural Semantic Parsing Jiaqi Guo1,∗ Qian Liu2,∗ Jian-Guang Lou3, Zhenwen Li4 Xueqing Liu5, Tao Xie4, Ting Liu1 1Xi’an Jiaotong University, Xi’an, China 2Beihang University, Beijing, China 3Microsoft Research, Beijing, China 4Peking University, Beijing, China 5Stevens Institute of Technology, Hoboken, New Jersey, USA [email protected], [email protected] [email protected], flizhenwen, [email protected] [email protected], [email protected] Abstract MR Geo ATIS Job Prolog - - 91.4 Meaning representation is an important com- ponent of semantic parsing. Although re- Lambda 90.4 91.3 85.0 searchers have designed a lot of meaning rep- FunQL 92.5 - - resentations, recent work focuses on only a SQL 78.0 69.0 - few of them. Thus, the impact of meaning Prolog 89.6 - 92.1 representation on semantic parsing is less un- Lambda --- derstood. Furthermore, existing work’s per- FunQL --- formance is often not comprehensively evalu- ated due to the lack of readily-available execu- SQL 82.5 79.2 - tion engines. Upon identifying these gaps, we Table 1: State-of-the-art performance for MRs on Geo, propose UNIMER, a new unified benchmark ATIS, and Job. The top table shows exact-match accu- on meaning representations, by integrating ex- racy whereas the bottom table shows execution-match isting semantic parsing datasets, completing accuracy. Most existing work focuses on evaluating the missing logical forms, and implementing only a small subset of dataset × MR pairs, leaving most the missing execution engines. The resulting of the table unexplored. A finer-grained table is avail- unified benchmark contains the complete enu- able in the supplementary material (Table8). meration of logical forms and execution en- gines over three datasets × four meaning rep- resentations. A thorough experimental study of neural networks techniques, significant improve- on UNIMER reveals that neural semantic pars- ing approaches exhibit notably different per- ments have been made in semantic parsing perfor- formance when they are trained to generate mance (Jia and Liang, 2016; Yin and Neubig, 2017; different meaning representations. Also, pro- Dong and Lapata, 2018; Shaw et al., 2019). gram alias and grammar rules heavily impact Despite the advancement in performance, we the performance of different meaning repre- identify three important biases in existing work’s sentations. Our benchmark, execution engines evaluation methodology. First, although multiple and implementation can be found on: https: MRs are proposed, most existing work is evaluated //github.com/JasperGuo/Unimer. on only one or two of them, leading to less com- 1 Introduction prehensive or even unfair comparisons. Table1 shows the state-of-the-art performance of semantic A remarkable vision of artificial intelligence is to parsing on different dataset × MR combinations, enable human interactions with machines through where the rows are the MRs and the columns are natural language. Semantic parsing has emerged the datasets. We can observe that while Lambda as a key technology for achieving this goal. In Calculus is intensively studied, the other MRs have general, semantic parsing aims to transform a nat- not been sufficiently studied. This biased evalua- ural language utterance into a logic form, i.e., tion is partly caused by the absence of target logic meaning repre- a formal, machine-interpretable forms in the missing cells. Second, existing work sentation (MR) (Zelle and Mooney, 1996; Dahl 1 often compares the performance on different MRs et al., 1994). Thanks to the recent development directly (Sun et al., 2020; Shaw et al., 2019; Chen ∗Work done during an internship at Microsoft Research. et al., 2020) without considering the confounding 1In this paper, we focus on grounded semantic parsing, where meaning representations are grounded to specific knowl- edge bases, instead of ungrounded semantic parsing. 1520 Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 1520–1540, November 16–20, 2020. c 2020 Association for Computational Linguistics role that MR plays in the performance,2 causing capture the semantic equivalence (Bunel et al., unfair comparisons and misleading conclusions. 2018). As different MRs have different degrees Third, a more comprehensive evaluation methodol- of syntactic difference, they suffer from this ogy would consider both the exact-match accuracy problem differently. Second, Grammar rules. and the execution-match accuracy, because two Grammar-based neural models can guarantee that logic forms can be semantically equivalent yet do the generated program is syntactically correct (Yin not match precisely in their surface forms. How- and Neubig, 2017; Wang et al., 2020; Sun et al., ever, as shown in Table1, most existing work is 2020). For a given set of logical forms in an MR, only evaluated with the exact-match accuracy. This there exist multiple sets of grammar rules to model bias is potentially due to the fact that execution them. We observe that when the grammar-based engines are not available in six out of the twelve neural model is trained with different sets of dataset × MR combinations. grammar rules, it exhibits a notable performance Upon identifying the three biases, in this paper, discrepancy. This finding alias with the one made we propose UNIMER, a new unified benchmark, by in traditional semantic parsers (Kate, 2008) that unifying four publicly available MRs in three of the properly transforming grammar rules can lead to most popular semantic parsing datasets: Geo, ATIS better performance of a traditional semantic parser. and Jobs. First, for each natural language utter- ance in the three datasets, UNIMER provides anno- In summary, this paper makes the following tated logical forms in four different MRs, including main contributions: Prolog, Lambda Calculus, FunQL, and SQL. We identify that annotated logical forms in some MR • We propose UNIMER, a new unified bench- × dataset combinations are missing. As a result, mark on meaning representations, by integrat- we complete the benchmark by semi-automatically ing and completing semantic parsing datasets translating logical forms from one MR to another. in three datasets × four MRs; we also imple- Second, we implement six missing execution en- ment six execution engines so that execution- gines for MRs so that the execution-match accuracy match accuracy can be evaluated in all cases; can be readily computed for all the dataset × MR • We provide the baseline results for two widely combinations. Both the logical forms and their ex- used neural semantic parsing approaches on ecution results are manually checked to ensure the our benchmark, and we conduct an empirical correctness of annotations and execution engines. study to understand the impact that program After constructing UNIMER, to obtain a pre- alias and grammar rule plays on the perfor- liminary understanding on the impact of MRs mance of neural semantic parsing; on semantic parsing, we empirically study the performance of MRs on UNIMER by using two 2 Preliminaries widely-used neural semantic parsing approaches (a seq2seq model (Dong and Lapata, 2016; Jia In this section, we provide a brief description of and Liang, 2016) and a grammar-based neural the MRs and neural semantic parsing approaches model (Yin and Neubig, 2017)), under the super- that we study in the paper. vised learning setting. 2.1 Meaning Representations In addition to the empirical study above, we further analyze the impact of two operations, i.e., We investigate four MRs in this paper, namely, program alias and grammar rules, to understand Prolog, Lambda Calculus, FunQL, and SQL, be- how they affect different MRs differently. First, cause they are widely used in semantic parsing and Program alias. A semantically equivalent program we can obtain their corresponding labeled data in may have many syntactically different forms. As at least one semantic parsing domain. We regard a result, if the training and testing data have a Prolog, Lambda Calculus, and FunQL as domain- difference in their syntactic distributions of logic specific MRs, since the predicates defined in them forms, a naive maximum likelihood estimation are specific for a given domain. Consequently, the can suffer from this difference because it fails to execution engines of domain-specific MRs need to be significantly customized for different domains, 2In (Kate et al., 2005; Liang et al., 2011; Guo et al., 2019), it was revealed that using an appropriate MR can substantially requiring plenty of manual efforts. In contrast, SQL improve the performance of a semantic parser. is a domain-general MR for querying relational 1521 MR Logical Form answer(A, ( flight(A) , tomorrow(A) , during day(A, B) , const (B, period (morning)), Prolog from(A, C) , const(C, city(Pittsburgh)), to(A, D), const (D, city(Atlanta)))) Lambda ( lambda A:e ( (flight A) ^ (during day A morning:pd) ^ (from A Pittsburgh:ci) Calculus ^ (to A Atlanta:ci) ^ (tomorrow A) ) ) answer ( flight ( tomorrow ( intersect ( during day ( period ( morning ) ) , FunQL from ( city ( Pittsburgh ) ) , to ( city ( Atlanta ) ) ) ) ) ) SELECT flight id FROM . WHERE city 1.city name = ‘pittsburgh’ AND city 2.city name = ‘atlanta’ AND date day 1.year = 1991 SQL AND date day 1.month number = 1 AND date day 1.day number = 20 AND departure time BETWEEN 0 AND 1200 Table 2: Examples of meaning representations for utterance “what flights do you have in tomorrow morning from pittsburgh to atlanta?” in the ATIS domain. databases. Its execution engines (e.g., MySQL) 2005). It abstracts away variables and encodes can be used directly in different domains. Table2 compositionality via its nested function-argument shows a logical form for each of the four MRs in structure, making it easier to implement an efficient the ATIS domain. execution engine for FunQL. Concretely, unlike Prolog has long been used to represent the meaning Prolog and Lambda Calculus, predicates in FunQL of natural language (Zelle and Mooney, 1996; Kate take a set of entities as input and return another set and Mooney, 2006).
Load more

Recommended publications
  • Semantic Parsing for Single-Relation Question Answering
    Semantic Parsing for Single-Relation Question Answering Wen-tau Yih Xiaodong He Christopher Meek Microsoft Research Redmond, WA 98052, USA scottyih,xiaohe,meek @microsoft.com { } Abstract the same question. That is to say that the problem of mapping from a question to a particular relation We develop a semantic parsing framework and entity in the KB is non-trivial. based on semantic similarity for open do- In this paper, we propose a semantic parsing main question answering (QA). We focus framework tailored to single-relation questions. on single-relation questions and decom- At the core of our approach is a novel semantic pose each question into an entity men- similarity model using convolutional neural net- tion and a relation pattern. Using convo- works. Leveraging the question paraphrase data lutional neural network models, we mea- mined from the WikiAnswers corpus by Fader et sure the similarity of entity mentions with al. (2013), we train two semantic similarity mod- entities in the knowledge base (KB) and els: one links a mention from the question to an the similarity of relation patterns and re- entity in the KB and the other maps a relation pat- lations in the KB. We score relational tern to a relation. The answer to the question can triples in the KB using these measures thus be derived by finding the relation–entity triple and select the top scoring relational triple r(e1, e2) in the KB and returning the entity not to answer the question. When evaluated mentioned in the question. By using a general se- on an open-domain QA task, our method mantic similarity model to match patterns and re- achieves higher precision across different lations, as well as mentions and entities, our sys- recall points compared to the previous ap- tem outperforms the existing rule learning system, proach, and can improve F1 by 7 points.
    [Show full text]
  • The Generative Lexicon
    The Generative Lexicon James Pustejovsky" Computer Science Department Brandeis University In this paper, I will discuss four major topics relating to current research in lexical seman- tics: methodology, descriptive coverage, adequacy of the representation, and the computational usefulness of representations. In addressing these issues, I will discuss what I think are some of the central problems facing the lexical semantics community, and suggest ways of best ap- proaching these issues. Then, I will provide a method for the decomposition of lexical categories and outline a theory of lexical semantics embodying a notion of cocompositionality and type coercion, as well as several levels of semantic description, where the semantic load is spread more evenly throughout the lexicon. I argue that lexical decomposition is possible if it is per- formed generatively. Rather than assuming a fixed set of primitives, I will assume a fixed number of generative devices that can be seen as constructing semantic expressions. I develop a theory of Qualia Structure, a representation language for lexical items, which renders much lexical ambiguity in the lexicon unnecessary, while still explaining the systematic polysemy that words carry. Finally, I discuss how individual lexical structures can be integrated into the larger lexical knowledge base through a theory of lexical inheritance. This provides us with the necessary principles of global organization for the lexicon, enabling us to fully integrate our natural language lexicon into a conceptual whole. 1. Introduction I believe we have reached an interesting turning point in research, where linguistic studies can be informed by computational tools for lexicology as well as an appre- ciation of the computational complexity of large lexical databases.
    [Show full text]
  • Semantic Parsing with Semi-Supervised Sequential Autoencoders
    Semantic Parsing with Semi-Supervised Sequential Autoencoders Toma´sˇ Kociskˇ y´†‡ Gabor´ Melis† Edward Grefenstette† Chris Dyer† Wang Ling† Phil Blunsom†‡ Karl Moritz Hermann† †Google DeepMind ‡University of Oxford tkocisky,melisgl,etg,cdyer,lingwang,pblunsom,kmh @google.com { } Abstract This is common in situations where we encode nat- ural language into a logical form governed by some We present a novel semi-supervised approach grammar or database. for sequence transduction and apply it to se- mantic parsing. The unsupervised component While such an autoencoder could in principle is based on a generative model in which latent be constructed by stacking two sequence transduc- sentences generate the unpaired logical forms. ers, modelling the latent variable as a series of dis- We apply this method to a number of semantic crete symbols drawn from multinomial distributions parsing tasks focusing on domains with lim- ited access to labelled training data and ex- creates serious computational challenges, as it re- tend those datasets with synthetically gener- quires marginalising over the space of latent se- ated logical forms. quences Σx∗. To avoid this intractable marginalisa- tion, we introduce a novel differentiable alternative for draws from a softmax which can be used with 1 Introduction the reparametrisation trick of Kingma and Welling (2014). Rather than drawing a discrete symbol in Neural approaches, in particular attention-based Σx from a softmax, we draw a distribution over sym- sequence-to-sequence models, have shown great bols from a logistic-normal distribution at each time promise and obtained state-of-the-art performance step. These serve as continuous relaxations of dis- for sequence transduction tasks including machine crete samples, providing a differentiable estimator translation (Bahdanau et al., 2015), syntactic con- of the expected reconstruction log likelihood.
    [Show full text]
  • Deriving Programming Theories from Equations by Functional Predicate Calculus
    Calculational Semantics: Deriving Programming Theories from Equations by Functional Predicate Calculus RAYMOND T. BOUTE INTEC, Ghent University The objects of programming semantics, namely, programs and languages, are inherently formal, but the derivation of semantic theories is all too often informal, deprived of the benefits of formal calculation “guided by the shape of the formulas.” Therefore, the main goal of this article is to provide for the study of semantics an approach with the same convenience and power of discov- ery that calculus has given for many years to applied mathematics, physics, and engineering. The approach uses functional predicate calculus and concrete generic functionals; in fact, a small part suffices. Application to a semantic theory proceeds by describing program behavior in the simplest possible way, namely by program equations, and discovering the axioms of the theory as theorems by calculation. This is shown in outline for a few theories, and in detail for axiomatic semantics, fulfilling a second goal of this article. Indeed, a chafing problem with classical axiomatic semantics is that some axioms are unintuitive at first, and that justifications via denotational semantics are too elaborate to be satisfactory. Derivation provides more transparency. Calculation of formulas for ante- and postconditions is shown in general, and for the major language constructs in par- ticular. A basic problem reported in the literature, whereby relations are inadequate for handling nondeterminacy and termination, is solved here through appropriately defined program equations. Several variants and an example in mathematical analysis are also presented. One conclusion is that formal calculation with quantifiers is one of the most important elements for unifying contin- uous and discrete mathematics in general, and traditional engineering with computing science, in particular.
    [Show full text]
  • SQL Generation from Natural Language: a Sequence-To- Sequence Model Powered by the Transformers Architecture and Association Rules
    Journal of Computer Science Original Research Paper SQL Generation from Natural Language: A Sequence-to- Sequence Model Powered by the Transformers Architecture and Association Rules 1,2Youssef Mellah, 1Abdelkader Rhouati, 1El Hassane Ettifouri, 2Toumi Bouchentouf and 2Mohammed Ghaouth Belkasmi 1NovyLab Research, Novelis, Paris, France 2Mohammed First University Oujda, National School of Applied Sciences, LaRSA/SmartICT Lab, Oujda, Morocco Article history Abstract: Using Natural Language (NL) to interacting with relational Received: 12-03-2021 databases allows users from any background to easily query and analyze Revised: 21-04-2021 large amounts of data. This requires a system that understands user Accepted: 28-04-2021 questions and automatically converts them into structured query language such as SQL. The best performing Text-to-SQL systems use supervised Corresponding Author: Youssef Mellah learning (usually formulated as a classification problem) by approaching NovyLab Research, Novelis, this task as a sketch-based slot-filling problem, or by first converting Paris, France questions into an Intermediate Logical Form (ILF) then convert it to the And corresponding SQL query. However, non-supervised modeling that LARSA Laboratory, ENSAO, directly converts questions to SQL queries has proven more difficult. In Mohammed First University, this sense, we propose an approach to directly translate NL questions into Oujda, Morocco SQL statements. In this study, we present a Sequence-to-Sequence Email: [email protected] (Seq2Seq) parsing model for the NL to SQL task, powered by the Transformers Architecture exploring the two Language Models (LM): Text-To-Text Transfer Transformer (T5) and the Multilingual pre-trained Text-To-Text Transformer (mT5). Besides, we adopt the transformation- based learning algorithm to update the aggregation predictions based on association rules.
    [Show full text]
  • Semantic Parsing with Neural Hybrid Trees
    Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17) Semantic Parsing with Neural Hybrid Trees Raymond Hendy Susanto, Wei Lu Singapore University of Technology and Design 8 Somapah Road, Singapore 487372 {raymond susanto, luwei}@sutd.edu.sg Abstract QUERY : answer(STATE) We propose a neural graphical model for parsing natural lan- STATE : exclude(STATE, STATE) guage sentences into their logical representations. The graph- ical model is based on hybrid tree structures that jointly rep- : state( ) : next to( ) resent both sentences and semantics. Learning and decoding STATE all STATE STATE are done using efficient dynamic programming algorithms. The model is trained under a discriminative setting, which STATE : stateid(STATENAME) allows us to incorporate a rich set of features. Hybrid tree structures have shown to achieve state-of-the-art results on STATENAME :(tx ) standard semantic parsing datasets. In this work, we propose a novel model that incorporates a rich, nonlinear featurization Which states do not border Texas ? by a feedforward neural network. The error signals are com- puted with respect to the conditional random fields (CRFs) objective using an inside-outside algorithm, which are then Figure 1: An example tree-structured semantic representa- backpropagated to the neural network. We demonstrate that tion and its corresponding natural language sentence. by combining the strengths of the exact global inference in the hybrid tree models and the power of neural networks to extract high level features, our model is able to achieve new explicit labelings between words and meaning representa- state-of-the-art results on standard benchmark datasets across tion tokens are not available in the training data.
    [Show full text]
  • Learning Logical Representations from Natural Languages with Weak Supervision and Back-Translation
    Learning Logical Representations from Natural Languages with Weak Supervision and Back-Translation Kaylin Hagopiany∗ Qing Wangy Georgia Institute of Technology IBM Research Atlanta, GA, USA Yorktown Heights, NY, USA [email protected] [email protected] Tengfei Ma Yupeng Gao Lingfei Wu IBM Research AI IBM Research AI IBM Research AI Yorktown Heights, NY, USA Yorktown Heights, NY, USA Yorktown Heights, NY, USA [email protected] [email protected] [email protected] Abstract Semantic parsing with strong supervision requires a large parallel corpus for train- ing, whose production may be impractical due to the excessive human labour needed for annotation. Recently, weakly supervised approaches based on reinforce- ment learning have been studied, wherein only denotations, but not the logical forms, are available. However, reinforcement learning approaches based on, e.g., policy gradient training, suffer reward sparsity and progress slowly. In this work, we explore enriching the policy gradient framework with a back-translation mecha- nism that boosts the reward signal. In this vein, we jointly optimize the expected reward of obtaining the correct denotation and the likelihood of reconstructing the natural language utterance. Initial experimental results suggest the proposed method slightly outperforms a state-of-the-art semantic parser trained with policy gradient. Introduction Semantic parsing is the task of converting natural language utterances to logical representations that a machine can understand and execute as problem solving steps. There has recently been a surge of interest in developing machine learning methods for semantic parsing, owing to the creation of large- scale text corpora. There has also been a proliferation of logical representations used in this setting, including logical forms [28], domain specific languages [26], python code [24], SQL queries [31], and bash commands [20].
    [Show full text]
  • A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing
    Don’t Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing Subendhu Rongali∗ Luca Soldaini University of Massachusetts Amherst Amazon Alexa Search Amherst, MA, USA Manhattan Beach, CA, USA [email protected] [email protected] Emilio Monti Wael Hamza Amazon Alexa Amazon Alexa AI Cambridge, UK New York, NY, USA [email protected] [email protected] ABSTRACT ArtistName Virtual assistants such as Amazon Alexa, Apple Siri, and Google Play the song don't stop believin by Journey Assistant often rely on a semantic parsing component to understand PlaySongIntent SongName which action(s) to execute for an utterance spoken by its users. Traditionally, rule-based or statistical slot-filling systems have been Intent: PlaySongIntent used to parse “simple” queries; that is, queries that contain a single Slots: SongName(don't stop believin), action and can be decomposed into a set of non-overlapping entities. ArtistName(Journey) More recently, shift-reduce parsers have been proposed to process more complex utterances. These methods, while powerful, impose specific limitations on the type of queries that can be parsed; namely, Figure 1: Semantic parsing of a “simple” query. Simple they require a query to be representable as a parse tree. queries define single action (intent) and can be decomposed In this work, we propose a unified architecture based on Se- into a set of non-overlapping entities (slots). quence to Sequence models and Pointer Generator Network to handle both simple and complex queries. Unlike other works, our 1 INTRODUCTION approach does not impose any restriction on the semantic parse Adoption of intelligent voice assistants such as Amazon Alexa, schema.
    [Show full text]
  • A Survey of Computational Semantics: Representation, Inference and Knowledge in Wide-Coverage Text Understanding Johan Bos* University of Groningen
    Language and Linguistics Compass 5/6 (2011): 336–366, 10.1111/j.1749-818x.2011.00284.x A Survey of Computational Semantics: Representation, Inference and Knowledge in Wide-Coverage Text Understanding Johan Bos* University of Groningen Abstract The aim of computational semantics is to capture the meaning of natural language expressions in representations suitable for performing inferences, in the service of understanding human language in written or spoken form. First-order logic is a good starting point, both from the representation and inference point of view. But even if one makes the choice of first-order logic as representa- tion language, this is not enough: the computational semanticist needs to make further decisions on how to model events, tense, modal contexts, anaphora and plural entities. Semantic representa- tions are usually built on top of a syntactic analysis, using unification, techniques from the lambda-calculus or linear logic, to do the book-keeping of variable naming. Inference has many potential applications in computational semantics. One way to implement inference is using algo- rithms from automated deduction dedicated to first-order logic, such as theorem proving and model building. Theorem proving can help in finding contradictions or checking for new infor- mation. Finite model building can be seen as a complementary inference task to theorem proving, and it often makes sense to use both procedures in parallel. The models produced by model generators for texts not only show that the text is contradiction-free; they also can be used for disambiguation tasks and linking interpretation with the real world. To make interesting inferences, often additional background knowledge is required (not expressed in the analysed text or speech parts).
    [Show full text]
  • Neural Semantic Parsing Graham Neubig
    CS11-747 Neural Networks for NLP Neural Semantic Parsing Graham Neubig Site https://phontron.com/class/nn4nlp2018/ Tree Structures of Syntax • Dependency: focus on relations between words ROOT I saw a girl with a telescope • Phrase structure: focus on the structure of the sentence S VP PP NP NP PRP VBD DT NN IN DT NN I saw a girl with a telescope Representations of Semantics • Syntax only gives us the sentence structure • We would like to know what the sentence really means • Specifically, in an grounded and operationalizable way, so a machine can • Answer questions • Follow commands • etc. Meaning Representations • Special-purpose representations: designed for a specific task • General-purpose representations: designed to be useful for just about anything • Shallow representations: designed to only capture part of the meaning (for expediency) Parsing to Special-purpose Meaning Representations Example Special-purpose Representations • A database query language for sentence understanding • A robot command and control language • Source code in a language such as Python (?) Example Query Tasks • Geoquery: Parsing to Prolog queries over small database (Zelle and Mooney 1996) • Free917: Parsing to Freebase query language (Cai and Yates 2013) • Many others: WebQuestions, WikiTables, etc. Example Command and Control Tasks • Robocup: Robot command and control (Wong and Mooney 2006) • If this then that: Commands to smartphone interfaces (Quirk et al. 2015) Example Code Generation Tasks • Hearthstone cards (Ling et al. 2015) • Django commands (Oda
    [Show full text]
  • Practical Semantic Parsing for Spoken Language Understanding
    Practical Semantic Parsing for Spoken Language Understanding Marco Damonte∗ Rahul Goel Tagyoung Chung University of Edinburgh Amazon Alexa AI [email protected] fgoerahul,[email protected] Abstract the intent’s slots. This architecture is popular due to its simplicity and robustness. On the other hand, Executable semantic parsing is the task of con- verting natural language utterances into logi- Q&A, which need systems to produce more com- cal forms that can be directly used as queries plex structures such as trees and graphs, requires a to get a response. We build a transfer learn- more comprehensive understanding of human lan- ing framework for executable semantic pars- guage. ing. We show that the framework is effec- One possible system that can handle such a task tive for Question Answering (Q&A) as well as is an executable semantic parser (Liang, 2013; for Spoken Language Understanding (SLU). Kate et al., 2005). Given a user utterance, an exe- We further investigate the case where a parser on a new domain can be learned by exploit- cutable semantic parser can generate tree or graph ing data on other domains, either via multi- structures that represent logical forms that can be task learning between the target domain and used to query a knowledge base or database. In an auxiliary domain or via pre-training on the this work, we propose executable semantic pars- auxiliary domain and fine-tuning on the target ing as a common framework for both uses cases domain. With either flavor of transfer learn- by framing SLU as executable semantic parsing ing, we are able to improve performance on that unifies the two use cases.
    [Show full text]
  • Arxiv:2009.14116V1 [Cs.CL] 29 Sep 2020
    A Survey on Semantic Parsing from the Perspective of Compositionality Pawan Kumar Srikanta Bedathur Indian Institute of Technology Delhi, Indian Institute of Technology Delhi, New Delhi, Hauzkhas-110016 New Delhi, Hauzkhas-110016 [email protected] [email protected] Abstract compositionality: evident by the choice of the for- mal language and the construction mechanism and Different from previous surveys in seman- lexical variation: grounding the words/phrases to tic parsing (Kamath and Das, 2018) and appropriate KB entity/relation. knowledge base question answering(KBQA) (Chakraborty et al., 2019; Zhu et al., 2019; Language Compositionality: According to Hoffner¨ et al., 2017) we try to takes a dif- Frege’s principle of compositionality: “the ferent perspective on the study of semantic meaning of a whole is a function of the meaning parsing. Specifically, we will focus on (a) meaning composition from syntactical struc- of the parts. The study of formal semantics ture (Partee, 1975), and (b) the ability of for natural language to appropriately represent semantic parsers to handle lexical variation sentence meaning has a long history (Szab, 2017). given the context of a knowledge base (KB). Most semantic formalism follow the tradition of In the following section after an introduction Montague’s grammar (Partee, 1975) i.e. there is of the field of semantic parsing and its uses a one-to-one correspondence between syntax and in KBQA, we will describe meaning represen- semantics e.g. CCG (Zettlemoyer and Collins, tation using grammar formalism CCG (Steed- 2005). We will not be delving into the concept of man, 1996).
    [Show full text]