University of Pennsylvania ScholarlyCommons Publicly Accessible Penn Dissertations 2017 Compositional Lexical Semantics In Natural Language Inference Ellie Pavlick University of Pennsylvania, [email protected] Follow this and additional works at: https://repository.upenn.edu/edissertations Part of the Computer Sciences Commons Recommended Citation Pavlick, Ellie, "Compositional Lexical Semantics In Natural Language Inference" (2017). Publicly Accessible Penn Dissertations. 2519. https://repository.upenn.edu/edissertations/2519 This paper is posted at ScholarlyCommons. https://repository.upenn.edu/edissertations/2519 For more information, please contact [email protected]. Compositional Lexical Semantics In Natural Language Inference Abstract The focus of this thesis is to incorporate linguistic theories of semantics into data-driven models for automatic natural language understanding. Most current models rely on an impoverished version of semantics which can be learned automatically from large volumes of unannotated text. However, many aspects of language understanding require deeper models of semantic meaning than those which can be easily derived from word co-occurrence alone. In this thesis, we inform our models using insights from linguistics, so that we can continue to take advantage of large-scale statistical models of language without compromising on depth and interpretability. We begin with a discussion of lexical entailment. We classify pairs of words according a small set of distinct entailment relations: e.g. equivalence, entailment, exclusion, and independence. We show that imposing these relations onto a large, automatically constructed lexical entailment resource leads to measurable improvements in an end-to-end inference task. We then turn our attention to compositional entailment, in particular, to modifier-noun composition. We show that inferences involving modifier-noun phrases (e.g. “red dress”, “imaginary friend”) are much more complex than the conventional wisdom states. In a systematic evaluation of a range of existing state-of-the-art natural language inference systems, we illustrate the inability of current technology to handle the types of common sense inferences necessary for human-like processing of modifier-noun phrases. We propose a data-driven method for operationalizing a formal semantics framework which assigns interpretable semantic representations to individual modifiers. We use our method in order to find instances of fine-grained classes involving multiple modifiers (e.g. “1950s American jazz composers”). We demonstrate that our proposed compositional model outperforms existing non-compositional approaches. Degree Type Dissertation Degree Name Doctor of Philosophy (PhD) Graduate Group Computer and Information Science First Advisor Chris Callison-Burch Keywords Artificial Intelligence, Computational Linguistics, Machine Learning, Natural Language Processing, Paraphrasing, Semantics Subject Categories Computer Sciences This dissertation is available at ScholarlyCommons: https://repository.upenn.edu/edissertations/2519 COMPOSITIONAL LEXICAL SEMANTICS IN NATURAL LANGUAGE INFERENCE Ellie Pavlick A DISSERTATION in Computer and Information Science Presented to the Faculties of the University of Pennsylvania in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy 2017 Supervisor of Dissertation Chris Callison-Burch, Associate Professor of Computer and Information Science Graduate Group Chairperson Lyle Ungar, Professor of Computer and Information Science Dissertation Committee Chris Callison-Burch, Associate Professor of Computer and Information Science Ido Dagan, Professor of Computer Science Mitch Marcus, Professor of Computer and Information Science Florian Schwarz, Associate Professor of Linguistics Lyle Ungar, Professor of Computer and Information Science COMPOSITIONAL LEXICAL SEMANTICS IN NATURAL LANGUAGE INFERENCE c COPYRIGHT 2017 Ellie Pavlick This work is licensed under the Creative Commons Attribution NonCommercial-ShareAlike 3.0 License To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/ Dedicated to my sisters, Cassie and Ginger. Your creativity and sense of humor is why I love language. iii ACKNOWLEDGEMENT I am incredibly lucky to have had such a supportive advisor, Chris Callison-Burch. Chris invested in me very early in my computer science career and had confidence in me well before I had any idea what I was doing. He has been my go-to mentor and my strongest advocate since. With every year, I have only become more excited about the field, and I credit that to Chris's consistent energy and encouragement. It is been a genuine privilege to work together. A huge thank you to my dissertation committee. Mitch Marcus has been a constant source of wisdom and guidance, and his personality, in my mind, will always be synonymous with Penn NLP and the wonderful intellectual environment I have experienced during my time here. Lyle Ungar's directness and pragmatism has been invaluable, and he has provided me with clarity and perspective when trying to articulate my own thoughts. Florian Schwarz gave me my first formal education in linguistics, and I am not exaggerating to say that that has had the single largest influence on the way I approach research. Annual meetings with Ido Dagan have been the most enlightening part of ACL conferences over the past several years{his drive is inspiring and every conversation I have had with him has reaffirmed my enthusiasm for the ambitiousness of NLP. I have been fortunate to work with many other incredible research mentors. I have learned so much from Ben Van Durme, Joel Tetreault, Peter Clark, and Niranjan Balasubramanian and owe each of them a huge thank you. Thank you especially to Marius Pasca, for engaging in every discussion with such depth and for challenging me on every opinion{working together made me a holistically better researcher, and I am extremely grateful. Of course, thank you also to my many coauthors and other collaborators, who have been a constant source of new ideas and great conversations. Finally, I am forever thankful to my family and friends who have been an unwavering support network. Thank you first to my incredible husband Steve, for being my best friend iv and for somehow managing to both keep me inspired and keep me grounded over the years. Thank you to my parents, Karin and Michael, for endless love and encouragement and for raising me to be confident and ambitious and unapologetic. Thank you to my sisters, Cassie and Ginger{growing up with you two is the best thing that ever happened to me. Thank you to Andrea, Jan, Howard, Nate, and Sherrie for all the love and support. Last, but never least, thank you to all the friends I have made at Penn and within the ACL community, and to the friends from my other lives at JHU, Peabody, and elsewhere, for all the happiness you've brought me. v ABSTRACT COMPOSITIONAL LEXICAL SEMANTICS IN NATURAL LANGUAGE INFERENCE Ellie Pavlick Chris Callison-Burch The focus of this thesis is to incorporate linguistic theories of semantics into data-driven models for automatic natural language understanding. Most current models rely on an im- poverished version of semantics which can be learned automatically from large volumes of unannotated text. However, many aspects of language understanding require deeper models of semantic meaning than those which can be easily derived from word co-occurrence alone. In this thesis, we inform our models using insights from linguistics, so that we can continue to take advantage of large-scale statistical models of language without compromising on depth and interpretability. We begin with a discussion of lexical entailment. We classify pairs of words according a small set of distinct entailment relations: e.g. equivalence, en- tailment, exclusion, and independence. We show that imposing these relations onto a large, automatically constructed lexical entailment resource leads to measurable improvements in an end-to-end inference task. We then turn our attention to compositional entailment, in particular, to modifier-noun composition. We show that inferences involving modifier-noun phrases (e.g. \red dress", \imaginary friend") are much more complex than the conven- tional wisdom states. In a systematic evaluation of a range of existing state-of-the-art nat- ural language inference systems, we illustrate the inability of current technology to handle the types of common sense inferences necessary for human-like processing of modifier-noun phrases. We propose a data-driven method for operationalizing a formal semantics frame- work which assigns interpretable semantic representations to individual modifiers. We use our method in order to find instances of fine-grained classes involving multiple modifiers (e.g. \1950s American jazz composers"). We demonstrate that our proposed compositional model outperforms existing non-compositional approaches. vi TABLE OF CONTENTS ACKNOWLEDGEMENT . iv ABSTRACT . vi LIST OF TABLES . xv LIST OF ILLUSTRATIONS . xviii CHAPTER 1 : Introduction . 1 1.1 Overview . 1 1.2 Outline of this Document . 8 CHAPTER 2 : Background and Related Work . 11 2.1 Definition of \Entailment" in Natural Language . 11 2.1.1 Entailment in Formal Linguistics . 11 2.1.2 Entailment in Natural Language Processing . 13 2.1.3 Types of Knowledge Tested by RTE . 14 2.1.4 RTE Systems and Approaches . 17 2.2 Lexical Entailment . 20 2.2.1 Word Denotations and Semantic Types . 20 2.2.2 Definition of Semantic Containment