QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering

Total Page:16

File Type:pdf, Size:1020Kb

QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering Michihiro Yasunaga Hongyu Ren Antoine Bosselut Percy Liang Jure Leskovec Stanford University {myasu,hyren,antoineb,pliang,jure}@cs.stanford.edu If it is not used for hair, a roundAbstract brush is an example of what? If it is not used for hair, a round brush is an example of what? A. hair brush B. bathroom C. art supplies* A. hair brush B. bathroom C. art supplies* D. showerThe problem E. hair salon of answering questions using D. shower E. hair salon QA contextknowledge from pre-trained language models QA context (LMs) and knowledgeQA context graphs (KGs) presents QA context two challenges: givenNode a QA context (question Node Question Choice Question Choice and answerEntity choice), methodsEntity need to (i) Entity Entity identify relevant knowledge from large KGs, and (ii) perform joint reasoning over the QA hair hair hair hair context and KG.AtLocation Here we proposebrush a new AtLocation brush model, QA-GNN, which addresses the above AtLocation Answer AtLocation Answer challenges throughRelatedTo two key innovations: (i) RelatedTo round art round art relevancebrush scoring, where we usesupply LMs to esti- brush supply mate the importance of KG nodes relative to UsedFor UsedFor the given QA context,painting and (ii)UsedFor joint reasoning, painting UsedFor Knowledgewhere graph we connect the QA context and KG Knowledge graph to form a joint graph, and mutually update Figure 1: Given the QA context (question and answer their representations through graph-based message passing. We evaluate QA-GNN on the choice; purple box), we aim to derive the answer by CommonsenseQA and OpenBookQA datasets, performing joint reasoning over the language and the and show its improvement over existing LM Graph Connectionknowledge (§3.1) graph (green box).Dev Acc. Contextualization (§3.2) Dev Acc. and LM+KG models, as well as its capability to No edge between Z and KG nodes 74.81 No contextualization 75.56 perform interpretable and structured reasoning,Connect Z to all KG nodes 76.38 w/ contextual embedding 76.31 Connect Z to QA entity nodes (final system) 76.54 w/ relevance score (final system) 76.54 e.g., correctly handling negation in questions. be noisy (Bordes et al., 2013; Guu etw/ both al., 2015). 76.52 How to reason effectively with both sources of GNN Attention & Message (§3.3) Dev Acc. 1 Introduction knowledge remains an important open problem.GNN Layers (§3.3) Dev Acc. Node type, relation, score-aware (final system) 76.54 L = 3 75.53 - type-aware 75.41 L = 4 76.34 Question answering systems must be able to access Combining LMs and KGs for reasoning (hence-L = 5 (final system) 76.54 - relation-aware 75.61 L = 6 76.21 relevant knowledge and reason over it. Typically,- score-awareforth, LM+KG) presents75.56 two challenges:L = given 7 75.96 knowledge can be implicitly encoded in large a QA context (e.g., question and answer choices; Graph Connectionlanguage (§3.1) models (LMs)Dev pre-trained Acc. Relevance on unstructuredscoring (§3.2) Dev Acc.Figure Graph 1 purple Connection box), (§3.1) methods needDev Acc. to (i)Relevance identify scoring (§3.2) Dev Acc. No edgetext between (Petroni Z and KG et nodes al., 2019;74.11 BosselutNothing et al., 2019), or ex-75.15informativeNo edge between knowledge Z and KG nodes from a large74.81 KGNothing (green 75.56 Connectplicitly Z to all KG represented nodes in structured76.38 w/ knowledgecontextual embedding graphs76.31box);Connect and (ii) Z to captureall KG nodes the nuance of76.38 the QAw/ contextcontextual embedding 76.31 Connect Z to QA entity nodes (final) 76.54 w/ relevance score (final) 76.54 Connect Z to QA entity nodes (final) 76.54 w/ relevance score (final) 76.54 (KGs), such as Freebase (Bollackerw/ both et al., 2008)76.52and the structure of the KGs to performw/ both joint 76.52 and ConceptNet (Speer et al., 2017), where entities reasoning over these two sources of information. GNN Attention & Message (§3.3) Dev Acc. GNN Attention & Message (§3.3) Dev Acc. are represented as nodes and relationsGNN Layers (§3.3) betweenDev Acc. Previous works (Bao et al., 2016; Sun et al.,GNN 2018 Layers; (§3.3) Dev Acc. Node type, relation, score-aware (final) 76.54 L = 3 75.53 Node type, relation, score-aware (final) 76.54 L = 3 75.53 - type-awarethem as edges. Recently,75.11 pre-trainedL = 4 LMs have76.34 Lin et- type-aware al., 2019) retrieve a subgraph75.41 from theL = 4 KG 76.34 - relation-aware 75.23 L = 5 (final) 76.54 - relation-aware 75.61 L = 5 (final) 76.54 demonstrated remarkable success inL = 6 many question76.21 by taking topic entities (KG entities mentionedL = 6 in 76.21 - score-aware 75.15 - score-aware 75.56 answering tasks (Liu et al., 2019L = ;7 Raffel et al.75.96, the given QA context) and their few-hop neighbors.L = 7 75.96 2020). However, while LMs have a broad coverage However, this introduces many entity nodes that of knowledge, they do not empirically perform well are semantically irrelevant to the QA context, on structured reasoning (e.g., handling negation) especially when the number of topic entities or hops (Kassner and Schütze, 2020). On the other hand, increases. Additionally, existing LM+KG methods KGs are more suited for structured reasoning (Ren for reasoning (Lin et al., 2019; Wang et al., 2019a; et al., 2020; Ren and Leskovec, 2020) and enable Feng et al., 2020; Lv et al., 2020) treat the QA explainable predictions e.g., by providing reasoning context and KG as two separate modalities. They paths (Lin et al., 2019), but may lack coverage and individually apply LMs to the QA context and graph If it is not used for hair, a round brush is an example of what? If it is not used for hair, a round brush is an example of what? If it is not used for hair, a round brush A. hair brush B. bathroom C. art supplies* A. hair brush B. bathroom C. art supplies* A. hair brush B. bathroom C. art supplies D. shower E. hair salon D. shower E. hair salon D. shower E. hair salon Language Context Language Context Language Context Context Node Context Context Node Node Question Choice Entity Question Choice Question Entity Entity Entity Entity hair hair hair hair hair AtLocation brush AtLocation brush AtLocation AtLocation AtLocation Answer AtLocation Answer RelatedTo RelatedTo RelatedTo round art round round art brush supply brush brush supply UsedFor UsedFor UsedFor painting UsedFor painting painting UsedFor Knowledge Graph Knowledge Graph Knowledge Graph TextEncoder Language e.g. LM MLP Context [q; a] Plausibility Z Z KG Score Retrieval Contextual GNN (§3) Pooling LM QA context Encoding MLP [q; a] Probability Joint Z Z KG Graph (§3.1) score Retrieval Reasoning (§3.3) Relevance Pooling Scoring (§3.2) QA-GNN Figure 2: Overview of our approach. Given a QA context (z), we connect it with the retrieved KG to form a joint graph (working graph;§3.1), compute the relevance of each KG node conditioned on z (§3.2; node shading indicates the relevance score), and perform reasoning on the working graph (§3.3). neural networks (GNNs) to the KG, and do not dling negation and entity substitution in questions): mutually update or unify their representations. This it achieves 4.6% improvement over fine-tuned LMs separation might limit their capability to perform on questions with negation, while existing LM+KG structured reasoning, e.g., handling negation. models are +0.6% over fine-tuned LMs. We also Here we propose QA-GNN, an end-to-end show that one can extract reasoning processes from LM+KG model for question answering that QA-GNN in the form of general KG subgraphs, not addresses the above two challenges. We first encode just paths (Lin et al., 2019), suggesting a general the QA context using an LM, and retrieve a KG method for explaining model predictions. subgraph following prior works (Feng et al., 2020). Our QA-GNN has two key insights: (i) Relevance 2 Problem Statement scoring: Since the KG subgraph consists of all few-hop neighbors of the topic entities, some entity We aim to answer natural language questions using nodes are more relevant than others with respect to knowledge from a pre-trained LM and a structured the given QA context. We hence propose KG node KG. We use the term language model broadly to be relevance scoring: we score each entity on the KG any composition of two functions, fhead(fenc(x)), subgraph by concatenating the entity with the QA where fenc, the encoder, maps a textual input x to a LM context and calculating the likelihood using a pre- contextualized vector representation h , and fhead trained LM. This presents a general framework to uses this representation to perform a desired task weight information on the KG; (ii) Joint reasoning: (which we discuss in §3.2). In this work, we specifi- We design a joint graph representation of the QA cally use masked language models (e.g., RoBERTa) LM context and KG, where we explicitly view the QA as fenc, and let h denote the output representa- context as an additional node (QA context node) and tion of a [CLS] token that is prepended to the input connect it to the topic entities in the KG subgraph sequence x, unless otherwise noted. We define the as shown in Figure 1. This joint graph, which we knowledge graph as a multi-relational graph G = term the working graph, unifies the two modalities (V;E).
Recommended publications
  • Technical Reference Manual for the Standardization of Geographical Names United Nations Group of Experts on Geographical Names
    ST/ESA/STAT/SER.M/87 Department of Economic and Social Affairs Statistics Division Technical reference manual for the standardization of geographical names United Nations Group of Experts on Geographical Names United Nations New York, 2007 The Department of Economic and Social Affairs of the United Nations Secretariat is a vital interface between global policies in the economic, social and environmental spheres and national action. The Department works in three main interlinked areas: (i) it compiles, generates and analyses a wide range of economic, social and environmental data and information on which Member States of the United Nations draw to review common problems and to take stock of policy options; (ii) it facilitates the negotiations of Member States in many intergovernmental bodies on joint courses of action to address ongoing or emerging global challenges; and (iii) it advises interested Governments on the ways and means of translating policy frameworks developed in United Nations conferences and summits into programmes at the country level and, through technical assistance, helps build national capacities. NOTE The designations employed and the presentation of material in the present publication do not imply the expression of any opinion whatsoever on the part of the Secretariat of the United Nations concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. The term “country” as used in the text of this publication also refers, as appropriate, to territories or areas. Symbols of United Nations documents are composed of capital letters combined with figures. ST/ESA/STAT/SER.M/87 UNITED NATIONS PUBLICATION Sales No.
    [Show full text]
  • A Structural Analysis of Mide Chants
    A Structural Analysis of Mide Chants GEORGE FULFORD McMaster University Introduction In this paper I shall investigate the relationship between words and im­ agery in seven song scrolls used by members of an Ojibwa religious society known as the Midewiwin. These texts were collected in the late 1880s by W.J. Hoffman for the Bureau of American Ethnology and subsequently published in their seventh Annual Report (Hoffman 1891).1 All the pictographs which I shall discuss were inscribed on birch bark and used by members of the Midewiwin to record chants used in their ceremonies. According to Hoffman (1891:192) these chants consisted of only a few words or short phrases. They were sung by single individuals — never in chorus — and were repeated over and over again, usually to the accompaniment of a wooden kettle drum. In a previous study (Fulford 1989) I analyzed patterns of structural variation among these pictographs and outlined how three complex symbols — the otter, bear and bird — evolved from clan emblems into pictographic markers. The focus of my earlier study was purely iconographic; in this paper I shall explore the verbal structure of Midewiwin chants in order to show some of the ways in which they were pictographically encoded. For the sake of convenience, I have limited my discussion to song scrolls sharing the otter symbol. Six of the seven scrolls that I shall examine contain this marking device. Although one (designated Scroll C in the appendix) lacks an otter, it displays many other formal similarities with Hoffman published transcriptions and translations of 23 songs performed at the White Earth Reservation in northwestern Minnesota.
    [Show full text]
  • 5892 Cisco Category: Standards Track August 2010 ISSN: 2070-1721
    Internet Engineering Task Force (IETF) P. Faltstrom, Ed. Request for Comments: 5892 Cisco Category: Standards Track August 2010 ISSN: 2070-1721 The Unicode Code Points and Internationalized Domain Names for Applications (IDNA) Abstract This document specifies rules for deciding whether a code point, considered in isolation or in context, is a candidate for inclusion in an Internationalized Domain Name (IDN). It is part of the specification of Internationalizing Domain Names in Applications 2008 (IDNA2008). Status of This Memo This is an Internet Standards Track document. This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 5741. Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at http://www.rfc-editor.org/info/rfc5892. Copyright Notice Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
    [Show full text]
  • Etir Code Lists
    eTIR Code Lists Code lists CL01 Equipment size and type description code (UN/EDIFACT 8155) Code specifying the size and type of equipment. 1 Dime coated tank A tank coated with dime. 2 Epoxy coated tank A tank coated with epoxy. 6 Pressurized tank A tank capable of holding pressurized goods. 7 Refrigerated tank A tank capable of keeping goods refrigerated. 9 Stainless steel tank A tank made of stainless steel. 10 Nonworking reefer container 40 ft A 40 foot refrigerated container that is not actively controlling temperature of the product. 12 Europallet 80 x 120 cm. 13 Scandinavian pallet 100 x 120 cm. 14 Trailer Non self-propelled vehicle designed for the carriage of cargo so that it can be towed by a motor vehicle. 15 Nonworking reefer container 20 ft A 20 foot refrigerated container that is not actively controlling temperature of the product. 16 Exchangeable pallet Standard pallet exchangeable following international convention. 17 Semi-trailer Non self propelled vehicle without front wheels designed for the carriage of cargo and provided with a kingpin. 18 Tank container 20 feet A tank container with a length of 20 feet. 19 Tank container 30 feet A tank container with a length of 30 feet. 20 Tank container 40 feet A tank container with a length of 40 feet. 21 Container IC 20 feet A container owned by InterContainer, a European railway subsidiary, with a length of 20 feet. 22 Container IC 30 feet A container owned by InterContainer, a European railway subsidiary, with a length of 30 feet. 23 Container IC 40 feet A container owned by InterContainer, a European railway subsidiary, with a length of 40 feet.
    [Show full text]
  • Management of Stroke in Neonates and Children: a Scientific Statement from the American Heart Association/American Stroke Association
    UCSF UC San Francisco Previously Published Works Title Management of Stroke in Neonates and Children: A Scientific Statement From the American Heart Association/American Stroke Association. Permalink https://escholarship.org/uc/item/0sw5j7c1 Journal Stroke, 50(3) ISSN 0039-2499 Authors Ferriero, Donna M Fullerton, Heather J Bernard, Timothy J et al. Publication Date 2019-03-01 DOI 10.1161/str.0000000000000183 Peer reviewed eScholarship.org Powered by the California Digital Library University of California AHA/ASA Scientific Statement Management of Stroke in Neonates and Children A Scientific Statement From the American Heart Association/American Stroke Association The American Academy of Neurology affirms the value of this statement as an educational tool for neurologists. Donna M. Ferriero, MD, MS, FAHA, Co-Chair; Heather J. Fullerton, MD, MAS, Co-Chair; Timothy J. Bernard, MD, MSCS; Lori Billinghurst, MD, MSc, FRCPC; Stephen R. Daniels, MD, PhD; Michael R. DeBaun, MD, MPH; Gabrielle deVeber, MD; Rebecca N. Ichord, MD; Lori C. Jordan, MD, PhD, FAHA; Patricia Massicotte, MSc, MD, MHSc; Jennifer Meldau, MSN; E. Steve Roach, MD, FAHA; Edward R. Smith, MD; on behalf of the American Heart Association Stroke Council and Council on Cardiovascular and Stroke Nursing Purpose—Much has transpired since the last scientific statement on pediatric stroke was published 10 years ago. Although stroke has long been recognized as an adult health problem causing substantial morbidity and mortality, it is also an important cause of acquired brain injury in young patients, occurring most commonly in the neonate and throughout childhood. This scientific statement represents a synthesis of data and a consensus of the leading experts in childhood cardiovascular disease and stroke.
    [Show full text]
  • K191201 Trade/Device Name
    November 15, 2019 Hiossen, Inc. Peter Lee QA/RA Manager 85 Ben Fairless Drive Fairless Hills, Pennsylvania 19030 Re: K191201 Trade/Device Name: EM SA Implant System Regulation Number: 21 CFR 872.3640 Regulation Name: Endosseous Dental Implant Regulatory Class: Class II Product Code: DZE Dated: November 14, 2019 Received: November 15, 2019 Dear Peter Lee: We have reviewed your Section 510(k) premarket notification of intent to market the device referenced above and have determined the device is substantially equivalent (for the indications for use stated in the enclosure) to legally marketed predicate devices marketed in interstate commerce prior to May 28, 1976, the enactment date of the Medical Device Amendments, or to devices that have been reclassified in accordance with the provisions of the Federal Food, Drug, and Cosmetic Act (Act) that do not require approval of a premarket approval application (PMA). You may, therefore, market the device, subject to the general controls provisions of the Act. Although this letter refers to your product as a device, please be aware that some cleared products may instead be combination products. The 510(k) Premarket Notification Database located at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm identifies combination product submissions. The general controls provisions of the Act include requirements for annual registration, listing of devices, good manufacturing practice, labeling, and prohibitions against misbranding and adulteration. Please note: CDRH does not evaluate information related to contract liability warranties. We remind you, however, that device labeling must be truthful and not misleading. If your device is classified (see above) into either class II (Special Controls) or class III (PMA), it may be subject to additional controls.
    [Show full text]
  • Partitions with Prescribed Hook Differences
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Elsevier - Publisher Connector Europ. J. Combinatorics (1987) 8, 341-350 Partitions with Prescribed Hook Differences GEORGE E. ANDREWS,· R. J. BAXTER, D. M. BRESSOUD,t W. H. BURGE, P. J. FORRESTER AND G. VIENNOT We investigate partition identities related to off-diagonal hook differences. Our results generalize previous extensions of the Rogers-Ramanujan identities. The identity of the related polynomials with constructs in statistical mechanics is discussed. 1. INTRODUCTION I. Schur's polynomial proof of the Rogers-Ramanujan identities [15] has been the starting point for several diverse studies. The polynomials Schur introduced were generalized to provide partition identities related to successive ranks of partitions [1], [2], [5], [10), [11], [12], [13]. Subsequently these same polynomials were found to be the key element in providing Rogers-Ramanujan type identities for Regime II of the hard hexagon model [4], [9; ch. 14]. In [6], three of us considered an extensive generalization of the hard hexagon model, and we found that polynomial approximations to certain sums of eigenvalues played an essen­ tial role. Again these polynomials were clearly of the same general structure as those introduced by Schur. In this paper we consider in full generality the families of polynomials that arose in [6]. The statistic of partitions that plays the main role is that of 'off-diagonal hook difference', a topic introduced in [13]. In Section 2 we provide the necessary combinatorial preliminaries. In Section 3 we prove a general partition theorem which generalized the results in [1], [2], [5], [10], [11].
    [Show full text]
  • BASHKIR MANUAL Nicholas Poppe (University of Washington), and Andreas Tietze (University of California, Los Angeles), Consulting Editors
    f,;7 . I i / Indiana University Publications Graduate School Uralic and Altaic Series THOMAS A. SEBEOK, Editor Billie Greene, Assistant to the Editor Fred W. Householder, Jr., John R. Krueger, Felix J. Oinas, Alo Raun, Denis Sinor, and Odan Schiltz, Associate Editors GyuIa Decsy (University of Hamburg), Lawrence Krader (Syracuse Univer­ sity), John Lotz (Columbia University), Samuel E. Martin (Yale University), BASHKIR MANUAL Nicholas Poppe (University of Washington), and Andreas Tietze (University of California, Los Angeles), Consulting Editors Vol. American Studies in Uralic Linguistics, Edited by the Indiana University Committee on Uralic Studies (1960) o.p. Vol. 2 Buriat Grammar, by Nicholas Poppe (1960) - $3.00 Vol. 3 The Structure and Development of the Finnish Language, by Lauri Hakulinen (trans. by John Atkinson) (1961) $7.00 Vol. 4 Dagur Mongolian: Grammar Texts, and Lexicon, by Samuel E. Martin (1961) $5.00 Vol. 5 An Eastern Cheremis Manual: Phonology, Grammar, Texts, and Glossary, by Thomas A. Sebeok and Frances J. Ingemann (1961) - $4.00 Vol. 6 The Phonology of Modern Standard Turkish, byR.B.Lees(1961) - $3.50 Vol. 7 Chuvash Manual: Introduction, Grammar, Reader, and Vocabulary, by John R. Krueger (1961) $5.50 Vol. 8 Buriat Reader, by James E. Bosson (supervised and edited by Nicholas Poppe) (1962) - $2.00 Vol. 9 Latvian and Finnic Linguistic Convergences, by Valdis J. Zeps (1962) - $5.00 Vol. 10 Uzhek Newspaper Reader (with Glossary), by Nicholas Poppe, Jr. (1962) - $2.00 Vol. 11 Hungarian Reader (Folklore and Literature) With Notes, Edited by John Lotz (1962) $1.00 (continued inside back cover) t._._ Vt5f ~ HL9~1 AMERICAN COUNCIL OF LEARNED SOCIETIES Research and Studies in Uralic and Altaic Languages BASHKIR MANUAL Descriptive Grammar and Texts Project No.
    [Show full text]
  • 2Nd Stage International Geopolitical Competition TEST
    2nd stage International Geopolitical Competition TEST 11 VII 2020 RULES • The test duration is 60 minutes and it contains 60 test questions. You shall put your answers in previously sent form and it shouldn’t be sent back later than midday (Warsaw time). • You are prohibited to use any help from other people. Your personal notes, books, or the Internet is allowed. • For correct response you will gain 1 point, for no answer there is no penalty, and for wrong one there is -1 point penalty. • You are not allowed to change previously given answers. To answer a question put X mark on a table in your printed form. 1. What does powermetrics as subdiscipline of geopolitics deal with? A. History of geopolitical doctrines B. Statistics of international agreements C. Biographies of outstanding geopoliticians D. Measuring the length of national borders E. Measuring the power of political units F. Measurement of the soldiers’ physical ability QA: prof.. dr hab.. M. Sułek 2. Who introduced the concept of soft power to international relations science? A. Niccoló Machiavelli B. Rudolf Kjellén C. John Mearsheimer D. Karl Haushofer E. Joseph Nye F. Henry Kissinger QA: prof.. dr hab.. M. Sułek 3. Smart power is the connected way of using of: A. Hard power and sharp power B. Hard power and soft power C. Hard power and sticky power D. Soft power and sticky power E. Sharp power and soft power F. Sharp power and sticky power QA: prof.. dr hab.. M. Sułek 4. The main determinant of the state’s international position is: A.
    [Show full text]
  • Delaware River Water Quality ^Ristol to Marcus Hook Pennsylvania August 1949 to December 1963 R«» W
    Delaware River Water Quality ^ristol to Marcus Hook Pennsylvania August 1949 to December 1963 r«» W. B. KEIGHTON ONTRIBUTIONS TO THE HYDROLOGY OF THE UNITED STATES GEOLOGICAL SURVEY WATER-SUPPLY PAPER 1809-O ^repared in cooperation with the :ity of Philadelphia '.JNITED STATES GOVERNMENT PRINTING OFFICE, WASHINGTON : 1965 UNITED STATES DEPARTMENT OF THE INTERIOR STEWART L. UDALL, Secretory GEOLOGICAL SURVEY Thomas B. Nolan, Director For sale by the Superintendent of Documents, U.S. Government Printing Office Washington, D.C. 20402 - Price 25 cents (paper cover) CONTENTS Page Abstract_______________________________________________________ 01 Introduction. _____________________________________________________ 1 Purpose and scope_____________________________________________ 4 Previous investigations and acknowledgments.____________________ 4 Some basic physical and hydrologic facts______.______________________ 5 Graphic summaries of water-quality facts__________________________ 9 Water temperature____________________________________________ 9 Hydrogen-ion concentration (pH)______________________________ 10 Dissolved-solids concentration_________________________________ 10 Composition of dissolved solids__________________________________ 19 Chloride___________________________________-_ 24 Dissolved oxygen._____________________________________________ 27 Suspended sediment.__________________________________________ 30 Tabular summaries of water-quality characteristics ____________________ 36 Local variations in water-quality patterns___________________-__--___-
    [Show full text]
  • Is Multihop QA in DIRE Condition? Measuring and Reducing Disconnected Reasoning
    Is Multihop QA in DIRE Condition? Measuring and Reducing Disconnected Reasoning Harsh Trivedi†∗ Niranjan Balasubramanian† Tushar Khot‡ Ashish Sabharwal‡ † Stony Brook University, Stony Brook, U.S.A. fhjtrivedi,[email protected] ‡ Allen Institute for AI, Seattle, U.S.A. ftushark,[email protected] Abstract Input Which country got independence when the cold war started? Has there been real progress in multi-hop The war started in 1950. question-answering? Models often exploit The cold war started in 1947. dataset artifacts to produce correct answers, France finally got its independence. without connecting information across multi- India got independence from UK in 1947. Set of Facts ple supporting facts. This limits our ability 30 countries were involved in World War 2. to measure true progress and defeats the pur- pose of building multi-hop QA datasets. We Disconnected Reasoning Output make three contributions towards addressing Answer SF Simple Combination this. First, we formalize such undesirable be- Answer havior as disconnected reasoning across sub- India India sets of supporting facts. This allows develop- No Interaction Supporting Facts (SF) ing a model-agnostic probe for measuring how much any model can cheat via disconnected reasoning. Second, using a notion of con- trastive support sufficiency, we introduce an Figure 1: Example of disconnected reasoning, a form automatic transformation of existing datasets of bad multifact reasoning: Model arrives at the answer that reduces the amount of disconnected rea- by simply combining its outputs from two subsets of soning. Third, our experiments1 suggest that the input, neither of which contains all supporting facts. there hasn’t been much progress in multifact From one subset, it identifies the blue supporting fact QA in the reading comprehension setting.
    [Show full text]
  • A Comparison of Circle and J Hook Performance Within the Grenadian Pelagic Longline Fishery Anthony G
    Nova Southeastern University NSUWorks HCNSO Student Theses and Dissertations HCNSO Student Work 4-25-2019 A Comparison of Circle and J Hook Performance within the Grenadian Pelagic Longline Fishery Anthony G. Burns Nova Southeastern University, [email protected] Follow this and additional works at: https://nsuworks.nova.edu/occ_stuetd Part of the Marine Biology Commons, and the Oceanography and Atmospheric Sciences and Meteorology Commons Share Feedback About This Item NSUWorks Citation Anthony G. Burns. 2019. A Comparison of Circle and J Hook Performance within the Grenadian Pelagic Longline Fishery. Master's thesis. Nova Southeastern University. Retrieved from NSUWorks, . (510) https://nsuworks.nova.edu/occ_stuetd/510. This Thesis is brought to you by the HCNSO Student Work at NSUWorks. It has been accepted for inclusion in HCNSO Student Theses and Dissertations by an authorized administrator of NSUWorks. For more information, please contact [email protected]. Thesis of Anthony G. Burns Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Science M.S. Marine Biology Nova Southeastern University Halmos College of Natural Sciences and Oceanography April 2019 Approved: Thesis Committee Major Professor: David Kerstetter Committee Member: Amy C. Hirons, Ph.D. Committee Member: Paul T. Arena, Ph.D. This thesis is available at NSUWorks: https://nsuworks.nova.edu/occ_stuetd/510 HALMOS COLLEGE OF NATURAL SCIENCES AND OCEANOGRAPHY A COMPARISON OF CIRCLE AND J HOOK PERFORMANCE WITHIN THE GRENADIAN PELAGIC LONGLINE FISHERY By: Anthony G. Burns Submitted to the Faculty of Halmos College of Natural Sciences and Oceanography in partial fulfillment of the requirements for the degree of Master of Science with a specialty in: Marine Biology Nova Southeastern University May 2019 Thesis of Anthony G.
    [Show full text]