DOCSLIB.ORG

Enhancing Neural Data-To-Text Generation Models with External Background Knowledge

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Enhancing Neural Data-To-Text Generation Models with External Background Knowledge Enhancing Neural Data-To-Text Generation Models with External Background Knowledge Shuang Chen1,∗ Jinpeng Wang2, Xiaocheng Feng1, Feng Jiang1;3, Bing Qin1, Chin-Yew Lin2 1 Harbin Institute of Technology, Harbin, China 2 Microsoft Research Asia 3 Peng Cheng Laboratory [email protected], fjinpwa, [email protected], fxcfeng, [email protected], [email protected] Abstract Infobox Description Recent neural models for data-to-text genera- Nacer Hammami (born December 28, 1980) is an Algerian football player who is tion rely on massive parallel pairs of data and currently playing for MC El Eulma in the text to learn the writing knowledge. They of- Algerian Ligue Professionnelle 1. ten assume that writing knowledge can be ac- quired from the training data alone. However, Entity Linking when people are writing, they not only rely Subject Relation Object Guelma country Algeria on the data but also consider related knowl- (Q609871) (P17) (Q262) edge. In this paper, we enhance neural data-to- defender instance of association football position text models with external knowledge in a sim- (Q336286) (P31) (Q4611891) …… … ple but effective way to improve the fidelity MC El Eulma league Algerian Ligue Professionnelle 1 of generated text. Besides relying on parallel (Q2742749) (P118) (Q647746) data and text as in previous work, our model External Background Knowledge from Wikidata attends to relevant external knowledge, en- Figure 1: An example of generating description from coded as a temporary memory, and combines a Wikipedia infobox. External background knowledge this knowledge with the context representa- expanded from the infobox is helpful for generation. tion of data before generating words. This al- lows the model to infer relevant facts which are not explicitly stated in the data table from an external knowledge source. Experimen- determines how to generate the text based on se- tal results on twenty-one Wikipedia infobox- lected contents. Traditionally, these two sub- to-text datasets show our model, KBAtt, con- problems have been tackled separately. In recent sistently improves a state-of-the-art model on years, neural generation models, especially the most of the datasets. In addition, to quantify encoder-decoder model, solve these two subprob- when and why external knowledge is effective, lems jointly and have achieved remarkable suc- we design a metric, KBGain, which shows a cesses in several benchmarks (Mei et al., 2016; strong correlation with the observed perfor- Lebret et al., 2016; Wiseman et al., 2017; Dusekˇ mance boost. This result demonstrates the rel- evance of external knowledge and sparseness et al., 2018; Nie et al., 2018). of original data are the main factors affecting Such end-to-end data-to-text models rely on system performance. massive parallel pairs of data and text to learn the 1 Introduction writing knowledge. They often assume that all writing knowledge can be learned from the train- Automatic text generation from structured data ing data. However, when people are writing, they (data-to-text) is a classic task in natural language will not only rely on the data contents themselves generation which aims to automatically gener- but also consider related knowledge, which is ne- ate fluent, truthful and informative texts based glected by previous methods. For example, as on structured data (Kukich, 1983; Holmes-Higgin, shown in Fig.1, an infobox about a person called 1994; Reiter and Dale, 1997). Data-to-text is of- Nacer Hammami is paired with its corresponding ten formulated into two subproblems: content se- biography description from the Wikipedia. How- lection which decides what contents should be in- ever, the information in the infobox is not enough cluded in the text and surface realization which to cover all the facts mentioned in the descrip- ∗Contribution during internship at Microsoft Research. tion. To generate this description from the in- 3022 Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3022–3032, Hong Kong, China, November 3–7, 2019. c 2019 Association for Computational Linguistics fobox, we need to expand information based on performance. external background knowledge from its related The contributions of our work can be summa- entities. For example, in the description: 1) “is rized as follows: an Algerian football player” indicates the nation- ality of Nacer Hammami which is not explicitly • We demonstrate that external knowledge base stated in the infobox. However, the place of birth, could be used to enhance the performance of Guelma, of Nacer Hammami is given, therefore neural data-to-text models. the nationality can be inferred from the knowledge • We propose a simple yet effective model, that Guelma is a place in Algeria. 2) “playing for KBAtt, to integrate external knowledge using MC El Eulma in the Algerian Ligue Profession- a dual-attention mechanism. nelle 1” describes the fact that MC El Eulma is a club in the Algerian Ligue Professionnelle 1 which • We design a metric, KBGain, to quantify is also not explicitly stated in the infobox which when and why external knowledge is effec- can be expanded from external knowledge base. tive. One may argue that neural models can • We contribute twenty infobox-to-text learn such knowledge when enough parallel co- datasets from a variety of domains. occurrence pairs such as (Guelma, Algerian) and (MC El Eulma, Algerian Ligue Professionnelle 1) are available. However even in such case, neu- 2 The Proposed Model ral models still tend to make mistakes for sparse Our model takes a data table D (e.g., a Wikipedia co-occurrence pairs as we will show in the experi- infobox) and a relevant external knowledge base ments section. (KB) containing a set of facts F as input and gen- In this paper we enhance neural-network- erates a natural language text y = y1; :::; yT con- based data-to-text generation models with external sisting of T words. To augment the infobox with knowledge in a simple but effective way. Besides external knowledge, we preserve the Wikipedia in- learning the association between data and text ternal hyperlink information in the field values of from parallel data-text pairs as in previous work, infobox, and track these hyperlinks to get their our model attends to relevant external knowledge, corresponding entities from Wikidata2 where we encoded as a temporary memory, and combines retrieve only one-hop facts. The backbone of our this knowledge with the context representation of model is an attention based sequence-to-sequence data before generating words. Specifically, both model (Bahdanau et al., 2014) equipped with copy infobox and background knowledge facts are en- mechanism (See et al., 2017). As shown in Fig.2, coded and a dual-attention mechanism is proposed the model consists of four main components: a to guide the decoder to generate text. table encoder, a KB encoder, the dual attention To verify the effectiveness of our proposed mechanism and a decoder. We describe each com- model, Knowledge Base enhanced Attention- ponent in the following sections. based sequence-to-sequence network (KBAtt), we conduct experiments on multiple Wikipedia 2.1 Table Encoder infobox-to-text datasets including WikiBio (Le- In Fig.2, the input data table D consists of sev- 1 bret et al., 2016) and 20 new datasets . Our ex- eral field name and field value pairs. We follow periment results show that KBAtt consistently im- (Sha et al., 2017; Liu et al., 2017) to tokenize the proves a state-of-the-art neural data-to-text model field values and transform the input table into a to achieve higher performances on most of the N flattened sequence f(ni; vi)gi=1, where each ele- datasets. To quantify when and why external ment is a token vi from a field value paired with knowledge is effective, we design a metric which its corresponding field name ni. To encode the shows a strong correlation with the observed per- flattened table, we map each (ni; vi) to vector n v n v formance boost. This result demonstrates the rel- xi = [e i ; e i ], where e i and e i are trainable evance of external knowledge and sparseness of 2We adopt Wikidata (dumps version: 20150831) as the original data are the main factors affecting system external knowledge base. Although we extend the infobox with external knowledge by using the Wikipedia hyperlink, 1Available at https://github.com/hitercs/ we can also apply entity linking to link input data to the WikiInfo2Text knowledge base in practice. 3023 Field Name Field Value Table Encoder Decoder ࢚࢝ Field Name ሺ࢔࢏ሻ Content ሺ࢜࢏ሻ Full name Nacer Hammami Full name Nacer ࢎ૚ softmax Place of birth Guelma Full name Hammami Playing position Defender Flatten Table ࢎ૛ ࢚ࢇ࢈࢒ࢋ …… GRU Table Attention ࢉ࢚ MLP Current team MC El Eulma Current team Eulma ࢎࡺ Entity Linking ࢊ࢚ Field Name ሺ࢔࢐ሻ Subject ሺ࢙࢐ሻ Relation ሺ࢘࢐ሻ Object ሺ࢕࢐ሻ KB Encoder Guelma country Algeria Place of birth ࢎ GRU (Q609871) (P17) (Q262) ૚ …… … … ࢎ KB Attention ࢑࢈ ࢐ ࢉ࢚ ࢉ࢚࢞ MC El Eulma league Algerian Ligue Professionnelle 1 ࢉ࢚ି૚ ࢊ࢚ି૚ ࢚࢝ି૚ Current team (Q2742749) (P118) (Q647746) ࢎࡺ Figure 2: A diagram of the knowledge base enhanced neural data-to-text generation model. First, we transform the table into a flattened sequence, extract entities mentioned in the field value of the infobox and link them to Wikidata where we can retrieve relevant facts. Then, the table contents and external knowledge base facts are carefully encoded. Finally, a single layer GRU decoder with a dual attention mechanism decides which part of information should be used for generation. word embeddings of ni and vi, and [:; :] is the con- to generate words conditioned on both table infor- catenation of vectors. Then each xi is encoded mation and background knowledge fact informa- into a hidden vector hi using a bi-directional GRU tion.
Load more

Recommended publications
  • Realising the Potential of Algal Biomass Production Through Semantic Web and Linked Data
    LEAPS: Realising the Potential of Algal Biomass Production through Semantic Web and Linked data ∗ Monika Solanki Johannes Skarka Craig Chapman KBE Lab Karlsruhe Institute of KBE Lab Birmingham City University Technology (ITAS) Birmingham City University [email protected] [email protected] [email protected] ABSTRACT In order to derive fuels from biomass, algal operation plant Recently algal biomass has been identified as a potential sites are setup that facilitate biomass cultivation and con- source of large scale production of biofuels. Governments, version of the biomass into end use products, some of which environmental research councils and special interests groups are biofuels. Microalgal biomass production in Europe is are funding several efforts that investigate renewable energy seen as a promising option for biofuels production regarding production opportunities in this sector. However so far there energy security and sustainability. Since microalgae can be has been no systematic study that analyses algal biomass cultivated in photobioreactors on non-arable land this tech- potential especially in North-Western Europe. In this paper nology could significantly reduce the food vs. fuel dilemma. we present a spatial data integration and analysis frame- However, until now there has been no systematic analysis work whereby rich datasets from the algal biomass domain of the algae biomass potential for North-Western Europe. that include data about algal operation sites and CO2 source In [20], the authors assessed the resource potential for mi- sites amongst others are semantically enriched with ontolo- croalgal biomass but excluded all areas not between 37◦N gies specifically designed for the domain and made available and 37◦S, thus most of Europe.
    [Show full text]
  • From Cataloguing Cards to Semantic Web 1
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Scientific Open-access Literature Archive and Repository Archeologia e Calcolatori 20, 2009, 111-128 REPRESENTING KNOWLEDGE IN ARCHAEOLOGY: FROM CATALOGUING CARDS TO SEMANTIC WEB 1. Introduction Representing knowledge is the basis of any catalogue. The Italian Ca- talogue was based on very valuable ideas, developed in the late 1880s, with the basic concept of putting objects in their context. Initially cataloguing was done manually, with typewritten cards. The advent of computers led to some early experimentations, and subsequently to the de�nition of a more formalized representation schema. The web produced a cultural revolution, and made the need for technological and semantic interoperability more evi- dent. The Semantic Web scenario promises to allow a sharing of knowledge, which will make the knowledge that remained unexpressed in the traditional environments available to any user, and allow objects to be placed in their cultural context. In this paper we will brie�y recall the principles of cataloguing and lessons learned by early computer experiences. Subsequently we will descri- be the object model for archaeological items, discussing its strong and weak points. In section 5 we present the web scenario and approaches to represent knowledge. 2. Cataloguing: history and principles The Italian Catalogue of cultural heritage has its roots in the experiences and concepts, developed in the late 1880s and early 1900s, by the famous art historian Adolfo Venturi, who was probably one of the �rst scholars to think explicitly in terms of having a frame of reference to describe works of art, emphasizing as the main issue the context in which the work had been produced.
    [Show full text]
  • Download Slides
    a platform for all that we know savas parastatidis http://savas.me savasp transition from web to apps increasing focus on information (& knowledge) rise of personal digital assistants importance of near-real time processing http://aptito.com/blog/wp-content/uploads/2012/05/smartphone-apps.jpg today... storing computing computers are huge amounts great tools for of data managing indexing example google and microsoft both have copies of the entire web (and more) for indexing purposes tomorrow... storing computing computers are huge amounts great tools for of data managing indexing acquisition discovery aggregation organization we would like computers to of the world’s information also help with the automatic correlation analysis and knowledge interpretation inference data information knowledge intelligence wisdom expert systems watson freebase wolframalpha rdbms google now web indexing data is symbols (bits, numbers, characters) information adds meaning to data through the introduction of relationship - it answers questions such as “who”, “what”, “where”, and “when” knowledge is a description of how the world works - it’s the application of data and information in order to answer “how” questions G. Bellinger, D. Castro, and A. Mills, “Data, Information, Knowledge, and Wisdom,” Inform. pp. 1–4, 2004 web – the data platform web – the information platform web – the knowledge platform foundation for new experiences “wisdom is not a product of schooling but of the lifelong attempt to acquire it” representative examples wolframalpha watson source:
    [Show full text]
  • Knowledge Extraction for Hybrid Question Answering
    KNOWLEDGEEXTRACTIONFORHYBRID QUESTIONANSWERING Von der Fakultät für Mathematik und Informatik der Universität Leipzig angenommene DISSERTATION zur Erlangung des akademischen Grades Doctor rerum naturalium (Dr. rer. nat.) im Fachgebiet Informatik vorgelegt von Ricardo Usbeck, M.Sc. geboren am 01.04.1988 in Halle (Saale), Deutschland Die Annahme der Dissertation wurde empfohlen von: 1. Professor Dr. Klaus-Peter Fähnrich (Leipzig) 2. Professor Dr. Philipp Cimiano (Bielefeld) Die Verleihung des akademischen Grades erfolgt mit Bestehen der Verteidigung am 17. Mai 2017 mit dem Gesamtprädikat magna cum laude. Leipzig, den 17. Mai 2017 bibliographic data title: Knowledge Extraction for Hybrid Question Answering author: Ricardo Usbeck statistical information: 10 chapters, 169 pages, 28 figures, 32 tables, 8 listings, 5 algorithms, 178 literature references, 1 appendix part supervisors: Prof. Dr.-Ing. habil. Klaus-Peter Fähnrich Dr. Axel-Cyrille Ngonga Ngomo institution: Leipzig University, Faculty for Mathematics and Computer Science time frame: January 2013 - March 2016 ABSTRACT Over the last decades, several billion Web pages have been made available on the Web. The growing amount of Web data provides the world’s largest collection of knowledge.1 Most of this full-text data like blogs, news or encyclopaedic informa- tion is textual in nature. However, the increasing amount of structured respectively semantic data2 available on the Web fosters new search paradigms. These novel paradigms ease the development of natural language interfaces which enable end- users to easily access and benefit from large amounts of data without the need to understand the underlying structures or algorithms. Building a natural language Question Answering (QA) system over heteroge- neous, Web-based knowledge sources requires various building blocks.
    [Show full text]
  • Datatone: Managing Ambiguity in Natural Language Interfaces for Data Visualization Tong Gao1, Mira Dontcheva2, Eytan Adar1, Zhicheng Liu2, Karrie Karahalios3
    DataTone: Managing Ambiguity in Natural Language Interfaces for Data Visualization Tong Gao1, Mira Dontcheva2, Eytan Adar1, Zhicheng Liu2, Karrie Karahalios3 1University of Michigan, 2Adobe Research 3University of Illinois, Ann Arbor, MI San Francisco, CA Urbana Champaign, IL fgaotong,[email protected] fmirad,[email protected] [email protected] ABSTRACT to be both flexible and easy to use. General purpose spread- Answering questions with data is a difficult and time- sheet tools, such as Microsoft Excel, focus largely on offer- consuming process. Visual dashboards and templates make ing rich data transformation operations. Visualizations are it easy to get started, but asking more sophisticated questions merely output to the calculations in the spreadsheet. Asking often requires learning a tool designed for expert analysts. a “visual question” requires users to translate their questions Natural language interaction allows users to ask questions di- into operations on the spreadsheet rather than operations on rectly in complex programs without having to learn how to the visualization. In contrast, visual analysis tools, such as use an interface. However, natural language is often ambigu- Tableau,1 creates visualizations automatically based on vari- ous. In this work we propose a mixed-initiative approach to ables of interest, allowing users to ask questions interactively managing ambiguity in natural language interfaces for data through the visualizations. However, because these tools are visualization. We model ambiguity throughout the process of often intended for domain specialists, they have complex in- turning a natural language query into a visualization and use terfaces and a steep learning curve. algorithmic disambiguation coupled with interactive ambigu- Natural language interaction offers a compelling complement ity widgets.
    [Show full text]
  • Knowit VQA: Answering Knowledge-Based Questions About Videos
    The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20) KnowIT VQA: Answering Knowledge-Based Questions about Videos Noa Garcia,1 Mayu Otani,2 Chenhui Chu,1 Yuta Nakashima1 1Osaka University, Japan, 2CyberAgent, Inc., Japan {noagarcia, chu, n-yuta}@ids.osaka-u.ac.jp otani [email protected] Abstract We propose a novel video understanding task by fusing knowledge-based and video question answering. First, we in- troduce KnowIT VQA, a video dataset with 24,282 human- generated question-answer pairs about a popular sitcom. The dataset combines visual, textual and temporal coherence rea- /HRQDUG+DYH\RXQRWLFHGWKDW+RZDUGFDQWDNHDQ\WRSLFDQGXVHLWWRUHPLQG soning together with knowledge-based questions, which need \RXWKDWKHZHQWWRVSDFH" of the experience obtained from the viewing of the series to be 6KHOGRQ,QWHUHVWLQJK\SRWKHVLV/HW¶VDSSO\WKHVFLHQWLILFPHWKRG answered. Second, we propose a video understanding model /HRQDUG2ND\+H\+RZDUGDQ\WKRXJKWVRQZKHUHZHVKRXOGJHWGLQQHU" by combining the visual and textual video content with spe- +RZDUG$Q\ZKHUHEXWWKH6SDFH6WDWLRQ2QDJRRGGD\GLQQHUZDVDEDJIXOO cific knowledge about the show. Our main findings are: (i) the RIPHDWORDI%XWKH\\RXGRQ¶WJRWKHUHIRUWKHIRRG\RXJRWKHUHIRUWKHYLHZ incorporation of knowledge produces outstanding improve- ments for VQA in video, and (ii) the performance on KnowIT 9LVXDO +RZPDQ\SHRSOHDUHWKHUHZHDULQJJODVVHV"2QH VQA still lags well behind human accuracy, indicating its 7H[WXDO :KRKDVEHHQWRWKHV:KRKDVEHHQWRWKHVSDFH"SDFH" +R+RZDUGZDUG usefulness for studying current video modelling limitations. 7HPSRUDO +RZGRWKH\+RZGRWKH\ILQLVKWKHFRQYHUVDWLRQ" ILQLVKWKHFRQYHUVDWLRQ"66KDNLQJKDQGVKDNLQJKDQGV Introduction .QRZOHGJH :KRRZQVWKHSODFHZKHUHWKH:KRRZQVWKHSODFHZKHUHWKH\DUHVWDQGLQJ"\DUHVWDQGLQJ"66WXDUWWXDUW Visual question answering (VQA) was firstly introduced in (Malinowski and Fritz 2014) as a task for bringing to- Figure 1: Types of questions addressed in KnowIt VQA. gether advancements in natural language processing and image understanding.
    [Show full text]
  • Directions in AI Research and Applications at Siemens Corporate
    AI Magazine Volume 11 Number 1 (1991)(1990) (© AAAI) Research in Progress of linguistic phenomena. The com- Directions in AI Research putational work concerns questions of adequate processing models and algorithms, as embodied in the and Applications at actual interfaces being developed. These topics are explored in the Siemens Corporate Research framework of three projects: The nat- ural language consulting dialogue and Development project (Wisber) takes up research- oriented topics (descriptive grammar formalisms and linguistically adequate grammar specification, handling of Wolfram Buettner, Klaus Estenfeld, discourse, and so on), the data-access Hans Haugeneder, and Peter Struss project (Sepp) focuses on the practi- cal application of state-of-the-art technology, and the work on gram- mar-development environments ■ Many barriers exist today that prevent particular, Prolog extensions; and (4) (Ape) centers on the problem of ade- effective industrial exploitation of current design and analysis of neural networks. quate tools for specifying linguistic and future AI research. These barriers can The lab’s 26 researchers are orga- knowledge sources and efficient pro- only be removed by people who are work- nized into four groups corresponding cessing methods. ing at the scientific forefront in AI and to these areas. Together, they provide Wisber is jointly funded by the know potential industrial needs. Siemens with innovative software The Knowledge Processing Laboratory’s German government and several research and development concentrates in technologies, appropriate applications, industrial and academic partners. the following areas: (1) natural language prototypical implementations of AI The goal of the project is to develop interfaces to knowledge-based systems and systems, and evaluations of new a knowledge-based advice-giving databases; (2) theoretical and experimen- techniques and trends.
    [Show full text]
  • A Framework for Ontology-Based Library Data Generation, Access and Exploitation
    Universidad Politécnica de Madrid Departamento de Inteligencia Artificial DOCTORADO EN INTELIGENCIA ARTIFICIAL A framework for ontology-based library data generation, access and exploitation Doctoral Dissertation of: Daniel Vila-Suero Advisors: Prof. Asunción Gómez-Pérez Dr. Jorge Gracia 2 i To Adelina, Gustavo, Pablo and Amélie Madrid, July 2016 ii Abstract Historically, libraries have been responsible for storing, preserving, cata- loguing and making available to the public large collections of information re- sources. In order to classify and organize these collections, the library commu- nity has developed several standards for the production, storage and communica- tion of data describing different aspects of library knowledge assets. However, as we will argue in this thesis, most of the current practices and standards available are limited in their ability to integrate library data within the largest information network ever created: the World Wide Web (WWW). This thesis aims at providing theoretical foundations and technical solutions to tackle some of the challenges in bridging the gap between these two areas: library science and technologies, and the Web of Data. The investigation of these aspects has been tackled with a combination of theoretical, technological and empirical approaches. Moreover, the research presented in this thesis has been largely applied and deployed to sustain a large online data service of the National Library of Spain: datos.bne.es. Specifically, this thesis proposes and eval- uates several constructs, languages, models and methods with the objective of transforming and publishing library catalogue data using semantic technologies and ontologies. In this thesis, we introduce marimba-framework, an ontology- based library data framework, that encompasses these constructs, languages, mod- els and methods.
    [Show full text]
  • Ontologies to Interpret Remote Sensing Images: Why Do We Need Them?
    Ontologies to interpret remote sensing images : why do we need them? Damien Arvor, Mariana Belgiu, Zoe Falomir, Isabelle Mougenot, Laurent Durieux To cite this version: Damien Arvor, Mariana Belgiu, Zoe Falomir, Isabelle Mougenot, Laurent Durieux. Ontologies to inter- pret remote sensing images : why do we need them?. GIScience and Remote Sensing, Taylor & Francis: STM, Behavioural Science and Public Health Titles, 2019, pp.1-29. 10.1080/15481603.2019.1587890. halshs-02079438 HAL Id: halshs-02079438 https://halshs.archives-ouvertes.fr/halshs-02079438 Submitted on 26 Mar 2019 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. GIScience & Remote Sensing ISSN: 1548-1603 (Print) 1943-7226 (Online) Journal homepage: https://www.tandfonline.com/loi/tgrs20 Ontologies to interpret remote sensing images: why do we need them? Damien Arvor, Mariana Belgiu, Zoe Falomir, Isabelle Mougenot & Laurent Durieux To cite this article: Damien Arvor, Mariana Belgiu, Zoe Falomir, Isabelle Mougenot & Laurent Durieux (2019): Ontologies to interpret remote sensing images: why do we need them?, GIScience & Remote Sensing To link to this article: https://doi.org/10.1080/15481603.2019.1587890 © 2019 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.
    [Show full text]
  • Question Answering
    Question Answering What is Ques+on Answering? Dan Jurafsky Ques%on Answering One of the oldest NLP tasks (punched card systems in 1961) Simmons, Klein, McConlogue. 1964. Indexing and ( ( Dependency Logic for Answering English Quesons. !"#$%&' )&*#'%+,-.'$/#0$ American Documentaon 15:30, 196-204 What do worms eat? Worms eat grass Horses with worms eat grass worms horses worms eat with eat eat grass worms grass what Birds eat worms Grass is eaten by worms birds worms eat eat 2 worms grass Dan Jurafsky Ques%on Answering: IBM’s Watson • Won Jeopardy on February 16, 2011! WILLIAM WILKINSON’S “AN ACCOUNT OF THE PRINCIPALITIES OF WALLACHIA AND MOLDOVIA” Bram Stoker INSPIRED THIS AUTHOR’S MOST FAMOUS NOVEL 3 Dan Jurafsky Apple’s Siri 4 Dan Jurafsky Wolfram Alpha 5 Dan Jurafsky Types of Ques%ons in Modern Systems • Factoid ques+ons • Who wrote “The Universal Declara4on of Human Rights”? • How many calories are there in two slices of apple pie? • What is the average age of the onset of au4sm? • Where is Apple Computer based? • Complex (narrave) ques+ons: • In children with an acute febrile illness, what is the efficacy of acetaminophen in reducing fever? • What do scholars think about Jefferson’s posi4on on dealing with pirates? 6 Dan Jurafsky Commercial systems: mainly factoid quesons Where is the Louvre Museum located? In Paris, France What’s the abbreviaon for limited L.P. partnership? What are the names of Odin’s ravens? Huginn and Muninn What currency is used in China? The yuan What kind of nuts are used in marzipan? almonds What instrument does
    [Show full text]
  • Ques+On Answering
    Queson Answering Debapriyo Majumdar Information Retrieval – Spring 2015 Indian Statistical Institute Kolkata Adapted from slides by Dan Jurafsky (Stanford) and Tao Yang (UCSB) Queson Answering One of the oldest NLP tasks (punched card systems in 1961) Simmons, Klein, McConlogue. 1964. Indexing and Dependency Logic for Answering English Ques%on: Poten%al-Answers: Questions. American Documentation 15:30, 196-204 What do worms eat? Worms eat grass Horses with worms eat grass worms horses worms eat with eat eat grass worms grass what Birds eat worms Grass is eaten by worms birds worms eat eat worms grass 2 Ques%on Answering: IBM’s Watson § Won Jeopardy on February 16, 2011! WILLIAM WILKINSON’S “AN ACCOUNT OF THE PRINCIPALITIES OF WALLACHIA AND MOLDOVIA” Bram Stoker INSPIRED THIS AUTHOR’S MOST FAMOUS NOVEL 3 Apple’s Siri § A seemingly “limited” set of few possible questions § Answers based on contextual parameters 4 Wolfram Alpha, Google 5 Wolfram Alpha But in this case, Google returns a standard list of document links 6 Types of Ques%ons in Modern Systems § Factoid questions – Answers are short – The question can be rephrased as “fill in the blanks” question Examples: – Who directed the movie Titanic? – How many calories are there in two slices of apple pie? – Where is Louvre museum located? § Complex (narrative) questions: – What precautionary measures should we take to be safe from swine flu? – What do scholars think about Jefferson’s position on dealing with pirates? 7 Paradigms for QA § IR-based approaches – TREC QA Track (AskMSR, ISI,
    [Show full text]
  • The Negative Impact of A-Box Materialization on Rdf2vec Knowledge Graph Embeddings
    More is not Always Better: The Negative Impact of A-box Materialization on RDF2vec Knowledge Graph Embeddings Andreea Ianaa, Heiko Paulheima aData and Web Science Group, University of Mannheim, Germany Abstract RDF2vec is an embedding technique for representing knowledge graph entities in a continuous vector space. In this paper, we investigate the effect of materializing implicit A-box axioms induced by subproperties, as well as symmetric and transitive properties. While it might be a reasonable assumption that such a materialization before computing embeddings might lead to better embeddings, we conduct a set of experiments on DBpedia which demonstrate that the materialization actually has a negative effect on the performance of RDF2vec. In our analysis, we argue that despite the huge body of work devotedon completing missing information in knowledge graphs, such missing implicit information is actually a signal, not a defect, and we show examples illustrating that assumption. Keywords RDF2Vec, Embedding, Reasoning, Knowledge Graph Completion, A-box Materialization 1. Introduction A straightforward assumption is that completing miss- ing knowledge in a knowledge graph before computing RDFvec [1] was originally conceived for exploiting knowl- node representations will lead to better results. However, edge graphs in data mining. Since most popular data in this paper, we show that the opposite actually holds: mining tools require a feature vector representation of completing the knowledge graph before computing an records, various techniques have been proposed for cre- RDF2vec embedding actually leads to worse results in ating vector space representations from subgraphs, in- downstream tasks. cluding adding datatype properties as features or creat- ing binary features for types [2].
    [Show full text]