Teaching Forge to Verbalize Dbpedia Properties in Spanish
Total Page:16
File Type:pdf, Size:1020Kb
Teaching FORGe to Verbalize DBpedia Properties in Spanish Simon Mille Stamatia Dasiopoulou Universitat Pompeu Fabra Independent Researcher Barcelona, Spain Barcelona, Spain [email protected] [email protected] Beatriz Fisas Leo Wanner Universitat Pompeu Fabra ICREA and Universitat Pompeu Fabra Barcelona, Spain Barcelona, Spain [email protected] [email protected] Abstract a tremendous amount of structured knowledge Statistical generators increasingly dominate has been made publicly available as language- the research in NLG. However, grammar- independent triples; the Linked Open Data (LOD) based generators that are grounded in a solid cloud currently contains over one thousand inter- linguistic framework remain very competitive, linked datasets (e.g., DBpedia, Wikidata), which especially for generation from deep knowl- cover a large range of domains and amount to edge structures. Furthermore, if built modu- billions of different triples. The verbalization of larly, they can be ported to other genres and LOD triples, i.e., their mapping onto sentences in languages with a limited amount of work, without the need of the annotation of a consid- natural languages, has been attracting a growing erable amount of training data. One of these interest in the past years, as shown by the organi- generators is FORGe, which is based on the zation of dedicated events such as the WebNLG Meaning-Text Model. In the recent WebNLG 2016 workshop (Gardent and Gangemi, 2016) challenge (the first comprehensive task ad- and the 2017 WebNLG challenge (Gardent et al., dressing the mapping of RDF triples to text) 2017b). As a result, a variety of new NLG systems FORGe ranked first with respect to the over- designed specifically for handling structured data all quality in human evaluation. We extend the have emerged, most of them statistical, as seen in coverage of FORGE’s open source grammati- cal and lexical resources for English, so as to the 2017 WebNLG challenge, although a number further improve the English outcome, and port of rule-based generators have also been presented. them to Spanish, to achieve a comparable qual- All systems focus on English, mainly because no ity. This confirms that, as already observed training data other than for English are available as in the case of SimpleNLG, a robust universal yet. Given the high cost for the creation of train- grammar-driven framework and a systematic ing data, this state of affairs is likely to persist for organization of the linguistic resources can be some time. Therefore, the question on the compet- an adequate choice for NLG applications. itiveness of rule-based generators arises. 1 Introduction One of the rule-based generators presented at The origins of Natural Language Generation WebNLG was FORGe (Mille and Dasiopoulou, (NLG) are in rule-based sentence/text genera- 2017), which ranked first with respect to over- tion from numerical data or deep semantic struc- all quality in the human evaluation. FORGe is tures. With the availability of large scale syn- grounded in the linguistic model of the Meaning- tactically annotated corpora and the lack of pub- Text Theory (Mel’cukˇ , 1988). The multistratal licly available knowledge repositories, the focus nature of this model allows for a modular or- had shifted to statistical surface generation. How- ganization of blocks of graph-transduction rules, ever, thanks to Semantic Web (SW) initiatives from blocks that are universal, i.e., multilingual, such as the W3C Linking Open Data Project,1 to blocks that are language-specific. The graph- transduction framework MATE (Bohnet and Wan- 1https://www.w3.org/wiki/SweoIG/ TaskForces/CommunityProjects/ ner, 2010) furthermore facilitates a systematic hi- LinkingOpenData erarchical rule writing and testing. SimpleNLG 473 Proceedings of The 12th International Conference on Natural Language Generation, pages 473–483, Tokyo, Japan, 28 Oct - 1 Nov, 2019. c 2019 Association for Computational Linguistics (Gatt and Reiter, 2009) demonstrated that a well- ing statistically the most appropriate output (Gar- defined generation infrastructure, along with a dent et al., 2017b; Belz et al., 2011). Template- transparent, easy to handle rule and structure for- based systems are very robust, but also limited in mat, is a key for its take up and use for creation terms of portability since new templates need to of generation modules for multiple languages. In be defined for every new domain, style, language, what follows, we aim to demonstrate that the etc. Statistical systems have the best coverage, but FORGe generator can also well serve as a multi- the relevance and the quality of the produced texts lingual portable text generator for verbalization of cannot be ensured. Furthermore, they are fully de- structured data and that its lexical and grammatical pendent on the available (still scarce and mostly resources can be easily extended to reach a higher monolingual) training data. The development of coverage of linguistic constructions. For this, we grammar-based systems is time-consuming and extend its publicly available resources for English, they usually have coverage issues. However, they so as to improve the quality of the English texts do not require training material, allow for a greater and port the resources to Spanish with a compara- control over the outputs (e.g. for mitigating er- ble output quality. rors or tuning the output to a desired style), and In the next section, we summarize the related the linguistic knowledge used for one domain or work. Section3 introduces FORGe. In Section language can be reused for other domains and lan- 4, we outline our work on the extension of the guages. In addition to these, a number of systems available English resources and on the adaptation actually address the whole sequence as one step, of FORGe to Spanish. Section5 presents the re- by combining approaches (i) and (iii) and filling sults of the automatic evaluation of the extended the slot values of pre-existing templates using neu- system, and Section6 a qualitative evaluation of ral network techniques (Nayak et al., 2017). the outputs in both languages. Section7, finally, In the WebNLG challenge (Gardent et al., draws some conclusions and presents the future 2017a), systems of types (ii) and (iii) have been work. presented. The task consisted in generating texts from up to 7 DBpedia triples from 15 categories, 2 Related work covering in total 373 distinct DBpedia properties. Nine categories appeared in the training data (‘As- The most prominent recent illustration of the tronaut’, ‘Building’, ‘University’, etc.), i.e., were portability of a generation framework is Sim- “seen”, and five categories were “unseen”, i.e., pleNLG. Originally developed for generation of they did not appear in the training data (‘Athlete’, English in practical applications (Gatt and Re- ‘Artist’, etc.). At the time of the challenge, the iter, 2009), in the meantime it has been ported to WebNLG dataset contained about 10K distinct in- generate, among others, in Brasilian Portuguese puts and 25K data-text pairs; a sample data-text (De Oliveira and Sripada, 2014), Dutch (de Jong pair is shown in Figure1. The neural genera- and Theune, 2018), German (Bollmann, 2011), tor ADAPT (Elder et al., 2018) performed best on Italian (Mazzei et al., 2016), and Spanish (Soto seen data, and FORGe on unseen data and over- et al., 2017). However, while SimpleNLG is a all. In what follows, we aim to improve the per- framework for surface generation, usually with a formance of FORGe on seen data for English and limited coverage, we are interested in a portable furthermore port it to Spanish. multilingual framework for large scale text gener- ation from structured data, more precisely, from 3 Overview of FORGe DBpedia properties (Lehmann et al., 2015). Although most existing NLG generators com- FORGe is an open-source generator implemented bine different techniques, there are three main ap- in terms of graph transducers; it covers the last proaches to generating texts from an input se- two typical NLG tasks (text planning and linguis- quence of structured data (Bouayad-Agha et al., tic generation). Following the Meaning-Text The- 2014; Gatt and Krahmer, 2018): (i) filling slot ory (Mel’cukˇ , 1988), FORGe is based on the no- values in predefined sentence templates (Androut- tion of linguistic dependencies, that is, the seman- sopoulos et al., 2013), (ii) applying grammars tic, syntactic and morphological relations between (rules) that encode different types of linguistic the components of the sentence. Input predicate- knowledge (Wanner et al., 2010), and (iii) predict- argument structures are mapped onto sentences by 474 A1 A2 subject leader object dpos=NP definiteness=DEF dpos=NP Reference 1: Charles Michel is the leader of Belgium where class=Person the German language is spoken. Antwerp is located in the A2 country and served by Antwerp International airport. Location Reference 2: Antwerp International Airport serves the city of Antwerp which is a popular tourist destination in Belgium. One of the languages spoken in Belgium is German, and the subject speak object leader is Charles Michel. dpos=NP lex=speak VB 02 definiteness=DEF Figure 1: Sample pair of data (subject-property-object) and human-produced texts (references). Figure 2: Sample PredArg templates corresponding to the leader (top) and language (bottom) properties. applying a series of rule-based graph transducers. The generator handles Semantic Web inputs by pertinent subject/object class labels,