Fact-based Text Editing Hayate Isoy∗ Chao Qiaoz Hang Liz yNara Institute of Science and Technology zByteDance AI Lab [email protected], fqiaochao, [email protected] Abstract Set of triples f(Baymax, creator, Douncan Rouleau), We propose a novel text editing task, referred (Douncan Rouleau, nationality, American), to as fact-based text editing, in which the goal (Baymax, creator, Steven T. Seagle), is to revise a given document to better de- (Steven T. Seagle, nationality, American), scribe the facts in a knowledge base (e.g., sev- (Baymax, series, Big Hero 6), eral triples). The task is important in practice (Big Hero 6, starring, Scott Adsit)g because reflecting the truth is a common re- quirement in text editing. First, we propose a Draft text method for automatically generating a dataset Baymax was created by Duncan Rouleau, a winner of for research on fact-based text editing, where Eagle Award. Baymax is a character in Big Hero 6 . each instance consists of a draft text, a revised Revised text text, and several facts represented in triples. Baymax was created by American creators We apply the method into two public table- Duncan Rouleau and Steven T. Seagle . Baymax is to-text datasets, obtaining two new datasets a character in Big Hero 6 which stars Scott Adsit . consisting of 233k and 37k instances, respec- tively. Next, we propose a new neural network Table 1: Example of fact-based text editing. Facts are architecture for fact-based text editing, called represented in triples. The facts in green appear in FACTEDITOR, which edits a draft text by re- both draft text and triples. The facts in orange are ferring to given facts using a buffer, a stream, present in the draft text, but absent from the triples. and a memory. A straightforward approach to The facts in blue do not appear in the draft text, but address the problem would be to employ an in the triples. The task of fact-based text editing is to encoder-decoder model. Our experimental re- edit the draft text on the basis of the triples, by deleting sults on the two datasets show that FACTE- unsupported facts and inserting missing facts while DITOR outperforms the encoder-decoder ap- retaining supported facts. proach in terms of fidelity and fluency. The results also show that FACTEDITOR conducts inference faster than the encoder-decoder ap- aims to revise the text by adding missing facts and proach. deleting unsupported facts. Table1 gives an exam- 1 Introduction ple of the task. As far as we know, no previous work did address arXiv:2007.00916v1 [cs.CL] 2 Jul 2020 Automatic editing of text by computer is an impor- the problem. In a text-to-text generation, given a tant application, which can help human writers to text, the system automatically creates another text, write better documents in terms of accuracy, flu- where the new text can be a text in another language ency, etc. The task is easier and more practical than (machine translation), a summary of the original the automatic generation of texts from scratch and text (summarization), or a text in better form (text is attracting attention recently (Yang et al., 2017; editing). In a table-to-text generation, given a table Yin et al., 2019). In this paper, we consider a new containing facts in triples, the system automatically and specific setting of it, referred to as fact-based composes a text, which describes the facts. The text editing, in which a draft text and several facts former is a text-to-text problem, and the latter a (represented in triples) are given, and the system table-to-text problem. In comparison, fact-based ∗ The work was done when Hayate Iso was a research text editing can be viewed as a ‘text&table-to-text’ intern at ByteDance AI Lab. problem. First, we devise a method for automatically cre- and copy mechanisms (Gu et al., 2016; Gulcehre ating a dataset for fact-based text editing. Recently, et al., 2016) has dramatically changed the land- several table-to-text datasets have been created and scape, and now one can perform the task rela- released, consisting of pairs of facts and corre- tively easily with an encoder-decoder model such sponding descriptions. We leverage such kind of as Transformer provided that a sufficient amount data in our method. We first retrieve facts and of data is available. For example, Li et al.(2018) their descriptions. Next, we take the descriptions introduce a deep reinforcement learning framework as revised texts and automatically generate draft for paraphrasing, consisting of a generator and an texts based on the facts using several rules. We evaluator. Yin et al.(2019) formalize the prob- build two datasets for fact-based text editing on lem of text edit as learning and utilization of edit the basis of WEBNLG (Gardent et al., 2017) and representations and propose an encoder-decoder ROTOWIRE, consisting of 233k and 37k instances model for the task. Zhao et al.(2018) integrate respectively (Wiseman et al., 2017) 1. paraphrasing rules with the Transformer model for Second, we propose a model for fact-based text text simplification. Zhao et al.(2019) proposes a editing called FACTEDITOR. One could employ method for English grammar correction using a an encoder-decoder model, such as an encoder- Transformer and copy mechanism. decoder model, to perform the task. The encoder- Another approach to text editing is to view the decoder model implicitly represents the actions for problem as sequential tagging instead of encoder- transforming the draft text into a revised text. In decoder. In this way, the efficiency of learning contrast, FACTEDITOR explicitly represents the and prediction can be significantly enhanced. Vu actions for text editing, including Keep, Drop, and Haffari(2018) and Dong et al.(2019) con- and Gen, which means retention, deletion, and duct automatic post-editing and text simplification generation of word respectively. The model utilizes on the basis of edit operations and employ Neu- a buffer for storing the draft text, a stream to store ral Programmer-Interpreter (Reed and De Freitas, the revised text, and a memory for storing the facts. 2016) to predict the sequence of edits given a se- It also employs a neural network to control the quence of words, where the edits include KEEP, entire editing process. FACTEDITOR has a lower DROP, and ADD. Malmi et al.(2019) propose a se- time complexity than the encoder-decoder model, quential tagging model that assigns a tag (KEEP and thus it can edit a text more efficiently. or DELETE) to each word in the input sequence Experimental results show that FACTEDITOR and also decides whether to add a phrase before outperforms the baseline model of using encoder- the word. Our proposed approach is also based decoder for text editing in terms of fidelity and on sequential tagging of actions. It is designed for fluency, and also show that FACTEDITOR can per- fact-based text editing, not text-to-text generation, form text editing faster than the encoder-decoder however. model. 2.2 Table-to-Text Generation 2 Related Work Table-to-text generation is the task which aims to 2.1 Text Editing generate a text from structured data (Reiter and Text editing has been studied in different settings Dale, 2000; Gatt and Krahmer, 2018), for exam- such as automatic post-editing (Knight and Chan- ple, a text from an infobox about a term in biol- der, 1994; Simard et al., 2007; Yang et al., 2017), ogy in wikipedia (Lebret et al., 2016) and a de- paraphrasing (Dolan and Brockett, 2005), sentence scription of restaurant from a structured represen- simplification (Inui et al., 2003; Wubben et al., tation (Novikova et al., 2017). Encoder-decoder 2012), grammar error correction (Ng et al., 2014), models can also be employed in table-to-text gen- and text style transfer (Shen et al., 2017; Hu et al., eration with structured data as input and gener- 2017). ated text as output, for example, as in (Lebret The rise of encoder-decoder models (Cho et al., et al., 2016). Puduppully et al.(2019) and Iso et al. 2014; Sutskever et al., 2014) as well as the atten- (2019) propose utilizing an entity tracking module tion (Bahdanau et al., 2015; Vaswani et al., 2017) for document-level table-to-text generation. 1The datasets are publicly available at https:// One issue with table-to-text is that the style of github.com/isomap/factedit generated texts can be diverse (Iso et al., 2019). Re- y0 AGENT-1 performed as PATIENT-3 on BRIDGE-1 mission that was operated by PATIENT-2. x^0 AGENT-1 served as PATIENT-3 was a crew member of the BRIDGE-1 mission. x0 AGENT-1 performed as PATIENT-3 on BRIDGE-1 mission. (a) Example for insertion. The revised template y0 and the reference template x^0 share subsequences. The set of triple templates T nT^ is f(BRIDGE-1, operator, PATIENT-2)g. Our method removes “that was operated by PATIENT-2” from the revised template y0 to create the draft template x0. y0 AGENT-1 was created by BRIDGE-1 and PATIENT-2. x^0 The character of AGENT-1, whose full name is PATIENT-1, was created by BRIDGE-1 and PATIENT-2. x0 AGENT-1, whose full name is PATIENT-1, was created by BRIDGE-1 and PATIENT-2. (b) Example for deletion. The revised template y0 and the reference template x^0 share subsequences. The set of triple templates T^ nT is f(AGENT-1, fullName, PATIENT-1)g.