Simple English Wikipedia: a New Text Simplification Task
Total Page：16
File Type：pdf, Size：1020Kb
Simple English Wikipedia: A New Text Simplification Task William Coster David Kauchak Computer Science Department Computer Science Department Pomona College Pomona College Claremont, CA 91711 Claremont, CA 91711 [email protected] [email protected] Abstract gadda et al., 2009; Vickrey and Koller, 2008; Chan- drasekar and Srinivas, 1997). Finally, models for In this paper we examine the task of sentence text simplification are similar to models for sentence simplification which aims to reduce the read- compression; advances in simplification can bene- ing complexity of a sentence by incorporat- fit compression, which has applications in mobile ing more accessible vocabulary and sentence structure. We introduce a new data set that devices, summarization and captioning (Knight and pairs English Wikipedia with Simple English Marcu, 2002; McDonald, 2006; Galley and McKe- Wikipedia and is orders of magnitude larger own, 2007; Nomoto, 2009; Cohn and Lapata, 2009). than any previously examined for sentence One of the key challenges for text simplification simplification. The data contains the full range is data availability. The small amount of simplifi- of simplification operations including reword- cation data currently available has prevented the ap- ing, reordering, insertion and deletion. We plication of data-driven techniques like those used provide an analysis of this corpus as well as in other text-to-text translation areas (Och and Ney, preliminary results using a phrase-based trans- lation approach for simplification. 2004; Chiang, 2010). Most prior techniques for text simplification have involved either hand-crafted rules (Vickrey and Koller, 2008; Feng, 2008) or 1 Introduction learned within a very restricted rule space (Chan- The task of text simplification aims to reduce the drasekar and Srinivas, 1997). complexity of text while maintaining the content We have generated a data set consisting of 137K (Chandrasekar and Srinivas, 1997; Carroll et al., aligned simplified/unsimplified sentence pairs by 1998; Feng, 2008). In this paper, we explore the pairing documents, then sentences from English 1 sentence simplification problem: given a sentence, Wikipedia with corresponding documents and sen- 2 the goal is to produce an equivalent sentence where tences from Simple English Wikipedia . Simple En- the vocabulary and sentence structure are simpler. glish Wikipedia contains articles aimed at children Text simplification has a number of important ap- and English language learners and contains similar plications. Simplification techniques can be used to content to English Wikipedia but with simpler vo- make text resources available to a broader range of cabulary and grammar. readers, including children, language learners, the Figure 1 shows example sentence simplifications elderly, the hearing impaired and people with apha- from the data set. Like machine translation and other sia or cognitive disabilities (Carroll et al., 1998; text-to-text domains, text simplification involves the Feng, 2008). As a preprocessing step, simplification full range of transformation operations including can improve the performance of NLP tasks, includ- deletion, rewording, reordering and insertion. ing parsing, semantic role labeling, machine transla- 1http://en.wikipedia.org/ tion and summarization (Miwa et al., 2010; Jonnala- 2http://simple.wikipedia.org 665 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:shortpapers, pages 665–669, Portland, Oregon, June 19-24, 2011. c 2011 Association for Computational Linguistics a. Normal: As Isolde arrives at his side, Tristan dies with her name on his lips. Simple: As Isolde arrives at his side, Tristan dies while speaking her name. b. Normal: Alfonso Perez Munoz, usually referred to as Alfonso, is a former Spanish footballer, in the striker position. Simple: Alfonso Perez is a former Spanish football player. c. Normal: Endemic types or species are especially likely to develop on islands because of their geographical isolation. Simple: Endemic types are most likely to develop on islands because they are isolated. d. Normal: The reverse process, producing electrical energy from mechanical, energy, is accomplished by a generator or dynamo. Simple: A dynamo or an electric generator does the reverse: it changes mechanical movement into electric energy. Figure 1: Example sentence simplifications extracted from Wikipedia. Normal refers to a sentence in an English Wikipedia article and Simple to a corresponding sentence in Simple English Wikipedia. 2 Previous Data 3 Simplification Corpus Generation We generated a parallel simplification corpus by aligning sentences between English Wikipedia and Wikipedia and Simple English Wikipedia have both Simple English Wikipedia. We obtained complete received some recent attention as a useful resource copies of English Wikipedia and Simple English for text simplification and the related task of text Wikipedia in May 2010. We first paired the articles compression. Yamangil and Nelken (2008) examine by title, then removed all article pairs where either the history logs of English Wikipedia to learn sen- article: contained only a single line, was flagged as a tence compression rules. Yatskar et al. (2010) learn stub, was flagged as a disambiguation page or was a a set of candidate phrase simplification rules based meta-page about Wikipedia. After pairing and filter- on edits identified in the revision histories of both ing, 10,588 aligned, content article pairs remained Simple English Wikipedia and English Wikipedia. (a 90% reduction from the original 110K Simple En- However, they only provide a list of the top phrasal glish Wikipedia articles). Throughout the rest of this simplifications and do not utilize them in an end- paper we will refer to unsimplified text from English to-end simplification system. Finally, Napoles and Wikipedia as normal and to the simplified version Dredze (2010) provide an analysis of the differences from Simple English Wikipedia as simple. between documents in English Wikipedia and Sim- ple English Wikipedia, though they do not view the To generate aligned sentence pairs from the data set as a parallel corpus. aligned document pairs we followed an approach similar to those utilized in previous monolingual Although the simplification problem shares some alignment problems (Barzilay and Elhadad, 2003; characteristics with the text compression problem, Nelken and Shieber, 2006). Paragraphs were iden- existing text compression data sets are small and tified based on formatting information available in contain a restricted set of possible transformations the articles. Each simple paragraph was then aligned (often only deletion). Knight and Marcu (2002) in- to every normal paragraph where the TF-IDF, co- troduced the Zipf-Davis corpus which contains 1K sine similarity was over a threshold or 0.5. We ini- sentence pairs. Cohn and Lapata (2009) manually tially investigated the paragraph clustering prepro- generated two parallel corpora from news stories to- cessing step in (Barzilay and Elhadad, 2003), but taling 3K sentence pairs. Finally, Nomoto (2009) did not find a qualitative difference and opted for the generated a data set based on RSS feeds containing simpler similarity-based alignment approach, which 2K sentence pairs. does not require manual annotation. 666 For each aligned paragraph pair (i.e. a simple 91/100 were identified as correct, though many of paragraph and one or more normal paragraphs), we the remaining 9 also had some partial content over- then used a dynamic programming approach to find lap. We also repeated the experiment using only that best global sentence alignment following Barzi- those sentences with a similarity above 0.75 (rather lay and Elhadad (2003). Specifically, given n nor- than 0.50 in the original data). This reduced the mal sentences to align to m simple sentences, we number of pairs from 137K to 90K, but the eval- find a(n, m) using the following recurrence: uators identified 98/100 as correct. The analysis throughout the rest of the section is for threshold a(i, j) = of 0.5, though similar results were also seen for the 8 a(i, j − 1) − skip penalty threshold of 0.75. > > Although the average simple article contained ap- > a(i − 1, j) − skip penalty <> a(i − 1, j − 1) + sim(i, j) proximately 40 sentences, we extracted an average max > a(i − 1, j − 2) + sim(i, j) + sim(i, j − 1) of 14 aligned sentence pairs per article. Qualita- > > a(i − 2, j − 1) + sim(i, j) + sim(i − 1, j) tively, it is rare to find a simple article that is a direct :> a(i − 2, j − 2) + sim(i, j − 1) + sim(i − 1, j) translation of the normal article, that is, a simple ar- ticle that was generated by only making sentence- where each line above corresponds to a sentence level changes to the normal document. However, alignment operation: skip the simple sentence, skip there is a strong relationship between the two data the normal sentence, align one normal to one sim- sets: 27% of our aligned sentences were identical ple, align one normal to two simple, align two nor- between simple and normal. We left these identical mal to one simple and align two normal to two sim- sentence pairs in our data set since not all sentences ple. sim(i, j) is the similarity between the ith nor- need to be simplified and it is important for any sim- mal sentence and the jth simple sentence and was plification algorithm to be able to handle this case. calculated using TF-IDF, cosine similarity. We set Much of the content without direct correspon- skip penalty = 0.0001 manually. dence is removed during paragraph alignment. 65% Barzilay and Elhadad (2003) further discourage of the simple paragraphs do not align to a normal aligning dissimilar sentences by including a “mis- paragraphs and are ignored. On top of this, within match penalty” in the similarity measure. Instead, aligned paragraphs, there are a large number of sen- we included a filtering step removing all sentence tences that do not align. Table 1 shows the propor- pairs with a normalized similarity below a threshold tion of the different sentence level alignment opera- of 0.5.