Italian and English Sentence Simplification
Total Page：16
File Type：pdf, Size：1020Kb
Italian and English Sentence Simplification: How Many Differences? Martina Fieromonte•, Dominique Brunato, Felice Dell’Orletta, Giulia Venturi • University of Pavia m.fi[email protected] Istituto di Linguistica Computazionale “Antonio Zampolli” (ILC–CNR) ItaliaNLP Lab - www.italianlp.it fdominique.brunato,felice.dellorletta,[email protected] Abstract ful for several reasons: it permits i) to detect and classify a set of necessary transformations in TS, The paper proposes a cross-linguistic anal- ii) to assess if a given corpus complies with user ysis of two parallel monolingual corpora requirements and simplification tasks and iii) to conceived for automatic text simplification evaluate the impact of simplification operations in two languages, Italian and English. The on target populations. If the corpus investigation aim is to find similarities and differences also encompasses a cross-linguistic comparison, in the process of simplification in two ty- it might also shed light on peculiarities and sim- pologically different languages. To carry ilarities underlying the process of simplification out the comparison, 1,000 sentences were across languages. However, so far this last is- extracted from the two corpora and anno- sue has been rather ignored with the exception of tated with a scheme previously used to an- Gonzalez-Dios et al. (2018), who compared how 1 notate simplification phenomena. macro-simplification operations derived from dif- 1 Introduction ferent annotation schemes are distributed in Ital- ian, Basque and Spanish parallel corpora. This pa- In recent years, the availability of parallel mono- per intends to explore this under-investigated per- lingual corpora has boosted the adoption of data- spective and proposes a cross-linguistic analysis of driven techniques for the task of automatic text two parallel monolingual corpora, i.e. the Italian simplification (ATS). These corpora are in general corpus PaCCSS-IT (Parallel Corpus of Complex– aligned at sentence level and consist of complex Simple Aligned Sentences for ITalian) (Brunato et sentences paired with their simple version. How- al., 2016) and the English Parallel Wikipedia Cor- ever, except for English which can rely on two pus (Coster and Kauchak, 2011). Through this large parallel corpora, i.e. the Parallel Wikipedia comparison, the paper tries to answer the follow- Corpus2 (Coster and Kauchak, 2011)(ParWik) and ing three questions: the Newsela corpus3 (Xu et al., 2015), these cor- 1. To what extent can an annotation scheme pora are scarce or rather small in other languages. conceived for the annotation of simplification in To reduce time and effort required for the con- one language be used to annotate simplifications struction of parallel corpora, some works tried new in other language? approaches to automatically or semi-automatically 2. Are there any differences or similarities in the collect such resources, e.g. Coster and Kauchak distribution and nature of simplification operations (2011),Yatskar et al. (2010), Brunato et al. (2016), in the two languages? Tonelli et al. (2016). Moreover to take advan- 3. If we find differences, to what extent do they tage of empirical data, most of these resources depend on language only, or on the type of cor- were annotated with rules aimed at identifying pora? the typologies of modifications an original sen- To answer these questions, 1,000 paired sen- tence goes through during the process of simpli- tences were extracted from the two corpora and fication. The inspection can be considered use- annotated with the scheme described in Brunato 1Copyright c 2019 for this paper by its authors. Use per- et al. (2016). This allows us to carry out a quan- mitted under Creative Commons License Attribution 4.0 In- titative and qualitative analysis focused on under- ternational (CC BY 4.0). 2http://www.cs.pomona.edu/∼dkauchak/simplification/ standing the nature of the modifications occurring 3https://newsela.com/data/ in the datasets. 2 Related work lying the construction of parallel corpora were also observed in Italian. For example, the comparison Given the relevance of parallel monolingual cor- reported in Tonelli et al. (2016) between a cor- pora in ATS, many projects have driven their atten- pus of Wikipedia edit stories and two corpora of tion on the development of these resources. The heterogeneous texts for young readers manually main approaches in the literature vary from the simplified according to different strategies (i.e. a manual simplification of original texts carried out structural and an intuitive one) proved the exis- by experts (see e.g. Xu et al. (2015) in English, tence of differences in terms of the linguistic level Bott and Saggion (2014) in Spanish, Brunato et al. affected by simplification. They concern for in- (2015) in Italian), to the alignment of already ex- stance the distribution of some simplification oper- isting text collections, containing same-topic doc- ations and the average of operations per sentence. uments written in two different styles, a complex As regards the first aspect, in manually simpli- and a simple one. It is the case of e.g. Coster and fied corpora, editors opted for a word-level lexical Kauchak (2011) and Tonelli et al. (2016), both re- substitution, while Wikipedia editors for a phrase- lying on the Wikipedia corpus but in a different level substitution. As regards the second aspect, way. The first is based on the alignment between the Wikipedia edit story corpus contains an aver- articles extracted from the standard and the Sim- age lower distribution of simplification per sen- ple English Wikipedia, a project started in 2001 tence. Though related to these works, our con- containing English Wikipedia pages written in ba- tribution differs in that it adds a cross-linguistic sic English; the latter relies on the edits that users level of comparison and also tries to provide an had made on the Italian Wikipedia and explicitly overview of possible factors affecting the distribu- marked as instances of simplification. A further tion and the nature of simplification operations in strategy was envisaged by Brunato et al. (2016), ATS corpora. who first collected a corpus of sentences sharing the same meaning from a large web corpus, and 3 Corpora and annotation scheme then ranked the most similar pairs according to their linguistic complexity assigned by an auto- Corpora. The corpora used in the analysis are the matic readability assessment system. Italian corpus PaCCSS-IT and the English Paral- In many cases, existing ATS corpora were also lel Wikipedia Corpus (ParWik). PaCCSS-IT is a annotated with rules to make explicit the most parallel corpus composed of about 63,000 paired frequent operations occurring in the process of sentences, obtained crawling the web. The cor- sentence simplification and distinguishing differ- pus is the result of a three-step approach strongly ent typologies of linguistic phenomena involved shaped by the level of simplification under investi- in sentence transformation. The classification gation, i.e. syntactic simplification, consisting in: of simplification operations is typically two-level i) an unsupervised step in which a great amount based, i.e. it contains a few macro-level opera- of sentences with overlapping lexicon and differ- tions and for some of them a more specific sub- ent syntactic structure was clustered according to 4 class which can depend on the size of the unit af- a similarity metric and automatically aligned ; ii) fected (e.g. sentence, phrase or word) or the lin- a supervised step aimed to train a classifier to pre- guistic level at which the operation applies (i.e. dict the sentence alignment and iii) a readability lexical, syntactic, discourse). Comparing ParWik assessment step aimed at assigning a readability with the manually simplified corpus Newsela, Xu score to the sentences in each pair. ParWik in- et al. (2015) also noticed that the approach adopted stead was obtained aligning two already existing to construct ATS resources has an impact on the text collections: the English Wikipedia and the type of simplification phenomena. For instance, Simple English Wikipedia. The authors aligned there are more differences between paired sen- paragraphs whose TF*IDF cosine similarity was tences before and after simplification in Newsela, over a threshold of 0.5. The final corpus consists suggesting that complex linguistic structures are of 167,000 aligned sentence pairs. often retained in ParWik. Simple sentences in Par- To summarize, the two corpora differ in the fol- Wik contains also longer words, together with a 4To be part of a cluster a sentence had to share all lemmas greater number of function words and punctuation. with PoS ‘noun’, ‘verb’, ‘numeral’, ‘personal pronoun’ and Similar differences related to the approach under- ‘negative adverb’. lowing aspects: i) language; ii) corpus collection can see, the first three most frequent operations in approach iii) domain of texts; iv) level of simplifi- PaCCSS-IT are: ‘deletion’, ‘verbal features’ and cation under investigation. ‘insertion’ and in ParWik ‘deletion’, ‘lexical sub- stitution’ and ‘insertion’. Excluding deletion, the PaCCSS-IT ParWik i) Italian English differences resulted to be statistically significant ii) Web crawling Wiki-based alignment for all operations, according to the Chi-squared iii) Web corpus Encyclopedic test (p value <0.05). iv) Mainly syntax Lexicon+Syntax At first glance, these results seem to suggest that language-specific factors affect the process of sim- Table 1: Corpora design criteria. plification. However, it is interesting to note that a Annotation of simplification operations. The qualitative analysis of these findings partially rules comparison was conducted on 1,000 sentence out this hypothesis, suggesting instead to interpret pairs randomly extracted from the two corpora. To the differences also in view of the other criteria make possible the comparison, the sentences were reported in Table 1. Specifically, the impact of annotated with the scheme in Table 2, previously language is limited to the different distribution of conceived to annotate PaCCSS-IT.