The International Corpus of Arabic: Compilation, Analysis and Evaluation
Total Page:16
File Type:pdf, Size:1020Kb
The International Corpus of Arabic: Compilation, Analysis and Evaluation Sameh Alansary Magdy Nagi Bibliotheca Alexandrina, P.O. Box 138, 21526, El Bibliotheca Alexandrina, P.O. Box 138, 21526, El Shatby, Alexandria, Egypt. Shatby, Alexandria, Egypt. Department of Phonetics and Linguistics, Faculty Computer and System Engineering Dept. Faculty of Arts, Alexandria University , El Shatby, Alexan- of Engineering, Alexandria University, dria, Egypt. Alexandria Egypt. [email protected] [email protected] Abstract 2nd language in other non-Arab countries as Ma- li and Turkey2. This paper focuses on a project for building The formal Arabic language, known as Clas- the first International Corpus of Arabic sical Arabic is the language in which the Qur’an (ICA). It is planned to contain 100 million is written and is considered to be the base of the analyzed tokens with an interface which al- syntactic and grammatical norms of the Arabic lows users to interact with the corpus data in a number of ways [ICA website]. ICA is a language. However, today it is considered more representative corpus of Arabic that has been of a written language than a spoken one. Modern initiated in 2006, it is intended to cover the Standard Arabic (MSA) is similar to Classical Modern Standard Arabic (MSA) language as Arabic, but it is an easier form. It is understood being used all over the Arab world. ICA has across the Arab world and it is used by television been analyzed by Bibliotheca Alexandrina presenters and politicians, it is the form used to Morphological Analysis Enhancer (BAM- teach Arabic as a foreign language. There are AE). BAMAE is based on Buckwalter Arabic different MSA varieties as the rate of similarity Morphological Analyzer (BAMA). Precision between every Arab country version of MSA and and Recall are the evaluation measures used Classical Arabic differs. This is one of the issues to evaluate the BAMAE system. At this point, Precision measurement ranges from that this paper will present. 95%-92% while recall measurement was Due to the fact that the need for Arabic cor- 92%-89%. This depends on the number of pus is increasing as well as the fact that the trials qualifiers retrieved for every word. The per- to build an Arabic corpus in the last few years centages are expected to rise by implement- were not enough to consider that the Arabic lan- ing the improvements while working on larg- guage has a real, representative and reliable cor- er amounts of data. pus, it was necessary to build such an Arabic corpus that is capable of supporting various lin- 1. Introduction guistic research in Arabic. Thus, ICA was in- spired by the difficulties that encountered Arabic Arabic is the largest member of the Semitic Language researches as a result of the lack of language family, most closely related to Arama- publicly available Arabic corpora. ic, Hebrew, Ugaritic and Phoenician. Arabic is Bibliotheca Alexandrina (BA) has initiated a one of the six official languages of the United 1 big project to build the “International Corpus of Nations and it is the main language of most of Arabic (ICA)”, a real trial to build a representa- the Middle East countries. Arabic ranks fifth in tive Arabic corpus as being used all over the Ar- the world's league table of languages, with an ab world to support research in Arabic. The In- estimated 206 million native speakers, 24 million ternational Corpus of Arabic is planned to con- as 2nd language speakers to add up to total of tain 100 million words. The collection of sam- 233 million and World Almanac estimates the ples is limited to written Modern Standard Ara- total speakers as 255 million. Arabic language is bic, selected from a wide range of sources and the official language in all Arab nations as designed to represent a wide cross-section of Ar- Egypt, Saudi Arabia and Algeria. Moreover, it is abic; it is stimulating the first systematic investi- also an official language in non-Arab countries as Israel, Chad and Eritrea. It is also spoken as a 2http://www.vistawide.com/languages/top_30_langua 1http://www.un.org/en/aboutun/languages.shtml ges.htm 8 Proceedings of the EMNLP 2014 Workshop on Arabic Natural Langauge Processing (ANLP), pages 8–17, October 25, 2014, Doha, Qatar. c 2014 Association for Computational Linguistics gation of the national varieties as being used all resulting in more than 148,000 tokens, 49,000 of over the Arab world (Alansary, et al. 2007). which have received the analytical processing8. There were some trials for building Arabic The raw text corpora trails as (KACST) King corpora. Some of them were annotated corpora Abdul-Aziz City for Science and Technology and others were raw texts corpora. Annotated Corpus9 contains 732,780,509 words representing corpora trails as Penn Arabic Treebank (PATB). 869,800 text files and 7,464,396 distinct words. It The LDC was sponsored to develop an Arabic contains a lot of classical Arabic texts; however, POS and Treebank of only 1,000,000 words. it is neither analyzed nor well planned. Ara- This corpus doesn’t contain any branched genres biCorpus 10 is a corpus that was developed by except 600 stories from the ANNAHAR News Dilworth Parkinson. It is a large corpus that Agency. The POS only annotated version of this could be accessed, but it is not analyzed. Words ANNAHAR corpus was released in 20043. The can be searched for in Arabic or Latin script. The output from Buckwalter’s Arabic Morphological website provides detailed instructions on the Analyzer is used as the starting point for the search. It contains 173,600,000 words in five morphological annotation and POS tagging of main categories or genres: Newspapers, Modern Arabic newswire text (Maamouri M., 2004). Literature, Nonfiction, Egyptian Colloquial, and Arabic Gigaword Corpus is an archive of news- Premodern. wire text data that depends on press only; it has In what follows, Section 2 reviews the ICA been compiled from Arabic news sources by data design, how it is compiled, discuss the copy- LDC4. The data coverage is limited, it was com- rights issue and what is the current ICA statistics. piled from Egypt, Lebanon, Tunisia, Saudi Ara- Section 3 describes the analysis stage of ICA, the bia and from outside the Arab world such as tool that is used in the analysis, why was it cho- England. NEMLAR Annotated Written Corpus sen followed by ICA evaluation and a compari- consists of about only 500, 000 words of Arabic son with another morphological disambiguator. text from 13 different categories, aiming to Section 4 gives a brief review on the ICA web- achieve a well-balanced corpus that offers a rep- site for the researchers to query its data. Conclu- resentation of the variety in syntactic, semantic sions and suggestions for further work are given and pragmatic features of modern Arabic lan- in section 5. guage5. The accuracy of the automatic analysis is around 95% (Atiyya M. et al, 2005). Its analysis 2. ICA Design & Compilation Stage features are limited, moreover its use is restrict- ed; it is not accessible for commercial use6. The ICA is similar to the International Corpus KALIMAT is a free multipurpose Arabic corpus, of English (ICE) in terms of concept rather than consists of 18,167,183 annotated words repre- in design. They are similar in trying to include senting 20,291 Arabic articles collected only the varieties of the language; the Modern Stand- from the Omani newspaper Alwatan. A morpho- ard Arabic (MSA) includes publications from logical analysis process on the data collection every Arab country that uses Arabic as official using AL Khalil 7 morphological analyser was language and it has been decided to include Ara- conducted to reach an accuracy of 96% (El-Haj bic publications from outside the Arab nations. M., 2013). Prague Arabic Dependency Treebank However, they are different in terms of corpus (PADT) version 1.0 distribution comprises over design criteria and data compilation. For exam- 113,500 tokens of data annotated analytically ple, on the one hand, Egyptian Modern Standard and provided with the disambiguating morpho- Arabic is the most widespread variety that is logical information. In addition, the release in- used to represent MSA in ICA corpus. On the cludes complete annotations of MorphoTrees other hand, in building ICE11 a fixed size from each variation was taken from any country that uses English as official language (one million words); however, balance in size does not always mean fixing a number of words for each varia- 3 https://catalog.ldc.upenn.edu/LDC2005T20 tion as will be clarified in the next section. 4https://catalog.ldc.upenn.edu/LDC2003T12 5http://catalog.elra.info/product_info.php?products_id =873 8https://catalog.ldc.upenn.edu/LDC2004T23 6http://catalog.elra.info/product_info.php?products_id 9http://www.kacstac.org.sa/Pages/default.aspx =873 10http://arabicorpus.byu.edu/search.php 7 http://alkhalil-morpho-sys.soft112.com/ 11http://ice-corpora.net/ICE/INDEX.HTM 9 It is important to realize that the creation of if they had been published outside as al-Hayat ICA is a "cyclical" process, requiring constant magazine which is published in London12. reevaluation during the corpus compilation. Con- Size criterion in the corpus design focuses on sequently, we are willing to change our initial the number of words. However, issues of size are corpus design if there are any circumstances also related to the number of texts from different would arise that requires such changes. genres, the number of samples from each text, and the number of words in each sample. Such 2.1 ICA Design decisions were taken based on how common the genre or the source is.