Advancement in Productivity of Arabic Into English Machine Translation Systems from 2008 to 2013

Full Length Research Paper

Advancement in productivity of Arabic into English Machine Translation Systems from 2008 to 2013

Awatif M. Abu-Al-Sha'r* and Ali F. AbuSeileek

Faculty of Educational Sciences, Al al-Bayt University, JORDAN.

Accepted 14 March, 2013

This paper attempts to compare between the advancements in the productivity of Arabic into English Machine Translation Systems between two years, 2008 and 2013. It also aims to evaluate the progress achieved by various systems of Arabic into English electronic translation between the two years. For tracing such advancement, a comparative analysis of translated texts taken from previous studies were re-evaluated according to certain criteria by focusing on the functional characteristics and sub- characteristics of the output: reliability, fidelity, terminology, and syntax. This re-evaluation proposes to develop metrics related to these four functional criteria. The data used in this study are the same five text-genres that have been evaluated by one of the researchers in 2008 that covered technical, legal, literary, journalistic, and economic topics translated from Arabic into English using seven Machine Translation systems: Google, Ajeeb, Professional Translator, 1-800-translate, World lingo, Tran Sphere, and An-Nakel. The translated texts were analyzed to identify errors, deficiencies, limitations, and loss of meaning. Results of this analysis were compared with those findings achieved in 2008 so as to bring together complementary innovative progress in the production of Machine Translation. The findings revealed certain advancement in the translated production of Google system compared with the deficiencies of the other six systems in all the functional characteristics of the output.

Key words: Machine translation evaluation, Arabic into English translation, functional characteristics of output.

INTRODUCTION

Advancement achieved in the field of Arabic into English Some systems that use statistical MT have gained Machine Translation could be related to the production of some level of acceptability such as Language Weaver, electronic translation systems. Hutchins (2003) claims in IBM (IBM Machine Translation), and Google. Other online his feature article that it is estimated that some 1000 commercial systems that exploit statistical MT offerings different MT packages are on sale (when each language are probably moving towards producing real work in this pair is counted separately). The availability of these field. Examples include the following MT systems: systems in the market for users has led to a rapid and an Systran, SDL, LEC, ProMT, Linguatec, Word Magic, urgent need for evaluation and investigation. Resear- Apptek, Sakhr (Tarjim), and many Japanese, Chinese, chers are eager to explore accurate ways to evaluate the and Korean MT systems that are mostly available in output of this production due to the great need for those regions (for more information, see Abu-Al-Sha’r, research that may approve standards and metrics for 2009). There is increasing usage of MT services, which evaluating the translated text from Arabic into English. are often free such as the well-known 'Babelfish' on Such evaluation should incorporate syntactic and AltaVista. As there is a demand for urgent receiving of semantic information into translation models. foreign-language information, and top quality output is not

*Corresponding author. E-mail: [email protected].

526 Educ. Res. Rev.

essential, a relatively acceptable output of MT is taken into consideration that there are various systems of beneficial at the present time. Hutchin (2003) assured Arabic into English electronic translation based on other this by saying that the Internet is providing the means for approaches. However, it is difficult to include all these more rapid delivery of quality translations to individuals systems in one study. Therefore, this study is limited to and to small companies, and a number of MT system using only seven MT systems, including Google, Ajeeb, vendors now offer translation services, usually 'adding Professional Translator, 1-800-translate, Worldlingo, value' by human post-editing. TranSphere, and An-Nakel. Developing systems for MT involves the development of new models. Gerber (2012) suggests a new model for MT as there are still many established groups interna- Advancement in Arabic into English MT evaluation tionally doing research on linguistic rule-based MT. Gerber's new model is divided into three eras of Debate about the demand of gaining an accurate Arabic development: The rule-based era from 1950 to 2000, the into English MT output has been a hot issue nowadays statistical era from 1995 till present, and the hypothetical because of the current events taking place in the Arab next MT paradigm that has not emerged yet. World. Such demand requires evaluation of the output The main concern of translation from Arabic into produced by the free electronic systems that offer English researchers is exploring a matric that improves translation. Similarly, advancement in the field of MT from accurate syntactic transformation. Their efforts have Arabic into English is proved to be prominent due to some success and have now gained popularity as a receiving a relatively accurate translation of text . Although research pursuit. In addition, other researchers' efforts in some Arabic MT systems have relatively achieved an MT have also paved the road for research concerned acceptable level of readability and reliability, the level of with empirical methods in natural language processing. transferring accurate data from Arabic into English still Research tracks dealt with 14 papers in the phrase- needs further processing by almost all MT systems. based framework and 10 in various syntax-based Since computers are not able to totally understand frameworks that report progress in this caliber. Marcu et Arabic, there are challenges of translating a text from al. (2012) stated in a recent survey that syntax-based Arabic into English that resembles the translation of a methods for statistical MT will continue to be an area of human. significant focus through about 2013. The same survey Arabic MT evaluation studies mainly focused on suggested that semantics, robust handling of different English into Arabic translation; and the majority of topics and topic shift, and handling of discourse structure research on improving Arabic into English MT from 2008 will begin to see results around 2018. These are till the present time focuses on syntax reordering promises that pursue research in the area of MT. (Habash, 2007; Marcu et al., 2012) due to the fact Studies (Al-Bahadily, 2001; Al-Otoum, 2006; Izwaini, that Arabic has a large set of morphological features. 2006) conducted so far have focused on analyzing the These features are usually suffixes or prefixes that can effectiveness of the productivity of MT systems in completely change the meaning of the word. Salem et al. translating texts. However, they do not focus on (2008) proved in a study how the characteristics of the comparing the advancement in the productivity of a Arabic language affect the development of a MT tool from certain system over time. This study is a mere Arabic to English. Several distinguishing features of comparison of the electronic MT output of 7 translation Arabic pertinent to MT are explored in detail with systems (Google, Ajeeb, Professional Translator, 1-800- reference to some potential difficulties that are presented translate, Worldlingo, TranSphere, and An-Nakel). It tries in their study. to assess the translation output in terms of functional Furthermore, there are Arabic words that have the characteristics (suitability, accuracy, and well-formed- meaning of a full sentence. Arabic also has free word ness) and sub-characteristics (fidelity, terminology, order that offers vast possibilities to express the same syntax, and readability). It compares between the sentence without changing the meaning of the sentence. advancements in the productivity of Arabic into English This is a huge challenge as it is impossible to get a MT systems between two years, 2008 and 2013. It also similar equivalent in English. Attia (2008) investigates aims to evaluate the progress achieved by various different methodologies to manage the problem of systems of Arabic into English electronic translation morphological and syntactic ambiguities in Arabic. He between the two years. More specifically, it aims to claims, "I built an Arabic parser using Xerox Linguistics answer the following two research questions: 1) Which Environment (XLE) which allows writing grammar rules functional characteristics and sub-characteristics of the and notations that follow the LFG [Lexical functional translation output (reliability, fidelity, terminology, and grammar] formalisms. I also formulate a description of syntax) gained more progress among the seven MT main syntactic structures in Arabic within the LFG systems under study?, and 2) What is the advancement framework" (p.1). It could be concluded here that the in the productivity of Arabic into English MT systems framework for the evaluation of MT that produces the between the two years, 2008 and 2013? It should be most accurate output could be adopted due to certain

Abu-Al-Sha'r and AbuSeileek 527

characteristics of Arabic. The evaluation of Arabic MT source text drawn from many different genres, both system, which measures the functional criteria of spoken and written, and translate it into fluent English readability, fidelity, terminology, and syntax, could be a while preserving all of the meaning present in the original target for researchers. Such evaluation proposes to Arabic text. Translation agencies will use their own best develop metrics related to the dimensions of reliability, practices to produce high quality translations. While we fidelity, terminology, and syntax, or the dimensions trust that each agency has its own mechanism of quality termed in the literature as acceptability, adequacy, and control, we provide the following specific guidelines so accuracy. that all translations are guided by some common Translation from Arabic into English is a complex and principles (p.7). demanding process where productivity could be determined by the quality and range of its dictionary These principles could be considered as tips for tran- entries. For example, Galley et al. (2009) confirm slation. Concerning MT, the NIST 2008 MT Evaluation Googler’s claim by saying "Although Arabic-to-English (MT-08) has conducted an evaluation on a data of Arabic translation quality has improved significantly in recent into English of documents drawn from newswire text years, pervasive problems remain [...]. One of them is the documents and web-based newsgroup documents as re-ordering of verb-initial clauses-especially matrix shown in Table 1. clauses-during translation […]. We have recently The data was translated by four professional translation developed a high-precision Arabic subject detector that companies independently generating high quality can be integrated into phrase-based translation pipelines. translations. Each translation agency was required to A characteristic feature of our work is the decision to have native speaker(s) of the source and target influence decoding directly instead of re-ordering the languages working on the translations. MT quality was Arabic input prior to translation. We have also created a measured automatically using an N-gram co-occurrence state-of-the-art Arabic parser that can be used for a statistic metric developed by IBM and known as BLEU. It variety of MT tasks" (p. 2). measures the sequence of N-words that it shares with MT programs cater to bring improvements for key one or more high quality reference translations or N- languages (e.g., Arabic and Chinese into English) to the grams. MT systems report that they have achieved level where English-speaking analysts can scan and progress in translating languages; see Appendix A about gather information from foreign language news. For advancements and future prospects of some MT Arabic-to-English, they have certainly succeeded, and the systems. One of the goals of this study is to verify these performance of research MT systems lags only slightly views. behind human first-draft translations. Gerber (2012) confirms that the vast majority of the evaluations conducted by the National Institute of Standards and RELATED LITERATURE Technology (NIST) and the Defense Advanced Research Projects Agency (DARPA) over the past 8 years have This part deals with research in the evaluation of been on translation of Arabic and Chinese newswire texts electronic systems of Arabic into English MT. Research in into English. Chinese has proven much more difficult, and Arabic into English MT evaluation has not started until the fact that the research campaigns on Arabic and 2001. The first evaluative study was conducted in Iraq by Chinese have been waged with the same algorithms and Al-Bahadily (2001). It compared the translation outputs of the comparable amounts of training data, has three Arabic systems: Al-Wafi, Al-Mutarjim Al-Arabey and demonstrated more powerfully that some language pair Al-Nakel Al-Arabi. The evaluation dealt with terminology, directions are much harder than others. grammatical aspects, and semantic analysis. The findings of the study stated the merits and the demerits of each system. Besides, it is revealed that there is a poor Arabic into English MT evaluation standards grammatical coverage of the three systems. Al-Otoum (2006) evaluated two Arabic into English MT Some standards that are used in evaluating Arabic into systems: TranSphere, and An-nakel. The Framework for English MT include the following: the Framework for the the Evaluation of MT in ISLE (FEMTI) was applied to Evaluation of Machine Translation in ISLE (FEMTI); the investigate the overall quality of translation produced by National Institute of Standards and Technology (NIST); these two systems with respect to the four functional and Bilingual Evaluation Understudy (BLEU) that uses a criteria: readability, fidelity, terminology, and syntax. The measurement scale to assess a system's performance in findings indicated that the output of both systems suffer Chinese and Arabic against human translation. GALE from low faithfulness, lexical problems, mistranslation of Program: Arabic to English Translation Guidelines terms, and inadequacy. (Gerber, 2012) states: Izwaini (2006) evaluated three systems of on-line translation to diagnose the problems of Arabic MT: The goal of the translation process is to take Arabic Google, Sakhr, and Systran, using two sets of texts

528 Educ. Res. Rev.

Table 1. Source data of Arabic into English.

Source language Sources Newsgroup / Web Arabic AAW, AFP, AHR, ASB, HYT, NHR, QDS, XIN, Assabah Xinhua News Agency Various web forums

(Arabic and English) as input. The study attempted to and quantified the effect of segmentation errors. With a detect the reasons of the translation problems, trying to human evaluation, their study also showed that ATB shed light on the areas where the right translation inter-annotator agreement remains low relative to the solution is missed. The investigation faced a wide range WSJ (Wall Street Journal) corpus. The findings of their of common linguistic problems as well as mode-specific study suggest that current parsing models would benefit problems. The output of the three systems of the five from better annotation consistency and enriched sentences of the study showed that there are two annotation in certain syntactic configurations. shortcomings in Google's translations regarding the Huck et al. (2011) studied several advanced techniques writing format and spelling. Sakhr got the credit for having and models for Arabic-to-English statistical MT. They diacritics in its system output. The literal translation by examined how the challenges imposed by this particular the Systran showed that so many words remaining language pair and translation direction can be without translation, as well as mistranslation of lexical successfully tackled within the framework of hierarchical items, wrong word order and awkward syntax, makes the phrase-based translation. They provided a concise post-editing needed for Systran's output so com- review, an empirical evaluation, and an in-depth analysis: prehensive and time consuming. The output is a distorted soft syntactic labels, a discriminative word lexicon model, language with no cohesion and coherence. Izwaini additional reordering, and shallow rules. They brought states, "Systran translations need to be investigated in a together complementary methods that previously have more detailed study to help in identifying and rectifying only been investigated in isolation and mostly on different the large number of errors produced" (p. 146). language pairs. Combinations of the methods yield Carpuat et al. (2010) studied the challenges raised by significant improvements over a baseline using a usual Arabic verb and subject detection and reordering in set of models. The resulting hierarchical systems perform Statistical MT. They evaluated translation quality using competitive on the large-scale NIST Arabic-to-English both BLEU and translation edit rate (TER) scores on translation task. three standard evaluation test sets from the NIST eva- Koehn et al. (2007) evaluated a MT system based on luations, which yield more than 4400 test sentences with the Interlingua approach, the Universal Network 4 reference translations. The method used remarkably Language (UNL) system, designed for Multilanguage yields statistically significant improvements in BLEU and translation. The dataset is evaluated for three metrics TER on the medium and large SMT systems at the 99% using the four references obtained from the professional confidence level. They showed that post-verbal subject translators. Their study aims at comparing the MT constructions are hard to translate because they have systems based on UNL against other systems. Also, it highly ambiguous reordering patterns when translated to serves to analyze the development of the system English. Their study also revealed that implementing understudy by comparing output at the sentence level. It reordering is difficult because the boundaries of verb- is observed that UNL results in the best score, followed subject constructions are hard to detect accurately, even by Google, Sakhr, and then Babylon. Results also with a state-of-the-art Arabic dependency parser. They revealed that UNL performed better than the three proposed a strategy that improved BLEU and TER systems on all metrics, especially when generating scores, even on a strong large-scale baseline and sentences with a complex structure. despite noisy parses. Data analysis showed that the Salem et al. (2008) tried to explore how the features of technique of VS reordering improves word alignment Arabic Language affect the development of MT. The coverage, which led to larger phrase-tables that improve paper concludes with a proposed model incorporating the the quality of translation. role and reference grammar (RRG) technique to achieve Green and Manning (2010) offered broad insight into this end. the underperformance of Arabic constituency parsing by The Stanford University’s Arabic-to-English Statistical analyzing the interplay of linguistic phenomena, anno- MT system for the 2009 NIST MT Open Evaluation tation choices, and model design. By establishing document describes Stanford University’s first entry into a significantly higher parsing baselines, they have shown NIST Arabic-English MT evaluation. It describes two main that Arabic parsing performance is not as poor as improvements over a previous Chinese-English sub- previously thought, but remains much lower than English. mission (Galley et al., 2009): a hierarchical lexicalized They described grammar state splits that significantly reordering model (Galley et al., 2009) and a technique for improve parsing performance, catalogued parsing errors, performing minimum error rate training (Cer et al., 2008)

Abu-Al-Sha'r and AbuSeileek 529

that outperforms the standard Powell method. Moreover, compared between the computer-assisted translation Carpuat et al. (2010) traced in their study the challenges (CAT) tools SDL Trados Studio 2009 (MT system), and of ambiguity in the output of MT from Arabic into English. Kilgray’s Memoq 5.0 (released in 2011) (MT system). The They show that post-verbal subject constructions are major functional areas of CAT tools, including support for hard to translate because they have highly ambiguous complex language scripts and file formats, file analysis reordering patterns when translated to English. Another and invoicing, project management and workflows, challenge in implementing reordering is difficult because translation memory and terminology management, the boundaries of verbal-subject constructions are hard to interoperability and QA. Results concluded, identify accurately, even with an advanced Arabic dependency parser. They propose to reorder verbal- "memoQ overwhelmingly offers superior features overall subject constructions into subject-verb order for statistical and greater efficiencies and productivity for translators in MT word alignment only which improves TER and BLEU a range of professional scenarios. Although Trados scores significantly, despite noisy parses and even on a excels in some areas such as tag placements, it is less strong large-scale baseline. Standard phrase-based SMT amenable to the contemporary need for interoperability systems memorize phrasal translation of verb and subject and flexibility demanded by exchanges in commercial constructions as observed in the training bi-text. They do translation than its rival memoQ. The rigidity of Trados’ not capture any generalizations between occurrences in unidirectional TMs and TBs are a major drawback for VS and SV orders, even for the same verbs. In addition, effective resource exchange whilst the lack of HTML text their distance-based reordering models are not well preview for complex language scripts is startling for a suited to handling complex reordering operations, which leading modern CAT tool " (p. 9). can include long distance dependencies, and may vary by context. Despite these limitations, phrase-based SMT Moreover, Miller and Vanni (2007) analyzed PLATO (MT systems have achieved competitive results in Arabic-to- system) assessments of clarity, coherence, lexical English benchmark evaluations. However, error analysis robustness, syntax, name-rendering, terminology, and shows that verbs are still often dropped or incorrectly morphology in a comparison of 10 input Arabic texts. translated, and subjects are split or garbled in translation. Analysis showed general lexical robustness, and This suggests that better syntactic modeling should divergent performance for clarity and name-rendering further improve SMT. We attempt to get a better under- were also observed. This indicates that the system may standing of translation patterns for Arabic verb construc- be reliable. The evaluation metrics incorporated in tions, particularly verb subject (VS) constructions, by PLATO discriminate between MT systems’ performance studying their reordering patterns and occurrence in a on type of text. Finally, a more recent study, Parton and hand aligned Arabic-English parallel tree bank. Results McKeown (2010), presented an algorithm for detecting shows that rules of VS reordering are not straightforward errors in MT focusing on content words. It was evaluated and so SMT should benefit from direct modeling of Arabic in the context of cross-lingual question answering VS translation. In order to detect VS constructions, we (CLQA), where the focus was on correcting the detected use our state-of-the-art Arabic dependency parser, which errors. The error detection algorithm could identify is essentially the CATIBEX baseline in our subsequent spuriously deleted content words with high accuracy. parsing work in Marton et al. (2010), and is further described there. VS subjects and their exact boundaries are hard to identify accurately. Concluding remarks Arabglot (2012) evaluated error analysis results from MT systems across genres (using Arabic) so as to Studies conducted so far have focused on analyzing the classify and count errors in texts machine translated by productivity language-to-language MT systems, problems the popular MT systems: Systran and Google Translate. in translating certain features and characteristics, or The error scheme focuses on semantic errors found in compared between the effectiveness of various two text genres that have been used, general politics, language-to-language MT systems. However, none of and technology. The results demonstrate how both MT them has focused on the advancement in productivity of systems perform and whether patterns across text genres Arabic into English MT systems. As this area is under may be suggested. The results revealed widely divergent researched, the present study focuses on comparing differences in performance across the MT systems, with between the advancement in productivity of Arabic into Google Translate vastly outperforming Systran in English MT systems between two years, 2009 and 2013. preserving source semantic content. While both systems It also compares between the functional characteristics scored higher error counts for the genre of technology and sub-characteristics of the output, including reliability, compared to politics, Google’s error rate in the political fidelity, terminology, and syntax in translated texts by the domain was only a tenth of those registered in Systran, seven MT systems (Google, Ajeeb, Professional and only a half in the technological domain. Translator, 1-800-translate, Worldlingo, TranSphere, and In another experiment, Arabglot (2012) analytically An-Nakel) which have some claims, prospects and views.

530 Educ. Res. Rev.

Table 2. Means of sentence scores to measure readability.

Mean of the scores of the 115 sentences of the five texts Ajeeb (Systran Professiona 1-800-translate An- Readability Google Worldlingo Transphere free translation) l translator Nakel scale (%) (Bablyon (%) (%) (%) (%) translator) (%) (%) 0 2.5 36 41 41 49 41 43 1 8.9 39.3 28.9 37 28 37 31.8 2 11.5 10.7 22 13.2 15.4 11 13.2 3 77.1 14 8.1 8.8 7.6 11 12 Total 100 100 100 100 100 100 100

They are listed in the following websites: made. It compares between the advancements in the productivity of the seven Arabic into English Machine (i) www.google.com Translation systems. It also aims to evaluate the progress (ii) http://tarjim.ajeeb.com/ajeeb achieved by the various systems of Arabic into English (iii) www.wordlingo.com electronic translation between two years, 2008 and 2013 . (iv) www.1-800-Translate .com (v) http://www.translation.net/nakel_translation.html (vi) http://www.translation.net FINDINGS (vii) www.professionaltranslation.com A 4-point scale from 0 to 3 was applied to measure the readability of the 115 sentences of the five texts under Procedures study, where 0 is the lowest and 3 is the highest as shown in Table 2. (3) indicates that the meaning is clear, This study intends to measure the four functional criteria apparent, acceptable and correctly conveyed from the of readability, fidelity, syntax, and terminology. Inter- first reading; (2) indicates that with some justification, the national Standards for Language Engineering ISLE of meaning of the translated sentence seems clear; (1) (FEMTI) is applied to the five texts that include 115 shows that the meaning is below 50% of being sentences (Appendix A). The texts were translated using understood; and (0) indicates that the meaning of the the seven MT systems (Google, Ajeeb, and Professional sentence is not clear. This scale of readability was used Translator, 1-800-translate, Worldlingo, TranSphere, and to measure the extent to which each sentence in the five An-Nakel). Different scales have been applied in this texts (115 sentences) is accurately transferred. Table 2 evaluation. A team of three raters who are specialists in presents the means of sentence scores to measure Arabic-English translation evaluated the texts. They readability by the seven MT systems under study. evaluated 10% of the texts together and discussed points Based on the findings in Table 2, the means of of disagreement until consensus was reached. Each then readability (clarity) of the translated texts for Google is evaluated the texts independently, and none of them more than 50%, whereas the means of readability of the knows about the rating done by the others. The inter-rater other 6 systems were below 50. These systems were reliability between them was .89 which is acceptable for Professional, Transphere, Ajeeb, Wordlingo, and 1-800- the purposes of this study. To measure readability and translate, An-Nakel; all are almost at the same level of fidelity, a 4-point scale starts from 0 that indicates the readability as the means ranges from 30 to 22%. The lowest score whereas 3 indicates that the meaning of the overall evaluation of readability in the seven systems sentence is clear despite the occurrences of other indicates that Google has achieved a high level of clarity mistakes, was applied to identify to what extent the in transferring the texts from Arabic into English. Table 2 content of the five texts is correctly transferred by the also shows that the output of Google is more acceptable seven MT systems. A 5-point scale was used to measure than the other systems whereas the output of both1-800- the grammatical correctness (syntax) of the translated translate and An-Nakelare is the least acceptable. 3 and output. To measure terminology a 2-point scale was used 2 on the scale are considered as they indicate the to determine whether the translation is correct. Then the positive degrees. overall results are arranged according to the most faithful To measure fidelity, a 4-point scale was also applied to results of the output on each matrix, regardless of the identify to what extent the information of the five texts is type of the text. Finally, a comparison between the faithfully and completely transferred by the seven MT findings of this study and that of Abu Alsha'r (2008) was systems. The scale in Table 3 shows that (3) is where al-

Abu-Al-Sha'r and AbuSeileek 531

Table 3. Means of sentence scores to measure fidelity.

Mean of the scores of the115 sentences of the six texts Ajeeb Professional 1-800-translate Scale of Google Worldlingo An-Nakel Transphere (Systran free translator (Bablyon fidelity (%) (%) (%) (%) translation) (%) (%) translator) (%) 0 3 29.8 38.3 33.3 31 29 30.8 1 6 44.7 44.2 44 46 43.2 41 2 15 15.5 10.5 15 14.4 17.2 17.2 3 76 10 7 7.7 8.6 10.6 11 Total 100 100 100 100 100 100 100

Table 4. Means of sentence scores to measure syntax.

Mean scores of the 115sentences to measure syntax Ajeeb Professional 1-800-translate Scale of Google Worldlingo Transphere An-Nakel (Systran free translator (Bablyon syntax (%) (%) (%) (%) translation) (%) (%) translator) (%) 1 4 42 36 30.5 37 38 33 2 7 34 41.1 40 35.6 31 27 3 14 8.2 10.2 12 17.4 20.8 23 4 40 11 8.7 11 6 6 11 5 35 4.8 4 6.5 4 4.2 6 Total 100 100 100 100 100 100 100

most all the information is faithfully conveyed from the is almost very high compared with the other systems first reading; (2) indicates that the information is relatively such as Transphere, Worldlingo, Ajeeb, and Professional faithful; (1) percentage of faithfulness in transferring the translator. information is below 50%; and (0) indicates that the The fourth and final matric to obtain a measurable information is not translated faithfully. This scale of fidelity value is to measure terminology which manifests the tries to measure the extent to which all information in accuracy of translation. A 2-point scale was used to each sentence in the five texts (115 sentences) is determine whether the translation is correct or not, where faithfully reproduced. Table 3 shows that faithfulness in (0) indicates the terms that are wrongly translated, but (1) transferring the text under study by Google has gained an is assigned for the terms that are correctly translated in unexpected level compared with the other six systems. It the five texts. The means of the translation of Google has is revealed that Google has proved a high level of exceeded the level of 50% which indicates that Google progress; whereas the other six systems produced produces an accurate choice of terms equivalent to the unfaithful translation as indicated by the figures in Table translated text, as shown in Table 5. It is also revealed 3. here the means of terminology of other two systems are Syntax includes almost all grammatical problems that more than 50% (An-Nakel and 1-800-translate). Ajeeb, have great effect on understanding the text. A 5-point Professional, and Transphere have mistranslated some scale was used to measure the grammatical correctness terms, but WorldLingo has mistranslated a good number (syntax) of the 115 translated sentences where (5) shows of terms. that the sentence is grammatically perfect; (4) is almost The percentages of the means of the four matrices perfect but with few minor problems; (3) is reasonably (readability, fidelity, syntax, and terminology) are grammatical, but has less serious problems; (2) is almost calculated so as to evaluate the output of the seven MT ungrammatical, but has a lot of serious problems that systems. Then these percentages are compared by the affect the meaning; and (1) shows that the sentence is functional characteristics of the General Software Quality completely ungrammatical fragment. (GSQ). These characteristics constitute 100% which is Table 4 reveals that translation of Google of the five divided into three criteria: suitability with 25%, accuracy texts has the least grammatical problems compared to 50% and well-formed with 25%. These have sub- the other systems. The percentage that Google has gained characteristics that include readability, fidelity, terminology,

532 Educ. Res. Rev.

Table 5. Means of sentence scores to measure terminology.

Mean of the scores of the115 sentences of all texts Ajeeb Professional 1-800-translate An- Scale of Google Worldlingo Transphere (Systran free translator (Bablyon Nakel terminology (%) (%) (%) translation) (%) (%) translator) (%) (%) 0 17 55 58 49 61 55 41 1 83 45 42 51 39 45 59 Total 100 100 100 100 100 100 100

Table 6. Percentage of the means of the overall evaluation of the seven MT systems.

Sub-functional characteristics 100%

Readability Fidelity Syntax Terminology MT System (%) (%) (%) (%) Google 88.6 91 75 83 An-Nakel 22 27.8 17 59 Transphere 25.2 28.2 10.2 45 translate1-800-translate (Bablyon translator) 22 9.2 17.5 51 Ajeeb (Systran free translation) 24.7 25.5 15.8 45 Professional translator 30.1 17.5 12.7 42 Worldlingo 23 23 10 39

Table 7. Means of the overall evaluation of the seven MT systems.

Title Sub-functional characteristics 25% each Total Readability Fidelity Syntax Terminology MT System (100%) (%) (%) (%) (%) Google 22.15 22.7 18.7 20.7 84.25 An-Nakel 5.5 6.9 4.2 14.7 31.3 Transphere 6.3 7.0 2.5 11.2 27 translate1-800-translate (Bablyon translator) 5.5 2.3 4.3 12.7 24.8 Ajeeb (Systran free translation) 6.1 6.3 3.9 11.2 27.5 Professional 7.5 4.3 3.1 10.5 25.4 Worldlingo 5.7 5.7 2.5 9.7 23.6

and syntax with 25% for each. percentage of the means of each functional sub- The final process is to do an overall evaluation of each characteristic of the seven systems, the means in Table 7 system; and then to compare the findings with these show the percentage of each sub-characteristic out of characteristics, so as to sort out the qualities of these 25%. However, the levels of accuracy, suitability, and seven systems under studying. Table 6 shows the means well-formedness are shown in Table 7 which indicates of the evaluation of the five texts translated by the seven that Google has gained the highest percentages in systems. The results indicate that the highest perfor- readability, syntax, and terminology; and Ajeeb gained mance in readability, fidelity, syntax, and terminology has the height in fidelity. been done by Google. To calculate the percentage of accuracy of the systems Table 6 also shows that the percentages of readability, under study, the sum of readability and terminology fidelity, and syntax are below 50% for all systems except shows the percentage of accuracy. The accuracy level of Google; whereas the percentage of terminology is higher the output of Google is the highest. This again ensures than 50% for all systems except Google. To calculate the that Google is the most accurate among the seven

Abu-Al-Sha'r and AbuSeileek 533

Table 8. Percentage of each criterion in the GSQ = 25.

Suitability Accuracy Well-formedness Total MT System (25%) (50%) (25%) (100%) Google 22.15 43.4 18.7 84.25 An-Nakel 5.5 21.6 4.2 31.3 Transphere 6.3 18.2 2.5 27 translate1-800-translate (Bablyon translator) 5.5 15 4.3 24.8 Ajeeb (Systran free translation) 6.1 17.5 3.9 27.5 Professional 7.5 14.8 3.1 25.4 Worldlingo 5.7 15.4 2.5 23.6

systems. The percentages of each criterion: readability, translation output (reliability, fidelity, terminology, and syntax, and terminology are calculated in terms of sui- syntax) gained more progress among the seven MT tability, accuracy, and well-formedness. Where readability systems under study?", a scrutinized look at the results of represents suitability and has the percentage of 25%; this study assures that Google has performed an accuracy represents both fidelity and terminology and outstanding advancement compared with the other therefore has 50%, and well-formedness represents systems. Tables 6 and 7 show the means of the overall syntax which has 25% (Table 8). evaluation of the seven MT systems according to the four The percentages of suitability, accuracy, and well- matrices: readability, fidelity, syntax, and terminology. formedness indicate that Google performs the best Each criterion represents 25% of the GSQ (General among the other six systems with respect to the criteria of Software Quality) which is 100%. The results manifested readability, terminology, fidelity, and syntax. Google that Google Translate system has the highest percentage translator has gained the highest percentage (84.25%), among the seven systems in the four evaluative criteria: followed by An-Nakel (31.3%), which is below 50%. readability is 22.15%; fidelity is 22.7%; syntax is 18.7%; Finally, Table 9 presents the sub-functional charac- and terminology is 20.7%. An-Nakel comes second teristics, readability, fidelity, syntax, terminology, and the where terminology got the higher percentage among the total mean in the productivity of Arabic into English MT other three criteria. These results go on-line with Al- systems in two years, 2008 and 2013. Based on Table 9, Otoum's terminology where An-Nakel is within the it can be concluded that Google had the highest average quality of other Arabic MT systems, whereas advancement in the productivity of Arabic into English MT Transphere is below the average quality of other Arabic systems from the year 2008 to 2013 in all sub-functional MT systems. These results explain why some characteristics, including readability, fidelity, syntax, translations are unreadable, awful, and inaccurate. terminology, and the total mean. However, there was a The present study also sought to compare between the quite low advancement in the productivity of all the other advancements of the output of Arabic into English MT MT systems in the translation of the five texts, in the four systems. The findings of this study revealed that Google sub-functional characteristics, except An-Nakel whose has achieved prominent advancement in the field of productivity of Arabic into English translation dropped electronic translation in all sub-functional characteristics, down unexpectedly (Table 9). readability, fidelity, syntax, terminology, and the total mean. On the other hand, the other systems (Ajeeb, Pro- fessional Translator, 1-800-translate, Worldlingo, Tran- DISCUSSION Sphere, and An-Nakel) had a quite limited advancement related to the same features in the productivity of Arabic This discussion is of two main concerns. The first deals into English Machine Translation systems between the with the results of this study that evaluate the progress two years, 2008 and 2013. These results match with achieved by the seven systems of Arabic into English those claims reported by Arabglot (2012) that electronic translation from 2008 till now; and the second demonstrated how both MT systems (Google and is concerned with comparing the output of analysis of the Systran) perform across the MT systems, where Google five translated texts taken from a previous study Translate vastly outperforms Systran in preserving according to certain criteria by focusing on the functional source semantic content. characteristics and sub-characteristics of the output: Comparing the progress achieved by the seven reliability, fidelity, terminology, and syntax. systems of Arabic into English electronic translation To answer the first question of the study "which under study from 2008 until now indicates the great functional characteristics and sub-characteristics of the advancement that Google has achieved. From Table 9, in

534 Educ. Res. Rev.

Table 9. The means of the overall evaluation of the seven MT systems in 2008 and 2013.

Sub-functional characteristics 25% each MT System Year Total (100%) Readability (%) Fidelity (%) Syntax (%) Terminology (%) 2008 6.5 6.5 6.2 14.5 33.7 Google 2013 22.15 22.7 18.7 20.7 84.25

2008 4.5 6.3 5.3 14 30.1 An-Nakel 2013 18.2 25.2 21.4 56 18.2

2008 5.1 6.2 3.7 11.5 26.5 Transphere 2013 6.3 7.0 2.5 11.2 27

2008 5.1 5.6 5 10.2 25.9 1-800-translate 2013 5.5 2.3 4.3 13.8 25.9

2008 4.3 6.6 3.5 10 24.4 Ajeeb 2013 6.1 6.3 3.9 11.2 27.5

2008 5.5 4.3 3.9 4.5 18.2 Professional 2013 7.5 4.3 3.1 10.5 25.4

2008 3.7 5.7 3 4.2 16.6 Worldlingo 2013 5.7 5.7 2.5 9.7 23.6

terms of functional characteristics (readability, fidelity, support some of Google Translate (2013) claims that: syntax, and terminology), it indicates that Google has got the highest percentage (readability 22.15% out of 25% It looks for patterns in hundreds of millions of documents which means a high acceptable level of suitability and to help decide on the best translation for the user. By fidelity, and terminology also gained a high percentage detecting patterns in documents that have already been which indicates an accurate output 43.4%; syntax has got translated by human translators, Google Translate can 18.7% which indicates a good level of well-formedness. make intelligent guesses as to what an appropriate However, many translation deficiencies have emerged in translation should be. This process of seeking patterns in the output of the other six systems. large amounts of text is called "Statistical Machine This leads to the conclusion that syntactic and lexical Translation". Since the translations are generated by problems still affect the overall quality of electronic machines, not all translation will be perfect. The more translation. All six systems of translation from Arabic into human-translated documents that Google Translate can English are incapable of handling complex sentences that analyze in a specific language, the better the translation carry indirect meaning. Hence, it is because of literal quality will be. translation, the translation output suffers from low This is why translation accuracy will sometimes vary percentage of faithfulness. This indicates that these across languages… Google Translate tests other systems, unlike Google, have no real contact with the languages, called "alpha languages" that may have less- issues and studies that are under investigation in the field reliable translation quality than our supported languages. of parsing. We are always working to support other languages and These results go in line with Google claims which will introduce them as soon as the translation quality assure that by detecting patterns in documents that have meets our standards. If you encounter a translation that already been translated by human translators, Google doesn't seem right, quite often Google Translate will have Translate can make intelligent guesses as to what an alternative results available. appropriate translation should be. In addition, the To view these, simply click the phrase in question. alternatives suggested by Google gear the focus towards When you click a better alternative translation, Google the urgent need to a perfect transfer of the translation Translate will learn from your feedback and continue to output in the present time. Finally, these findings may improve over time.

Abu-Al-Sha'r and AbuSeileek 535

CONCLUSION, LIMITATIONS AND IMPLICATIONS Attia M (2008). Handling Arabic morphological and syntactic ambiguity within the LFG framework with a view to Machine Translation. School of Languages, Linguistics and Cultures, The University of The comparison of the evaluation of the output of the Manchester, UK. seven different MT systems from Arabic into English Carpuat M, Marton Y, Habash N (2010). Improving Arabic-to-English based on the functional characteristics of evaluation statistical Machine Translation by reordering post-verbal subjects for revealed that six systems are still below the average alignment. Columbia University, Center for Computational Learning Systems. Proceedings of the ACL 2010 Conference Short Papers, quality of acceptable translation. However, the efforts and pp. 178–183. Association for Computational Linguistics. advancements achieved by Google proved that it is Carpuat M, Marton Y, Habash N (2010). Reordering matrix post-verbal possible to come over most of the challenges that hinder subjects for arabic-to-english smt. In Proceedings of the Conference the process of producing better translation. This study Traitement Automatique des Langues Naturelles (TALN). Cer D, Jurafsky D, Manning Ch (2008). Regularization and search for showed that even if the degree of accuracy, well- minimum error rate training. ACL 2008 Third Workshop on Statistical formedness, and suitability is not within the level, Google Machine Translation. found a relative solution by offering acceptable and Galley M, Green S, Cer D, Chang P, Manning Ch (2009). Stanford helpful options that may lead to better and more University's Arabic-to-English Statistical Machine Translation System for the 2009 NIST evaluation. NIST 2009 Open Machine Translation convenient output than that produced by the systems Evaluation Workshop. under study. The findings of this study also indicate that Gerber L (2012). Machine Translation: Ingredients for productive and Google advancement in producing acceptable Arabic stable MT deployments - Part 2. Retrieved November 14, 2012 from translation has surpassed expectations, due to the better www.translationdirectory.com. Google On-line Translation Page. (2013). http://www.google.ae/ understanding of the special features and characteristics language_tools?hl=envisited June-2012-January 2013. of Arabic language and to applying the most appropriate Green S, Manning Ch (2010). Better Arabic Parsing: Baselines, processing approaches. It is recommended here that Evaluations, and Analysis. Proceeding COLING '10 Proceedings of such advancement could be more accurate if Googlers the 23rd International Conference on Computational Linguistics, Association for Computational Linguistics Stroudsburg, PA, USA keep on up with most of the results of such studies in the pp.394-402. area of linguistics and electronic translation. Habash N (2007). Arabic Morphological representations for Machine The findings of this study should be interpreted Translation. In A. van den Bosch and A. Soudi, editors, Arabic cautiously for the following reasons. Firstly, the results computational morphology knowledge-based and empirical methods. London: Springer. are limited to Arabic to English translation only. This may Huck M, Vilar D, Stein D, Ney H (2011). Advancements in Arabic-to- motivate other studies in the opposite direction (English English Hierarchical Machine Translation. Proceedings of the 15th to Arabic) or other pairs of languages. Secondly, the International Conference of the European Association for Machine translated texts are limited to certain genres (technical, Translation 30-31. Centre for Computational Linguistics, Katholieke Universiteit Leuven Leuven, Belgium. legal, literary, journalistic, and economic). Therefore, Izwaini S (2006). Problems of Arabic Machine Translation: Evaluation of other studies may be conducted to include other genres. three systems. Proceedings of the International Conference on the Thirdly, there are many Arabic-English MT systems Challenge of Arabic for NLP/MT. The British Computer Society based on other approaches, essentially Rule-based (BSC), London pp.118-148 Koehn P, Hoang H, Birch A, Callison-Burch C, Federico M, Bertoldi N, Machine Translation (RBMT). However, the findings of Cowan B, Shen W, Moran C, Zens R, Dyer C, Bojar O, Constantin A, the study are limited to the seven MT systems (Google, Herbst E (2007). Moses: Open source toolkit for Statistical Machine Ajeeb, Professional Translator, 1-800-translate, World- Translation. In Proceedings of the 45th Annual Meeting of the lingo, TranSphere, and An-Nakel), as it is difficult to Association for Computational Linguistics. Demonstration Session, ACL 07, Prague, Czech Republic pp. 177-180. include all MT systems. This may motivate other studies Marcu D, Frazer A, Wong W, Knight K (2012). Language Weaver about other MT systems. Finally, the analysis is limited to Arabic-English MT. Language Weaver, Inc. focusing on certain functional characteristics and sub- Marton Y, Habash N, Rambow O (2010). Improving Arabic dependency characteristics of the output (reliability, fidelity, termi- parsing with lexical and inflectional morphological features. In Proceedings of the 11th Meeting of the North American Chapter of nology, and syntax). Similar studies may be devised to the Association for Computational Linguistics (NAACL) workshop on include other linguistic features. Statistical Parsing of Morphologically Rich Languages (SPMRL), Los Angeles, Language Weaver Arabic-English MT pp.178-183. Miller K, Vanni M (2007). Formal v. informal: Register-differentiated REFERENCES Arabic MT evaluation in the PLATO Paradigm. Natural Language Processing pp.161-166. Abu-Al-Sha’r, AM (2009). Output evaluation of available Arabic into Parton K, McKeown K (2010). MT error detection for cross-lingual English electronic Machine Translation systems. Translation Studies question answering. Proceedings of the 23rd International in the New Millennium 7:1-25. Conference on Computational Linguistics: Posters pp.946-954. Al-Bahadily, HKM (2001). Comparative study of Machine Translation Salem Y, Hensman A, Nolan B (May 2008). Towards Arabic to English systems. Unpublished M.Sc thesis. Baghdad: Saddam University. Machine Translation . ITB J. 17(1-17). Al-Otoum NA (2006). An evaluation of two Arabic into English Machine Translation systems: A comparative study. Yarmouk University, Jordan: Unpublished MA Thesis. Arabglot (2012). An evaluative comparison of SDL Trados Studio 2009 and MemoQ 5.0. Retrieved December 11, 2012 from www.arabglot.com.

536 Educ. Res. Rev.

Appendix A

This appendix is divided into 3 parts as follows: a) Sample texts extracts in Arabic followed by human translation. b) Output translation of the seven systems translation into English in 2013. c) Output translation of the seven systems translation in October, 2008.

a) Sample texts extracts in Arabic followed by human translation.

1) Technical Text: اب اد اد: ال ارض ون ن ات أ و ب ود وود اواع ارى اوع ج . رم اف ات اد ا أ ن د ان رن ن ن ا ارر. Human translation: Virus Hepatitis: The Pathogenic factor consists of two types of viruses (a) and (b). It is believed that there are other types; one of which is (c). Despite the antigenic differences of certain qualities, they cause two kinds of diseases among human beings both of which are clinically identical.

2) Literary Text: - ود أدھم أ دة م ر ن ل . - وط أن ن د وا وأوا ّدر او. Human translation: Adham has found in Umaymah a type of happiness unknown to him before. (missing ) 3) Economic Text: Exchange Rate اذا دث ز ازان ا دو زدة ات ا ط ادول ارى ن ن ر ارف ل ا ارع. Human translation: If there is a deficit in the account balance of a country as a result of the excess of demanded foreign currencies over what the other states demand of such currencies, the rate of exchange will tend to rise. 4) Legal Text: وات ھ ارام اب وت ا ا: ادام،ال ا اؤدة،ال اؤد،ال ا اؤ،ال اؤت(م 14 وت ارد) وا ھ ارام اب وت ا ا اس،ارا م ارط .(م 15 وت ارد ). ا ات ارام اب وت ادر ا: اس ادري وارا . - Human translation: Felonies are the crimes punishable by one of the following felonies penalties: death, hard labor, life sentence, life detention, temporary hard labor, temporary detention (Article 14 Jordanian Penalties), and the misdemeanors the crimes punisher on it in the penalties the delinquencies as follow: the prison, the fine then bind in a bail. (M 15 Jordanian Penalties).

5) Journalistic Text اؤال ارك وار, ھو ذا م ھل ا ارا اد , وھ زاد اور ارا اط, را اور ار . اوت اذي ر , إران ر , و اوت اذي را ادور ار , زاد ادور ارا . و اوت اذي ن ارس وش ن ره ن ادور ارا ا ر ارھب أن . واراق, ن ارس د ن د إران ق ارب ارھب. Human translation: The confusing and puzzling question is: why that evident strategic fact, i.e. the augmenting Iranian presence and declining Turkish presence in the region, is ignored? At the time Turkey is moving Northward, Iran is heading towards the West, and while the Turkish role is retreating, the Iranian role is increasing. And at the time President Push expresses his satisfaction with the role Iranian security played in combating terrorism in Afghanistan and Iraq, the former President Mohammad Khatami proclaims the Iranian contribution to war against terrorism.

Abu-Al-Sha'r and AbuSeileek 537

b) Output of translation of some of the seven systems translation in 2013.

System Text 1 translations Technical text 1 Ajeeb Liver infection is atheism: The pathogenic laborer kinds from the mother-in-law ['a] and in in and presence kinds of last believes in from her the kind [j]. In spite of disagreement of the characteristics [aalmstDdyt] to that they causes the stubborn human similar illness from [aalnaaHyh] clinical.

2 Google Acute inflammation of the liver: The pathogen are two types of sludges A and B believe the existence of other types, including type C. Despite the different qualities, but they Zbban antigenic in humans these two diseases are identical from clinically. 3 Translator1 - Liver 800 infection is atheism: The pathogeniclaborer kinds from the mud 'a and in andpresence kinds (Systran) of last believes in from her thekind j. In spite of disagreement of thecharacteristics aalmstDdyt to that they causesthe stubborn human similar illness fromaalnaaHyh clinical 4 Professional .acute hepatitis: pathogen types of sludges a and b and believed that other types of type c. Despite the different qualities but they cause almstdadet when two identical human clinically 5 Worldlingo The pathogenic laborer kinds from the mother-in-law ['a] and in in and presence kinds of last believes in from her the kind [j]. In spite of disagreement of the characteristics [aalmstDdyt] to that they causes the stubborn human similar illness from [aalnaaHyh] clinical .

6 Transphere Sharp hepatitis. The factor the disease two kinds of fever A and B and think of the existence of another kinds from it kind it G. 7 Al -nakel The sharp hepatitis. The factor of the disease is two fevers A and B and we believe another kinds among of it the kind c. Despite a difference anticode characteristics he made every effort that he causes two similar diseases concerning clinical at the human being.

System Text 1 translations and for simplicity announced . ل Ajeeb Found the black in Umayma a happinessdid not know 1 of. Happiness in saying and situations until light .in او

2 Google Adham found in Omaima HE did not know before. - Simplicity and declared his happiness Testify and conditions of scarce even by his brothers, Options: (options are checked where the translation is ambiguous) - Adham found in Omaima happiness did not know before. - Simplicity announced his happiness heard and conditions until scarce his brothers. 3 Translator1 -800 - finder attacks in 'amymthappiness does not introduce her before. - and for his simplicity (Systran) happiness in his announcedabout his statements and his situationscorroded tnddr in him his sisters .2 4 Professional Adham found in amiya h.e. unknown before. For simplicity I pleased him and his brothers are scarce even conditions. 2 5 Worldlingo finder attacks in ['amymt] happiness does not introduce her before. - and for his simplicity happiness in his announced about his statements and his situations corroded [tnddr] in him [aaxwth].2 6 Transphere Adham found a happiness in Umayma did not know before. And for simplicity declared of happiness in saying and situations until brothers joking in it. 7 Al -nakel Adham finds in Omayma happiness he doesn't know it previously. To announce publicly from His Grace by his catchwords and its circumstances until joking by it its brothers.

538 Educ. Res. Rev.

c) A sample of the output translation of some of the seven MT systems in October, 2008.

No. Source language: Google 1-800 translate An- Nakel Transphere Sentence Metra husband He was a full However a black then As for Adham was a ن زو رع 1 was heart with husband the turning he was a husband is husband with a heart full اب و love and good with the love and brimful by the lover of love and good ن ارة companionship the kind treatment, and the beauty of association. and as I have and as the And likewise I/she/you And wandering started to و إدارة 2 occupied management of filled it diverting the occupy place perform in اوت ن زء management time time distracted it stopping from part of an heart and felt that the ن ھ ار on the part of from the innocent its places of innocent time does not pass in an اد ن ل Mlahih innocent in part of its nightclubs entertainment in the eye blink, and the day the garden before in the garden garden previously, follows by the night, before, frequentation comedies of the garden from before Gil love the rest of the love has , indeed the love has , Love took the rest of his د ل اب 3 his day cheated by the rest penetrated the day و of its day remainder of its and Domineer , and it tyrannized , and he was and rule tyrannically in it واد 4 even forgotten over it until it forgot overcome it until and until forgot himself. himself himself forgetful its breath. ,Successive days And happy days The happy days Followed happy days . ووات أم ھ 5 and carefree life came in succession followed and spread over and extended over day in succession, and stretched over what وادت وق 6 an estimated what the mockers and she was extended estimated the cynical در روان و Radwan, Jalil appreciated upon what fate Radwan and Abbas and س و ل ,Abbas, and cynics Rdouanou Abbas Radhouan and Abbas Jalil ارون and Galilr and Jalil mockers but it hit the end so But it collided with , but it bumped at the but collided in the end in و ارطت 7 wise and calm the end with that end against that wise that wise quietude as the ذك ending Waterfall wise quietness as calm as out flowing foamed outflow ادوء ام flowing water in the willing flowing foamy waters of the effervescent cascade ه ال the river wishing waterfalls end Al waterfall foamy churn waters be finished in the اد ارا .Allemsebdeh Mzbda in the firm are concluded in composed river also ازدة ار .sober. river sedate river ارن He returned the And the inquiry The question settles And wondering started to ود اؤل 8 question has been returned that its place in a black occupy place perform in ل at the heart occupies a place heart an heart ب ادھم Adham, , natives that time He felt that the time , then he felt that time and felt that the time , ر ن ازن 9 does not pass in a does not pass in a doesn't move in a does not pass in an eye ر ت flash, followed blink and that the moment, and the day blink, and the day follows ن و أن ار ,Allbel that day day was followed by follows it the night by the night ال Allb and that if the and the whisper if it and the confidential and the apostrophe if be و إن اة إذا 10 intimacy continued communicated to talk when I/she/you interconnected to other وات إ ر to the end-lost all not an end it lost was interconnected to an end lost all the دت ل meaning each meaning except an end then meanings I/you led all meaning,