Approaching Machine from – a perspective on commonalities, potentials, differences

Oliver Culoˇ Faculty of Translation Studies, Linguistics, and Cultural Studies University of Mainz 76726 Germersheim, Germany [email protected]

Abstract and even desirable, especially in the light of recent developments. On the one hand, paradigms in MT The exchange between Translation Studies have been shifting into a direction in which lin- (TS) and (MT) has guistic and translational knowledge is being rein- been relatively rare. However, given recent tegrated in various ways in hybrid architectures, cf. developments in both fields like increased e.g. (Eisele et al., 2008), by means of adding mor- importance of post-editing and reintegra- phological or word order information , e.g. (Koehn tion of linguistic and translational knowl- and Hoang, 2007; Collins et al., 2005), adding syn- edge into hybrid systems, it seems desir- tactic information, e.g. (Quirk et al., 2005; Ding able to intensify the exchange. This paper and Palmer, 2005), or adapting models to domains, aims to contribute to bridging the gap be- e.g. (Koehn and Schroeder, 2007; Bertoldi and tween the two fields. I give a brief account Federico, 2009). On the other hand, TS has seen of the changing perspective of TS schol- a rise of empirical, often corpus-based research in ars on the field of translation as a whole, various areas, e.g. (Hansen-Schirra et al., 2012; including MT, leading to a more open con- Oakes and Ji, 2012; Rojo and Ibarretxe-Antunano, cept of translation. I also point out some 2013), which in method and communication style potential for knowledge transfer from TS certainly is more accessible to researchers from to MT, the idea here centring around the MT. Last but not least, post-editing – where hu- adoption of text-centric notions from TS mans and the machine meet – is of growing impor- both for the further development of MT tance in the translator’s world. systems and the study of post-editing phe- nomena. The paper concludes by suggest- This paper thus addresses some points with re- ing further steps to be taken in order to fa- gard to fostering exchange between TS and MT. cilitate an intensified future exchange. Section 2 gives an account of some emerging views towards a more open concept of the phe- 1 Introduction nomenon called translation. Section 3 makes sug- gestions how MT could benefit from adopting Translation Studies (TS) and Machine Translation text-centred notions prominent in TS. Section 4 (MT) share core goals, the most prominent among presents some initial findings from post-editing them being the study and accomplishment of trans- studies, indicating a case of how post-editing influ- lation between two languages. Still, exchange has ences the process of translation, thus pointing out been remarkably rare between the two disciplines the need to further study these two processes in a in the past decades. contrastive manner. Section 5 discusses the obser- Despite possible reasons for misunderstandings vations presented and makes suggestions with re- and scepticism, some of them being discussed in spect to potential further directions in the endeav- section 2, this paper intends to show that intensi- our to bridge the gap between TS and MT. Section fied exchange between the two fields is possible 6 then concludes the paper. c 2014 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- Having been written by someone who is aware BY-ND. of some of the developments in MT but usually

199 is concerned with more human-centric issues of translators, translation scholars, etc., a scepticism translation, it is conceivable that in this paper some which he connects to feelings of uncertainty in of the latest and greatest developments in MT have a progressively digitalised world. He proposes a been missed. While inevitably this opinion piece view on MT as a tool available to humans; hu- is shaped in many ways by my personal view of mans, then, would still be the agents in the trans- MT, the main goal of this contribution is to give an lation process, as someone has to design and use account of a potential common ground of MT and MT systems. Rozmyslowicz’s view might indeed TS as well as to promote further discussion on this help solve the dilemma of intentionality and agen- topic. tivity and tear down some of the walls having been erected over time. After all, nobody would think 2 Towards a cluster concept of of declaring lexicography as useless or not of in- translation terest to TS, and if dictionaries are merely “tools” available to us in the translation process, then so At first sight, the image that one could come up can be MT. with for MT and TS is that of unequal twins. While being concerned with the same core goals, the ap- Ultimately, though, it will be necessary to rede- proaches to translation taken by the two fields dif- fine TS in a way which will not rule out MT as fer. While MT is often associated with a some- a field of interest to translation scholars. Cronin what mechanistic view of language and seems (2012), for instance, goes so far as to define trans- more interested in “how to make things work”, lation as a technology by itself and describes the TS emphasises the importance of cultural factors progressing digitalisation as a mere change in the and discusses problems such as (un)translatability nature of translation. This does not say much, or the dichotomy between freedom and loyalty in however, about the different perspectives on and translation. In short, TS at least is much con- approaches to translation and their relation to each cerned with the “things that don’t work” as with other. More promisingly, Tymoczko (2005) puts those that work. Also, MT discourse traditionally forward a view on translation as a cluster concept, shows many characteristics of fields like engineer- i.e. an open concept in which the various clusters ing, e.g. the frequent use of mathematical sym- (e.g. linguistic and theory, var- bols, while TS communication is more discursive ious national or regional traditions, etc.) are con- in nature. Last but not least, the entity “at work” nected by family resemblances. Tymoczko also in the translation process is a very different one. emphasises that the translation concept will in fu- Following Catford’s (1965, 31) definitions, I see ture inevitably extend further due to the ongoing the translation machine as a device operating with technological changes. Her view underlines the di- co-textually based algorithms, whereas the human versity of approaches to translation, perspectives translator follows looser, more contextually based on it, etc, and, by the very meaning of diversity, rules and norms which can deliberately be bent or does not bear any aspect of dominance1 of one side ignored. over the other. Present-day mainstream TS theory liberates the human translator from merely being an inter- In this paper, I will adopt the views expressed by operater decoding messages in one language and Tymoczko. If MT is related to human-centric TS encoding them in another. Translation is seen as by family resemblances, it is necessary to identify an intentional human act with the goal of produc- the common ground of MT and TS. In the follow- ing a text in a target language with a specific rela- ing, I will discuss areas which may be of value to tion to the target culture. Rozmyslowicz (in press) both, by means of knowledge transfer, exchange, discusses this conceptualisation as a cause for a or joint research. Of course, only a fraction of pos- theoretical dilemma: By this definition of transla- sible topics can be addressed here. tion which emphasises the aspects agentivity and intentionality, MT is in fact discarded as a type of translation, as the criterion of intentionality is something a machine does not match. 1As opposed to such concepts like acceptance or tolerance Rozmyslowicz aims at helping to overcome which I understand to presuppose certain structures of power the scepticism towards MT that exists amongst or dominance, or the struggle for it

200 3 The translation unit text and its and behaving like a target culture text. In terms implications for Machine Translation of functional translation theory, one could charac- terise MT as a kind of translation which generally Translators have benefitted from many technical aims at being instrumental and functionally con- innovations in MT. Translation memories, term stant (i.e. retaining text function). databases, and parallel corpora have radically changed the translator’s workplace in the past With text as a key concept for translation, text decades; MT proper is set to equally become part type is one of the factors that comes into focus. of the translation process. This can also work the While it is useful to think of translation happening other way around, as will be argued in the follow- in different domains with all the effects described ing, with the translation world – or in this case TS – above, two different text types in the same domain holding things in store that may be valuable to MT. may be of very different nature – even more so in We will look at how TS uses the concept text to two different languages, thus adding the factor of model translation, and how MT could benefit from translation direction to the set of relevant factors. adopting notions associated with this concept. Let us look, for instance, at the business domain. A financial report will be very formal both in En- MT has been using notions like domain which, glish and German. Shareholder letters, however, quite obviously, have an effect on lexis, phraseol- exhibit various differences in style and grammar: ogy, and grammar, and of course also on transla- English shareholder letters are of much more col- tion. To just pick out one example: Words like loquial style. Emotive expressions like “We can Mutter should be translated differently depending make it!” remain untranslated in from on the domain a text is rooted in. In general lan- English to German, as they are not deemed appro- guage, Mutter will mostly mean ‘mother‘, in engi- priate (Culoˇ et al., 2011). Also English resorts neering it would rather be translated as ‘(screw-)- to less formal phrasing than German, regarding nut’. Other notions connected to the concept text e.g. the verb phrase, with English simply using that seem underrepresented in mainstream MT are forms of the verb be where German uses formulaic text type, translation direction, and text status. Be- expressions such as betragen ‘amount to’ (Culo,ˇ fore we turn to investigate these notions, a brief 2010). overview of one type of translation theory, func- tional theory, will serve to highlight the relevance Some of the differences in style between En- of the concept text for translation. glish and German shareholder letters can also be quantified in terms of grammar. Part of the CroCo 3.1 Text-centric factors in (Hansen-Schirra et al., 2012) was the study Studies: the examples of text type, of grammatical properties of originals and transla- translation direction, text status tions. The corpus compiled for the study contained In functional translation theory (Nord, 1997; a parallel part with texts from 8 registers like com- House, 1997; Nord, 2006), the notion of text is puter manuals (INSTR), shareholder letters, or po- predominant. The text as a whole is taken to be the litical essays (ESSAY), both with English originals main translation unit, and factors like cultural and translated to German (E2G) or vice versa (G2E). situational context as well as purpose of the trans- Each register contains at least ten texts totalling lation are decisive factors in the process. A text around 30,000 tokens. can retain or change its function, either by some- One study within this project investigated the one’s intention (e.g. when toning down a pamphlet shifts of grammatical function that occur in trans- and translating it as political program) or because lation. The study was performed on data which it is differently received in the target culture than were automatically aligned on word level, manu- in the source culture. The function of the text is ally aligned on sentence level, and manually an- marked on various linguistic levels, from orthog- notated with grammatical functions. All the in- raphy (e.g. progressive vs. conservative spelling stances in which two aligned content words (i.e. in German) to text structure; in other words, the noun, verb, adjective, or adverb) did not appear translation unit is not a horizontal, but a vertical in the same grammatical functions in original and phenomenon (Nord, 2011). Moreover, translations translation were counted as indicative of a gram- can be either documentary, highlighting features matical shift. Figure 1 shows how the proportion of the source text, or instrumental, i.e. appearing of subject-to-object shifts in relation to all subject

201 shifts varies depending on translation direction and effect on the performance of a translation model register. was to be expected. Similarly, an adaptation of MT systems to the patent domain, not only to the lexis, but also to its “various stylistic and formatting pe- culiarities” (Ceaus¸u et al., 2011, 25) – conforming to the concept of text type – results in significant gains in the system’s performance. The study of linguistic features of translated texts has also been applied to MT products, e.g. by Lapshinova-Koltunski (2013). She compares the distribution of features like nominality vs. verbal- ity in various types of translations such as human Figure 1: The proportion of subject-to-object shifts translations from scratch, translations made with in all cases of subject shifts for the registers ES- CAT tools, and translations made by statistical MT SAY and INSTR and for the two translation direc- systems. She finds, for instance, that output from tions E2G and G2E statistical MT systems tends to be more nominal than output produced by using a translation mem- ory system. She also presents a pilot experiment of Lexico-grammatical features such as objects in how this method of comparison can be extended to theme position do not only behave differently with more complex features such as verb-last vs. verb- respect to translation direction, but also with re- second position for passives in German. Such a text status spect to . In other words, originals and metric could be used complementary to existing translations in one language differ in the distribu- metrics which are sentence-bound and relate to ref- tion of these features. For instance, Teich (2003) erence translations, such as BLEU (Papinieni et al., shining through observes of grammatical features: 2002) or METEOR (Banerjee and Lavie, 2005). German texts translated from English over-exhibit From the viewpoint of TS, Lapshinova- passive forms when compared to original German Koltunski’s method constitutes a text-wide (and texts. Diwersy et al. (in press) analyse a broad thus text-centric) metric, examining how machine range of lexico-grammatic features for English and translations behave with respect to certain fea- German originals and translations (amongst oth- tures. This metric could be useful for studying ers) and make similar observations for features like in greater depth the performance of MT systems objects in theme (i.e. sentence initial) position: which are already adapted to a certain domain or English translations over-exhibit these, while Ger- text type, as in the case of patent MT, or which in man translations under-exhibit them when com- general achieve well higher than average results, pared to original texts in the same language. comparing them not only to translations, but also 3.2 Existing and potential applications of to original language. text-centric concepts In a second step, the products of post-editing could be analysed using this metric and contrasted How can such findings as those cited above be of with the feature analysis of the preceding MT value to MT? That factors like translation direction product, in order to investigate whether and how and text status can be made fruitful for MT pur- the post-editing process influences the outcome of poses has been demonstrated e.g. by Kurokawa et the translation process in contrast to a human from- al. (2009). In their experiments, the authors found scratch translation. The following section deals that they were able to train an equally performant with this question from the viewpoint of lexical translation model on a fifth of the data size when consistency. classifying the training data according to whether they were from originals or translations prior to us- 4 The influence of post-editing on the ing them in the training phase, as opposed to train- translation process ing their model on all data available regardless of their status. When considering the findings on the Post-Editing (O’Brien, 2010), i.e. the task of cor- different lexico-grammatical behaviour depending recting MT output, is a process in which human on translation direction and text status, a positive translators and the machine meet. As O’Brien

202 notes, post-editing and revision are similar but dif- imprisoned for the killing of four of-his ferent tasks: They differ in such dimensions as patients.” the types of errors translators are faced with or the time available. There are studies on the efficiency Besides issues of lexical choice and grammar, of post-editing, e.g. with regard to gains in pro- there is a noteworthy problem with lexical consis- cessing time and/or errors typically changed in the tency in the post-editing product. The MT system post-editing process, e.g. (Groves and Schmidtke, had in both cases translated nurse into the Ger- 2009; De Almeida and O’Brien, 2010). The man word Krankenschwester which indicates a fe- changes observed are typically local phenomena, male nurse, though the text refers to a male nurse. like inserting missing articles or correcting termi- The post-editor failed to edit the first occurrence of nology. However, as pointed out above, in terms nurse such that it reflects in German that this is a of functional translation theory, the main unit of male nurse (Krankenpfleger rather than Kranken- translation is the text. The following example, an schwester). The second occurrence was edited ac- individual observation from an ongoing pilot study cordingly, facilitated by the fact that the gender of on post-editing, shall highlight that the rendering the nurse is made explicit by the pronoun his in the of textual features may, too, be influenced by the same sentence. post-editing process. When looking at the distribution of these errors as shown in Table 4, the picture seems quite clear: In a recent pilot study, students and profession- This specific error only occurs in the post-editing als were asked to translate, blind-edit (i.e. edit the task, in four out of eight cases; it does so for stu- MT product without the source text as reference) dents and professionals alike. The playback of the and post-edit short snippets from newspaper texts. translation sessions reveals that in the human trans- The products from the three processes were con- lation task four of the translators first translated trasted with regard to lexico-grammatical errors as nurse as Krankenschwester (female nurse) and re- well as with regard to global translation strategies vised it during the translation of the rest of the text. like ensuring lexical consistency. The remaining four translators read the whole text Consistency in translation is ensured by vari- first or performed a search on the topic in the inter- ous strategies like determining a terminology to be net before they started translating. Therefore, they used, backtracking during translation, or including translated nurse correctly right from the start. We a drafting phase in the translation workflow. As get very similar results for the blind editing: Four post-editing already constitutes something like a of the editors changed other words/phrases first, (first) drafting phase, one would hope that it would before they realised that Krankenschwester was aid the goal of reaching consistency in a text. Let not correct, while the other three editors started us look at the following sentence pair which con- editing after reading the complete MT output and sists of an original English title of a newspaper ar- corrected Krankenschwester right away. ticle plus its first sentence, one post-edited trans- lation into German, and the gloss of the German translation: Table 1: Number of inconsistent translations for human translation (HT), blind editing (ED), and Killer nurse receives four life sentences. post-editing (PE) Hospital nurse C.N. was imprisoned for HT ED PE life today for the killing of four of his (fe)male nurse inconsist. 0 (8) 0 (7) 4 (8) patients. (source text) The point to be made here is thus not that MT Killer-Krankenschwester zu viermal “got it wrong”. It is more remarkable that half of lebenslanger Haft verurteilt. Der the post-editors did not seem to care or manage to Krankenpfleger C.N. wurde heute auf correct this striking inconsistency. Similar obser- Lebenszeit eingesperrt fur¨ die Totung¨ vations are currently being made with regard to ter- von vier seiner Patienten. (post-edited) minological consistency in a follow-up study us- Lit. “Killer female-nurse to four times ing not general language texts, but LSP texts such life-long imprisonment sentenced. The as technical documentation; this data is still being male-nurse C.N. was today for lifetime evaluated, though.

203 At this time, I can only speculate about the rea- such inconsistencies in post-editing may occur due sons. It might be that working with two texts in to lack of familiarity with the task. But one might parallel (the source and the MT output) results in a as well reply to this that if the task and the prob- cognitive load which makes it harder to perform lems were understood and taught well, such incon- other operations. Another possibility is that the sistencies and other potential problems should be post-editors relied more on the MT output than minimised right from the start (cf. e.g. O’Brien, they would admit or even be aware of. We might 2002 ). even be looking at a combination of these fac- On a more general level, I would suggest several tors; however, this remains mere speculation at this steps to be taken in order to continue establishing point, as I am not aware of any study which inves- a common ground for TS and MT: tigates such a phenomenon in depth. In any case, something about the post-editing process seems • identify more common areas of interest different enough to lead to such errors. While con- • identify concepts and methods that can be sistency is an important textual criterion, the text- shared oriented scrutiny of the data from this study will extend to other textual factors like, for instance, • define, create, or learn a common or at least the grammatical marking of text function (e.g. ad- mutually understandable terminology dressee vs. content orientation by means of avoid- ing resp. using impersonal constructions etc.). • find platforms for exchange, e.g. common workshops, publication platforms etc.

5 Discussion With this paper, I have attempted to contribute to the first two points. This paper has approached translation as an open concept which includes MT as an area of inter- 6 Conclusion est for translation scholars; a view that has been voiced before and has positively evolved in the past Both TS and MT have seen developments in the years, as described in section 2. past years which have paved new ways for poten- The goal of this paper is to be another step on tial collaboration. This paper has addressed some the way to more intense exchange and collabora- commonalities, potentials, and differences for and tion between TS and MT. The sections 3 and 4 de- between the two disciplines from the perspective part from text-centric notions prominent in TS and of TS. I have laid out some possibilities for knowl- show how some of the phenomena described and edge transfer and further collaborative research studied by means of these can be of common inter- both in corpus-based translation research as well est to both disciplines. In section 3, I discuss some as process-based research on the human translation examples of how factors like translation direction process and post-editing. At the end, some more have already successfully been applied in MT. I general suggestions as to how exchange could be then propose to extend one of these approaches to intensified were made. The views stated and sug- make it a text-centric metric for the distribution of gestions made in this paper are inevitably influ- linguistic features in MT products and to subse- enced by the perspective of the author rooted in quently use this metric to study the influence of the human-centric translation and are certainly incom- post-editing task on the outcome of the translation plete. In any case, MT scholars are more than wel- process in contrast to from-scratch translations. come to join the discourse. In section 4, I show that the post-editing task can have an influence on the translation process References when seen on a textual level and with respect to the global strategy of ensuring lexical consistency. In Banerjee, Satanjeev and Alon Lavie. 2005. METEOR: an automatic metric for MT evaluation with im- consequence, this finding emphasising that the two proved correlation with human judgments. In Pro- processes seem to differ enough to deserve being ceedings of the ACL Workshop on Intrinsic and Ex- studied further; in fact, we might learn a lot more trinsic Evaluation Measures for Machine Transla- about both kinds of processes by further contrast- tion and/or Summarization, pages 65–72. ing them. With respect to the findings presented Bertoldi, Nicola and Marcello Federico. 2009. Domain in section 4, one might be inclined to criticise that adaptation for statistical machine translation with

204 monolingual resources. In Proceedings of the fourth Groves, Declan and Dags Schmidtke. 2009. Identifi- workshop on statistical machine translation, pages cation and analysis of post-editing patterns for MT. 182–189, Athens, Greece. Association for Computa- In MT Summit XII: proceedings of the twelfth Ma- tional Linguistics. chine Translation Summit, pages 429–436, Ottawa, Canada. Catford, John C. 1965. A linguistic theory of transla- tion. An essay in applied linguistics. Oxford Univer- Hansen-Schirra, Silvia, Stella Neumann, and Erich sity Press, Oxford. Steiner. 2012. Cross-linguistic Corpora for the Study of Translations. Insights from the Language Ceaus¸u, Alexandru, John Tinsley, Jian Zhang, and Pair English-German. De Gruyter, Berlin. Andy Way. 2011. Experiments on domain adap- tation for patent machine translation in the PLuTO House, Juliane. 1997. Translation quality assessment. project. A model revisited. Gunter Narr Verlag, Tubingen.¨

Collins, Michael, Philipp Koehn, and Ivona Kucerovˇ a.´ Koehn, Philipp and Hieu Hoang. 2007. Factored trans- 2005. Clause restructuring for statistical machine lation models. In Proceedings of EMNLP-CoNLL, translation. In Proceedings of the 43rd annual pages 868–876. meeting on association for computational linguis- tics, pages 531–540. Association for Computational Koehn, Philipp and Josh Schroeder. 2007. Experi- Linguistics. ments in domain adaptation for statistical machine translation. In ACL Workshop on Machine Transla- Cronin, Michael. 2012. The translation age: transla- tion 2007. tion, technology, and the new instrumentalism. In Venuti, Lawrence, editor, The translation studies Kurokawa, David, Cyril Goutte, and Pierre Isabelle. reader, pages 469–482. Routledge. 2009. Automatic detection of translated text and its impact on machine translation. Proceedings. MT Culo,ˇ Oliver, Silvia Hansen-Schirra, Karin Maksymski, Summit XII, The twelfth Machine Translation Sum- and Stella Neumann. 2011. Empty links and cross- mit International Association for Machine Transla- ing lines: querying multi-layer annotation and align- tion hosted by the Association for Machine Transla- ment in parallel corpora. Translation: Computation, tion in the Americas. Corpora, Cognition. Special Issue Parallel Corpora: Annotation, Exploitation, Evaluation, 1(1):75–104. Lapshinova-Koltunski, Ekaterina. 2013. VARTRA: a comparable corpus for the analysis of translation Culo,ˇ Oliver. 2010. Valency, translation and the syn- variation. In Proceedings of the 6th Workshop on tactic realisation of the predicate. In Vitas, Duskoˇ Building and Using Comparable Corpora, pages 77– and Cvetana Krstev, editors, Proceedings of the 29th 86, Sofia, Bulgaria. International Conference on Lexis and Grammar (LGC), pages 73–82, Belgrade, Serbia. Nord, Christiane. 1997. Translating as a purposeful activity. Functionalist approaches explained. Num- De Almeida, Giselle and Sharon O’Brien. 2010. ber 1 in Translation Theories Explained. Jerome, Analysing post-editing performance: correlations Manchester. with years of translation experience. In Proceedings of the 14th annual conference of the European asso- Nord, Christiane. 2006. Translating for communica- ciation for machine translation, St. Raphael,¨ France, tive purposes across culture boundaries. Journal of pages 27–28. translation studies, 9(1):43–60.

Ding, Yuan and Martha Palmer. 2005. Machine trans- Nord, Christiane. 2011. Vertikal statt horizontal: die lation using probabilistic synchronous dependency bersetzungseinheit aus funktionaler sicht. In Funk- insertion grammars. In Arbor, Ann, editor, Proceed- tionsgerechtigkeit und Loyalitat.¨ Theorie, Methode ings of the 43rd annual meeting of the ACL, pages und Didaktik des funktionalen Ubersetzens¨ , vol- 541–548. ume 32 of TRANSUD¨ Arbeiten zur Theorie und Praxis des Dolmetschens. Frank & Time, Berlin. Diwersy, Sascha, Stefan Evert, and Stella Neumann. in press. A weakly supervised multivariate approach Oakes, Michael P. and Meng Ji, editors. 2012. Quan- to the study of language variation. In Szmrecsanyi, titative methods in corpus-based translation studies: Benedikt and Bernhard Walchli,¨ editors, Aggregat- a practical guide to descriptive translation research. ing Dialectology, Typology, and Register Analysis. Studies in corpus linguistics; v. 51. John Benjamins Linguistic Variation in Text and Speech, Linguae & Pub. Co, Amsterdam ; Philadelphia. Litterae. De Gruyter, New York/Berlin. O’Brien, Sharon. 2002. Teaching post-editing: a pro- Eisele, Andreas, Christian Federmann, Hans Uszkoreit, posal for course content. In Sixth EAMT Workshop, Saint-Amand Herve,´ Martin Kay, Michael Jelling- pages 99–106, Manchester, U.K. haus, Sabine Hunsicker, Teresa Herrmann, and Yu Chen. 2008. Hybrid architectures for multi- O’Brien, Sharon. 2010. Introduction to : engine machine translation. In Translating and the who, what, how and where to next? In Proceedings Computer 30, London, UK. of AMTA 2010, Denver, Colorado.

205 Papinieni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Com- putational Linguistics, pages 311–318, Philadelphia, USA. Quirk, Christopher, Arul Menezes, and Colin Cherry. 2005. Dependency treelet translation: syntactically informed phrasal SMT. In Arbor, Ann, editor, Pro- ceedings of the 43rd annual meeting of the ACL, pages 271–279. Rojo, Ana and Iraide Ibarretxe-Antunano, editors. 2013. Cognitive linguistics and translation: ad- vances in some theoretical models and applications. Number 23 in Applications of Cognitive Linguistics. De Gruyter Mouton, Berlin. Rozmyslowicz, Tomasz. in press. Machine transla- tion: A problem for translation theory. New voices in translation studies. Teich, Elke. 2003. Cross-linguistic variation in sys- tem and text. A methodology for the investigation of translations and comparable texts, volume 5 of Text, Translation, Computational Processing. Mouton de Gruyter, Berlin/New York. Tymoczko, Maria. 2005. Trajectories of research in translation studies. Meta, 50(4):1082–1097.

206