Multi-Retranslation corpora: Visibility, variation, value, and virtue ...... Tom Cheesman Department of Languages, Swansea University, UK Kevin Flanagan Department of Languages, Swansea University, UK and SDL Research, Bristol, UK Stephan Thiel Bauhaus University Weimar, Germany and Studio Nand, Berlin Jan Rybicki Institute of English Studies, Jagiellonian University, Poland Robert S. Laramee Department of Computer Science, Swansea University, UK Jonathan Hope Department of English, Strathclyde University, UK Avraham Roos Amsterdam School of Culture and History, University of Amsterdam, the Netherlands ...... Abstract Variation among human translations is usually invisible, little understood, and under-valued. Previous statistical research finds that translations vary most where the source items are most semantically significant or express most ‘attitude’ (affect, evaluation, ideology). Understanding how and why translations vary is important for translator training and translation quality assessment, for cultural research, and for machine translation development. Our experimental project began with the intuition that quantitative variation in a corpus of historical retranslations might be used to project quasi-qualitative annotations onto the Correspondence: translated text. We present a web-based system which enables users to create Tom Cheesman, Department parallel, segment-aligned multi-version corpora, and provides visual interfaces of Languages, Swansea for exploring multiple translations, with their variation projected onto a base University, SA2 8PP, UK. E-mail: text. The system can support any corpus of variant versions. We report experi- [email protected] ments using our tools (and stylometric analysis) to investigate a corpus of forty

Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017. ß The Author 2016. Published by Oxford University 739 Press on behalf of EADH. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/ licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited. doi:10.1093/llc/fqw027 Advance Access published on 23 August 2016 Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017 T. Cheesman et al.

German versions of a work by Shakespeare. Initial findings lead to more ques- tions than answers......

1 Introduction language—introduces further variation. If transla- tions are reprinted or otherwise re-used, they tend Our project began with a simple observation and an to be changed again. Venuti (2004) argues that re- intuition. The observation: in any set of multiple translations (more than most translations) ‘create 4 translations in a given language, variation among value’ in the target culture. A first translation of a them varies through the course of the text. Some foreign work creates awareness of it. If retranslations text units or chunks (at any level from word, say, follow, the work becomes assimilated to the target up to chapter or character part in a play) are more culture. If retranslations multiply, each both re- variously translated than others. The intuition: this inforces the value and status of the work in the variation can be used to project an annotation onto target culture, and extends the range of competing the translated text, indicating where and how the interpretations surrounding it. Retranslations there- extent of translation variation varies. This is the es- fore throw up questions going well beyond linguistic sence of our online system. It uses a ‘Translation and cultural transfer, concerning ‘the values and in- Array’ (a parallel multi-translation corpus, aligned stitutions of the translating culture’, and how these to a ‘base text’ of the translated work) to achieve are defended, challenged, or changed (Venuti, 2004, ‘Version Variation Visualization’. Here, ‘version’ p. 106). encompasses any text which can be at least partly Within Translation Studies, ‘retranslation stu- aligned with others. But the website strapline is: dies’ is underdeveloped, despite its fundamental im- ‘Explore great works with their world-wide portance for translation, linguistics, and translations’.1 communication, as well as comparative, trans- If multiple translations of a work exist, then the national cultural studies. As Munday (2012) work is enduringly popular and/or prestigious, ca- argues, retranslations are important resources, be- nonical or classic, in the translating culture: typically cause no single utterance or text exists in isolation ‘great works’ of scripture, literature, philosophy, from alternative forms it might have taken. Any etc.2 Interest in comparing such works’ multiple extant text is surrounded by a ‘penumbra’ of ‘un- translations is surprisingly limited. Some large selected forms’ (Munday, 2012, p. 13, citing Grant, aligned retranslations corpora are publicly accessible 2007, pp. 183–4); so any translation is surrounded online (works of scripture),3 but user access is lim- by ‘shadow translations’ (Johannson, 2011, p. 3, ited to two parallel texts, and no analytic tools are citing Matthiessen, 2001, p. 83). Sets of translations provided. No similar resources exist for any secular by different translators (or the same translators at works at all, yet. This reflects the notorious ‘invisi- different moments) make visible at least some bility’ of translators and translations in general otherwise unselected forms. This offers scope for (Venuti, 2008). A key aim of our project is to studying ‘the value orientations that underlie these make them visible. selections’ (Munday, 2012, p. 13). Our project seeks Retranslations are successive translations of the to go even further: from the how and why of vari- ‘same’ source work, often somehow dependent on ation among translations, back to the varying cap- precursor (re)translations. The source works con- acity of the translated text to provoke variation. cerned are mostly unstable texts in their original The article is organized as follows: Section 2 re- language: what translators translate varies and views related work, including statistical studies in changes. And so does how they do it. The gamut translation variation. Section 3 presents our soft- runs from word-for-word renderings to very free ware project, covering our Aligner, Corpus adaptations or rewritings with little obvious relation Overviews (including stylometric analysis), and to the source. Relay translation—via a third our key innovation: an interface deploying ‘Eddy

740 Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017

Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017 Multi-retranslations

and Viv’ algorithms to explore translation variation. analysis of eight English versions of Zola’s novel Section 4 presents findings of experiments using the Nana is typically vague: software. Section 5 offers concluding comments. (...) specific conditions (...) explain the similarities and differences (...). The condi- tions comprise broad social forces: changing 2 Related Work ideologies and changing linguistic, literary, and translational norms; as well as more spe- There has been little digital work on larger retrans- cific situational conditions: the particular con- lations corpora, involving works of wide intrinsic text of production and the translator’s interest, and none designed to facilitate access to preferences, idiosyncrasies, and choices. multiple translations, and the translated work, to- (Brownlie, 2006, p. 167) gether with algorithmic analyses. Ja¨nicke et al. (2015) take in some ways a similar approach, but The basic lesson is that translation is a humanities their ‘TRAViz’ interface offers a very different mode subject. Translators are writers. As Baker warns: of text visualization, is monolingual (shows no Identifying linguistic habits and stylistic pat- translated text), and works best with more limited terns is not an end in itself: it is only worth- variation and shorter texts (see Section 3.3). while if it tells us something about the cultural Lapshinova-Koltunski (2013) describes a parallel and ideological positioning of the translator, multi-translation corpus designed to support com- or of translators in general, or about the cog- putational linguistic analyses of differences between nitive processes and mechanisms that contrib- professional translations, student translations, ute to shaping our translational behaviour. Machine Translation (MT) outputs, and edited We need then to think of the potential motiv- MT outputs. Shei and Pain (2002) proposed a simi- ation for the stylistic patterns that might lar parallel corpus, with an interface designed for emerge from this type of study. translator training. These projects only offer access (Baker, 2000, p. 258) to filtered segments of the text corpus, and do not envisage exploring variation among retranslations. Her comment is cited by Li et al. (2011, p. 157), Altintas, Can, and Patton (2007) used two time- in their computationally assisted study of two separated (c.1950, c.2000) collections of published English translations of Xueqin Cao’s 5 translations of the same seven English, French, or Hongloumeng. They conclude: Russian literary classics into Turkish, to quantify corpus-assisted translation research can go aspects of language change. This raises the question beyond proving the obvious or the already whether such translations ‘represent’ their language. known as long as meta- or para-texts are Corpus-based Translation Studies (Baker, 1993; available for the analysis. The extent and Kruger et al., 2011) has established that translated depth of such analysis of course depends on language differs from untranslated language. We the amount of information available in the also know from decades of work in Descriptive form of meta- or other texts. Translation Studies (Morini, 2014; Toury, 2012) (Li et al., 2011, p. 164) that retranslations vary for complex genre-, market-, subculture-specific and institutional fac- Genuine understanding of cultural materials re- tors, and individual psychosocial factors, involving quires knowledge and critical understanding of the translators and others with a hand in the work many other materials, to assess how multi-scale (commissioners, editors), and their uses of re- human factors shape texts and the effects they sources including source versions and prior have (had) in their cultural world. (re)translations. Non-digital studies in retranslation underline the There is no consensus on defining such factors importance of such shaping factors. Deane-Cox and their interrelations. The conclusion of a manual (2014) and O’Driscoll (2011) both recently

Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017 741

Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017 T. Cheesman et al.

investigated large sets of English retranslations of typically do not have ready translation solutions 19th-century French novels. They detail at length and require some ‘‘artistic creativity’’ on the part the historical contexts of each retranslation, its pro- of translators’, and that this necessary ‘freedom’ duction and reception, and analyse short samples makes translation fundamentally ‘‘‘non-comput- linguistically or stylistically. Deane-Cox’s overall ar- able’’ or ‘‘non-algorithmic’’’ (Babych and Hartley, gument disproves the ‘Retranslation Hypothesis’ 2004, p. 835, citing Penrose, 1989). They conclude put forward by Antoine Berman (1990, p. 1). that there are: Berman argued that over time, successive retransla- fundamental limits on using data-driven tions should tend to translate the source text more approaches to MT, since the proper transla- accurately. In fact—as we will see—this may hold tion for the most important units in a text for a first few retranslations, but when they multi- may not be present in the corpus of available ply, the hypothesis no longer holds. This is partly translations. Discovering the necessary trans- because retranslators who come late in a series must lation equivalent might involve a degree of be more inventive, to distinguish their work from inventiveness and genuine intelligence. that of precursors and rivals. The desire for distinc- (Babych and Hartley, 2004, p. 836) tion is a great motivator (Mathijssen, 2007; Hanna, 2016). Critical translation studies pays close atten- Munday (2012, pp. 131–54) studied seventeen tion to such specific contextual factors, viewing each English translations of an extract from a story by translation as an act of intervention in a particular Jorge Luis Borges: two published translations and moment in a particular place in the geographical fifteen commissioned from advanced trainee trans- and social world, and a trace of a translator’s (and lators. Four in five lexical units varied. Invariance associated agents’) both conscious and unconscious was associated with ‘simple, basic, experiential or choices (Munday, 2012, p. 20). As Munday argues, denotational processes, participants and relations’ translation is essentially an evaluative act. (p. 143). Variation mainly occurred in ‘lexical ex- Translator’s decisions are based on evaluations of pression of attitude’, i.e. affect/emotion, judgment/ the source text, of the implicit values of its author ethics, appreciation, or evaluation (p. 24). Variation and intended audience, and of the expectations and was greatest at ‘critical points’, where ‘attitude-rich’ values of the intended audience of the translation. words and phrases ‘carry the attitudinal burden of the text’ and communicate ‘the central axiological 2.1 Statistical Studies values of the protagonists, narrator or writer’ (p. 146)—again, in effect, the semantically most sig- Statistical studies of differences between translations nificant items. confirm this perspective, and also rain on the MT Translations vary most at points of greatest se- parade. They show that variation is greatest both in mantic and evaluative/attitudinal salience. MT has a the most semantically significant units of a text, and long way to go, then. Its problems include identify- in the units which are most expressive of values and ing attitude, affect, or evaluation in a text to be affect. Babych and Hartley (2004) measured the sta- translated. In a chapter on MT and pragmatics, bility of alternative translations at word and phrase Farwell and Helmreich (2015) discuss lexical and level in English versions of 100 French news stories syntactic differences in 125 Spanish newswire art- by two professional translators. They found a strong icles translated into English by two professional statistical correlation between instability and the translators: 40% of units differed, and 41% of dif- scores of linguistic items in the source text for sali- ferences could be attributed to the translators’ dif- ence (tf.idf score) or significance (S-score; see ferent ‘assumptions about the world’ (rather than Babych et al., 2003). The more important an item assumption-neutral paraphrasing, or error). One is for a text’s meaning, the less translators tend to example is this headline: agree about translating it (though each one is con- sistent in using their selected terms). Babych and Acumulacio´ndevı´veres por anuncios sı´smicos Hartley deduce that ‘highly significant units en Chile

742 Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017

Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017 Multi-retranslations

Translation 1: Hoarding caused by earthquake works can be very successful (i.e. ‘good’, for many predictions in Chile people) without being at all complete or accurate. A Translation 2: Stockpiling of provisions be- related question is: Why do most retranslations have cause of predicted earthquakes in Chile brief lives (just one publication, or media or per- (Farwell and Helmreich, 2015, p. 171) formance use), while others—backed by some insti- tutional authority—become canonical, and have The translations make vastly different ideolo- many editions, revisions, and re-uses, over gener- gical, political, evaluative assumptions. ‘Hoarding’ ations? Does the answer lie in linguistic, textual suggests a panicky, irrational population, respond- qualities of the translation, measured in terms of ing to rumours of an unlikely event. ‘Stockpiling’ its relation to the original work? Or in some quali- (by the population, or the civil authorities?) is a ties of it, measured in relation to alternative versions prudent response to credible (scientific experts’?) or other target culture corpora? Or does it lie solely warnings. It is impossible—without ‘meta- or in institutional factors? para-texts’—to disentangle whether the translators Our project does not comprehensively address impute different values to the mind of the source these questions. It grew out of a particular piece of text creator, or to its intended readers, or to the translation criticism, and the intuition that digital anticipated readers of the target text, and/or tools could be developed to explore patterns in vari- whether they express their own psychological and ation among multiple (re)translations, in themselves, ideological values. ‘Acumulacio´n’, here, has major in relation to target cultural contexts, and in relation evaluative implications which could not be pre- to the translated work. Before knowing any of the dicted without area-specific political and economic above-mentioned studies, Cheesman wanted to find expertise. Perhaps a multi-retranslation corpus ways to compare a large collection of German trans- could be used to discover which items provoke vari- lations and adaptations of Shakespeare’s play, The ation, as a proxy for such knowledge? If not, what Tragedy of Othello, The Moor of Venice (see corpus would it discover? overviews in section 3.2 below).6 His interest was as a researcher in German and comparative literature and 3 Project Description culture. He had worked on a recent, controversial version of Othello (Cheesman, 2010), and wondered A multi-retranslation corpus will contain versions of how it related to others. He manually examined over various kinds; complete, fragmentary, edited, thirty translations (1766–2010) of a very small adapted versions; versions derived from (a version sample: a fourteen-word rhyming couplet, a ‘critical of) the original-language translated work, or from moment’ which is rich in affect, evaluation, and am- 7 intermediaries in the translating language, and/or biguity (Cheesman, 2011). His study showed how other languages; versions in various media; for vari- differences among the translations traced a 250-year- ous audiences (popular, scholarly, restricted); in long conversation about human issues in the work— mono-, bi-, or plurilingual formats; from various gender, race, class, political power, interpersonal periods and places; produced and received under power, and ethics. Could digital tools help to explore various economic, political, institutional, and cul- such questions and communicate their interest to a tural-linguistic conditions. An obvious lay question wider public? The couplet he had selected was clearly is: Which one is best? But the problem is already more variously translated than most passages in clear: By what criteria, or whose, do we judge? the play. So he wondered if we could devise an algo- Models for assessing professional translations rithmic analysis which would identify all the most (House, 1997) are predicated on full and precise variously translated passages, to steer further rendering of the source, but work less well with cre- research. ative genres, where such ‘fidelity’ is often subordi- A proof-of-concept toolset (‘Translation Array nated to effect in the target culture. Retranslations Prototype’) was built, using as test data a corpus of poetry, plays, novels, religious, or philosophical of thirty-eight hand-curated digital texts of

Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017 743

Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017 T. Cheesman et al.

German translations and adaptations of part of the have an arbitrary number of attributes. For a play: Othello, Act 1, Scene 3. This is about 3,400 play these might be ‘type’ (with values such as continuous words of the play’s 28,000, in English: ‘Speech’, ‘Stage Direction’), or ‘Speaker’ (with 392 lines and 92 speeches (in Neill’s 2006 edition). values such as ‘Othello’, ‘Desdemona’), and so The restricted sample size was due to restricted re- on. sources for curating transcriptions, and translation (Flanagan in: Cheesman et al., 2012) copyright limitations. Versions were procured from Hand- or machine-made attributes such as libraries, second-hand book-sellers, and theatre ‘irony’, ‘variant from source x’, ‘crux’, ‘body part publishers (who distribute texts not available y’, ‘affect z’, ‘syllogism’, ‘trochee’, and ‘enjambe- through the book trade). Digital transcription ment’ are equally possible. But all would require stripped out original formatting and paratexts (pref- time-consuming tagging. In fact, we have worked aces, notes, etc). The transcriptions were minimally annotated, marking up speech prefixes, speeches, only with ‘type: Speech’. Segment positions are and stage directions. The brief for the programmers stored as character offsets within documents, and (Flanagan and Thiel) was to build visual web inter- texts can be edited without losing this information faces enabling the user to: align a set of versions with (transcription errors keep being discovered). a base text and so create a parallel multi-version Segmented documents are aligned in an interactive corpus;8 obtain overviews of corpus metadata and WYSIWYG tool, where an ‘auto-align’ function aligned text data; navigate parallel text displays; aligns all the next segments of specified attribute. apply an algorithmic analysis to explore the differ- For Othello, every speech prefix, speech and ‘other’ ing extent to which base text segments provoke vari- string is automatically pre-defined as a segment of ation among translations; customize this analysis that type. Any string of typographic characters in a and create various forms of data output to support speech can be manually defined as a segment and cultural analyses. aligned. Thiel and colleagues at Studio Nand built visual interfaces on top of Prism, including parallel- 3.1 Aligner text views tailor-made for dramatic texts (base text An electronic Shakespeare text was manually col- and any translation), and the ‘Eddy and Viv’ view lated with a recent edition, to give us a base text discussed below (Section 4). Thiel (2014b) docu- inclusive of historic variants.9 Then we needed to ments the design process. He also sketched a scal- align it segmentally with the versions. Existing open able, zoomable multi-parallel view of base text and tools for working with text variants (e.g. Juxta col- all aligned versions, an overview model which re- 10 mains to be developed as an interface for combined lation software) lack necessary functionality; so do 11 existing computer-assisted translation tools; per- reading and analysis (Thiel, 2014a). haps such software could be adapted; at any rate we built a web-based tool from scratch. The devel- 3.2 Corpus overviews oper, Flanagan, explains its two main components: Visual overviews of a corpus support distant read- Ebla: stores documents, configuration details, ings of text and/or metadata features. We devised segment and alignment information, calcu- three. An online, interactive time-map of historical lates variation statistics, and renders docu- geography shows when and where versions were ments with segment/variation information. written and published (performances are a desider- Prism: provides a web-based interface for up- atum); it identifies basic genres (published books for loading, segmenting and aligning documents, readers, books for students, theatre texts), and pro- then visualizing document relationships. vides bio-bibliographical information (Thiel, 2012). Areas of interest in a document are demar- A stylometric diagram is discussed in Section 3.2.2 cated using segments, which also can be (Fig. 2). ‘Alignment maps’ depict the information nested or overlapped. Each segment can created by segment alignment (Fig. 1).

744 Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017

Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017 on 27 November 2017 by Swansea University user Downloaded fromhttps://academic.oup.com/dsh/article-abstract/32/4/739/2669776 iia coasi nteHmnte,Vl 2 o ,2017 4, No. 32, Vol. Humanities, the in Scholarship Digital Multi-retranslations

Fig. 1 Alignment maps of thirty-five German Othello 1.3 (1766–2010) 745 T. Cheesman et al.

3.2.1 Alignment maps strength represents degree of similarity. These lines Alignment maps, developed by Thiel, are ‘barcode’- (edges) can indicate intertextual relations: depend- type maps which show how a translation’s constitu- ency of some kind, including potential plagiarism. ent textual parts (here: speeches) align with a similar Directionality can be inferred from date labels on map of the base text. Figure 1 shows thirty-five such nodes. For example, the version by Bodenstedt maps, in chronological sequence. Each left-hand (1867) (near top centre) was revised in the strongly block represents the English base text of Othello connected version by Ru¨diger (1983). This confirms 1.3, the right-hand block represents a German data on his title page. Other results, as we will see, text, and the connecting lines represent alignments are more surprising: spurs to further research. in the system. Within each block, horizontal bars The x/y axes are not meaningful. The analysis represent speeches (in sequence top to bottom) involves hundreds of counts using differing param- and thickness represents their length, measured in eters: the diagram is a design solution to the prob- words; Othello’s longest speech in the scene (and lem of representing high-dimensional data in a two- the play) is highlighted. Small but significant differ- dimensional plane. Removing or adding even one ences in overall length can be noticed: translations version produces a different layout and can re-ar- tend to be longer than the translated texts, so it is range clusters. Moreover, the analysis process is so interesting to spot versions which are complete yet complex that we cannot specify which text features more concise, such as Gundolf (1909). We can see lie behind the results. Broadly, though, the diagram which versions, in which passages, make cuts, can be read historically, right to left: a highly formal reduce, expand, transpose, or add material which poetic theatre language gives way to increasingly in- could not be aligned with the base text. In the formal, colloquial style. centre of the figure, the German translation Nine versions are revisions, editions or rewritings (Felsenstein and Stueber, 1964) of the Italian li- of the canonical translation by Baudissin (originally bretto (by Boito) of Verdi’s opera Otello (1887) is 1832, in the famed ‘Schlegel-Tieck’ Shakespeare edi- a good example of omission, addition, and trans- tion; see: Sayer, 2015). Most are quite strongly con- position. Omissions and additions are also evident nected and closely clustered, but the apparent in the recent stage adaptations on the bottom line. stylometric variety is a surprise. The long, weak Zimmer (2007), like Boito, assigns Othello’s long line connecting the cluster to the heavily revised speech to multiple speakers. In our online system, stage adaptation by Engel (1939) (upper left) is to these maps serve as navigational tools alongside the be expected, but the length and weakness of the texts in Thiel’s parallel-text views. Each bar repre- connection with Wolff’s (1926) published edition senting a speech is also tagged with the relevant (lower right) is more of a surprise. His title page speech prefix, so any character’s part can be high- indicated a modestly revised canonical text, but styl- lighted and examined. Aligned segments are rapidly, ometry suggests something more radical is going smoothly synched in these interfaces, assisting ex- on.13 ploratory bilingual reading. Above all, this analysis reveals the salience of his- torical period. Distinct clusters are formed by all the 3.2.2 Stylometric network diagram early C19 versions (mid-right), arguably all the late Figure 2 depicts a stylometric analysis of relative C19 versions (top), most of the late C20 versions Most Frequent Word frequencies in 7,000-word (lower left), and all the C21 versions (far left). The chunks of forty German versions of Othello, carried C21 versions are all idiosyncratic adaptations (cf. out by Rybicki using the Stylo script and the Gephi Fig. 1, bottom line). It is surprising to see how simi- visualization tool.12 The network diagram shows (1) lar they appear, in stylometric terms, relative to the relations of general similarity between versions, rep- rest of the corpus. And what do the strong links resented by relative proximity (clustering), and (2) among them indicate? Mutual influence, plagiarism, similarities in particular sets of frequency counts, common external influence? What about the lines represented by connecting lines; their thickness or leading from Gundolf (1909) (low centre) across to

746 Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017

Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017 Multi-retranslations

Fig. 2 Stylometric analysis of forty German Othellos Node label key: Translator_Date. Prefix: Baud ¼ version of Baudissin (1832). Suffixes: _Pr ¼ prose study edition. No suffix ¼ other book. _T ¼ theatre text (no book trade distribution). _X ¼ theatre text, not performed (only version by a woman).

Swaczynna (1972), to Laube (1978), to Gu¨nther Differences in the use of German represented by (1995)? Gu¨nther is the most celebrated living distances across the rest of graph cannot be due to German Shakespeare translator: do these lines any general historical changes in the language. They trace his debts to less famous precursors? Period reflect changes in the specific ways German is used outliers are also interesting. Zeynek (?-1948) ap- by translators of Shakespeare for the stage, and/or pears to be writing a C19 style in the 1940s. The for publications aimed at people who want to read unknown Schwarz (1941) is curiously close to the his work for pleasure. famous Fried (1972). Rothe (1956) (extreme bottom left) is writing in a late C20 style in the 1950s. This 3.3 The ‘Eddy and Viv’ interface throws interesting new light on the notorious ‘Rothe Overviews are invaluable, but the core of our system case’ of the Weimar Republic and Nazi years: he was is a machine for examining differences at small victimized for his ‘liberal’, ‘modern’ approach to scale. The machine implements an algorithm we translation (Von Ledebur, 2002). called ‘Eddy’,14 to measure variation in a corpus Genre is salient, too. A very distinct cluster, of translations of small text segments. Eddy’s find- bottom right, includes all versions designed for ings are then aggregated and projected onto the base study and written in prose (rather than verse). text segments by the algorithm ‘Viv’ (‘variation in This includes our two earliest versions (1766 and variation’). In an interface built by Thiel, on the 1779) and two published 200 years later (1976, basis of Flanagan’s work, users view the scrollable 1985). Strongly interconnected, weakly connected base text (Fig. 3: left column) and can select any with any other versions, this cluster demonstrates previously defined and aligned segment: this calls the flaw in the approach of Altintas et al. (2007). up the translations of it, in a scrollable list (Fig. 3:

Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017 747

Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017 T. Cheesman et al.

Fig. 3 Eddy and Viv interface (Colour online)

right columns). The list can be displayed in various considered as representing an axis in N-di- sequences (transition between sequences is a pleas- mensional space, where N is the length of ingly smooth visual effect) by selecting from a the corpus word list. For each version, a menu: order by date; by the translator’s surname; point is plotted within this space whose co- by length; or (as shown in Fig. 3) by Eddy’s algo- ordinates are given by the word frequencies in rithmic analysis of relative distinctiveness. Eddy the version word list for that version. (Words metrics are displayed with the translations, and not used in that version have a frequency of also represented by a yellow horizontal bar which zero.) The position of a notional ‘average’ is longer, the higher the relative value. translation is established by finding the cen- We defined ‘segment’, by default, as a ‘natural’ troid of that set of points. An initial ‘Eddy’ chunk of dramatic text: an entire speech, in semi- variation value for each version is calculated automated alignment. Manual definition of seg- by measuring the Euclidean distance between ments (any string within a speech) is possible, but the point for that version and the centroid. defining and aligning such segments in forty ver- Flanagan in Cheesman et al. (2012–13) sions is time-consuming. In future work we intend This default Eddy algorithm is based on the to use the more standard definition: segment ¼ vector space model for information retrieval. sentence (not that this would simplify alignment, Given a set S of versions {a, b, c ...} where each since translation and source sentence divisions fre- version is a set of tokens {t , t , t ...t }, we create a quently do not match). Eddy compares the wording 1 2 3 n set U of unique tokens from all versions in S (i.e. a of each segment version with a corpus word list: corpus word list). For each version in S we construct here the corpus is the set of aligned segment ver- vectors of attributes A, B, C ...where each attribute sions. No stop words are excluded; no stemming, is the occurrence count within that version of the lemmatization, or parsing is performed. Flanagan corresponding token in U, that is: explains how the default Eddy algorithm works: Xjaj Each word in the corpus word list [the set of A ¼ aj ¼ Ui unique words for all versions combined] is j¼1

748 Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017

Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017 Multi-retranslations

We construct a further vector Z to represent the same. The Duke’s couplet beginning ‘If virtue no centroid of A, B, C ... such that delighted beauty lack...’ (the subject of Cheesman’s initial studies), has the darkest under- ðÞA þ B þ C ... Z ¼ i i i lay. As we knew, translators (and editors, per- jSj formers, and critics) interpret this couplet in Then, for a version a, the default Eddy value is widely varying ways. In the screenshot, the Duke’s calculated as: couplet has been selected by the user: part of the list vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u of versions can be seen on the right. MTs back into uXjUj English are provided, not that they are always t 2 Eddy ¼ jZi Aij helpful. i¼1 Unlike the TRAViz system (Ja¨nicke et al., 2015), ours does not represent differences between versions This default Eddy formula is used in the experi- in terms of edit distances, and translation choices in ments reported below, coupled with a formula for terms of dehistoricized decision pathways. Our Viv as the average (arithmetic mean) of Eddy values. system preserves key cultural information (historical Other versions of the formulae can be selected by sequence). It can better represent very large sets of users,15 e.g. an alternative Eddy value based on an- highly divergent versions. The TRAViz view of two gular distance, calculated as:  lines from our Othello corpus (Ja¨nicke et al., 2015, 1 AZ Figure 17) is a bewilderingly complex graph. With 2cos IAIIZI Eddy ¼ highly divergent versions of longer translation texts,  TRAViz output is scarcely readable. Crucially there Work remains to be done on testing the different is no representation of the translated base text. The algorithms, including the necessary normalization Eddy and Viv interface is (as yet) less adaptable to for variations in segment length.16 other tasks, but better suited to curiosity-driven 17 Essentially, Eddy assigns lower metrics to word- cross-language exploration. ings which are closer to the notional average, and higher metrics to more distant ones. So, Eddy ranks 4. versions on a cline from low to high distinctiveness, Experiments with Eddy and Viv or originality, or unpredictability. It sorts common- 4.1 Eddy and ‘Virtue? A fig!’ or-garden translations from interestingly different To illustrate Eddy’s working, Table 1 shows Eddy ones. results, in simplified rank terms (‘high’, ‘low’, or Viv shows where translators most and least dis- unmarked intermediate), for thirty-two chrono- agree, by aggregating Eddy values for versions of the logically listed versions of a manually aligned seg- base text segment, and projecting the result onto the ment with a very high Viv value: ‘Virtue? A fig!’ base text segment. Viv metrics for segments are dis- (Othello 1.3.315). An exclamation is always, in played if the text is brushed, and relative values are Munday’s terms, ‘attitude-rich’, burdened with shown by a colour annotation (floor and ceiling can affect; this one is a ‘critical point’ for several reasons. be adjusted). As shown in Fig. 3, the base text is ‘Virtue’ is a very significant term in the play, and annotated with a colour underlay of varying tone. crucially ambiguous: in Shakespeare’s time it meant Lighter tone indicates relatively low Viv (average not only ‘moral excellence’ but also ‘essential Eddy) for translations of that segment. Darker nature’, or ‘life force’, and ‘manliness’.18 The tone indicates higher Viv. Shakespeare’s text can speaker here is Iago, responding to Roderigo, who now be read by the light of translations has just declared that he cannot help loving the (Cheesman, 2015). heroine, Desdemona: ‘... it is not in my virtue to Sometimes it is obvious why translators disagree amend it’. Roderigo means: not in my nature, my more or less. In Fig. 3, Roderigo’s one-word speech power over myself, my male strength. But Iago’s ‘Iago -’ has a white underlay: every version is the response implies the moral meaning, too. Then,

Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017 749

Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017 on 27 November 2017 by Swansea University user Downloaded fromhttps://academic.oup.com/dsh/article-abstract/32/4/739/2669776 .Cheesman T. 750 Table 1 ‘Virtue? A fig!’ in thirty-two German translations (1766–2012)

iia coasi nteHmnte,Vl 2 o ,2017 4, No. 32, Vol. Humanities, the in Scholarship Digital Numbers ‘Eddy’ Length Translations Back-translations Sources Intertexts rank rank tal. et 1 Tugend? Pfifferling. Virtue? [Not worth a] chanterelle. Wieland 1766 (S) (P) 2 H Tugend!—Den Henker auch! Virtue!—[To Hell with] the executioner too! Eschenburg (ed. Eckert) 1779 (S) 3 L L Tugend? Possen! Virtue? Buffoonery! Schiller and Voss 1805 (P) Cf. #4, #11 4 L Tugend? Narrenspossen! Virtue? Fools’ buffoonery! Benda 1826/Ortlepp 1839 5 L Tugend! Abgeschmackt! Virtue! Vulgar! Baudissin [Schlegel-Tieck] 1832 (P) 6 Tugend? Zum Henker! Virtue? To the executioner! [¼ be damned!] Bodenstedt 1867 7 Tugend? Leeres Gefasel! Virtue? Mindless drivel! Jordan 1867 8 Tugend? Wischiwaschi! Virtue? Drivel! Gildemeister 1871 9 L L Tugend! Aeh! Virtue! Ugh! Vischer 1887 10 Tugend! Pfeif drauf! Virtue! Whistle on it! [¼ Don’t give a Gundolf 1909 damn for it] 11 L L Tugend! Possen! Virtue! Buffoonery! Baudissin (ed. Wolff) 1926 12 L Tugend! Dummheit! Virtue! Stupidity! Engel 1939 (T) 13 Energie? Ein Schmarren! Energy? Nonsense! [dialectal: S German] Schwarz 1941 (T) Cf. #23 14 L Tugend! Ach was! Virtue! Oh come on! Baudissin (ed. Brunner) 1947 (S) 15 H H Nicht die Kraft! Zum lachen! Not the strength! Laughable! Zeynek ?-1948 (T) 16 L ‘Tugend’? Quatsch! ‘Virtue’? Nonsense! Flatter 1952 17 Tugend? Weiße Ma¨use! Virtue? White mice! Rothe 1956 18 Macht? Dummes Zeug! Power? Stuff and nonsense! Schaller 1959 19 Tugend? Keine Feige wert! Virtue? Not worth a fig! Schro¨der 1962 20 Tugend? fick drauf Virtue? fuck it Swaczynna 1972 (T) 21 H H In deiner Macht? Ach was! In your power? Oh come on! Fried 1972 (P) Cf. #23, #27... 22 Tugend! Ein Quark! Virtue! Quark! [soft cheese/nonsense] Lauterbach 1973 (T) Cf. #27 23 Macht? Schmarren! Power? Nonsense! [dialectal: S German] Engler 1976 (S) 24 H H Nicht in deiner Macht? Son Quatsch! Not in your power? What nonsense! Laube 1978 (T) Cf. #27 25 Tugend! Ein Dreck! Virtue! Filth! [Crap] Ru¨diger 1983 (T) 26 L L Tugend? Quatsch Virtue? Nonsense Bolte and Hamblock 1985 (S) 27 H H Nicht in deiner Macht? Quark! Not in your power? Quark! Gu¨nther 1995 (P) Cf. #31, #32. 28 H Da kannst du lange beten You can pray a long time [¼ Not until the Motschach 1992 (T) cows come home] 29 L Affenkram Ape-rubbish [¼ Crap!] Buhss 1996 (T) 30 H H Charakter? Am Arsch der Charakter! Character? Character my arse! Zaimoglu and Senkel 2003 31 H H Nicht in deiner Macht? Quatsch! Not in your power? Nonsense! Leonard 2010 (T) 32 Nicht in deiner Macht! Not in your power! Steckel 2012 H and L indicate highest (H) and lowest (L) seven Eddy value rankings and length rankings. Alternative translations to ‘Tugend’ ¼ ‘virtue’ are underscored. Sources: ¼now in print. (S) ¼ study text. (T) ¼ no book trade distribution (theatre text). Intertexts: (P) ¼ prestigious, influential. Multi-retranslations

the phrase ‘A fig!’ is gross sexual innuendo. ‘Fig’ can be generated: it shows Eddy average rising in meant vagina. The expression derives from this corpus since about 1850. This may be a peculi- Spanish and refers to an obscene hand gesture: in- arity of German Shakespeare. It may be an artefact tense affect (see Neill, 2006, p. 235). (The expression of the method. But it is conceivable that, with fur- ‘I don’t give/care a fig!’ was once commonplace, and ther work, the period of an unidentified translation often used euphemistically for ‘fuck’, a word might be predicted by examining its Eddy metrics. Shakespeare never uses.) Eddy and Viv results for any selected segments, The lowest and highest seven Eddy rankings are based on the full corpus or a selected subset of ver- indicated. Eddy’s lowest-scoring translation is sions, can be retrieved and explored in several forms ‘Tugend? Quatsch’ (#16, #26). ‘Tugend’ is the of chart, table, and data export. The interactive modern dictionary translation of (moral) ‘virtue’. ‘Eddy Variation’ chart, for example, facilitates com- ‘Quatsch’ is a harmless expression of disagreement: parisons between one translator’s work and that of a bowdlerized translation (bowdlerization is clear in any set of others (e.g. her precursors and rivals). It most versions here).19 The Eddy score is low be- plots Eddy results for selected versions against seg- cause most translations (until 1985) use ‘Tugend’ ment position in the text; any version’s graph can be and several also use ‘Quatsch’. Eddy’s highest displayed or not (simplifying focus on the transla- score is for ‘Charakter? Am Arsch der Charakter!’ tion of interest); when a node is brushed, the rele- (#30). This is Zaimoglu’s controversial adaptation vant bilingual segment text is displayed. of 2003, with which Cheesman’s work on Othello Eddy’s weaknesses are evident in Table 1, too. It began (2010). No other translation uses those fails to highlight the only one-word translation words, including the preposition ‘am’ and article (#29), or the one giving ‘fig’ for ‘fig’ (#19), or the ‘der’. ‘Charakter’ accurately translates the main one with the German equivalent of ‘fuck’ (#20), ex- sense of Shakespeare’s ‘virtue’ here, and ‘Arsch’ pressing the obscenity which remains concealed fairly renders ‘A fig!’ This is among the philologic- from most German readers and audiences. We still ally informed translations of ‘virtue’ (as ‘energy’, need to sort ordinary translations from extraordin- ‘strength’, ‘power’), a series which begins with ary and innovative ones in more sophisticated ways. Schwarz (1941) (#13). It is also among the syntac- Eddy also fails to throw light directly on genetic and tically expansive translations, with colloquial speech other intertextual relations. Some are indicated in rhythms, which begin with Zeynek (?-1948) (#15).20 the ‘Intertexts’ column in Table 1: the probable in- Both series become predominant following the pres- fluence of some prestigious retranslations is appar- tigious Fried (1972) (#21). ent in several cases, as is the possible influence of Reading versions both historically and with some obscure ones. Such dependency relations re- Eddy, in our interface, makes for a powerful tool. quire different methods of analysis and representa- Here the historical distribution of Eddy rankings tion. Stylometric analysis (Section 3.2.2) provides confirms what we already know about changes in pointers. More advanced methods must also en- Shakespeare translation. The lowest mostly appear compass negative influence, or significant non-imi- up to 1926. The highest mostly appear since 1972 tation. Table 1 shows—and this result is typical (recall Figure 2: lower left quadrant). Ranking by too—that the canonical version (#5), the most length in typographical characters is not often often read and performed German Shakespeare useful, but with such a short segment its results text from 1832 until today, is ‘not’ copied or even are interesting, and similar to Eddy’s. Most of the closely varied. That is no doubt because of risk to a shortest are up to 1947, and most of the longest retranslator’s reputation. Retranslators must differ- since 1972: that shift towards more expansive, col- entiate their work from what the public and the loquial translations, again. specialists know (Hanna, 2016). Similar historical Eddy results are found for The tool we built is a prototype. Eddy is admit- many segments in our corpus. An ‘Eddy History’ tedly imperfect. But its real virtue lies in the power it graph, plotting versions’ average Eddy on a timeline, gives to Viv, enabling us to investigate to what

Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017 751

Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017 T. Cheesman et al.

extent base text features and properties might cor- global text features (e.g. here: Othello, Desdemona, relate with differences among translations. Even that Iago). is only a start, as Flanagan points out: Hypothesis 4 relates to ‘low’ Viv findings. It was somehow disappointing to find that speeches by the Ebla can be used to calculate different kinds of hero Othello and the heroine Desdemona, including variation statistics for base text segments passages which generate much editorial and critical based on aligned corpus content. These can discussion, had moderate, low, or very low Viv potentially be aggregated-up for more coarse-grained use. The results can be navi- scores. Famous passages where Othello tells his life gated and explored using the visualization story and how he fell in love with Desdemona, or functionality in Prism. However, translation where Desdemona defies her father and insists on variation is just one of the corpus properties going to war with Othello, surely present key chal- that could be investigated. Once aligned, the lenges for retranslators. Perhaps passages which have data could be analysed in many other ways. been much discussed by commentators and editors (Flanagan in: Cheesman et al., 2012–13) pose less of a cognitive and interpretive challenge, as the options are clearly established.23 This hypothesis 4.2 ‘Viv’ in Venice could be investigated by marking up passages with a metric based on the extent of associated annotation An initial Viv analysis of Othello 1.3, involving all in editions and/or frequency of citation in other cor- the ninety-two natural ‘speech’ segments, was re- 21 pora. For now, we have speculated that the hero’s ported (Cheesman, 2015). It found that the ‘highest’ Viv-value segments tended to be (1) near and heroine’s speeches in this particular scene do the start of the scene, (2) spoken by the Duke of exhibit common attitudinal, not so much linguistic, Venice, who dominates that scene, but appears in but dramatic features. In the low-Viv segments, the no other, and (3) rhyming couplets (rather than characters can be seen to be taking care to express blank verse or prose). There are twelve rhyming themselves particularly clearly; even if very emo- couplets in the scene; two are speech segments; tional, they are controlling that emotion to control both were in the top ten of ninety-two Viv results. a dramatic situation. Perhaps translators respond to No association was found between Viv value and this ‘low affect’ by writing less differently? But it is perceptible attitudinal intensity, or any linguistic difficult to quantify such a text feature and so check features. We did find some high-Viv segments asso- Viv results against any ‘ground truth’. ciated with specific cross-cultural translation chal- There is another possible explanation: in the lenges. Highest Viv was a speech by Iago with the most ‘canonical’ parts of the text (here: the hero’s phrase ‘silly gentleman’, which provokes many dif- and heroine’s parts), retranslators perhaps tread a ferent paraphrases. But some lower-Viv segments careful line between differentiating their work and present similar difficulties, on the face of it. There limiting their divergence from prestigious precur- 24 was no clear correlation. sors. Such ‘prestige cringe’ would relate to the Still, four hypotheses emerged for further above-mentioned negative influence, or non-imita- research. tion of the most prestigious translations (Section Hypothesis 1: Based on rhyming couplets having 3.4). Precursors act, paradoxically, as both negative high Viv-value: retranslators diverge more when and positive constraints on retranslators. they have additional poetic-formal constraints.22 Hypothesis 4: in the most canonical constituent Hypothesis 2: Based on finding (1) above: retran- parts of a work, Viv is low, as retranslators tend to slators diverge more at the start of a text or major combine willed distinctiveness with caution, limit- chunk of text (i.e. at the start of a major task). ing innovation. Hypothesis 3: Based on finding (2) above: retran- In the initial analysis, the groups of speeches as- slators diverge more in translating a very salient, signed highest and lowest Viv values had suspi- local text feature in a structural chunk (in this ciously similar lengths. Clearly the normalization scene: the part of the Duke) and less in translating of Eddy calculations for segment length leaves

752 Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017

Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017 on 27 November 2017 by Swansea University user Downloaded fromhttps://academic.oup.com/dsh/article-abstract/32/4/739/2669776

Table 2 ‘Viv values’ in two liners in Othello 1.3 generated by twenty German versions iia coasi nteHmnte,Vl 2 o ,2017 4, No. 32, Vol. Humanities, the in Scholarship Digital Multi-retranslations

(continued) 753 T. Cheesman et al.

something to be desired. The next and latest analysis focused on segments of similar length to investigate our hypotheses.

4.3 ‘Viv’ in two liners Table 2 shows the grammatically complete two-line verse passages in Othello 1.3, plus prose passages of equivalent length,25 in Viv value rank order. A sub- corpus of twenty translations was selected for better comparability.26 The text assigned to each major character part here is reasonably representative of their overall part in the scene, counted in lines: Brabantio (sample eighteen lines [nine couplets]/ total sixty-one lines) 0.3, Desdemona (10/31) 0.32, Duke (22/67) 0.33, Iago (14/65) 0.21, Othello (20/ 108) 0.19. Hypothesis 1 seems to be confirmed, though more work needs to be done to prove it conclu- sively: high Viv value correlates with poetic-formal constraint. In the column ‘Form’ in Table 2, blank verse is the default. Unsurprisingly, rhyming coup- lets appear mostly in the top half of the table, including five of the top ten items. Translators enjoy responding to the formal challenge of rhym- ing couplets in self-differentiating ways; and they must so respond, or else they very obviously plagi- arize, because these items are rare in the text and highly noticeable, for audiences or readers. Hypothesis 2 is not confirmed: scanning the column ‘Running order’, there is no sign that trans- lators differentiate their work more at the start of the scene, as they embark on a new chunk of the task. That could have been interesting for psycho- linguistic and cognitive studies of translation (Halverson, 2008). Hypothesis 3 seems to be confirmed, but we need much more evidence to be sure we have discovered a general pattern. Scanning the column ‘Speaker’, the Duke’s segments are more variously translated than those of other speakers. Even if we exclude rhyming couplets, the Duke is over-represented in

Continued the upper part of the table. Brabantio and Iago also have some very high-Viv lines, but their segments are distributed evenly up and down the table. Not so

Table 2 with the Duke, who is the salient, local text feature in this scene and no other.

754 Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017

Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017 Multi-retranslations

Hypothesis 4 also seems provisionally confirmed. the best way to account for the regularity of Othello is strikingly low-Viv, mostly. Desdemona the novelistic cycle—but ‘generation’ is itself a tends to be low- to mid-Viv. Translations of their very questionable concept. Clearly, we must parts differ ‘less’ than other parts, at this scale. Why? do better. We do not know. It could be ‘prestige cringe’ (Moretti, 2003, p. 82) (Section 4.2). But it could also be specific to this So too with ‘Translation Arrays’ and ‘Version text. Othello in particular refuses ‘affect’ in this Variation Visualization’: we must do better. scene, as he does throughout the first half of the We wanted to demonstrate that this sort of ap- play: he is in command of everything, including proach opens up interesting possibilities for future his emotions. He echoes a much discussed line research.28 Of course one big difference between just spoken by Desdemona (‘I saw Othello’s visage Moretti’s work and ours so far is one of scale. His in his mind’, 1.3.250) when he says to the Duke and team works with tens or hundreds of thousands of assembled Senators that he wants her to go to war texts and metadata items. We are working with a with him, but: few dozen versions of one play, in one target lan- I therefore beg it not, guage, because that is what we have got,29 and only a To please the palate of my appetite, fragment of the play, because we chose to make the Nor to comply with heat—the young affects texts publicly accessible, which entails copyright re- In me defunct—and proper satisfaction. strictions (and some expense). Our approach re- But to be free and bounteous to her mind: quires time-consuming text curation (correction of (...) digital surrogates against page images),30 permission (Othello 1.3.258–63) acquisition, and manual segmentation and align- ment processes (more sophisticated approaches This is one of the play’s cruxes—passages which 31 editors deem corrupt and variously resolve (here, including machine learning will speed these up). ‘me’ is often changed to ‘my’, ‘defunct’ to ‘distinct’, Moretti experimentally ‘operationalizes’ pre- and the punctuation revised).27 Translators also re- digital critical concepts such as ‘character-space’ or solve this passage variously, depending in part on ‘tragic collision’ (Moretti, 2013), by measuring which edition(s) they work with; but—as measured quantities in texts: digital proxies or analogues. by Viv—not very variously, compared with other Eddy and Viv, on the other hand, are measuring passages. Can it be that textual ‘affect’ is relatively relational corpus properties which have no obvious less, because that is the kind of character, the mind, pre-digital analogue. What could they be proxies the ‘virtue’ Othello is projecting? for? Eddy makes visible certain kinds of resemblance and difference, certain sequences, patterns of influ- ence and distinctiveness. Critically understanding 5 these still depends on understanding ‘para- and Concluding Comments meta-texts’ (Li, Zhang and Liu, 2011). Viv’s contri- bution is even less certain: we won’t know whether Findings which only confirmed what was already its results correspond to anything ‘real’ about trans- known would be truly disappointing (though we lated texts’ qualities, or those of translations, or of do need some such confirmation, to have any translators, until we have studied many more cases. faith in digital tools). Digital literary studies Eddy and Viv analysis, as implemented, is crude. should provoke thought. A classic example is We can imagine training next-generation Eddy on Moretti’s discovery of a rhythm of 25–30 years in human-evaluated variant translations. We can en- the emergence and disappearance of C19 novelistic visage experiments with lemmatization, stopword genres, which he uneasily ascribed to a cycle of bio- exclusion, parsing, morphosyntactical tagging,32 di- logical-sociocultural ‘generations’: verse automated segment definitions, text analytics, I close on a note of perplexity: faute de mieux, and plugging in other corpora for richer analyses. some kind of generational mechanism seems When does a translator’s use of language mimic a

Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017 755

Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017 T. Cheesman et al.

pre-existing style, when is it innovative, in what www.lrec-conf.org/proceedings/lrec2004/pdf/707.pdf way? We can map texts to Wordnets, historical dic- (accessed 16 January 2016). tionaries and thesauri. We can model topics, analyse Babych, B., Hartley, A., and Atwell, E. (2003). Statistical sentiments. We can explore consistency and coher- modelling of MT output corpora for information ex- ence within translations, usage of less common traction. In Archer, D., Rayson, P., Wilson, A., and words, word-classes, word-sets, grammatical, rhet- McEnery, T. (eds), Proceedings of the Corpus orical, poetic, prosodic, metrical, metaphorical fea- Linguistics 2003 Conference, Lancaster University, 28– tures, and so on. We can generate intertextual and 31 March 2003, pp. 62–70. http://ucrel.lancs.ac.uk/pub- phylogenetic trees. We can perhaps adjust Viv for lications/CL2003/papers/babych.pdf (accessed 16 January 2016). historical sequence, and weight for the complex ef- fects of influence, imitation, and intentional non- Baker, M. (1993). Corpus linguistics and translation stu- imitation. Given multi-lingual parallel corpora, we dies: implications and applications. In Baker, M., can project a cross-cultural Viv. The more sophisti- Francis, G., and Tognini-Bonelli, E. (eds), Text and Technology: In Honour of John Sinclair. Amsterdam cated the analysis, the greater its scope, the greater and Philadelphia: John Benjamins, pp. 233–250. the cost of text preparation and annotation, and the greater the challenge in creating visual interfaces Baker, M. (2000). Towards a methodology for investigating the style of a literary translator. Target, 12(2): 241–66. which offer value to non-programmers. For text re- sources on a scale which might justify such invest- Baudissin, W. (1832). Shakspeares dramatische Werke. ment, we must next look to scripture. Then we will Vol. 8. Berlin: Reimer. need experts in God’s domain, as well. Berman, A. (1990). La Retraduction comme espace de traduction. Palimpsestes, 13: 1–7. Bodenstedt, F. (1867). Othello, der Mohr von Venedig. Funding Leipzig: Brockhaus. Bolte, H. and Hamblock, D. (1985). Othello: Englisch- This work was supported by Swansea University Deutsch. Stuttgart: Philipp Reclam. (Research Incentive Fund and Bridging the Gaps), Brownlie, S. (2006). Narrative theory and retranslation and the main phase of software development was theory. Across Languages and Cultures, 7(2): 145–70. funded by a 6-month Research Development Buhss, W. (1996). William Shakespeare Othello, Venedigs Grant in 2012 under the Digital Transformations Neger. Berlin: Henschel Schauspiel Theaterverlag. theme of the Arts and Humanities Research Cheesman, T. (2010). Shakespeare and Othello in Filthy Council (UK), reference AH/J012483/1. Hell: Zaimoglu and Senkel’s Politico-Religious Tradaptation. Forum for Modern Language Studies, 46(2): 207–20. References Cheesman, T. (2011). Thirty Times ‘More Fair Than Black’: Othello Re-translation as Political Re-statement. Algee-Hewitt, M., Allison, S., Gemma, M., Heuser, Angermion, 4: 1–52. R., Moretti, F., and Walser, H. (2016). Canon/ archive: large-scale dynamics in the literary field. Cheesman, T. (2015). Reading originals by the light of Literary Lab Pamphlet,Vol.11.http://litlab.stan- translations. Shakespeare Survey, 68: 87–98. ford.edu/LiteraryLabPamphlet11.pdf (accessed 16 Cheesman, T. (2016). Othello 1.3: ‘Far More Fair Than January 2016). Black’. In Smith, B. R. (ed.), The Cambridge Guide to Altintas, K., Can, F., and Patton, J. M. (2007). Language the Worlds of Shakespeare, vol. 2. The World’s change quantification using time-separated parallel Shakespeare, 1660-Present. Cambridge: Cambridge translations. Literary and Linguistic Computing, 22(4): University Press, pp. 1156–61. 375–93. Cheesman, T. and the VVV Project Team (2012). Babych, B. and Hartley, A. (2004). Modelling legitimate Translation Sorting with Eddy and Viv. http://www. translation variation for automatic evaluation of MT scribd.com/doc/101114673/Eddy-and-Viv (accessed 16 quality. Proceedings of LREC 2004, pp. 833–6. http:// January 2016).

756 Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017

Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017 Multi-retranslations

Cheesman, T., Flanagan, K., and Thiel, S. (2012–13). Halverson, S. L. (2008). Psycholinguistic and cognitive Translation Array Prototype 1: Project Overview. approaches. In Baker, M. and Saldanha, G. (eds), http://www.delightedbeauty.org/vvvclosed/Home/ Routledge Encyclopedia of Translation Studies. London Project (accessed 16 January 2016). and New York: Routledge, pp. 211–16. Cheesman, T., Flanagan, K., Thiel, S., and Rybicki, J. Hanna, S. (2016). Bourdieu in Translation Studies: The (2016), Five maps of translations of Shakespeare. In Socio-cultural Dynamics of Shakespeare Translation in Wiggin, B. and Macleod, C. (eds), Un/Translatables: Egypt. London and New York: Routledge. New Maps for Germanic Literatures. Evanston, IL: Hope, J. and Witmore, M. (2014). The language of Northwestern University Press, forthcoming. Macbeth. In Thompson, A. (ed.), Macbeth: The State Deane-Cox, S. (2014). Retranslation: Literature and of Play. London: Bloomsbury (Arden), pp. 183–208. Reinterpretation. London: Bloomsbury. House, J. (1997). Translation Quality Assessment: A Model Eder, M., Kestemont, M., and Rybicki, J. (2016). Revisited.Tu¨bingen: Narr. Stylometry with R: a package for computational text Hutchings, T. (2015). Studying apps: research analysis. R Journal, 16(1), forthcoming. approaches to the digital Bible. In Cheruvallil- Engel, E. (1939). William Shakespeare Othello. Berlin: Contractor, S. and Shakkour, S. (eds), Digital Felix Bloch Erben. Methodologies in the Sociology of Religion.London: Engler, B. (1976). Othello: Englisch-deutsche Bloomsbury, chapter 9. Studienausgabe. Munich: Franke. Ja¨nicke, S., Geßner, A., Franzini, G., Terras, M., Mahony, Farwell, D. and Helmreich, S. (2015). Pragmatics-based S. and Scheuermann, G. (2015). TRAViz: a visualization machine translation. In Chan, S. (ed.), The Routledge for variant graphs. Digital Scholarship in the Humanities, Encyclopedia of Translation Technology. Abingdon/New 30(Suppl 1): i83–99. http://dsh.oxfordjournals.org/ York: Routledge, pp. 167–85. content/30/suppl_1/i83 (accessed 16 January 2016). Felsenstein, W. and Stueber, C. (1964). Giuseppe Verdi: Johannson, S. (2011). Between Scylla and Charybdis: on Othello. Milan and Frankfurt am Main: Ricordi. individual variation in translation. Languages in Contrast, 11(1): 3–19. Flatter, R. (1952). Othello der Mohr von Venedig. Sonderabdruck fu¨rBu¨hnenzwecke. Munich: Theater- Kruger, A., Wallmach, K., and Munday, J. (2011). Verlag Desch. Corpus-based Translation Studies: Research and Applications. London: Continuum. Fried, E. (1972). Hamlet/Othello. Berlin: Wagenbach. Lapshinova-Koltunski, E. (2013). VARTRA: a compar- Geng, Z., Laramee, R. S., Cheesman, T., Ehrmann, E., able corpus for analysis of translation variation. and Berry, D. M. (2011). Visualizing translation vari- Proceedings of the 6th Workshop on Building and ation: Shakespeare’s Othello. Advances in visual com- Using Comparable Corpora, Sofia, Bulgaria, August 8, puting. Lecture Notes in Computer Science, 6938: 653–63. 2013, pp. 77–86. http://aclweb.org/anthology/W13- Geng, Z., Laramee, R.S., Flanagan, K., Thiel, S., and 2510 (accessed 16 January 2016). Cheesman, T. (2015). ShakerVis: visual analysis of seg- Laube, H. (1978). William Shakespeare Othello Der Mohr ment variation of German translations of Shakespeare’s von Venedig u¨bersetzt und bearbeitet. Frankfurt am Othello. Information Visualization, 14(4): 273–88. Main: Verlag der Autoren. Grant, C. B. (2007). Uncertainty and Communication: Lauterbach, E. S. (1973). William Shakespeare, Othello, New Theoretical Investigations. Basingstoke: Palgrave der Mohr von Venedig. Aus dem Englischen von E. S. Macmillan. Lauterbach unter Mitarbeit von Benita Gleisberg. Gundolf, F. (1909). Shakespeare in deutscher Sprache. Vol. Berlin: Henschel Schauspiel Theaterverlag. 1. Berlin: Bondi, 1920. Li, D., Zhang, C., and Liu, K. (2011). Translation style Gu¨nther, F. (1995). William Shakespeare. Othello. and ideology: a corpus-assisted analysis of two English Zweisprachige Ausgabe. Munich: Deutscher translations of Hongloumeng. Literary and Linguistic Taschenbuch Verlag. Computing, 26(2): 153–166. Hadas, E. (2015). Word2Dream: A Reader’s Companion. Long, L. (2007). Revealing commentary through com- http://eranhadas.com/word2dream (accessed 16 parative textual analysis: the realm of Bible translations. January 2016). Palimpsestes, 20: 35–57.

Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017 757

Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017 T. Cheesman et al.

Matthiessen, C.M.I.M. (2001). The environments of Rybicki, J. (2012). The great mystery of the (almost) in- translation. In Steiner, E. and Yallop, C. (eds), Beyond visible translator: stylometry in translation. In Oakley, Content: Exploring Translation and Multilingual Text M. and Ji, M. (eds.), Quantitative Methods in Corpus- Production. Berlin and New York: Mouton de Based Translation Studies. Amsterdam: John Benjamins, Gruyter, pp. 41–124. pp. 231–48. Mathijssen, J. (2007). The Breach and the Observance: Rybicki, J. and Heydel, M. (2013). The stylistics and styl- Theatre Retranslation as a Strategy of Artistic ometry of collaborative translation: Woolf’s ‘Night and Differentiation, with Special Reference to Day’ in Polish. Literary and Linguistic Computing, Retranslations of Shakespeare’s Hamlet (1777-2001). 28(4): 708–17. PhD dissertation, Utrecht University. Sayer, J. (2015). Wolf Graf Baudissin (1789-1878): Life and Moretti, F. (2003). Graphs, maps, trees: abstract models Legacy.Mu¨nster: LIT. for literary history 1. New Left Review, 24: 67–93. Schaller, R. (1959). Shakespeares Werke. Vol. 4. Berlin: Moretti, F. (2013). ‘Operationalizing’: or, the Function of Ru¨tten & Loening. Measurement in Modern Literary Theory. Literary Lab Schro¨der, R. A. (1962). Shakespeare/deutsch. Berlin, Pamphlet, 6. http://litlab.stanford.edu/LiteraryLab Frankfurt am Main: Suhrkamp. Pamphlet6.pdf (accessed 16 January 2016). Schwarz, H. (1941). Othello, der Maure von Venedig. Morini, M. 2014. The Pragmatic Translator: An Integral Typescript. Shakespeare-Bibliothek Mu¨nchen. Theory of Translation. London: Bloomsbury. Shei, C-C. and Pain, H. (2002). Computer-assisted teach- Motschach, H. (1992). William Shakespeare Othello. ing of translation methods. Literary and Linguistic Munich: Drei Masken. Computing, 17(3): 323–43. Mueller, M. (2003-14). About Metadata and the Query Steckel, F. (2012). Die Trago¨die von Othello, dem Mohren Potential of the Digital Surrogate. http://wordhoard. von Venedig. Frankfurt am Main: Verlag der Autoren. northwestern.edu/userman/index.html (accessed 16 January 2016). Swaczynna, W. (1972). Die Trago¨die von Othello, dem Mohren von Venedig. Cologne: Jussenhoven & Fischer. Munday, J. (2012). Evaluation in Translation. Critical Points of Translation Decision-making. Abingdon; New Thiel, S. (2010). Understanding Shakespeare: towards a visual York: Routledge. form for dramatic texts and language. http://www.under- standing-shakespeare.com (accessed 16 January 2016). Neill, M. (ed.) (2006). The Oxford Shakespeare: Othello. Oxford: Oxford University Press. Thiel, S. (2012). Othello map. http://othellomap.nand.io/ (accessed 16 January 2016). O’Driscoll, K. (2011). Retranslation through the Centuries: Jules Verne in English. Oxford: Peter Lang. Thiel, S. (2014a). Visualizing translation variation. http:// transvis.s3-website-eu-west-1.amazonaws.com/# or www. Paloposki, O. and Koskinen, K. (2010). Reprocessing tinyurl.com/transvis2014 (accessed 16 January 2016). texts: the fine line between retranslating and revising. Across Languages and Cultures, 11(1): 29–48. Thiel, S. (2014b). Visualizing Translation Variation: Designing Tools for Literary Scholars in Translation Penrose, R. (1989). The Emperor’s New Mind: Concerning Studies and Linguistics. Masters Dissertation. Bauhaus Computers, Minds and the Laws of Physics. Oxford: University Weimar. Oxford University Press. Thiel, S. (2015). Macbeth Loglikelihoods. http://macbeth. Roos, A. (2015). Making a Clean Breast of English s3-website-eu-west-1.amazonaws.com/or www.tinyurl. Passover Haggadah Translations: Data Visualization com/macbthe (accessed 16 January 2016). of Bowdlerization in Haggadah Translations of Ezekiel 16:7. Unpublished chapter of PhD dissertation, Toury, G. (2012). Descriptive Translation Studies—and University of Amsterdam. Beyond. Revised edition. Amsterdam and Philadelphia: Benjamin. Rothe, H. (1956). Der Elisabethanische Shakespeare. Vol. 4. Baden-Baden: Holle. Venuti, L. (2004). Retranslations: The Creation of Value. Bucknell Review, 47(1): 25–38. Ru¨diger, R. (1983). William Shakespeare Othello, der Mohr von Venedig Trago¨die. In Anlehnung an die Venuti, L. (2008). The Translator’s Invisibility: A History U¨bersetzung von Friedrich von Bodenstedt Nach dem of Translation. Revised edition. London and New York: Original neu u¨bersetzt. Berlin: Felix Bloch Erben. Routledge.

758 Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017

Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017 Multi-retranslations

Von Ledebur, R. (2002). Der Mythos vom deutschen 6. For details of the forty plus German texts used, see Shakespeare: die Deutsche Shakespeare-Gesellschaft www.delightedbeauty.org (‘German’ page). zwischen Politik und Wissenschaft 1918-1945. Cologne 7. ‘If virtue no delighted beauty lack, / Your son-in-law and Weimar: Bo¨hlau. is far more fair than black’ (Othello 1.3.287–8). Wachsmann, M. (2005). William Shakespeare, Die Multilingual translations of this are crowd-sourced Trago¨die von Othello, dem Mohr von Venedig. Berlin: by Cheesman at: www.delightedbeauty.org. Gustav Kiepenheuer Bu¨hnenvertriebs-Gmbh. 8. This remains less easy than we would wish. Roos is working with Eran Hadas on a more user-friendly Wang, Q. and Li, D. (2012). Looking for translators’ fin- corpus-creation, segmenting and aligning interface, gerprints: a corpus-based study on chinese translations in the course of a study of English translations of of ulysses. Literary and Linguistic Computing, 27(1): the Hebrew Haggadah from the C18 to now, also 81–93. using tools such as TRAViz (Ja¨nicke et al., 2015) Wolff, M. J. (1926). Shakespeares Werke u¨bertragen nach and Word2Dream (Hadas, 2015). See Roos, 2015, Schlegel-Tieck. Vol. 14. Berlin: Volksverband der and http://www.tinyurl.com/JewishDH. Bu¨cherfreunde, Wegweiser-Verlag. 9. Cheesman collated MIT’s ‘Moby’ Shakespeare (http:// Zaimoglu, F. and Senkel, G. (2003).William Shakespeare shakespeare.mit.edu) with Neill’s edition (2006) for Othello. Bearbeitung.Mu¨nster: Monsenstein und added dialogue and modern spellings. We chose to Vannerdat. sample Othello 1.3 partly because the English text is stable between editions, at the level of speeches and Zeynek, T. (?-1948). Shakespeare: Othello Der Mohr von speech prefixes, if not at the level of wording (except Venedig. Munich: Ahn und Simrock Bu¨hnen und at 1.3.275–6—see Neill, 2006, p. 232); also for its var- Musikverlag. iety of major character parts. Zimmer, H. (2007). Othello steht im Sturm: Jugendstu¨ck 10. http://www.juxtasoftware.org. Juxta helps map phyl- frei nach Shakespeare. Weinheim: Deutscher ogeny, with the aim of (re)constructing an original or Theaterverlag Weinheim. an authoritative edition. We cannot study retransla- tions with any such aim. There is no right translation. There may be a canonical translation, but users feel free to revise it, because it is ‘just’ a translation. Notes 11. The potential value of this interface to support ex- 1. ‘Version Variation Visualization: Translation Array plorations of text-analytic features is illustrated by the Prototype 1’ at http://www.delightedbeauty.org/ ‘Macbthe’ interface (Thiel, 2015): users explore a vvvclosed. Further project links: www.tinyurl.com/ zoomable map of ‘Macbeth’ with a log likelihood vvvex. Alternative prototype tools were also built: see lemma table, following the impetus of Hope and Geng et al., 2011, 2015. See further: Cheesman, 2015, Witmore (2014). See also Thiel’s (2010) earlier work. 2016, and Cheesman et al., 2016. 12. See: Eder et al. (2016) and stylometric translations 2. The existence of multilingual (re)translations can indicate analyses by Rybicki (2012) and Rybicki and Heydel both popularity and prestige, as in publishers’ blurbs for (2013). novels ‘translated into X languages’. For the Stanford 13. On the ‘fine line between retranslation and revision’ Literary Lab, translations index popularity (Algee- see: Paloposki and Koskinen, 2010. There is no re- Hewitt et al., 2016, p. 3). But ‘multiple’ retranslations search on Wolff, or indeed on most of the translators oftenalsomeanprestige:someareincludedininstitu- here. tional curricula, reviewed in ‘high-brow’ media, etc. 14. Cheesman namedP Eddy after (1) a formula he primi- 3. For example, 1,096 versions of the Bible in 781 lan- tively devised as ‘ D’, adapting tf.idf formulae (see: guages at www.bible.com or approx. 170 versions of Cheesman and the VVV Project Team, 2012, p. 3), (2) the Quran in forty-seven languages at http://al-quran. his brother Eddy, and (3) the idea that retranslations info. See Long (2007) and Hutchings (2015). are metaphorical ‘eddies’ in cultural historical flows. 4. Venuti (2004) focuses on retranslations which deliber- 15. Formulae available: A: Euclidean distance; B: ately challenge pre-existing translations. Our corpus is Cheesman’s original, primitive formula; C: Viv as not so restricted. standard deviation of Eddy; D: Dice’s coefficient; E: 5. See also Wang and Li (2012): digitally supported ana- angular distance. lysis of two Chinese translations of James Joyce’s 16. ‘A normalisation needs to be applied to compensate Ulysses. for the effect of text length, [so] we calculated

Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017 759

Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017 T. Cheesman et al.

variation for a large number of base text segments of 23. We thank a DSH referee for pointing out this varying lengths, then plotted average [Euclidean] possibility. Eddy value against segment length. We found a loga- 24. Roos (2015) similarly finds lower Viv value in Bible rithmic relationship between the two, and arrived at a quotations (the most canonical segments) in normalisation function that gives an acceptably con- Haggadah translations. sistent average Eddy value regardless of text length’ 25. Based on the two-line verse segments found manu- (Flanagan in Cheesman et al., 2012–13). Eddy for- ally, the length range was set at 60–100 characters. mula E (angular distance) appears to address the Iago’s lengthy prose speeches include more examples length normalization problem to some extent. than were segmented and aligned. 17. Stephen Ramsay commented on the ‘graceful and 26. Baudissin (five versions, 1855–2000) was added to the illuminating’ interface that ‘prompts various kinds corpus previously used, to recognize this translation’s of ‘‘noticing’’ and encourages an essentially playful enduring relevance. and exploratory approach to the ‘‘data’’’ (personal 27. See: Neill, 2006, p. 231. The MIT text (from an 1860s correspondence, 26 May 2014). edition) is quoted, but with Neill’s line-numbering. 18. Neill glosses ‘virtue’ as ‘moral excellence’, ‘manly 28. We also envisage training applications. An interface strength and courage’, and ‘inherent nature’ at enabling trainee translators and trainers to compare 1.3.287; ‘power, strength of character’ at 1.3.315 versions would have great practical value, as an ad- (Neill, 2006, p. 233 and 235—see there also for ‘fig’). junct to a computer-assisted translation system and/ 19. Roos (2015) uses Eddy and Viv to explore bowdler- or an assessment and feedback system. ization in English Haggadah texts. 29. Shakespeare retranslations are found at scattered sites. 20. Zeynek died in 1948; his translations are undated. Larger, curated corpora are accessible in Czech and 21. Stylometry and common sense recommended nar- Russian: c.400 aligned texts (twenty-two versions of rowing the corpus to give less ‘noisy’ results. I Hamlet) at http://www.phil.muni.cz/kapradi; c.200 excluded prose study versions, adaptations with ex- texts (twelve versions of Hamlet) at http://rus-shake. tensive omissions, contractions, expansions and add- ru/translations. itions, C18 and C19 versions, including all versions of 30. The term ‘surrogate’ is taken from Mueller (2003–14). Baudissin (1832), leaving fifteen versions: Gundolf Ideally our system would include page images. (1909), Schwarz (1941), Zeynek (?-1948), Flatter 31. Roos is working on this with Eran Hadas. (1952), Rothe (1956), Schaller (1959), Schro¨der 32. Difficulties include in-text variants (e.g. in critical (1962), Fried (1972), Swaczynna (1972), Laube editions, or translators’, directors’, and actors’ (1978), Ru¨diger (1983), Motschach (1992), Gu¨nther copies) and orthographic variations (archaic and vari- (1995), Buhss (1996), Wachsmann (2005). ously modernized forms; ad hoc forms fitting met- 22. The norm in German Shakespeare translation is that rical rules; other non-standard forms). Rather than formal variation in the original (prose, blank verse, standardize texts to facilitate comparisons, the ma- rhymed verse, or another metrical scheme) should be chine should learn to recognize underlying replicated or analogously marked. Roos (2015) reports equivalences. similar findings for the Haggadah: rhyming verse sec- tions have higher Viv, if translators use rhyme.

760 Digital Scholarship in the Humanities, Vol. 32, No. 4, 2017

Downloaded from https://academic.oup.com/dsh/article-abstract/32/4/739/2669776 by Swansea University user on 27 November 2017