Tracing Antisemitic Language Through Diachronic Embedding Projections: France 1789-1914

Rocco Tripodi Massimo Warglien Simon Levis Sullam Deborah Paci Ca’ Foscari University of Venice {rocco.tripodi, warglien, levissmn, deborah.paci}@unive.it

Abstract seems to be harder than expected. In fact, while Bolukbasi et al.(2016) and Zhao et al.(2018) We investigate some aspects of the history of demonstrated that it is possible to debias specific antisemitism in France, one of the cradles of gendered-words, even after the debiasing proce- modern antisemitism, using diachronic word dure, the geometry of the embeddings remains embeddings. We constructed a large corpus of French books and periodicals issues that con- almost the same with respect to non gendered- tain a keyword related to Jews and performed words (Gonen and Goldberg, 2019), preserving a diachronic word embedding over the 1789- their original bias. 1914 period. We studied the changes over In this work, we turn these biases to the histo- time in the semantic spaces of 4 target words rian’s advantage and shed light on some aspects of and performed embedding projections over 6 the history of antisemitism in France during the so streams of antisemitic discourse. This allowed called long XIX century, between the French Rev- us to track the evolution of antisemitic bias in the religious, economic, socio-politic, racial, olution and the First Word War, using diachronic ethic and conspiratorial domains. Projections word embedding. This technique allows to cap- show a trend of growing antisemitism, espe- ture diachronic conceptual changes and to analyse cially in the years starting in the mid-80s and stereotyped historical biases. We tracked how his- culminating in the Dreyfus affair. Our analysis torical events and publications influenced the con- also allows us to highlight the peculiar adverse struction of the collective imaginary related to the bias towards Judaism in the broader context of Jewish question. other religions. We assume that words do not have a fixed mean- 1 Introduction ings. They can be used in different contexts to evoke a great variety of meanings using different Word embeddings are widely used in many Nat- connotational nuances. These multiple meanings ural Language Processing (NLP) tasks. They pro- are acquired (or lost) over time in correspondence vide a machine-interpretable representation of lex- to specific socio-political events. For example, ical features. Their effectiveness in representing one of the meanings of the word usurier (i. e.; words semantics consists essentially in the abil- money lender), as reported by the French Histor- ity of learning association patterns in the training ical Dictionary, refers to: the financial activities arXiv:1906.01440v1 [cs.CL] 4 Jun 2019 dataset. For this reason the learned representations of the Jews [who since the Middle Ages were], contain human-like biases (Caliskan et al., 2017). the only ones authorised to lend on pawns (Rey These biases can be detected easily and can be re- et al., 2010). This association derives from the fact lated to gender, ethic or racial aspects (Garg et al., that especially between the XVI and the XIX cen- 2018; Voigt et al., 2017). tury, this word acquired a negative connotation, Since the use of word embedding is ubiquitous nurtured by anti-Jewish prejudice and stereotyping in many commercial products such as search en- developing from the idea of an illegitimate interest gines and machine translators, the research com- attached to this activity. This image, as the above munity has introduced different techniques to de- mentioned definition explains, was also fixed in bias them (Bolukbasi et al., 2016; Zhao et al., the collective imaginary by Shylock, the Jewish 2018), especially under the gender dimension. protagonist in Shakespeare’s Merchant of Venice Despite these efforts debiasing word embeddings (1598). In this work, we trace the conceptual changes bossarsky et al.(2015): it states that prototypical of words related to the Jewish question. We col- words, words that are near to the centroid of a clus- lected a large corpus for this purpose, composed ter in a semantic space, change slower than words of thousands of books and newspapers published that are in a peripheral position. The laws of con- in France between 1789 and 1914. We used di- formity, innovation and prototipicality have been achronic word embedding to represent the data, questioned by Dubossarsky et al.(2017), who used measures of local changes in the semantic space controlled conditions to test them. of different words, and embedding projections Different works that tried to measure, directly to quantify biases in different semantic spheres. or indirectly, cultural drifts have been proposed re- The measurement of local changes is particularly cently. Garg et al.(2018) analysed gender and eth- suited for our study because we do not want to nic stereotypes in the United States during the 20th identify new meanings in the words related to the and 21st centuries, using word embeddings trained Jewish question, instead we want to trace how on the Google Books and Corpus of Historical the context of their use changed and how these American English (COHA) corpora. Kozlowski changes affected the representation of Jews at the et al.(2018) used diachronic word embeddings time of the rise of modern antisemitism. Measur- to conduct macro-cultural investigation of social ing biases over time is particularly interesting be- stereotypes. Kutuzov et al.(2017) attempted to cause it allows to connect them with antisemitic model the dynamics of wordwide armed conflicts streams as identified by historians in the field (Wil- using word embeddings trained on a news corpus. son, 1982) and operationalised by us. Zhao et al.(2017) analyzed the amplification ef- fect that learning models present on the gender di- 2 Related Work mension when trained on biased data.

Models for capturing diachronic conceptual 3 Motivations and historical background changes are associated with the distributional hypothesis (Harris, 1954; Firth, 1957; Weaver, We have looked at linguistics representation of 1955): the semantics of a word is defined by the Jews in 19th century France, which was one of context in which it is used. Following this as- the cradles of modern antisemitism in Europe, i.e. sumption, different models have been presented, of the mostly secularized and racial transformation based on co-occurrence vectors (Sagi et al., 2009; of the centuries-old Christian prejudice against the Gulordava and Baroni, 2011; Basile et al., 2016) Jews (Katz, 1980). or word embeddings (Kim et al., 2014; Kulkarni Since the entry and gradual integration of the et al., 2015; Hamilton et al., 2016a). Jews in French society after the Revolution of These works are brought together by the idea 1789, the appearance of anti-Jewish texts, the of analysing the contexts in which a word occurs rise of public controversies, or the burst of cases and have culminated in the measures of seman- and scandals in which Jews were supposedly in- tic shift and cultural drift, proposed by (Hamilton volved marked the emergence and spread of the et al., 2016a) and the law prototipicality proposed Jewish question on the French scene, in what by Dubossarsky et al.(2015). Semantic shifts have been called antisemitic moments or episodes are regular linguistic processes such as semantic (Birnbaum, 2011). Especially during the Third widening (e.g., dog, that in Middle English was Republic, beginning in 1870, references to Jews used to refer to dogs of a particular breed) (Bloom- entered the French public discourse in relation to field, 1933). This measure was used to derive two a supposed growing influence of the Jews on polit- laws of semantic change: the law of conformity: ical and economic affairs, the rise of anticlerical- semantic change scales with a negative power of ism in the face of Catholic France (for which Jews word frequency; and the law of innovation: poly- were considered responsible), the accusation of an semous words have significantly higher rates of se- alliance between Jews and Freemasonry. mantic change (Hamilton et al., 2016b). Cultural This process reached its climax with the Drey- drifts involve local changes to a lexical form’s use fus affair (1894), the unfounded accusation against (e.g.: the changes in the meaning of the word cell: a French army officer to have sold intelligence in- prison cell → cell phone)(Hamilton et al., 2016a). formation to the German enemy (Dreyfus would The law of prototipicality was introduced by Du- be exonerated in 1906): the affair caused the heavy spread of antisemitic accusations and anti-Jewish accusations, prejudices and stereotypes would movements of opinion (Wilson, 1982). Different christallize - or reach its climax - in the Dreyfus streams of antisemitism ran accross French society affair. We suggest that the usage in print (books throughout this time, together with a pro-Jewish and periodicals) of the term juif or other terms re- reaction driven by the supporters of Dreyfus’ in- lated to the Jewish question, all characterised by nocence (Kalman, 2010). an adverse bias, was especially connected to an- The publication in 1845 (republished in 1886) tisemitic tendencies. However, we should note of Alfred Toussenel’s Les Juifs rois de l’epoque that this vocabulary was also present at the time caused especially the rise of the so-called eco- in Biblical and theological scholarship, art and art- nomic antisemitism, which accused the Jews of historical publications, fictional and theatrical lit- an increasing economic and financial influence, of erature, medical treatises and the rising social sci- which the Rothschilds were considered the protag- ences. References to Jews in the public discourse onists and became a symbol. This accusation was were therefore not necessarily mobilised in a po- later confirmed by the supposed Jewish role in the litical context with explicit antisemitics aims. Our crash of the Catholic bank Union Gen´ erale´ (1882) investigation asks whether using diachronic word and in the Panama corruption scandal (1892), to- embeddings trained on a large corpus confirms gether with the revival of nationalism tied to the the chronological development of antisemitic lan- Boulangist crisis (Sternhell, 1998). These events guage which historians have described on a quali- generated a resurgence of antisemitism. In re- tative level (and if it sheds light on different, pre- sponse to the growing secularization and anticler- viously ignored, antisemitic moments). We also icalism, French Catholics revived an ancient tradi- examine the relevance of the semantic areas or tion of religious antisemitism, marked in this time streams in relation to the Jew which we have iden- by the appearance of works such as Gougenot des tified based on (Wilson, 1982), and we show the Mousseaux’s Le Juif, le juda¨ısme et la juda¨ısation trends through time of unfavourable biases to- des peuples chretiens´ (1869) and by the anti- wards Jews in the period considered. Jewish campaigns of Catholic periodicals such as L’Univers and La Croix. 4 The Corpus and the Embeddings In 1886 the journalist Edouard Drumont pub- 4.1 The corpus lished the hugely successful La France juive. Es- The corpus1 was constructed downloading from sai d’histoire contemporaine, which described a Gallica, the online library of the Bibliotheque` Na- French society under a greedy Jewish influence tionale de France2, the raw text of all the resources and control, painting in the style of a novelist (in- that contain a keyword related to Jews (see ap- spired by Balzac and by contemporary feuilletons pendixA for the complete list of keywords) and or serialized novels) the contours of Jewish con- have been published between 1789-1914. The re- spiracies. Although the subtitle of the work sug- search was further restricted to those resources gests an essay of contemporary history, on read- that have an OCR quality higher that 98%. The re- ing it is as if one is before an enormous cauldron sulting corpus contains 54.403 books and 245.188 of common place assumptions on Jews which in- periodicals issues. It is important to notice here cludes Catholic, social, racial, economic, and con- that we downloaded the full text of a book or news- spiratorial anti-Semitism. The success of his work paper issue even if a keyword appeared only once depended on the waves it made in the intellectual in it. milieu of the era and its impact on the popular Figures 1a and 1b indicate the distribution of re- masses attracted by the synthesis of anti-Semitism sources per year in the periodicals and books sub- of the right, of a church worried about laicisation, corpora, respectively, together with the total num- and anti-Semitism of the left, anti-capitalist and ber of resources in Gallica. The resources distri- laical. This and other books by Drumont mixed bution per year is not homogenous in neither sub- Catholic, socio-political, ethic and conspirationist corpora: publications increase significantly year antisemitism, accusing Jews of all sorts of reli- gious offenses, political machinations, moral per- 1The metadata of the corpus, the embeddings and the versions and secret plots (Kauffmann, 2008). code used for the experiments can be downloaded from https://github.com/roccotrip/antisem. The combination of these streams of anti-Jewish 2https://gallica.bnf.fr/ (a) Periodicals distribution (b) Books distribution (c) Tokens distribution (d) Num. of tokens in each bin

Figure 1: Distribuition of resources in the corpus and time bins division. by year. Several hypotheses can explain this pro- 5 Analysis liferation of documents over time. One straightfor- In this section we analyse the resulting embed- ward hypothesis can be related to increasing im- dings. First we study the changes in the seman- portance of Jews in the French public debate with tic space of 4 target words. Then we analyse the the proliferation of anti-Semitic movements and biases of the same words for 6 different dimen- newspapers such as La Croix, La libre parole, La sions, each of which corresponds to a predeter- Lutte antijuive and L’Intransigeant, just to name a mined stream. few. Yet, a second hypothesis can be related to the fact that the print industry grew over time. In fact, 5.1 Local changes many newspapers and publishers were founded af- The first analysis that we conducted is the ter 1825. For example, Hachette, the publisher measurement of the changes in the seman- with the largest number of books in our corpus tic space of the words used to refer to (1558), was founded in 1826. The newspapers Le Jews: juif (noun/adjective, masculine, singular), Figaro was founded in 1826, L’Univers in 1833 juifs (noun/adjective, masculine, plural), juive and Le Temps in 1861. Figure 1a and 1b, plotting (noun/adjective, feminine, singular) and juives our corpus compared to the whole Gallica one, (noun/adjective, feminine, plural). For this mea- seems to suggest that the second hypothesis is the surement we used the local neighborhood measure most plausible. In fact, the quantities of resources proposed by Hamilton et al.(2016a). To compute in our corpora follow a trend similar to those ob- this measure it is necessary to create a second or- served in the whole Gallica. der vector, s, according to equation1,

t (t) (t) (t) si = cos-sim(w , w )∀wj ∈ Nk(w )∪ i j i (1) 4.2 The embeddings (t+1) Nk(wi ),

Figure 1c shows the distribution of tokens per year ( ) where N w t represents the k-nearest neigh- distinguishing periodicals and books. The greater k( i ) bours (k nn) at time (t) (according to cosine part of the data is from the periodicals, giving to − similarity) of a target word w and w is the em- the corpus a focus on the contemporaneity. Given i ∗ bedding corresponding to word w . Once these this distribution it is impossible to train a model ∗ vectors are constructed we compute the cosine dis- using equally sized time bins. For this reason, tance, d, among them to quantify their differences, we decided to group the data into approximately with equation2, equal bins in terms of tokens. The resulting divi- sion comprehend 26 time bins of 450 millions ≈ d st1, st2 1 cos-sim st1, st2 . (2) tokens each (see Figure 1d). ( i i ) = − ( i i ) For each bin we trained a word2vec skip-gram The results of this experiment are presented in model (Mikolov et al., 2013) using a window size Figure2 for all the morphological variants of the 3 of 5 words on both sides, a word vector of 300 di- word juif, using k = 100 . What emerges clearly mensions and removing the words that occur less 3We noticed that the general trend of the curves in Figure than 25 times. 2 does not change much using different values of k (10, 25, (a) juif (b) juifs (c) juive (d) juives

Figure 2: Local neighborhood measure. The y axes indicates the cosine distance of the second-order vector constructed for each time period compared to the 1789 (blu line) and the preceding time period (red line).

juif juifs juive juives ⤸ 1841 ¹ ⤸ 1861 ¹ ⤸ 1874 ¹ ⤸ 1870 ¹ laquedem juive crucifient juif huguenots juda¨ıque syriennes negociantes´ mecr´ eant´ juda¨ıque schismatiques israelites¨ favorite musulmane iraniennes samaritaines rogatons rabin juda¨ısants juive opera syrienne musulmanes refugi´ ees´ blasphemateur´ bouddhiste fetichistes´ rabbins rigoletto heroine´ israelites¨ ascetes` ⤸ 1886 ¹ ⤸ 1870 ¹ ⤸ 1886 ¹ ⤸ 1880 ¹ ghetto juda¨ıque juda¨ısants juif drumont iranienne israelites´ epous´ ees´ deicides´ rabin her´ etiques´ synagogues antisemitisme´ apostasie´ musulmanes lutheriennes´ francmac¸on wanderghen cabalistes talmud circoncis lithuanienne femmes turques aryen anabaptiste luciferiens´ sanhedrin´ the´atrale` puritaine cel´ ebrations´ dissolues ⤸ 1893 ¹ ⤸ 1897 ¹ ⤸ 1893 ¹ ⤸ 1897 ¹ deicide´ talmud antisemites´ samaritains juiverie synagogue juif dissolues youtre bouddhiste youtres talmud satanisme heroine´ youtres baptisees´ francmac¸on sodomite youpins idolatresˆ monogamique lapidee´ antijuives prostituaient youpins anabaptiste enjuives´ pharisiens opprimee´ persecutrice´ antisemitiques´ ascetes` ⤸ 1897 ¹ ⤸ 1905 ¹ ⤸ 1901 ¹ ⤸ 1905 ¹ youtre rabin juda¨ısants synagogues stigmatisant dragonnade massacrees´ courtisannes semite´ usurier hellenisants´ talmud antijuive torturee´ terrorisees´ pa¨ıennes juda¨ısant shylock diaspora pharisiens antinationale puritaine diaspora prostituaient antisemite´ anabaptiste massacrant ismaelites´ dreyfusiste anabaptiste deport´ ees´ emigr´ ees´

Table 1: Words that have been introduced (left column ⤸) or eliminated (right column ¹) for our 4 target words in time periods with a high local neighborhood distance, compared to 1789. from these figures is that there are certain peri- synagogue and sanhedrin´ (i.e., the Jewish coun- ods of time in which the relation among a target cil) are replaced by more negatively connotated word and its local neighbourhood changed consis- words such as ghetto, deicides´ , antisemites´ and an- tently. What we noticed from them is that besides tijuives. From the few words presented in Table1 changes in the relative similarity among two words one can also notice a possible rise of antisemitic what changes more is the k-nn itself, with the in- prejudice (or at least of antisemitic language), with troduction or elimination of specific words. the introduction of specific words in the vocabu- Some of the words that were introduced (or lary specifically tailored to connote Jewish people eliminated) to (from) the k-nn of relevant time pe- in a derogatory way. Youtre and youpin are slang riods (according to local neighbourhood measure) racist insults negatively connoting the Jew. They are presented in Table1. The words in this ta- appear increasingly during the period 1880-1900. ble are ordered according to the cosine similarity Other terms with a negative connotation that en- with the target word. We can see an elimination tered the semantic area of our target words are of words related to the religious domain for all the juda¨ısants (i.e., judaizers), enjuives´ (i.e., strongly target words that we used, terms like rabbin (i.e., influenced by the Jewish spirit) and francmac¸on rabbi), talmud (i.e. the study of the Jewish law), (i.e., freemason). These terms, as we will see in the next section, are related to the idea of a Jewish 50, 100) and that fixing k = 100 gives a good representation of the variations over time. Increasing this value gives high conspiracy against the world. This is a clear ex- fluctuations and introduces many irrelevant words. ample of the growth of the antisemitic vocabulary. The analysis of the word juive is especially in- about gender. In this work we do not want to teresting. The word drumont entered its space in project words only according to a single direc- the time period 1886-1889. It refers to Eduard´ tion but we want to analyse different adverse and Drumont, a well known antisemite who published (or) favourable biases, comparing them over time. one of the bestsellers of the antisemitism (La For this reason, we defined six different seman- France juive), in 1886, and was the editor of an tic axes, that correspond to six antisemitic streams antisemitic newspaper (La libre Parole), founded (S)(Wilson, 1982). in 1892. We can also notice that in the semantic For each stream, s ∈ S, we identi- space of the word juive there are different words fied a set of n antonyms pairs, zs = neg pos neg pos related to theatre. This probably derives from {(a1 , a1 ), ..., (an , an )} to construct literary and theatre representations of Jewish fe- the bias subspace in the embedding. To avoid male characters, as well as references to supposed selection biases we selected the antonyms pairs Jewish inappropriate moral and sexual behaviours. starting from a positive seed word, that is Among the theatre representations we may recall highly representative for the stream, and used that of La Juive, first shown in 1835, one of the a knowledge base to collect its synonyms and most popular French operas of the 19th century, the corresponding antonyms (see appendixB for which tells the story of an impossible love affair the complete list of antonyms used). We noticed between a Christian man and a Jewish woman. that computing the PCA of each subspaces the The fictional Jew, invariably seen as an outsider, corresponding explained variance is concentrated provides a mirror for the phobias and obsessions of on the first component and that it is stable over French society at a time when old Jew hatred be- time. For example, the first component of the comes politicised, when anti-Semitism begins to racial stream has an explained variance of 0.34 permeate French ideology (Weinberg, 1983; Hall- (mean) with a standard deviation of 0.012. man, 2007; Samuels, 2009). The six different semantic areas, which may We can also see the introduction of the word correspond to related antisemitic discourses are: aryen (i.e.: aryan) in 1886. This word entered the 1. religious: antisemitism based on theological semantic space in a syntagmatic relation with the doctrines or narratives, and on religious prej- word juif and, as we will see in the next section, udices and accusations. The seed word is be- the period in which it entered is characterised by liever (unbeliever); a strong antisemitism characterised by an intensi- fication of racial and socio-political stereotypes. 2. economic: antisemitism based on a supposed Jewish role in the economy or on stereotypes concerning Jews’ economic behaviours. The seed word is generosity (greed); Figure 3: Semantic axis and projections. 3. socio-political: antisemitism based on malev- olent, e.g. anti-national, political behaviours 5.2 Embedding projections or on supposedly threatening Jewish actions. 5.2.1 The streams The seed word is honor (shame); To quantify biases in word embeddings semantic 4. racial: antisemitism based on the definition spaces it is common to project a specific word of Jews as a race, considered inferior. The vector on a semantic axis (Bolukbasi et al., 2016; seed word is pure (impure); Caliskan et al., 2017). The semantic axis can be computed as g = wi − wj and its projection as the 5. conspiratorial: antisemitism based on con- ˆ dot product b = w⋅g, assuming that the vectors are spiracy theories. The seed word is loyal (dis- normalised, the projection is equal to the cosine loyal); similarity. The higher the values of the projection, 6. ethic: antisemitism based on Jewish sup- the more biased the word is toward that direction. posed unethical or perverse morals or be- In previous literature (Bolukbasi et al., 2016) haviours. The seed word is moral (immoral); the gender direction (e.g., he⃗ − she⃗ ) was used to project words related to occupations in or- To quantify the biases for all the time we com- der to quantify if these words embed information puted the mean bias, b, for each stream as the Figure 4: Projections of our 4 target words to the 6 semantic axes. Positive values indicates the adverse bias. arithmetic mean of the individual biases, ˆb on each ing in the 1880s, before the Dreyfus affair. We axis, according to equation3: also notice an unexpected peak in adverse bias between 1855 and 1866, in connection with the 1 n b wi, s wi w neg w pos , (3) French Second Empire (1851-1870). The seman- ( ) = Q ⋅ ( aj − aj ) n j=1 tic areas or streams in relation to the Jew identified where n is the number of antonyms pairs in stream on the basis of (Wilson, 1982) seem relevant for antisemitic mo- s, Given the ordering of the antonyms in the com- the description of adverse bias in ments. The highest adverse bias characterises the putation of the bias axis (g = waneg − wapos ) we j j religious semantic area, followed by the economic define an adverse bias when b is positive and a and ethic areas. The religious adverse bias shows favourable bias when b is negative. a peak starting in 1855, after the establishment of An example of semantic axis constructed with Napoleon III’s Second Empire, a time of renewed the pair disloyal as negative word and loyal as allegiance to the Catholic Church and in 1895 at positive, is presented in Figure3( disloyal⃗ − the beginning of the Dreyfus affair. Also the eco- disloyal⃗ ). From this figure we can see that words nomic adverse bias shows a peak starting in 1855, that have a high projection value are words very perhaps because of the increase of economic dis- similar to the negative word, on the other hand, course on Jews following the publication of Tou- words with a low projection are very similar to the ssenel’s Les Juifs rois de l’epoque´ , and again coin- word on the other side. The projection tells us if a ciding with the establishment of the Second Em- word is closer to one extreme or the other. Unbi- pire. Another peak comes with the Dreyfus af- ased words should have a projection close to 0. fair. The ethic adverse bias peaks in the period 5.2.2 Biases related to Jews 1830-1855, diminishes afterward and peaks again The results of this experiment are presented in Fig- toward the end of the Dreyfus affair. ure4. The adverse bias is always high for the Racial, conspiratorial and especially sociopolit- words juif, juifs and juive. For the word juives ical semantic areas show a steady adverse bias and only on few cases it is negative. Adverse and an increase mostly after 1886, i.e. after the publi- favourable biases are measured with positive and cation of Drumont’s La France juive (1886). The negative measures respectively. conspiratorial adverse bias also peaks – like the re- Our analysis confirms the chronological devel- ligious, the economic and the ethic adverse bias – opment of antisemitic moments identified by histo- in 1855. rians, with a steady increase of adverse bias start- The singular juif prevails in the conspiratorial Figure 5: Cumulative bias projections compared to different religious groups.

Figure 6: Cumulative bias projections for other words used to refer to Jews. and socio-political semantic areas, which seem to a general picture of the biases at each time step. entail general statements about the Jew. This ten- Juif and catholic have a completely opposite dency has been noticed by historians as typical of bias: exclusively adverse in the first case, entirely modern antisemitism and has been called singular- favorable in the latter. Confronting juif and protes- isation (Miccoli, 2003). This underlines that there tant we notice a similar bias, adverse in the first are features common to all [Jews], because in all case, favorable in the latter. But the favorable and every one there emerges something which con- bias of Protestant is much more reduced than that stitutes a common and exclusive feature of the Jew of catholic. Confronting juifs and protestantes, as the enemy to be defeated (Miccoli, 2003). On both show an adverse bias (lower in the case of the other hand, the plural juifs prevails in the eco- Protestants). The adverse bias concerns protes- nomic and ethic areas, as implying collective be- tantes especially in relation to the religious do- haviours of Jews. main. Musulman and musulmans also show an ad- Racial, sociopolitical and conspiratorial seman- verse bias concentrated in the religious sphere. If tic areas show a steady adverse bias and increase we look at racial stream, this grows for juif(s) ref- especially after 1886. As the racist vision of the erence to protestants is absent; while there is an Jew increases, it is turned increasingly into a po- occasional emergence of musulman, with an ad- litical vision, and it is also nourished by a con- verse bias between 1789 and 1840, when ques- spirationist worldview, which will culminate in the tions of citizenship are being defined (France con- Dreyfus affair. quers Egypt in 1798 and in 1834 Algeria is an- nexed to France; in 1870 the Cremieux´ Decree 5.2.3 Comparative biases concerning granted French citizenship to Algerian Jews but different religious groups not to Muslims), and a favourable bias in 1891- The results of this experiment are presented in 95 (in 1890 a bill is proposed for the granting of Figure5. They show a comparison with three French citizenship to Algerian muslims, see Weill, different religious groups: Catholic, (catholique), 2005). The last increase is probably also con- Protestant (protestante) and Muslim (musulman). nected with the availability of a larger quantity of The plots sum positive and negative biases to give digitised North-African press in the corpus. (a) (b) (c)

Figure 7: Target words frequency.

5.2.4 Comparative bias concerning different remains steady during and after the affair. words used to refer to Jews The highest adverse bias characterises the re- The results of this experiment are presented in Fig- ligious semantic area, followed by the economic ure6. Israelite and israelites do not show a partic- and ethic spheres. The conspiratorial and sociopo- ular bias as the terms are often used euphemisti- litical areas show an adverse bias more often as- cally (including by Jews themselves), i.e. pre- sociated with the singular juif, as if they provoked ferred to the more direct and connotated juif and categorical statements. Adverse bias in the eco- juifs. These terms refers to the cultural assimila- nomic and ethic areas is expressed through the plu- tion and social integration of Jews into French so- ral juifs as describing collective behaviours. ciety, as described by Honore´(1981). The confrontation between juif and catholic The slang and derogatory youpin spread starting shows an entirely adverse bias in the first case and around 1886; its shows an exclusively adverse bias an entirely favorable bias in the latter case. The and a trend similar to juif, as if the terms juif and adverse bias towards other minorities, i.e. Protes- youpin were interchangeable. tants and Muslims concerns the religious semantic area. No bias concerning protestant emerges in the 5.3 Target words frequency racial semantic area, while a negative and positive Even if the corpus has been constructed selecting bias emerge in relation to Muslims at times when documents containing words related to the Jew- the question of French citizenship is being defined. ish question, we noticed that the frequencies of As one evaluates the presence of the word juif, words related to other religious groups is higher and the semantic areas surrounding it, one should for catholique and catholiques and slightly lower also consider that these may emerge in texts which for the words protestant, protestantes, musulman are not antisemitic per se, but still contribute to the and musulmanes. The frequencies of all the target spread of images of Jews, with specific biases. We wordsare reported in Figure 7a, 7b and 7c. refer here especially to literary texts. We suggest that the adverse bias in various se- 6 Conclusions mantic areas may be associated with antisemitic discourses, but this association should be further References to Jews increase throughout the 19th explored though an examination of the historical century, as Jews were integrated within French so- context (for example that of antisemitic moments) ciety and these references appear to be mostly as- or an analysis of the textual sources which spread sociated with an adverse bias in all semantic ar- the words associated with the Jew. eas. The adverse bias grows starting in the mid- 1880s, i.e. in the second half of the Third Repub- Acknowledgments lic, when the rise of anticlericalism and socialism was associated with Jews by the conservative and The authors of this work have received fund- catholic public opinion. Around this time the pub- ing from the European Union’s Horizon lication of Drumont’s La France juive provokes 2020 research and innovation programme under an adverse bias towards Jews clearly associated to grant agreement No 732942. The experiments antisemitic discourse in all semantic areas, which have been run on the SCSCF cluster of Ca’ Fos- prepares the outburst of the Dreyfus affair, and it cari University. References Proceedings of the Conference on Empirical Meth- ods in Natural Language Processing. Conference on Pierpaolo Basile, Annalina Caputo, Roberta Luisi, and Empirical Methods in Natural Language Process- Giovanni Semeraro. 2016. Diachronic analysis of ing, volume 2016, page 2116. NIH Public Access. the italian language exploiting google ngram. CLiC it, page 56. William L Hamilton, Jure Leskovec, and Dan Juraf- Pierre Birnbaum. 2011. The anti-semitic moment: A sky. 2016b. Diachronic word embeddings reveal tour of France in 1898. Chicago University Press. statistical laws of semantic change. arXiv preprint arXiv:1605.09096. Leonard Bloomfield. 1933. Language. George Allen and Unwin LTD. Zellig S Harris. 1954. Distributional structure. Word, 10(2-3):146–162. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Jean-Paul Honore.´ 1981. Le vocabulaire de Man is to computer programmer as woman is to l’antisemitisme´ en france pendant l’affaire dreyfus. homemaker? debiasing word embeddings. In Ad- Mots. Les langages du politique, 2(1):73–92. vances in neural information processing systems, pages 4349–4357. Julie Kalman. 2010. Rethinking antisemitism in nineteenth-century France. Cambridge University Aylin Caliskan, Joanna J Bryson, and Arvind Press. Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Jacob Katz. 1980. From prejudice to destruction: anti- Science, 356(6334):183–186. Semitism, 1700-1933. Harvard University Press. Haim Dubossarsky, Yulia Tsvetkov, Chris Dyer, and Edouard´ Drumont Eitan Grossman. 2015. A bottom up approach to Gregoire´ Kauffmann. 2008. . Per- category mapping and meaning change. In Net- rin. WordS, pages 66–70. Yoon Kim, Yi-I Chiu, Kentaro Hanaki, Darshan Hegde, Haim Dubossarsky, Daphna Weinshall, and Eitan and Slav Petrov. 2014. Temporal analysis of lan- Grossman. 2017. Outta control: Laws of semantic guage through neural language models. arXiv change and inherent biases in word representation preprint arXiv:1405.3515. models. In Proceedings of the 2017 conference on empirical methods in natural language processing, Austin C Kozlowski, Matt Taddy, and James A Evans. pages 1136–1145. 2018. The geometry of culture: Analyzing mean- ing through word embeddings. arXiv preprint John R Firth. 1957. A synopsis of linguistic theory, arXiv:1803.09288. 1930-1955. Studies in Linguistic Analysis. Vivek Kulkarni, Rami Al-Rfou, Bryan Perozzi, and Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and Steven Skiena. 2015. Statistically significant de- James Zou. 2018. Word embeddings quantify tection of linguistic change. In Proceedings of the 100 years of gender and ethnic stereotypes. Pro- 24th International Conference on World Wide Web, ceedings of the National Academy of Sciences, pages 625–635. International World Wide Web Con- 115(16):E3635–E3644. ferences Steering Committee.

Hila Gonen and Yoav Goldberg. 2019. Lipstick on a Andrey Kutuzov, Erik Velldal, and Lilja Øvrelid. pig: Debiasing methods cover up systematic gender 2017. Temporal dynamics of semantic relations biases in word embeddings but do not remove them. in word embeddings: an application to predict- arXiv preprint arXiv:1903.03862. ing armed conflict participants. arXiv preprint arXiv:1707.08660. Kristina Gulordava and Marco Baroni. 2011. A dis- tributional similarity approach to the detection of semantic change in the google books ngram cor- Giovanni Miccoli. 2003. Antiebraismo, antisemitismo: un nesso fluttuante. In Les racines chretiennes´ de pus. In Proceedings of the GEMS 2011 workshop ´ on geometrical models of natural language seman- l’antisemitisme´ politique, pages 1000–1021. Ecole tics, pages 67–71. franc¸aise de Rome.

Diana R Hallman. 2007. Opera, Liberalism, and An- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- tisemitism in Nineteenth-Century France: The Pol- rado, and Jeff Dean. 2013. Distributed representa- itics of Halevy’s´ La Juive. Cambridge University tions of words and phrases and their compositional- Press. ity. In Advances in neural information processing systems, pages 3111–3119. William L Hamilton, Jure Leskovec, and Dan Jurafsky. 2016a. Cultural shift or linguistic drift? comparing Alain Rey, Chantal Tanet, and Marianne Tomi. 2010. two computational measures of semantic change. In Dictionnaire historique de la langue franc¸aise. Eyal Sagi, Stefan Kaufmann, and Brady Clark. 2009. • Judas (i.e: Judass) Semantic density analysis: Comparing word mean- ing across time and phonetic space. In Proceedings • Moloch (i.e: the biblical name of a Canaanite of the Workshop on Geometrical Models of Natu- god associated with child sacrifice) ral Language Semantics, pages 104–111. Associa- tion for Computational Linguistics. • Ahasverus (i.e: a mythical immortal man Maurice Samuels. 2009. Inventing the Israelite: Jew- whose legend began to spread in Europe in ish Fiction in Nineteenth-Century France. Stanford the 13th century. The original legend con- University Press. cerns a Jew who taunted Jesus on the way to Zeev Sternhell. 1998. La droite revolutionnaire:´ les the Crucifixion and was then cursed to walk origines francaises du fascisme 1885-1914. Galli- the earth until the Second Coming.) mard. B Bias axes Rob Voigt, Nicholas P Camp, Vinodkumar Prab- hakaran, William L Hamilton, Rebecca C Hetey, The list of antonyms used to compute the bias Camilla M Griffiths, David Jurgens, Dan Jurafsky, axes. Note that the translation of the antonyms and Jennifer L Eberhardt. 2017. Language from pairs is provided only for the singular. We used a police body camera footage shows racial dispari- ties in officer respect. Proceedings of the National public resource (http://www.synonyms-fr.com) to Academy of Sciences, 114(25):6521–6526. collect antonyms relations. Warren Weaver. 1955. Translation. Machine Transla- Religious angel, devil; sacred, profane; pious, tion of Languages, 14:15–23. atheist; pious, pagan; pious, idolater; pious, im- Henry H Weinberg. 1983. The image of the jew in late pious; sacred, cursed; venerable, abject; faithful, nineteenth-century french literature. Jewish Social unfaithful; believer, unbeliever; religious, irreli- Studies, 45(3/4):241–250. gious; dedicated, atheist. Stephen Wilson. 1982. Ideology and experience: anti- Economic give, appropriate; generosity, greed; semitism in France at the time of the Dreyfus affair. generous, greedy; generous, miserly; generous, Fairleigh Dickinson University Press; London: As- sociated University Presses. stingy.

Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Or- Socio-political prodigal, greedy; honest, rabble; donez, and Kai-Wei Chang. 2017. Men also like honor, shame; friendly, hostile; loyal, deceitful; shopping: Reducing gender bias amplification us- socialist, capitalist; friend, enemy; ally, antago- ing corpus-level constraints. In Proceedings of the nist; conservative, progressive. 2017 Conference on Empirical Methods in Natural Language Processing. Racial normal, strange; superiority, inferiority; Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai- equality, inequality; pleasant, unpleasant; benign, Wei Chang. 2018. Learning gender-neutral word wicked; worthy, infamous; sympathy, hate; ac- embeddings. arXiv preprint arXiv:1809.01496. cepted, refused, better, worse; national, foreign; pure, impure; upper, lower; pure, filthy; clean, A Keywords dirty. • Juif (i.e: Jew - masculine, singular) Conspiratorial loyal; spy; honesty, treason; • Juive (i.e: Jew - feminine, singular) loyal, disloyal; clear, mysterious; obvious, oc- cult; sincere, deceitful; sincere, unfair; benefactor, • Judaisme (i.e: Judaism) criminal; clear, secret; friendly, threatening; clear, dark. • Israelite¨ (i.e: Israelite) Ethic chastity, lust; modest, intriguing; decent, • Israel¨ (i.e: Israel) indecent; virtuous, lascivious; faithful, unfaith- ful; moral, immoral; honest, dishonest; chaste, de- • Israelitisme¨ (i.e: Israelitism) praved; chaste, fleshly; pure, degenerate. • Mosa¨ısme (i.e: religions referred to the mes- sage of Moses)

• Talmud (i.e: Talmud)