Arxiv:1904.08386V1 [Cs.LG] 17 Apr 2019 in Works of Literature

Casting Light on Invisible Cities: Computationally Engaging with Literary Criticism Shufan Wang Mohit Iyyer University of Massachusetts, Amherst University of Massachusetts, Amherst [email protected] [email protected] Abstract Adelma (cities & the dead) Eusapia (cities & the dead) … An old man was loading a basket of … And to make the leap from life to sea urchins on a cart; I thought I death less abrupt, the inhabitants Literary critics often attempt to uncover mean- recognized him …he looked like a have constructed an identical copy of fisherman who, already old when I their city, underground… They say that ing in a single work of literature through care- was a child, could no longer be every time they go below they find among the living… Adelma is the city something changed in the lower ful reading and analysis. Applying natural lan- where you arrive dying and where Eusapia; the dead make innovations guage processing methods to aid in such lit- each finds again… in their city; not many, but surely… erary analyses remains a challenge in digital Zobeide (cities & desire) Isidora (cities & memory) humanities. While most previous work fo- …men of various nations had an When a man rides a long time through cuses on “distant reading” by algorithmically identical dream. They saw a woman wild regions he feels the desire for a running at night through an unknown city… seashells… perfect telescopes discovering high-level patterns from large col- city… They dreamed of pursuing her… and violins…He was thinking of all decided to build a city like the one in these things when he desired a city. lections of literary works, here we sharpen the the dream… they settled, waiting for Isidora, therefore, is the city of his focus of our methods to a single literary theory that scene to be repeated… dreams… about Italo Calvino’s postmodern novel Invis- ible Cities, which consists of 55 short descrip- Figure 1: Calvino labels the thematically-similar cities tions of imaginary cities. Calvino has provided in the top row as cities & the dead. However, although a classification of these cities into eleven the- the bottom two cities share a theme of desire, he assigns matic groups, but literary scholars disagree as them to different groups. to how trustworthy his categorization is. Due to the unique structure of this novel, we can computationally weigh in on this debate: we use recent advances in text representation learning leverage pretrained contextualized representa- to test a single literary theory about the novel In- tions to embed each city’s description and use visible Cities by Italo Calvino. unsupervised methods to cluster these embeddings. Additionally, we compare results of Framed as a dialogue between the traveler our computational approach to similarity judg- Marco Polo and the emperor Kublai Khan, Invis- ments generated by human readers. Our work ible Cities consists of 55 prose poems, each of is a first step towards incorporating natural lan- which describes an imaginary city. Calvino cat- guage processing into literary criticism. egorizes these cities into eleven thematic groups that deal with human emotions (e.g., desires, 1 Introduction memories), general objects (eyes, sky, signs), Literary critics form interpretations of meaning and unusual properties (continuous, hidden, thin). arXiv:1904.08386v1 [cs.LG] 17 Apr 2019 in works of literature. Building computational Many critics argue that Calvino’s labels are not models that can help form and test these interpre- meaningful, while others believe that there is a dis- tations is a fundamental goal of digital humani- tinct thematic separation between the groups, in- ties research (Benzon and Hays, 1976). Within cluding the author himself (Calvino, 2004). The natural language processing, most previous work unique structure of this novel — each city’s de- that engages with literature relies on “distant read- scription is short and self-contained (Figure1)— ing” (Jockers, 2013), which involves discover- allows us to computationally examine this debate. ing high-level patterns from large collections of As the book is too small to train any models, stories (Bamman et al., 2014; Chaturvedi et al., we leverage recent advances in large-scale lan- 2018). We depart from this trend by showing that guage model-based representations (Peters et al., computational techniques can also engage with lit- 2018a; Devlin et al., 2018) to compute a repre- erary criticism at a closer distance: concretely, we sentation of each city. We feed these representations into a clustering algorithm that produces ex- actly eleven clusters of five cities each and evalu- learned clusters Zoe Dorothea Zirma ate them against both Calvino’s original labels and Fedora Clustering crowdsourced human judgments. While the over- algorithm Dorothea all correlation with Calvino’s labels is low, both Zoe Fedora computers and humans can reliably identify some Zirma avg mean pooling thematic groups associated with concrete objects. While prior work has computationally analyzed ELMo a single book (Eve, 2019), our work goes be- yond simple word frequency or n-gram counts by Travelers return from the city of Zirma… leveraging the power of pretrained language models to engage with literary criticism. Admittedly, Figure 2: We first embed each city by averaging token our approach and evaluations are specific to Invis- representations derived from a pretrained model such ible Cities, but we believe that similar analyses as ELMo. Then, we feed the city embeddings to a clus- of more conventionally-structured novels could tering algorithm and analyze the learned clusters. become possible as text representation methods improve. We also highlight two challenges of applying computational methods to literary criti- group and the content of its descriptions. Bloom cisms: (1) text representation methods are imper- (2002) claims that the “cities are totally inter- fect, especially when given writing as complex as changeable”; Springer(1985) agrees, stating that Calvino’s; and (2) evaluation is difficult because “even the categories themselves seem both cho- there is no consensus among literary critics on a sen and assigned arbitrarily”. Teichert(1985) con- single “correct” interpretation. tends that “the catalogue is superimposed on, but does not cover, the elusive, fluid mass of an un- 2 Literary analyses of Invisible Cities written world”. While out of scope for our computational anal- Before describing our method and results, we first ysis, many possible theories exist regarding why review critical opinions on both sides of whether the groupings appear largely incoherent. For in- Calvino’s thematic groups meaningfully charac- stance, Boeck(2004) posits that the structural in- terize his city descriptions. coherence exists because all of the cities actually describe different facets of Marco Polo’s home- The groups are meaningful: Some scholars town of Venice. Breiner(1988) argues instead that believe that the thematic grouping imposed by Calvino’s labels “may refer only to a projection Calvino reflects properties of the cities he de- of the Khan’s occupational thirst for order, unre- scribes; Vrbani(2012), for example, argues that lated to the structure of the text”, while Knowles Calvino’s structure are “ontologically grounded (2015) hypothesizes that the mismatch is one of in different ways”. Buitendijk(2018) further many obstacles that readers need to “untangle” to provides examples of cities with the same label understand the central substance of the novel. that are clearly thematically similar, pointing at the “cities of desire” as “informed by 20th cen- 3 A Computational Analysis tury theories of desires associated with Sigmund Freud”. Calvino(2004) himself claims that he cre- We focus on measuring to what extent computers ates most categorizations of cities with clear la- can recover Calvino’s thematic groupings when bels in mind, especially the cities of memory and given just raw text of the city descriptions. At desire, which he deemed as “fundamental corner- a high level, our approach (Figure2) involves stones” of the novel. However, many critics argue (1) computing a vector representation for every that authorial intent is irrelevant when analyzing city and (2) performing unsupervised clustering of literature (Wimsatt and Beardsley, 1946; Barthes, these representations. The rest of this section de- 1994). scribes both of these steps in more detail. The groups are arbitrary: On the other hand, 3.1 Embedding city descriptions a large body of criticism focuses on the appar- While each of the city descriptions is relatively ent mismatch between a city’s assigned thematic short, Calvino’s writing is filled with rare words, complex syntactic structures, and figurative lan- Method Purity Accuracy 1 guage. Capturing the essential components of Random 0.32 33.3 each city in a single vector is thus not as simple GloVe 0.35 35.9 as it is with more standard forms of text. Nev- BERT 0.40 39.3 ertheless, we hope that representations from lan- ELMo 0.42 44.6 guage models trained over billions of words of Human - 48.8 text can extract some meaningful semantics from these descriptions. We experiment with three dif- Table 1: Results from cluster purity and accuracy on ferent pretrained representations: ELMo (Peters the “odd-one-out” task suggests that Calvino’s the- et al., 2018a), BERT (Devlin et al., 2018), and matic groups are not completely arbitrary. GloVe (Pennington et al., 2014). To produce a single city embedding, we compute the TF-IDF set of N data points, weighted element-wise mean of the token-level 1 X 2 purity = max jm \ dj: representations. For all pretrained methods, we N d2D additionally reduce the dimensionality of the city m2M embeddings to 40 using PCA for increased com- patibility with our clustering algorithm. 4 Evaluating clustering assignments While the results from the above section allow 3.2 Clustering city representations us to compare our three computational methods against each other, we additionally collect human Given 55 city representations, how do we group judgments to further ground our results.

Load more