A Computational Approach for Semantic Change Research

HEBREW UNIVERSITY OF JERUSALEM Semantic change at large: A computational approach for semantic change research Thesis for the degree "Doctor of Philosophy" in Brain Sciences: Computation and Information Processing by: Haim DUBOSSARSKY Submitted to the Senate of the Hebrew University of Jerusalem April 2018 iii This work was carried out under the supervision of Doctor Eitan GROSSMAN and Professor Daphna WEINSHALL v HEBREW UNIVERSITY OF JERUSALEM Abstract Edmond and Lily Safra Center for Brain Sciences Doctor of Philosophy Semantic change at large: A computational approach for semantic change research by Haim DUBOSSARSKY The search for hitherto undiscovered regularities and laws of semantic change is today made possible by two recent and independent developments: first, the upsurge in the availability of digitized historical corpora of texts; and sec- ond, novel computational methods that allow the automatic semantic processing of these corpora. Compared to traditional research methods in semantic change, this dynamic duo of massive corpora and innovative computational methods has the potential to lead to novel insights about semantic change, both in discovering regular patterns of change and in identifying their explanatory factors, based on large-scale data-driven analysis. In a series of studies, I demonstrate that this approach is not only feasible, but also fruitful for semantic change research. My research papers show that this approach extends the scope of research beyond known phenomena of semantic change, uncovering several regularities of semantic change, and complements existing theories with more objective and reliable analyses. However, these studies have also demonstrated the importance of methodological issues. As this field of research has just emerged, fundamental methodological concerns were raised which must be addressed in order to make an objective, reliable, and genuine contribution to the research field. Two papers were dedicated to tackle these methodological issues, and their results promise to put this computational approach on a solid methodological footing right from the beginning. The studies not only call into question earlier results, but also provide clear, feasible, and replicable validation routines to ensure objective and reliable testing. Importantly, these methodological accentuations may benefit the NLP community at large, as they have applications that go beyond semantic change research. vii Letter of Contribution Chapter 2 - Verbs change more than nouns: A bottom-up computational approach to semantic change. Haim Dubossarsky, Daphna Weinshall and Eitan Grossman. HD proposed the idea for the main analysis, processed the corpora, trained the model, and performed the critical analysis. DW supervised the analyses. HD and EG interpreted the results and proposed a linguistic theory for them. HD and EG wrote the manuscript: HD the technical parts to communicate them to the linguistic community, and EG the linguistic-theoretical parts. Chapter 3 - A bottom up approach to category mapping and meaning change. Haim Dubossarsky, Yulia Tsvetkov, Chris Dyer and Eitan Grossman HD proposed the theoretical research question. YT and CD positioned it in the right computation-theoretic background, and suggested the model that would allow to investigate this question. HD processed the corpora, trained the model, made the diachronic comparisons, and devised the critical analysis. HD, YT and CD interpreted the results on the computational side, and EG contributed the pivotal linguistic interpretation. HD and EG wrote the manuscript, and YT wrote part of the methodology. Chapter 4 - Outta Control: Laws of Semantic Change and Inherent Biases in Word Representation Models. Haim Dubossarsky, Eitan Grossman and Daphna Weinshall. HD speculated about the magnitude of the problem in the current research, and together with DW outlined the required experiments to test it as a research question. HD planned and carried out the experiments, analyzed their results, replicated the results of previous studies, and conducted the critical comparisons. HD interpreted the results together with DW. DW contributed to the theoretical part. HD, DW and EG wrote the manuscript, and emphasize its critical importance to the research field. Chapter 5 - Coming to Your Senses: on Controls and Evaluation Sets in Polysemy Research. Haim Dubossarsky, Matan Ben-Yosef, Eitan Grossman and Daphna Weinshall. HD identified the misconception in the current research, and suggested to phrase it as a research hypothesis. HD and DW devised the set of experiments to investigate the hypothesis. HD collected and preprocessed the corpora, trained the models, and conducted the critical comparisons. HD and MB replicated the results of previous models, and manipulating their code for the critical simulations. HD analyzed the results from all the experiments and simulations. DW contributed to the theoretical part. HD and DW interpreted the results, and drew conclusions. HD, DW and EG wrote the manuscript, and positioned the conclusions in the right research context. ix “In the beginning was the Word, and the Word was with God, and the Word was God.” John, 1:1. xi Contents Abstractv 1 Introduction1 1.1 Semantic change...........................1 1.2 Research paradigms in semantic change.............2 1.2.1 Overview..........................2 1.2.2 The traditional paradigm.................3 1.2.3 The quantitative paradigm................5 1.2.4 The computational paradigm...............6 1.3 Research objectives......................... 10 1.4 Current work............................ 12 References................................. 14 2 The Diachronic Word-Class Effect 17 3 The Diachronic Prototypicality Effect 39 4 Avoiding methodological pitfalls in semantic change research 45 5 Sense-specific word representation, a misconception? 57 6 Discussion and Conclusions 69 References................................. 74 1 Chapter 1 Introduction At the heart of this PhD dissertation is a new research paradigm for the study of semantic change, i.e., changes in the meaning of lexical items. This paradigm is characterized by a focus on large-scale, data-driven analyses based on modern computational tools, in order to address research questions stemming from linguistic research. This approach complements existing paradigms, and in so doing, enriches the toolbox of semantic change research. At the same time, this dissertation addresses fundamental methodological issues that are crucial not only for the emerging field of NLP research on semantic change, but for research in NLP more broadly. In particular, this dissertation constitutes a contribution to methodological issues inherent to research in the framework of DISTRIBUTIONAL SEMANTICS. The structure of this chapter is as follows. Section 1.1 provides a definition of semantic change. Section 1.2 presents essential background about semantic change research. Section 1.3 states our research objectives in light of the needs that are derived from current research state. Section 1.4 briefly describes our research papers, and how they contribute to the research field in accordance with the objectives outlined before. 1.1 Semantic change The meanings of words – or, more precisely, lexical items – are liable to change over time. For example, girl originally used to denote a child of either sex, but since the 15th century only refers to a young female (Bybee 2015, p. 202). Broadcast used to describe a method of scattering seeds less than a hundred years ago, but nowadays refers to the transmission of information by radio or television. The phenomenon of changes in the meaning of words over time is part of what is called semantic change, and is known at least since the work of Reisig (1839). Semantic change, to use a classic definition, may 2 Chapter 1. Introduction be described as “innovations which change the lexical meaning rather than the grammatical function of a form” (Bloomfield 1933, p. 425)1. A formal definition of semantic change would be as follows: given a historical corpus that contains texts from n different time periods in a chrono- logical order [ , , ..., ], semantic change is any difference between the Ct1 Ct2 Ctn meanings associated with a given word at t1, t2 ... tn. A model of semantic change aims first of all to discover which words changed their meaning between any two time-periods. Given this information, additional tasks can be undertaken in order to answer further questions, such as the size or extent of the changes, their characteristics, and their dependence on other factors. 1.2 Research paradigms in semantic change 1.2.1 Overview Until recently, research on semantic change was largely based on a traditional paradigm that used philological tools informed by linguistic analysis. For the most part, linguists interested in semantic change have studied changes in the denotation or connotation of individual words or constructions (or small groups thereof) as they occur in historical corpora. The introduction of modern quantitative and computational tools to the field did not change the basically TOP-DOWN nature of this research paradigm. While a top-down research approach allows a fine-grained examination of semantic change, it also imposes limitations on what one can learn about semantic change. The fundamental limitation is the size and the representa- tiveness of the dataset to be generalized over. Since previous studies were done on a SMALL SCALE and were based on ultimately anecdotal

Load more