Supervised Visualization of Vocabulary Knowledge Towards Explainable Support of Second Language Learners

Supervised Visualization of Vocabulary Knowledge towards Explainable Support of Second Language Learners Yo Ehara Shizuoka Institute of Science and Technology, 2200-2, Toyosawa, Fukuroi, Shizuoka, Japan. [email protected] Abstract easy to understand and, hence, are well studied. However, in second language learning, most gaps are related to mean- In second language learning, it is crucial to identify gaps in ing and semantics and are inherently abstract. Hence, visu- knowledge of the language between second language learn- alizing these gaps is essential to make these gaps intuitively ers and native speakers. Such a gap exists even when learning a single word in a second language. As the semantic broad- understandable. ness of a word differs from language to language, language The broadness of a word, or how a word can be used in learners must learn how broadly a word can be used in a lan- the language to express different concepts, is one such ab- guage. For example, certain languages use different words for stract gap (Read 2000). Because the meaning of a word dif- “period” in “a period of time” or “period pains” yet both are fers from language to language, when learning a word in a nouns. Learners whose native languages are such languages second language, there typically exists a gap between what typically have only partial knowledge of a word, even though learners think the word means and how the word is actu- they think they know the word “period,” producing a gap be- ally used in the language. Polysemous words are examples tween them and native speakers. Language learners typically want explanations for these word usage differences, which that are easy to understand: “book” can mean an item asso- even native speakers find it difficult to explain and find it ciating with reading, or it can mean to make a reservation. costly to annotate. To support language learners in noticing Other than these examples, to which the part-of-speech tag- these challenging differences easily and intuitively, this pa- ging techniques in natural language processing (NLP) seem per proposes a novel supervised visualization of the usages applicable, some examples are more subtle: some languages of a word. In our method, the usages of an inputted word in always use different words for “time” in “in a short time” large corpora written by native speakers are visualized, tak- or “for a time,” in which the word “time” refers to a period, ing the semantic proximity between the usages into account. and “time and space” or “time heals all wounds,” in which Then, for the single inputted word, our method makes a per- “time” is used as an abstract concept. In another example, sonalized prediction of word usages that each learner may many languages use different words for “period” in “a pe- know, based on his/her results of a quick vocabulary test, which takes approximately 30 minutes. The experiment re- riod of time”, and “period” in “period pains”. In this way, sults show that our method produces better usage frequency the granularity of the word’s senses should be distinguished counts than raw usage frequency counts in predicting vocab- for second language acquisition, as it varies from word to ulary test responses, implying that word usage prediction is word. accurate. Polysemous words encode different concepts in one word: hence, they have been one of the central topics in knowl- Introduction edge engineering. A substantial amount of work has been conducted to automatically recognize polysemous words for Acquiring a second language requires repeated efforts to practical applications by using machine learning, includ- narrow the gap between language learners’ knowledge of the ing those in the previous AAAI-MAKE workshops (Ram- language and that of native speakers. Making such gaps intu- prasad and Maddox 2019; Hinkelmann et al. 2019; Lau- itively understandable greatly helps language learners self- renzi et al. 2019). However, even among few such applica- teach the language and also helps researchers build effec- tions for second language acquisition (Heilman et al. 2007; tive language tutoring systems. Some gaps such as vocabu- Dias and Moraliyski 2009) in the artificial intelligence (AI) lary size, or time spent in language learning are intuitively community, the challenging problem of different granularity of the word’s senses in second language acquisition has not Copyright c 2020 held by the author(s). In A. Martin, K. Hinkel- been addressed. In second language acquisition, as learners mann, H.-G. Fill, A. Gerber, D. Lenat, R. Stolle, F. van Harmelen (Eds.), Proceedings of the AAAI 2020 Spring Symposium on Com- are typically not linguistic experts, i.e., novices, hence, sys- bining Machine Learning and Knowledge Engineering in Practice tems to support their learning need to be intuitively under- (AAAI-MAKE 2020). Stanford University, Palo Alto, California, standable. Our goal is to make the gaps among word usages USA, March 23-25, 2020. Use permitted under Creative Commons intuitively understandable, even for novice language learn- License Attribution 4.0 International (CC BY 4.0). ers. To this end, this paper proposes a novel supervised visualization method for word usages to assist in learning the different usages of a word. Our method first searches all usages of the target word in a large corpus written by native speakers. Then, it calculates the vector representation of each usage, or occurrence, of each word by using a contex- tualized word embedding method (Devlin et al. 2019). Con- textualized word embedding methods (Peters et al. 2018; Devlin et al. 2019) are recently proposed methods to embed each occurrence of a word, capturing the context of each usage of the word. Then, our method is trained to visualize the contextual- ized word embedding vectors by plotting each usage as a point in a two-dimensional space. Unlike a typical visualization method that merely projects the vectors to a two- dimensional space, our method is trained to fit and visu- ally explain a given supervision dataset. This means that the same vectors are visualized in different ways if the supervision dataset differs. Here, the supervisions are a vocabulary test result dataset that consists of a matrix-format data, recording which learner answered correctly/incorrectly to which word question. The method visualizes the areas a Figure 1: Usage of “haunt” predicted to be familiar to the learner. learner user may know by classifying each usage point in the visualization into known/not known to the learner. This classification is conducted in a personalized manner because learners’ language skills and specialized fields are different. The learner only needs to take a 30-minute vocabulary test for this purpose. Figure 1 shows an example visualization using our method. “To haunt” has two different meanings in English, the first being “to chase” and the other “to curse,” or to be af- fected by ghosts or misfortune. Each point shows the usage Figure 2: An example of concordancer. of the word in a corpus written by native speakers. The differences in point colors indicate whether they are predicted to be known to the learner. The right side of the figure, within the dotted curve, is predicted to be known to the learner. In Related Work this way, our method visualizes the semantic area the learner Explainable machine learning studies knows. While deep learning-based methods outperformed conven- Our contribution is as follows: tional machine learning methods such as support vector ma- chines (SVMs) in many tasks, parameters of deep learn- • For second language vocabulary learning, we propose ing methods are typically more difficult to interpret com- a novel supervised visualization model that captures pared to those in conventional models. To this end, in the word broadness via a personalized prediction of learner’s machine learning and artificial intelligence community, a knowledge of usages. number of methods have been proposed to extract explanations from trained machine-learning models, or training • As our visualization uses a vocabulary test result dataset models taking explainability into accounts (Ribeiro, Singh, as supervisions, learners can understand which usage of and Guestrin 2016; Koh and Liang 2017; Lundberg and Lee the inputted word is predicted to be known/not known to 2017; Ribeiro, Singh, and Guestrin 2018). him/her. Unlike previous methods that output automatic However, the purpose of these methods is to explain explanation of machine-learning models, our method is machine-learning models to help machine-learning engi- much more intuitive and novice-friendly for language neers and researchers in understanding the models. Obvi- learners in the sense that language learners do not need ously, second language learners are usually not machine- to know about machine learning models. learning engineers and researchers. Therefore, methods of these studies have different purposes, and it is difficult to • We evaluated our method in terms of predictive accuracy apply these methods to help their understanding of the mod- of vocabulary test result dataset and achieved better re- els. Language learners are typically even not interested in sults compared to baselines. the models. Rather, learners’ interests reside in understanding their current learning status and what they should learn to improve it. Hence, to meet learners’ needs, a model is de- Concordancer studies sirable for a learner to see his/her current learning status and While our proposed method is novel as a visualization, what he/she needs to learn in the near future. software tools that search the usages of an inputted word for educational purposes and display them itself are not Word Embedding Visualization Studies novel: such software is known as concordancers. Concor- dancers target learners, educators, and linguists as primary Word embedding techniques are techniques that have been users.

Load more