VCWE: Visual Character-Enhanced Word Embeddings Chi Sun, Xipeng Qiu,∗ Xuanjing Huang Shanghai Key Laboratory of Intelligent Information Processing, Fudan University School of Computer Science, Fudan University 825 Zhangheng Road, Shanghai, China fsunc17,xpqiu,
[email protected] Abstract et al., 2014; Cao and Rei, 2016; Sun et al., 2016a; Wieting et al., 2016; Bojanowski et al., 2016). Chinese is a logographic writing system, and Compositionality is more critical for Chinese, the shape of Chinese characters contain rich since Chinese is a logographic writing system. In syntactic and semantic information. In this pa- Chinese, each word typically consists of fewer per, we propose a model to learn Chinese word characters and each character also contains richer embeddings via three-level composition: (1) a convolutional neural network to extract the semantic information. For example, Chinese char- intra-character compositionality from the vi- acter “休” (rest) is composed of the characters for sual shape of a character; (2) a recurrent neural “º” (person) and “(” (tree), with the intended network with self-attention to compose char- idea of someone leaning against a tree, i.e., rest- acter representation into word embeddings; ing. (3) the Skip-Gram framework to capture non- Based on the linguistic features of Chinese, re- compositionality directly from the contextual cent methods have used the character informa- information. Evaluations demonstrate the su- tion to improve Chinese word embeddings. These perior performance of our model on four tasks: word similarity, sentiment analysis, named en- methods can be categorized into two kinds: tity recognition and part-of-speech tagging.1 1) One kind of methods learn word embeddings with its constituent character (Chen et al., 2015), 1 Introduction radical2 (Shi et al., 2015; Yin et al., 2016; Yu et al., 2017) or strokes3 (Cao et al., 2018).