The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI-21) Bridging the Domain Gap: Improve Informal Language Translation via Counterfactual Domain Adaptation Ke Wang,1,2 Guandan Chen,3 Zhongqiang Huang,3 Xiaojun Wan,1,2 Fei Huang3 1Wangxuan Institute of Computer Technology, Peking University 2The MOE Key Laboratory of Computational Linguistics, Peking University 3DAMO Academy, Alibaba Group fwangke17,
[email protected] fguandan.cgd, z.huang,
[email protected] Abstract A natural solution to this problem is domain adaptation, Despite the near-human performances already achieved on which aims to adapt NMT models trained on one or more formal texts such as news articles, neural machine transla- source domains (formal texts in our case) to the target do- tion still has difficulty in dealing with ”user-generated” texts main (informal texts in our case). Domain adaptation meth- that have diverse linguistic phenomena but lack large-scale ods can be roughly categorized into model-based and data- high-quality parallel corpora. To address this problem, we based ones (Chu and Wang 2018). The model-based meth- propose a counterfactual domain adaptation method to better ods make explicit changes to NMT model such as designing leverage both large-scale source-domain data (formal texts) new training objectives (Luong, Pham, and Manning 2015; and small-scale target-domain data (informal texts). Specifi- Chu, Dabre, and Kurohashi 2017), modifying the model ar- cally, by considering effective counterfactual conditions (the chitecture (Kobus, Crego, and Senellart 2017; Johnson et al. concatenations of source-domain texts and the target-domain 2017; Dou et al.