Context-aware Adversarial Training for Name Regularity Bias in Named Entity Recognition Abbas Ghaddar, Philippe Langlais†, Ahmad Rashid and Mehdi Rezagholizadeh Huawei Noah’s Ark Lab, Montreal Research Center, Canada †RALI/DIRO, Université de Montréal, Canada
[email protected],
[email protected] [email protected],
[email protected] Abstract Gonzales, Louisiana LOC In this work, we examine the ability of NER GonzalesP ER is a small city in Ascension models to use contextual information when Parish, Louisiana. predicting the type of an ambiguous entity. Obama, Fukui We introduce NRB, a new testbed carefully LOC designed to diagnose Name Regularity Bias ObamaP ER is located in far southwestern of NER models. Our results indicate that all Fukui Prefecture. state-of-the-art models we tested show such Patricia A. Madrid a bias; BERT fine-tuned models signifi- MadridP ER won her first campaign in 1978 .. cantly outperforming feature-based (LSTM- LOC CRF) ones on NRB, despite having com- Asda Jayanama parable (sometimes lower) performances on P ER standard benchmarks. AsdaORG joined his brother, Surapong ... To mitigate this bias, we propose a novel model-agnostic training method which Figure 1: Examples extracted from Wikipedia (ti- adds learnable adversarial noise to some tle in bold) that illustrate name regularity bias entity mentions, thus enforcing models to focus more strongly on the contextual in NER. Entities of interest are underlined, gold signal, leading to significant gains on types are in blue superscript, model predictions are NRB. Combining it with two other training in red subscript, and context information is high- strategies, data augmentation and parameter lighted in purple.