Detecting Inappropriate Messages on Sensitive Topics that Could Harm a Company’s Reputation∗ Nikolay Babakovz, Varvara Logachevaz, Olga Kozlovay, Nikita Semenovy, and Alexander Panchenkoz zSkolkovo Institute of Science and Technology, Moscow, Russia yMobile TeleSystems (MTS), Moscow, Russia fn.babakov,v.logacheva,
[email protected] foskozlo9,
[email protected] Abstract texts which are not offensive as such but can ex- press inappropriate views. If a chatbot tells some- Not all topics are equally “flammable” in terms thing that does not agree with the views of the com- of toxicity: a calm discussion of turtles or pany that created it, this can harm the company’s fishing less often fuels inappropriate toxic di- alogues than a discussion of politics or sexual reputation. For example, a user starts discussing minorities. We define a set of sensitive top- ways of committing suicide, and a chatbot goes ics that can yield inappropriate and toxic mes- on the discussion and even encourages the user to sages and describe the methodology of collect- commit suicide. The same also applies to a wide ing and labeling a dataset for appropriateness. range of sensitive topics, such as politics, religion, While toxicity in user-generated data is well- nationality, drugs, gambling, etc. Ideally, a chat- studied, we aim at defining a more fine-grained bot should not express any views on these subjects notion of inappropriateness. The core of inap- propriateness is that it can harm the reputation except those universally approved (e.g. that drugs of a speaker. This is different from toxicity are not good for your health).