Towards Conversational Recommendation over Multi-Type Dialogs
Zeming Liu1,∗ Haifeng Wang2, Zheng-Yu Niu2, Hua Wu2, Wanxiang Che1,† Ting Liu1 1Research Center for Social Computing and Information Retrieval, Harbin Institute of Technology, Harbin, China 2Baidu Inc., Beijing, China {zmliu, car, tliu}@ir.hit.edu.cn {wanghaifeng, niuzhengyu, wu hua}@baidu.com
Abstract Almost all these work focus on a single type of dialogs, either task oriented dialogs for recommen- We propose a new task of conversational rec- dation, or recommendation oriented open-domain ommendation over multi-type dialogs, where conversation. Moreover, they assume that both the bots can proactively and naturally lead a conversation from a non-recommendation di- sides in the dialog (especially the user) are aware alog (e.g., QA) to a recommendation dialog, of the conversational goal from the beginning. taking into account user’s interests and feed- In many real-world applications, there are multi- back. To facilitate the study of this task, ple dialog types in human-bot conversations (called we create a human-to-human Chinese dialog multi-type dialogs), such as chit-chat, task oriented dataset DuRecDial (about 10k dialogs, 156k dialogs, recommendation dialogs, and even ques- utterances), which contains multiple sequen- tion answering (Ram et al., 2018; Wang et al., 2014; tial dialogs for every pair of a recommenda- tion seeker (user) and a recommender (bot). Zhou et al., 2018b). Therefore it is crucial to study In each dialog, the recommender proactively how to proactively and naturally make conversa- leads a multi-type dialog to approach recom- tional recommendation by the bots in the context mendation targets and then makes multiple rec- of multi-type human-bot communication. For ex- ommendations with rich interaction behavior. ample, the bots could proactively make recommen- This dataset allows us to systematically investi- dations after question answering or a task dialog to gate different parts of the overall problem, e.g., improve user experience, or it could lead a dialog how to naturally lead a dialog, how to interact from chitchat to approach a given product as com- with users for recommendation. Finally we es- tablish baseline results on DuRecDial for fu- mercial advertisement. However, to our knowledge, ture studies.1 there is less previous work on this problem. To address this challenge, we present a novel 1 Introduction task, conversational recommendation over multi- type dialogs, where we want the bot to proac- In recent years, there has been a significant in- tively and naturally lead a conversation from a crease in the work of conversational recommen- non-recommendation dialog to a recommendation dation due to the rise of voice-based bots (Chris- dialog. For example, in Figure1, given a starting takopoulou et al., 2016; Li et al., 2018; Reschke dialog such as question answering, the bot can take
arXiv:2005.03954v3 [cs.CL] 22 May 2020 et al., 2013; Warnestal, 2005). They focus on how into account user’s interests to determine a recom- to provide high-quality recommendations through mendation target (the movie
亚洲电影⼤奖最佳⼥ comment actor actor 很好看哦！(Do you want to see her movie
Figure 1: A sample of conversational recommendation over multi-type dialogs. The whole dialog is grounded on knowledge graph and a goal sequence, while the goal sequence is planned by the bot with consideration of user’s interests and topic transition naturalness. Each goal specifies a dialog type and a dialog topic (an entity). We use different colors to indicate different goals and use underline to indicate knowledge texts. a goal sequence to lead the dialog, and the goals conduct dialog management to control the dialog are unknown to the users. When we address this flow, which determines a recommendation target as task, we will encounter two difficulties: (1) how the final goal with consideration of user’s interests to proactively and naturally lead a conversation to and online feedback, and plans appropriate short- approach the recommendation target, (2) how to term goals for natural topic transitions. To our iterate upon initial recommendation with the user. knowledge, this goal-driven dialog policy mecha- nism for multi-type dialog modeling is not studied To facilitate the study of this task, we create a in previous work. The responding module produces human-to-human recommendation oriented multi- responses for completion of each goal, e.g., chat- type Chinese dialog dataset at Baidu (DuRecDial). ting about a topic or making a recommendation to In DuRecDial, every dialog contains multi-type the user. We conduct an empirical study of this dialogs with natural topic transitions, which cor- framework on DuRecDial. responds to the first difficulty. Moreover, there are rich interaction variability for recommendation, This work makes the following contributions: corresponding to the second difficulty. In addition, • We identify the task of conversational recom- each seeker has an explicit profile for the modeling mendation over multi-type dialogs. of personalized recommendation, and multiple di- alogs with the recommender to mimic real-world • To facilitate the study of this task, we cre- application scenarios. ate a novel dialog dataset DuRecDial, with To address this task, inspired by the work of rich variability of dialog types and domains Xu et al.(2020), we present a multi-goal driven as shown in Table1. conversation generation framework (MGCG) to handle multi-type dialogs simultaneously, such as • We propose a conversation generation frame- QA/chitchat/recommendation/task etc.. It consists work with a novel mixed-goal driven dialog of a goal planning module and a goal-guided re- policy mechanism. sponding module. The goal-planning module can Datasets↓ Metrics→ #Dialogs #Utterances Dialog types Domains User profile Facebook Rec (Dodge et al., 2016) 1M 6M Rec. Movie No REDIAL (Li et al., 2018) 10k 163k Rec., chitchat Movie No GoRecDial (Kang et al., 2019) 9k 170k Rec. Movie Yes OpenDialKG (Moon et al., 2019) 12k 143k Rec. Movie, book No CMU DoG (Zhou et al., 2018a) 4k 130k Chitchat Movie No IIT DoG (Moghe et al., 2018) 9k 90k Chitchat Movie No Wizard-of-wiki (Dinan et al., 2019) 22k 202k Chitchat 1365 topics from Wikipedia No OpenDialKG (Moon et al., 2019) 3k 38k Chitchat Sports, music No DuConv (Wu et al., 2019) 29k 270k Chitchat Movie No KdConv (Zhou et al., 2020) 4.5k 86k Chitchat Movie, music, travel No DuRecDial 10.2k 156k Rec., chitchat, Movie, music, movie star, Yes QA, task food, restaurant, news, weather
Table 1: Comparison of our dataset DuRecDial to recommendation dialog datasets and knowledge grounded dialog datasets. “Rec.” stands for recommendation.
2 Related Work where each dialog contains in-depth discussions on multiple topics. In comparison with them, our Datasets for Conversational Recommendation dataset contains multiple dialog types, clear goals To facilitate the study of conversational recommen- to achieve during each conversation, and user pro- dation, multiple datasets have been created in pre- files for personalized conversation. vious work, as shown in Table1. The first rec- ommendation dialog dataset is released by Dodge Models for Conversational Recommendation et al.(2016), which is a synthetic dialog dataset Previous work on conversational recommender sys- built with the use of the classic MovieLens ratings tems fall into two categories: (1) task-oriented dataset and natural language templates. Li et al. dialog-modeling approaches in which the systems (2018) creates a human-to-human multi-turn rec- ask questions about user preference over prede- ommendation dialog dataset, which combines the fined slots to select items for recommendation elements of social chitchat and recommendation (Christakopoulou et al., 2018, 2016; Lee et al., dialogs. Kang et al.(2019) provides a recommen- 2018; Reschke et al., 2013; Sun and Zhang, 2018; dation dialogue dataset with clear goals, and Moon Warnestal, 2005; Zhang et al., 2018b); (2) non-task et al.(2019) collects a parallel Dialog ↔KG cor- dialog-modeling approaches in which the models pus for recommendation. Compared with them, learn dialog strategies from the dataset without pre- our dataset contains multiple dialog types, multi- defined task slots and then make recommendations domain use cases, and rich interaction variability. without slot filling (Chen et al., 2019; Kang et al., Datasets for Knowledge Grounded Conver- 2019; Li et al., 2018; Moon et al., 2019; Zhou et al., sation As shown in Table1, CMU DoG (Zhou 2018a). Our work is more close to the second cat- et al., 2018a) explores two scenarios for Wikipedia- egory, and differs from them in that we conduct article grounded dialogs: only one participant has multi-goal planning to make proactive conversa- access to the document, or both have. IIT DoG tional recommendation over multi-type dialogs. (Moghe et al., 2018) is another dialog dataset for Goal Driven Open-domain Conversation movie chats, wherein only one participant has ac- Generation Recently, imposing goals on open- cess to background knowledge, such as IMDB’s domain conversation generation models having at- facts/plots, or Reddit’s comments. Dinan et al. tracted lots of research interests (Moon et al., 2019; (2019) creates a multi-domain multi-turn conversa- Li et al., 2018; Tang et al., 2019b; Wu et al., 2019) tions grounded on Wikipedia articles. OpenDialKG since it provides more controllability to conversa- (Moon et al., 2019) provides a chit-chat dataset be- tion generation, and enables many practical appli- tween two agents, aimed at the modeling of dialog cations, e.g., recommendation of engaging entities. logic by walking over knowledge graph-Freebase. However, these models can just produce a dialog Wu et al.(2019) provides a Chinese dialog dataset- towards a single goal, instead of a goal sequence as DuConv, where one participant can proactively lead done in this work. We notice that the model by Xu the conversation with an explicit goal. KdConv et al.(2020) can conduct multi-goal planning for (Zhou et al., 2020) is a Chinese dialog dataset, conversation generation. But their goals are limited Ground-truth Seeker profile profile of the Task Knowledge built so far recommender can acquire seeker profile informa- seeker S templates graph Sk k Pi-1 tion only through dialogs with the seekers. Knowledge graph Inspired by the work of doc- ...... ument grounded conversation (Ghazvininejad et al., 嗯嗯，周迅真的很优秀。(Anyway, Xun Zhou is really excellent.) 2018; Moghe et al., 2018), we provide a knowledge graph to support the annotation of more informa- 那你要看看她演的《⻛声》吗？(Do you want to see her movie
Matching Score gc Fusion unit −1 ( = | , , , ) HGFU based Decoder
Matcher Cue word GRU Standard GRU
x gc kc
xy gc kc
Knowledge Selector Knowledge Selector xy g c Attention Attention k k
k k y x k k k k Bi-directional GRU BERxTy Bi-directional GRU Bi-directional GRU Bi-directional GRU Bi-directional GRU
Knowledge Encoder C-R Encoder Goal Encoder Knowledge Encoder Goal Encoder Context Encoder
context X context X target Y Y k k k t n n n a o o o r k k k g w w w n n n
(b) Retrieval-based response model e o o o l l l (c) Generation-based response model t e e e
w w w d d d l l l g g g e e e e e e d d d 1 2 3 g g g e e e 1 2 3 t−1 P < 0.5 Context X CNN g P (l|X, g ) Multi-task t c ≥ 0.5 classification
Goal completion Current goal prediction estimation
(a) Goal-planning module
Figure 3: The architecture of our multi-goal driven conversation generation framework (denoted as MGCG). of the goal-planning module, while the tuples of we modify the original retrieval model to suit our [context, a ground-truth goal, K, response] will be task by emphasizing the use of goals. used for training of the responding module. As shown in Figure3(b), our response ranker consists of five components: a context-response 4.2 Goal-planning Model representation module (C-R Encoder), a knowl- As shown in Figure3(a), we divide the task of edge representation module (Knowledge Encoder), goal planning into two sub-tasks, goal completion a goal representation module (Goal Encoder), a estimation, and current goal prediction. knowledge selection module (Knowledge Selec- Goal completion estimation For this subtask, tor), and a matching module (Matcher). we use Convolutional neural network (CNN)(Kim, The C-R Encoder has the same architecture as 2014) to estimate the probability of goal comple- BERT (Devlin et al., 2018), and it takes a context tion by: X and a candidate response Y as segment a and PGC (l = 1|X, gt−1). (1) segment b in BERT, and leverages a stacked self- Current goal prediction If gt−1 is not com- attention to produce the joint representation of X pleted (PGC < 0.5), then gc = gt−1, where gc and Y , denoted as xy. is the goal for Y . Otherwise, we use CNN based Each related knowledge knowledge is also en- multi-task classification to predict the current goal i coded as a vector by the Knowledge Encoder using by maximizing the following probability: a bi-directional GRU (Chung et al., 2014), which −→ ←− ty tp 0 sk can be formulated as k = [h ; h ], where T gt = arg max PGP (g , g |X, G , Pi , K), (2) i Tk 0 k gty ,gtp −→ ←− denotes the length of knowledge, hTk and h0 rep- gc = gt, (3) resent the last and initial hidden states of the two directional GRU respectively. where gty is a candidate dialog type and gtp is a candidate dialog topic. The Goal Encoder uses bi-directional GRUs to encode a dialog type and a dialog topic for goal 4.3 Retrieval-based Response Model representation (denoted as gc). In this work, conversational goal is an important For knowledge selection, we make the context- guidance signal for response ranking. Therefore, response representation xy attended to all knowl- edge vectors ki and get the attention distribution: described in (Yao et al., 2017), which is a standard GRU based decoder enhanced with external knowl- exp(MLP([xy; gc]) · ki) p(ki|x, y, gc) = (4) P edge gates. In addition to the loss LKL(θ), the j exp(MLP([xy; gc]) · kj ) generator uses the following losses: We fuse all related knowledge information into a P single vector kc = i p(ki|x, y, gc) ∗ ki. NLL Loss: It computes the negative log- We view kc, gc and xy as the information from likelihood of the ground-truth response knowledge source, goal source and dialogue source (LNLL(θ)). respectively, and fuse the three information sources into a single vector via concatenation. Finally we BOW Loss: We use the BOW loss proposed by calculate a matching probability for each Y by: Zhao et al.(2017), to ensure the accuracy of the fused knowledge kc by enforcing the rel- p(l = 1|X, Y, K, gc) = softmax(MLP([xy; kc; gc])) (5) evancy between the knowledge and the true 4 response. Specifically, let w = MLP(kc) ∈ 4.4 Generation-based Response Model R|V |, where |V | is vocabulary size. We de- To highlight the importance of conversational goals, fine: exp(wyt ) we also modify the original generation model by p(yt|kc) = . (9) PV exp(w ) introducing an independent encoder for goal repre- v=1 v sentation. As shown in Figure3(c), our generator Then, the BOW loss is defined to minimize: consists of five components: a Context Encoder, a m Knowledge Encoder, a Goal Encoder, a Knowledge 1 X L (θ) = − logp(y |k ) (10) Selector, and a Decoder. BOW m t c t=1 Given a context X, conversational goal gc and knowledge graph K, our generator first encodes Finally, we minimize the following loss function: them as vectors with the use of above encoders
(based on bi-directional GRUs). L(θ) = α · LKL(θ) + α · LNLL(θ) + LBOW (θ) (11) We assume that using the correct response will be conducive to knowledge selection. Then mini- where α is a trainable parameter. mizing KLDivloss will make the effect of knowl- edge selection in the prediction stage (not use the 5 Experiments and Results correct response) close to that of knowledge selec- 5.1 Experiment Setting tion with correct response. For knowledge selec- We split DuRecDial into train/dev/test data by ran- tion, the model learns knowledge-selection strategy domly sampling 65%/10%/25% data at the level through minimizing the KLDivLoss between two of seekers, instead of individual dialogs. To eval- distributions, a prior distribution p(ki|x, gc) and uate the contribution of goals, we conduct an ab- a posterior distribution p(ki|x, y, gc). It is formu- lation study by replacing input goals with UNK lated as: for responding model. For knowledge usage, we exp(ki · MLP([x; y; gc])) p(ki|x, y, gc) = (6) conduct another ablation study, where we remove PN exp(k · MLP([x; y; g ])) j=1 j c input knowledge by replacing them with UNK. exp(k · MLP([x; g ]) p(k |x, g ) = i c (7) i c PN 5.2 Methods5 j=1 exp(kj · MLP([x; gc]) N S2S We implement a vanilla sequence-to-sequence 1 X p(ki|x, y, gc) LKL(θ) = p(ki|x, y, gc)log (8) model (Sutskever et al., 2014), which is widely N p(ki|x, gc) i=1 used for open-domain conversation generation. In training procedure, we fuse all related MGCG R: Our system with automatic goal knowledge information into a vector kc = planning and a retrieval based responding model. P i p(ki|x, y, gc) ∗ ki, same as the retrieval-based MGCG G: Our system with automatic goal method, and feed it to the decoder for response planning and a generation based responding model. generation. In testing procedure, the fused knowl- P 4The BOW loss is to introduce an auxiliary loss that re- edge is estimated by kc = p(ki|x, gc) ∗ ki with- i quires the decoder network to predict the bag-of-words in the out ground-truth responses. The decoder is imple- response to tackle the vanishing latent variable problem. mented with the Hierarchical Gated Fusion Unit 5Please see Appendix 2. for model parameter settings. Methods↓ Metrics→ Hits@1/Hits@3 F1/ BLEU2 PPL DIST-2 Knowledge P/R/F1 S2S- gl.- kg. 6.78% / 24.55% 23.97 / 0.065 27.31 0.011 0.275 / 0.209 / 0.216 S2S+gl.- kg. 8.03% / 27.71% 24.78 / 0.077 24.82 0.012 0.287 / 0.223 / 0.231 S2S+gl.+kg. 8.37% / 27.67% 24.66 / 0.072 23.96 0.011 0.295 / 0.239 / 0.253 MGCG R- gl.- kg. 19.58% / 42.75% 33.22 / 0.207 - 0.171 0.344 / 0.301 / 0.306 MGCG R+gl.- kg. 19.77% / 42.99% 33.78 / 0.223 - 0.185 0.351 / 0.322 / 0.309 MGCG R+gl.+kg. 20.33%/ 43.61% 33.93 / 0.232 - 0.187 0.349 / 0.331 / 0.316 MGCG G- gl.- kg. 13.26% / 36.07% 33.11 / 0.189 18.51 0.037 0.386 / 0.349 / 0.358 MGCG G+gl.- kg. 14.21% / 38.91% 35.21 / 0.213 17.78 0.049 0.393 / 0.352 / 0.351 MGCG G+gl.+kg. 14.38% / 39.70% 36.81 / 0.219 17.69 0.052 0.401 / 0.377 / 0.383
Table 4: Automatic evaluation results. +(-)gl. represents “with(without) conversational goals”. +(-)kg. represents “with(without) knowledge”. For “S2S +gl.+kg.”, we simply concatenate the goal predicted by our model, all the related knowledge and the dialog context as its input.
Turn-level results Dialog-level results Methods↓ Metrics→ Fluency Appro. Infor. Proactivity Goal success rate Coherence S2S +gl. +kg. 1.08 0.23 0.37 0.94 0.37 0.49 MGCG R +gl. +kg. 1.98 0.60 1.28 1.22 0.68 0.83 MGCG G +gl. +kg. 1.94 0.75 1.68 1.34 0.82 0.91
Table 5: Human evaluation results at the level of turns and dialogs.
5.3 Automatic Evaluations ber of topic candidates is very large (around 1000), leading to the difficulty of topic prediction. As Metrics For automatic evaluation, we use several shown in Table4, for response generation, both common metrics such as BLEU (Papineni et al., MGCG R and MGCG G outperform S2S by a 2002), F1, perplexity (PPL), and DISTINCT (DIST- large margin in terms of all the metrics under the 2) (Li et al., 2016) to measure the relevance, flu- same model setting (without gl.+kg., with gl., or ency, and diversity of generated responses. Follow- with gl.+kg.). Moreover, MGCG R performs better ing the setting in previous work (Wu et al., 2019; in terms of Hits@k and DIST-2, but worse in terms Zhang et al., 2018a), we also measure the perfor- of knowledge F1 when compared to MGCG G.8 It mance of all models using Hits@1 and [email protected] might be explained by that they are optimized on Here we let each model to select the best response different metrics. We also found that the methods from 10 candidates. Those 10 candidate responses using goals and knowledge outperform those with- consist of the ground-truth response generated by out goals and knowledge, confirming the benefits humans and nine randomly sampled ones from of goals and knowledge as guidance information. the training set. Moreover, we also evaluate the knowledge-selection capability of each model by 5.4 Human Evaluations calculating knowledge precision/recall/F1 scores as done in Wu et al.(2019). 7 In addition, we also Metrics: The human evaluation is conducted at the report the performance of our goal planning mod- level of both turns and dialogs. ule, including the accuracy of goal completion es- For turn-level human evaluation, we ask each timation, dialog type prediction, and dialog topic model to produce a response conditioned on a given 9 prediction. context, the predicted goal and related knowledge. Results Our goal planning model can achieve ac- The generated responses are evaluated by three an- curacy scores of 94.13%, 91.22%, and 42.31% for notators in terms of fluency, appropriateness, infor- goal completion estimation, dialog type prediction, mativeness, and proactivity. The appropriateness and dialog topic prediction. The accuracy of dialog measures if the response can complete current goal topic prediction is relatively low since the num- and it is also relevant to the context. The informa- tiveness measures if the model makes full use of 6Candidates (including golden response) are scored by PPL knowledge in the response. The proactivity mea- using the generation-based model, then candidates are sorted based on the scores, and Hits@1 and Hits@3 are calculated. 8We calculate an average of F1 over all the dialogs. It 7When calculating the knowledge precision/recall/F1, we might result in that the value of F1 is not between P and R. compare the generated results with the correct knowledge. 9Please see Appendix 3. for more details. sures if the model can successfully introduce new Methods→ S2S MGCG R MGCG G topics with good fluency and coherence. Metrics↓Types↓ +gl. +kg. +gl. +kg. +gl. +kg. For dialogue-level human evaluation, we let each #Failed Rec. 106/7 95/18 93/20 gl./ Chitchat 120/93 96/117 80/133 model converse with a human and proactively make #Com- QA 66/5 61/10 60/11 recommendations when given the predicted goals pleted Task 45/4 36/13 39/10 and related knowledge.10 For each model, we col- gl. Overall 337/109 288/158 272/174 Rec. 0 8 7 lect 100 dialogs. These dialogs are then evaluated Chitchat 9 25 33 #Used QA 5 10 15 by three persons in terms of two metrics: (1) goal kg. success rate that measures how well the conversa- Task 0 3 2 Overall 14 46 57 tion goal is achieved, and (2) coherence that mea- sures relevance and fluency of a dialog as a whole. Table 6: Analysis of goal completion and knowledge All the metrics has three grades: good(2), fair(1), usage across different dialog types. bad(0). For proactivity, “2” indicates that the model introduces new topics relevant to the context, “1” means that no new topics are introduced, but knowl- used knowledge is proportional to goal success rate edge is used, “0” means that the model introduces across different dialog types or different methods, new but irrelevant topics. For goal success rate, “2” indicating that the knowledge selection capabil- means that the system can complete more than half ity is crucial to goal completion through dialogs. of the goals from goal planning module, “0” means Moreover, the goal of chitchat dialog is easier to the system can complete no more than one goal, complete in comparison with others, and QA and otherwise “1”. For coherence, “2”/“1”/“0” means recommendation dialogs are more challenging to that two-thirds/one-third/very few utterance pairs complete. How to strengthen knowledge selec- are coherent and fluent. tion capability in the context of multi-type dialogs, Results All human evaluations are conducted by especially for QA and recommendation, is very three persons. As shown in Table5, our two sys- important, which will be left as the future work. tems outperform S2S by a large margin, especially in terms of appropriateness, informativeness, goal 6 Conclusion success rate and coherence. In particular, S2S tends to generate safe and uninformative responses, fail- We identify the task of conversational recommen- ing to complete goals in most of dialogs. Our two dation over multi-type dialogs, and create a dataset systems can produce more appropriate and informa- DuRecDial with multiple dialog types and multi- tive responses to achieve higher goal success rate domain use cases. We demonstrate usability of this with the full use of goal information and knowledge. dataset and provide results of state of the art models Moreover, the retrieval-based model performs bet- for future studies. The complexity in DuRecDial ter in terms of fluency since its response is selected makes it a great testbed for more tasks such as from the original human utterances, not automati- knowledge grounded conversation (Ghazvininejad cally generated. But it performs worse on all the et al., 2018), domain transfer for dialog modeling, other metrics when compared to the generation- target-guided conversation (Tang et al., 2019a) and based model. It might be caused by the limited multi-type dialog modeling (Yu et al., 2017). The number of retrieval candidates. Finally, it can be study of these tasks will be left as the future work. seen that there is still much room for performance improvement in terms of appropriateness and goal success rate, which will be left as the future work. Acknowledgments
5.5 Result Analysis We would like to thank Ying Chen for dataset an- notation and thank Yuqing Guo and the reviewers In order to further analyze the relationship between for their insightful comments. This work was sup- knowledge usage and goal completion, we provide ported by the National Key Research and Devel- the number of failed goals, completed goals, and opment Project of China (No. 2018AAA0101900) used knowledge for each method over different di- and the Natural Science Foundation of China (No. alog types in Table6. We see that the number of 61976072). 10Please see Appendix 4. for more details. References Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016. A diversity-promoting ob- Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang jective function for neural conversation models. In Tseng, Inigo˜ Casanueva, Stefan Ultes, Osman Ra- NAACL-HLT, pages 110–119. madan, and Milica Gasiˇ c.´ 2018. MultiWOZ - a large-scale multi-domain wizard-of-Oz dataset for Raymond Li, Samira Ebrahimi Kahou, Hannes Schulz, task-oriented dialogue modelling. In EMNLP. Vincent Michalski, Laurent Charlin, and Chris Pal. 2018. Towards deep conversational recommenda- Qibin Chen, Junyang Lin, Yichang Zhang, Ming Ding, tions. In NIPS. Yukuo Cen, Hongxia Yang, and Jie Tang. 2019. To- wards knowledge-based recommender dialog sys- Nikita Moghe, Siddhartha Arora, Suman Banerjee, and tem. In ACL. Mitesh M. Khapra. 2018. Towards exploiting back- ground knowledge for building conversation sys- Konstantina Christakopoulou, Alex Beutel, Rui Li, tems. In EMNLP. Sagar Jain, and Ed H. Chi. 2018. Q and r: A two- stage approach toward interactive recommendation. Seungwhan Moon, Pararth Shah, Anuj Kumar, and Ra- In KDD. jen Subba. 2019. Opendialkg: Explainable conver- sational reasoning with attention-based walks over Konstantina Christakopoulou, Katja Hofmann, and knowledge graphs. In ACL. Filip Radlinski. 2016. Towards conversational rec- ommender systems. In KDD. Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic eval- Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, uation of machine translation. In ACL, pages 311– and Yoshua Bengio. 2014. Empirical evaluation of 318. gated recurrent neural networks on sequence model- ing. In arXiv preprint arXiv:1412.3555. Ashwin Ram, Rohit Prasad, Chandra Khatri, Anu Venkatesh, Raefer Gabriel, Qing Liu, Jeff Nunn, Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Behnam Hedayatnia, Ming Cheng, Ashish Nagar, Kristina Toutanova. 2018. Bert: Pre-training of deep Eric King, Kate Bland, Amanda Wartick, Yi Pan, bidirectional transformers for language understand- Han Song, Sk Jayadevan, Gene Hwang, and Art Pet- ing. In arXiv preprint arXiv:1810.04805. tigrue. 2018. Conversational AI: the science behind the alexa prize. In CoRR, abs/1801.03604. Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2019. Wizard Kevin Reschke, Adam Vogel, and Daniel Jurafsky. of wikipedia: knowledge-powered conversational 2013. Generating recommendation dialogs by ex- agents. In Proceedings of ICLR. tracting information from user reviews. In ACL.
Jesse Dodge, Andreea Gane, Xiang Zhang, Antoine Yueming Sun and Yi Zhang. 2018. Conversational rec- Bordes, Sumit Chopra, Alexander H. Miller, and ommender system. In SIGIR. Arthur Szlam andJason Weston. 2016. Evaluating Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. prerequisite qualities for learning end-to-end dialog Sequence to sequence learning with neural networks. systems. In ICLR. In NIPS.
Marjan Ghazvininejad, Chris Brockett, Ming-Wei Jianheng Tang, Tiancheng Zhao, Chengyan Xiong, Xi- Chang, Bill Dolan, Jianfeng Gao, Wen tau Yih, and aodan Liang, Eric P Xing, and Zhiting Hu. 2019a. Michel Galley. 2018. A knowledge-grounded neural Target-guided open-domain conversation. In ACL. conversation model. In AAAI. Jianheng Tang, Tiancheng Zhao, Chenyan Xiong, Xi- Dongyeop Kang, Anusha Balakrishnan, Pararth Shah, aodan Liang, Eric P. Xing, and Zhiting Hu. 2019b. Paul Crook, Y-Lan Boureau, and Jason Weston. Target-guided open-domain conversation. In Pro- 2019. Recommendation as a communication game: ceedings of ACL. Self-supervised bot-play for goal-oriented dialogue. In EMNLP. Zhuoran Wang, Hongliang Chen, Guanchun Wang, Hao Tian, Hua Wu, and Haifeng Wang. 2014. Policy Yoon Kim. 2014. Convolutional neural networks for learning for domain selection in an extensible multi- sentence classification. In Proceedings of the 2014 domain spoken dialogue system. In EMNLP. Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP), Doha, Qatar. Associa- Pontus Warnestal. 2005. Modeling a dialogue strategy tion for Computational Linguistics. for personalized movie recommendations. In The Beyond Personalization Workshop. Sunhwan Lee, Robert Moore, Guang-Jie Ren, Raphael Arar, and Shun Jiang. 2018. Making personal- Wenquan Wu, Zhen Guo, Xiangyang Zhou, Hua Wu, ized recommendation through conversation: Archi- Xiyuan Zhang, Rongzhong Lian, and Haifeng Wang. tecture design and recommendation methods. In 2019. Proactive human-machine conversation with AAAI Workshops. explicit conversation goal. In ACL. Jun Xu, Haifeng Wang, Zheng-Yu Niu, Hua Wu, and • Gender: We randomely select ”male” or ”fe- Wanxiang Che. 2020. Knowledge graph grounded male” as the seeker’s gender. goal planning for open-domain conversation genera- tion. In AAAI. • Age range: We randomly choose one from Lili Yao, Yaoyuan Zhang, Yansong Feng, Dongyan the 5 age ranges. Zhao, and Rui Yan. 2017. Towards implicit content- introducing for generative short-text conversation • Residential city: We randomly choose one systems. In EMNLP. from 55 cities in China as the seeker’s residen- tial city. Zhou Yu, Alexander I. Rudnicky, and Alan W. Black. 2017. Learning conversational systems that inter- • Occupation status: We randomly choose one leave task and non-task content. In IJCAI. from ”student”, ”worker” and ”retirement” Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur based on above age range. Szlam, Douwe Kiela, and Jason Weston. 2018a. Per- sonalizing dialogue agents: I have a dog, do you • Domain preference: We randomly select one have pets too? In ACL. or two domains as ones that the seeker likes Yongfeng Zhang, Xu Chen, Qingyao Ai, Liu Yang, (e.g., movie, food), and one domain as the and W. Bruce Croft. 2018b. Towards conversational one that the seeker dislikes (e.g., news). It search and recommendation: System ask, user re- will affect the setting of task templates for this spond. In CIKM. seeker. Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural • Seed entity preference: We randomly select dialog models using conditional variational autoen- one or two entities from KG entities of the coders. In ACL. domains preferred by the seeker as his/her Hao Zhou, Chujie Zheng, Kaili Huang, Minlie Huang, preference at entity level. It will affect the and Xiaoyan Zhu. 2020. Kdconv: A chinese setting of task templates for this seeker. multi-domain dialogue dataset towards multi-turn knowledge-driven conversation. In Proceedings of • Rejected entity list and accepted entity list: ACL. Both of them are empty at the beginning, and Kangyan Zhou, Shrimai Prabhumoye, and Alan W they will be updated as the conversation pro- Black. 2018a. A dataset for document grounded con- gresses; the two lists will affect the recommen- versations. In Proceedings of EMNLP. dation results of subsequent conversations to some extent. Li Zhou, Jianfeng Gao, Di Li, and Heung-Yeung Shum. 2018b. The design and implementation of Collection of Knowledge graph (KG) The do- xiaoice, an empathetic social chatbot. In CoRR, abs/1812.08989. main of knowledge graph include stars, movies, music, news, food, POI(Point of Interest), weather. Appendix • Stars: including the introduction, achieve- 1. Dataset collection process ments, awards, comments, birthday, birth- 1.1 Collection of seeker profiles/knowledge place, height, weight, blood type, constella- graph/task templates tion, zodiac, nationality, friends, etc Collection of seeker profile The attributes of • Film: including film rating, comments, re- seeker profiles are shown as follows: name, gender, gion, leading role, director, category, evalua- age range, city of residence, occupation status, and tion, award, etc seeker preference. Seeker preference includes : do- main preference, seed entity preference, entity list • Music: singer information, comments, etc rejected by the seeker, entity list accepted by the • News: including the topic, content, etc seeker. • Food: including the name, ingredients, cate- • Name: We generate the first-name Chinese gory, etc character(or last-name Chinese character) by randomly sampling Chinese characters from a • POI: including restaurant name, average set of candidate characters used as first name price, score, order quantity, address, city, spe- (or last name) for the gender of the seeker. cialty, etc • Weather: historical weather of 55 cities from ensure that at least two workers enter the task at the July 2017 to August 2019 same time, we arrange multiple workers to log in the annotation platform. During annotation, each Collection of task templates First we manually conversation is randomly assigned to two workers, annotate a list of around 20 high-level goal se- one of whom plays the role of BOT and the other quences as candidates. Most of these goal se- plays the role of User. Two workers conduct anno- quences include 3 to 5 high-level goals. Here each tation based on the seeker profile, knowledge graph high-level goal contains a dialog type and a domain and task templates. (not entity or chatting topic). Then for each seeker, Due to the complexity of our task design, the we select appropriate high-level goal sequences quality of data annotation may have a high vari- from the above list, which contains the domains ance. To address this problem, we provide a strict that fall into the seeker’s preferred domain list. data annotation standard to guide the workers to To collect goal sequences at entity level, we first annotate in the way we expect them to be. After use the seed entities of the seeker to enrich the in- data annotation, multiple data specialists will re- formation of above high-level goal sequences. If view it. As long as one specialist thinks it does not the seed entities are not enough, or there is no seeds meet the requirements, it will be marked back and for some domains in the high-level goal sequences, re-annotation until all specialists think that it fully we select some entities from KG for each goal do- meets requirements. main based on embedding based similarity scores of the seed entities (of current seeker) and the can- 2. Model Parameter Settings didate entity. Then we obtain goal sequences at All models are implemented using PaddlePaddle.11 entity level. Finally we use some rules to generate The parameters of all the modules are shown in a description for each goal (e.g., which side, the Table7. 12 seeker or the recommender, to start the dialog, how to complete the goal). Thus we have task templates 3. Turn-level Human Evaluation Guideline for guidance of data annotation. Fluency measures if the produced response itself To introduce diverse interaction behavior for rec- is fluent: ommendation, we design some fine-grained inter- • score 0 (bad): unfluent and difficult to under- action operations, e.g., the seeker may reject the stand. initial recommendation, or mention a new topic, or ask a question about an entity, or simply ac- • score 1 (fair): there are some errors in the cept the recommendation. Each interaction oper- response text but still can be understood. ation corresponds to a goal. We randomly sam- • score 2 (good): fluent and easy to understand. ple one of above operations and insert it into the entity-level goal sequences to diversify recommen- Appropriatenss measures if the response can re- dation dialogs. The entities associated with the spond to the context: above interaction operations are selected from the • score 0 (bad): Sub-dialogs for Rec and KG based on their similarity scores with current chitchat: not semantically relevant to the con- seeker’s seed entites. If the entity will be accepted text or logically contradictory to the context. by the seeker as described in the task templates Sub-dialogs for task-oriented: No necessary (including entity-level goal sequence and its de- slot value is involved in the conversation. Sub- scription), then its similarity score with the seeker’s dialogs for QA: Incorrect answer. seed entites should be relatively high. If the entity will be rejected by the seeker as described in the • score 1 (fair): relevant to the context as a task templates, then its similarity score with the whole, but using some irrelevant knowledge, seeker’s seed entites should be relatively low. or not answering questions asked by the users.
1.2 Dataset annotation process • score 2 (good): otherwise. We first release a small amount of data for training, 11It is an open source deep learning plat- and then carry out video training for annotation form(https://www.paddlepaddle.org.cn/) 12Due to S2S model uses the same parameters as problems. After that, a small amount of data is OpenNMT(https://github.com/OpenNMT/OpenNMT-py), its released again to select the final task workers. To parameters are not listed. module Parameter value • score 2 (good): more than half goals are Goal-planning model Embedding Size 256 achieved with full use of knowledge and goal Hidden Size 256 Batch Size 128 information. Learning Rate 0.002 Optimizer Adam Retrieval-based model Dropout 0.1 Coherence measures the overall fluency of the Embedding Size 512 Hidden Size 512 whole dialogue: Batch Size 32 Learning Rate 0.001 • score 0 (bad): two-thirds responses irrelevant Optimizer Adam or logically contradictory to the previous con- Weight Decay 0.01 Proportion Warmup 0.1 text. Generation-based model Embedding Size 300 Hidden Size 800 • score 1 (fair): less than one-third responses Batch Size 16 irrelevant or logically contradictory to the pre- Learning Rate 0.0005 Grad Clip 5 vious context. Dropout 0.2 Beam Size 10 • score 2 (good): very few response irrelevant Optimizer Adam or logically contradictory to the previous con- text. Table 7: Model parameter settings. 5. Case Study Informativeness measures if the model makes full Figure4 shows the conversations generated by the use of knowledge in the response: models via conversing with humans, given the con- versation goal and the related knowledge. It can • score 0 (bad): no knowledge is mentioned at be seen that our knowledge-aware generator can all. use more correct knowledge for diverse conversa- • score 1 (fair): only one knowledge triple is tion generation. Even though the retrieval-based mentioned in the response. method can also produce knowledge-grounded re- sponses, the used knowledge is relatively few and • score 2 (good): more than one knowledge inappropriate. The seq2seq model can’t success- triple is mentioned in the response. fully complete the given goal, as the knowledge is not fully used as our proposed knowledge-aware Proactivity measures if the model can introduce generator, making the generated conversation less new knowledge/topics in conversation: diverse and sometimes dull. • score -1 (bad): some new topics are intro- duced but irrelevant to the context. • score 0 (fair): no new topics/knowledge are used. • score 1(good): some new topics relevant to the context are introduced.
4. Dialogue-level Human Evaluation Guideline Goal Completion measures how good the given conversation goal is finished: • score 0 (bad): less than half goals are achieved.. • score 1 (fair): less than half goals are achieved with minor use of knowledge or goal informa- tion. Goal 问答(张柏芝)---->闲聊(明星)---->电影推荐(左右逢源)---->再见(再见) sequence QA(Cecilia Cheung) ----> Chitchat(Star) ----> Movie recommendation(Left Right Love Destiny) ----> Goodbye(Goodbye)
Knowledge 165cm 1980-5-24 身高 Height 生日 （ ） （Birthday）
Wu 主演 钟无艳（ Actress 主演 Yen （ ） Cecilia Actress 左右情缘 ） 张柏芝（ （ ） Left Right Love Cheung） （ 评论 Destiny） Comment 评论 （ ） Comment 星座 （ ） Constellation 评分 （ ） 评论 Rating Comment （ ） 很法式的一部短片 （ ） (A short film with so much french Very style) 非常过瘾的港片！（ 双子座 The first beauty addictive Hong Kong Gemini 香港第一美女（ （ ） in Hong Kong 8.3 movie!） ）
S2S MGCG_R MGCG_G
User:你知道电影『钟无艳』主演是谁吗? （Who is the main User:你知道电影『钟无艳』主演是谁吗? （Who is the main actress of User:你知道电影『钟无艳』主演是谁吗? （Who is the main actress of actress of Wu Yen?） Wu Yen?） Wu Yen?） Bot：你有喜欢的明星吗？(Do you have a favorite star?) Bot：是张柏芝哦 (Cecilia Cheung) Bot：张柏芝 （Cecilia Cheung ） User:我喜欢张柏芝啊 (I like Cecilia Cheung) User:谢谢啊，你好聪明。 (Thank you. You're smart.) User:谢谢啊，你好聪明。(Thank you. You're smart.) Bot：是钟无艳的歌！ （It's Wu Yen's song） Bot：是吗？真有眼光呢，她可是香港第一美女呢 (Is it? You have Bot：你是她的粉丝吗？ （Are you a fan of her?） User:你说的是什么啊？ (What are you talking about?) good taste. She is the first beauty in Hong Kong.） User:是啊，我很喜欢张柏芝。(Yes, I like Cecilia Cheung very much. ) Bot：是啊，还有一部电影叫《恋爱大赢家》，你看过吗？(Yes, User:除了漂亮，我很喜欢她演戏时的忘我与投入。（Apart from Bot：她是香港第一美女呢，你真有眼光呢。(She is the first beauty in there is another movie called "The Love Winner". Have you seen it?) beauty, I like her selflessness and devotion in acting.） Hong Kong. You have good taste.） User:暂时没看过啊。(Not yet.) Bot：你看过她演的那部电影《钟无艳》吗？ (Have you seen her User:除了漂亮，我很喜欢她演戏时的忘我与投入（Apart from beauty, I Bot：好的，拜拜 (OK, goodbye.) movie "Wu Yen"?) like her selflessness and devotion in acting） User:再见 (Bye) User:嗯，她很漂亮。 (Well, she’s very beautiful.) Bot：给你推荐一部张柏芝主演的电影《左右情缘》，很法式的一部短 Bot：好的，拜拜。(OK, goodbye.) Bot：你会喜欢上的的。(You'll like it) 片。(I'd like to recommend you "Left Right Love Destiny" starring Cecilia User:暂时没看过啊。(I haven't seen it yet.) Cheung, which is a short film with so much french style.) Bot：好的，再见 (OK, goodbye.) User:这个电影评分多少呢？（What's the rating of this movie?） User:嗯，我比较喜欢。（Well, I like it very much.） Bot：评分是8.3呢。(The rating is 8.3.) Bot：再见。(Bye.) User:那还不错，我抽时间要看看。（That's well. I'll see it later.） Bot：嗯嗯，相信您会喜欢的。(Mm-hmm. I'm sure you'll like it.) User:嗯，我有点事，回见。(Well, I have something to do. See you later.) Bot：好的，再见 (OK, goodbye.)
Figure 4: Conversations generated by three different models: texts in red color represent correct knowledge being appropriate in current context, while texts in blue color represent inappropriate knowledge. Texts in purple color indicate that the use of knowledge is correct, but the response is not appropriate.