A Data-Driven Model of Explanations for a Chatbot That Helps to Practice Conversation in a Foreign Language

A data-driven model of explanations for a chatbot that helps to practice conversation in a foreign language

Sviatlana Höhn AI Minds Vianden, Luxembourg [email protected]

Abstract conversations where one of the speakers is more knowledgeable in some matters than the other, for This article describes a model of other- instance in mastering professional terminology or initiated self-repair for a chatbot that helps communication in a second language not yet fully to practice conversation in a foreign lan- mastered. Therefore it is crucial for conversational guage. The model was developed using a agents acting in such environments to recognize corpus of instant messaging conversations and to handle repair initiations properly. between German native and non-native Repair sequences where the machine is the speakers. Conversation Analysis helped to trouble-speaker are in focus of this article. The create computational models from a small learner initiates a repair in response to something number of examples. The model has been not (fully) understood, and the machine explains. validated in an AIML-based chatbot. Un- This type of repair corresponds to other-initiated like typical retrieval-based dialogue sys- self-repair with a linguistic trouble source where tems, the explanations are generated at the language learner is the recipient of the trouble run-time from a linguistic database. talk (OISRL). CommICALL research is mainly grounded in 1 Introduction Second Language Acquisition (SLA) theory (Pe- Conversational agents tailored for communication tersen, 2010; Wilske, 2014). The model of expla- with language learners are studied in the area nation sequences, so called negotiations of mean- of Communicative Intelligent Computer-Assisted ing introduced by (Varonis and Gass, 1985) re- Language Learning (CommICALL). Starting with ceived a lot of attention and was highly re-used the idea of creating a machine that behaves like a in subsequent CALL research (Fredriksson, 2012; language expert in an informal chat, speciﬁc in- Satomi Kawaguchi, 2012). The model includes teractional practices need to be described where a trigger, an indicator, a response and a reaction linguistic identities of interaction participants be- to response. However, this model has been criti- come visible. Such practices include repair with cized for its view on repair as something "marring linguistic trouble source where non-native speak- the ﬂow" of a conversation and for being inappli- ers address troubles in comprehension or produc- cable to non-institutional settings (Markee, 2000). tion (Danilava et al., 2013). Although repair in native/non-native speaker talk Repair is a building block of conversation that has been intensively studied in Conversation Anal- helps to deal with troubles in understanding and ysis (CA) (Markee, 2000; Gardner and Wagner, production of talk. Depending on who produced 2004; Hosoda, 2006), the results have not been a trouble source and who initiates a repair we dis- operationalized for an implementation in a Com- tinguish between self-initiated and other-initiated mICALL system. Therefore, this article has two repair. A repair can be carried out by the same objectives: speaker who produced the trouble source or by the 1. Identify typical interactional resources em- other speaker (self-repair and other-repair). ployed for initiation and carry-out of repair Because there is a preference for self-repair, using methods of Conversation Analysis. other-initiated self-repair is the most frequent repair type. It may become even more frequent in 2. Create a computation models of the repair

395 Proceedings of the SIGDIAL 2017 Conference, pages 395–405, Saarbrucken,¨ Germany, 15-17 August 2017. c 2017 Association for Computational Linguistics of the type OISRL to be implemented in a tion) and misunderstandings (the system matched CommICALL application. user’s input to a wrong representation) (Dzikovska et al., 2009; Meena et al., 2015). These re- We use a dataset of German native/non-native in- pair types correspond to other-initiated self-repair stant messaging conversations (Höhn, 2015) to when the user is the trouble-speaker. analyze practices of repair in native/non-native Clarification requests in AI and NLP publica- speaker informal chat. All repair sequences have tions should not be confused with clarification re- been annotated. Collections of similar cases have quests in SLA publications where this term is used been built. Interactional resources used by lan- to refer to only a particular form of corrective feed- guage learners for repair initiations have been an- back (Lyster et al., 2013), or to a dialogue move in alyzed. Patterns of repair initiations have been ob- meaning negotiations (Varonis and Gass, 1985). tained through generalization. In this way, rules Emphasising the importance of correct recogni- for recognition of repair initiations have been cre- tion of user’s clarification requests, Purver(2004) ated. An implementation case study was set up to provides a study of various types of clarification validate the resulting computational models in an requests, see also follow-up publications (Purver, AIML-based chatbot. 2006; Ginzburg et al., 2007; Ginzburg, 2012). 2 Repair in Conversational Agents Purver(2004) uses the HPSG framework to cover the main classes of the identified classification Non-native speakers are usually not considered scheme. Because different functions might be ex- as the main user group of general-purpose dia- pressed by a clarification request of the same form, logue systems. The assumption dominates that Purver(2004) analyses the clarification readings human users understand everything what an agent to cover the correspondence between the form and may say. This assumption is reflected in the two the meaning of the repair initiations. However, main problems addressed by research on repair several points for critiques arise. For instance, for conversational agents: dealing with user’s self- some utterances may be formatted as repair ini- corrections which may make speech recognition tiations but have a different interactional func- difficult and managing system’s lack of information, such as expressing surprise and topicaliza- tion in order to satisfy user’s request. tion (not listed as possible readings). In addition, These two research areas may be found under repair initiations designed to deal with troubles keywords self-repairs, sometimes speech repairs in understanding are put together with strategies (Zwarts et al., 2010) or disfluencies (Shriberg, for dealing with troubles in production (e.g. gap 1994; Martin and Jurafsky, 2009), and clarifica- fillers). From the CA perspective, Purver(2004)’s tion dialogues or clarification requests, CRs in AI gap fillers correspond to self-initiated other-repair, and NLP publications. What is referred to by the thus are sequentially completely different. There- term self-repair in speech recognition domain cor- fore, modifications in the classification proposed responds to user’s self-initiated self-repair in CA by (Purver, 2004) are needed in order to better terminology. comply with studies in CA, and therefore better re- Shriberg(1994) uses the term reparandum to flect the state-of-the-art in CA-informed dialogue refer to what is called trouble source in CA. The research. model considers pauses (moment of interruption) Different types of causes for clari- and lexicalised means to focus on the replacement Example 2.1. fication used in (Schlangen, 2004, Ex. (12)). (editing terms). These are interactional recourses used by speakers to signal trouble in production a. A I ate a Pizza with chopsticks the other day and to pre-announce a coming replacement. B A Pizza with chopsticks on it? The term clarification dialogues is mostly used b. A Please give me a double torx. B What’s a torx? to describe repairs dealing with insufficient infor- c. A Please give me a double torx. mation available for a system after speech recog- B Which one? d. A Every wire has to be connected to a power nition and language understanding (Kruijff et al., source. 2008; Jian et al., 2010; Buß and Schlangen, 2011). B Each to a different one, or can it be the same for The term miscommunication was introduced to every wire? distinguish between non-understandings (the sys- Schlangen(2004) analyses communication prob- tem could not match user’s input to a representa- lems leading to clarification requests focusing on

396 trouble source types (what caused the communica- source, that is to signal trouble and to reference tion problem). Schlangen(2004) makes clear that the trouble source. Turn formats are specifically a more fine-grained classification of causes for re- important for the future recognition of repair initi- questing clarification in dialogue may be needed, ations by chatbots. specifically, a model distinguishing between different cases in Example 2.1. 3.1 Repair initiations From the CA perspective, speakers’ linguistic Two abstract types of repair other-initiations were and professional identities and preferences play a identified in the dataset: statements of non- role in speaker’s selection of a specific format of understanding where a part of partner’s utter- a repair initiation. Speaker B in Example 2.1.b. ance is marked as unclear, and candidate under- positions herself as a novice in torx matters with standings where the own version of understand- her repair initiation, while speakers B in Exam- ing of the problematic unit is provided. Non- ples 2.1.c. positions herself as knowledgeable in understandings require an explanation of the trou- torx matters. In addition, utterances may be de- ble source in the repair while candidate under- signed as repair initiations, but may in fact have standings require a yes/no answer. a different function. For instance, the repair ini- Repair other-initiations were found at two dis- tiation produced by B in Example 2.1.a. may be tinct types of position: immediate and delayed. analysed as a joke not requiring any explanation. The first type comes immediately after the trou- Other-initiated self-repair when the machine is ble source turn. The second type comes later than the trouble-speaker is explored in (Gehle et al., the adjacent turn. Sequentially, both correspond 2014). Based on a corpus of video-recorded to the next-turn repair initiation or second posi- human-robot-interactions in a museum, the au- tion repair described in CA literature as the first thors analyse interactional resources used by mu- structurally specified place for other-initiated re- seum visitors to signal troubles in understanding pair (Schegloff, 2000; Liddicoat, 2011). Delayed robot’s talk and dealing with misunderstandings. repair initiations occur because speakers in chat It was observed that people deal with different can produce turns simultaneously and follow dis- sorts of trouble similarly. tinct interleaved conversation threads. There is a The potential user of a CommICALL system is dependency between the position of the repair ini- a language learner who may have troubles in com- tiation and the interactional recourses for repair prehension. While user-initiated repair has been initiation. Some resources are used exclusively in subject of research of studies in human-robot in- the immediate position. teraction and general dialogue systems, not much Example 3.1. Open class repair initiation attention has been paid to it in CommICALL. This article seeks to contribute to the research on repair 615 L08 danke. good night) thank you. good night in CommICALL by a microanalytic study of se- 617 N04 gn8 :-) quences of other-initiated self-repair when the na- 618 L08 ??? tive speaker is the trouble-speaker. Based on the ??? [repair initiation] 619 N04 gn8 ist ein zusammengeschrumpftes "gute results of the empirical study, the problem of com- Nacht" (lies: "g" = "gut" und "n8" = "N-Acht") putational modeling of system’s reaction to the gn8 is an abbreviation of "good night" (read: "g"="good" and "n8" = "n-ight") learner’s repair initiation will be approached. The 620 N04 oder englisch, g=good, n-eight machine will need to recognize repair initiations, or English, g=good, n-eight to extract the trouble source and to deliver an ap- 621 L08 aach sooo)) I see propriate response. The the study contributes to language understanding for dialogue systems tar- In Example 3.1, the learner initiates a repair by geting language learners and has implications for posting three question marks directly after the user and expert models for CommICALL. trouble source turn. The native speaker N04 is able to locate the trouble source, which is the abbrevi- 3 Practices of repair in chat ation. In Example 3.1, the reference to the trouble source is realised by the immediate adjacent posi- This section analyses interactional resources used tion, and signaling trouble with comprehension is by the non-native speakers in chat in order realised by the questions marks. to other-initiate repair with a linguistic trouble Candidate understanding is another possibility

397 to mark a unit of an utterance as not (completely) service can be used for definition work. Turn 376 clear. Example 3.2 shows a fragment of a chat contains an expression that the learner does not where the native speaker N04 uses the word über- (fully) understand: "in sachen essen". This expres- fülltes to describe an event in Munich (turn 222). sion is being formally made to a trouble source in The learner L08 checks her understanding of this the repair initiation in turns 377 and 378. Turn 377 term in turn 223 by copying the trouble source and locates the trouble source and marks the expres- providing her own understanding of the word. The sion as unclear. Turn 378 contains an instruction trouble source is referenced through its repetition of what kind of explanation is desired. in the repair initiation. Signalling trouble is re- Example 3.3. In Sachen Essen: repair is carried alised through the comparison token, the candidate out with the help of machine translation. understanding and the question mark. Example 3.2. Many many people 376 N03 gibt es irgendwas moskau typisches in sachen essen? is there something of food which is typical for 221 L08 ja ich habe über Oktoberfest gehört, etwas moscow? lustiges und buntes)) 377 L07 in sachen essen??? yes I have heard about Oktoberfest, something in things food??? funny and colourful 378 L07 übersetze bitte))) 222 N04 ja, und teures und überfülltes ;-) translate please [smile] yes, and expensive and overfilled 379 N03 какая пища является типичным Москве? 223 L08 ))überfülltes bedeutet "viele viele Leute"? which food is typical for Moscow? overfilled means "many many people"? 224 N04 genau exactly 4 Empirical findings The repair initiations produced by the learners in the dataset always try to resolve problems with the Regarding repair initiations, it was found that: meaning, none of them was concerned with the (1) Questioning is the practice to initiate repair form by itself. in chat, confirming the results in the academic literature for oral interaction (Dingemanse et al., 3.2 Repair carry-out 2014). Other practices are declarations of lack Repair carry-out strategies depend on the type of of understanding such as unklar and ich verstehe the trouble source and the repair initiation for- nicht. mat and include confirmations / disconfirmations, (2) Devices for signalling are question marks, definition work and paraphrasing of the trouble dashes, explicit statements of non-understanding source. Direct definition work can be replaced or and presenting candidate understandings. extended by a hyperlink to an example or a demon- (3) References to trouble sources may be re- stration of an instance of the trouble source. alised through the adjacent position, demonstra- If the trouble source is an abbreviation, the def- tive expressions and full or partial repeats. inition work contained a full spelling of the abbre- (4) Though all repair initiations were second- viated words and their explanation. For chat ab- position initiations, they were not all immediate. breviations, a full reading of the abbreviation was Delayed repair initiation require more specific ref- normally provided and enough for explanation, as erencing to trouble source, open-class repair initi- Example 3.1 demonstrates. Problematic abbrevia- ations cannot be used in a delayed second position. tion were always repeated in the dataset, followed (5) Repetition-based repair initiations may con- by the full spelling or reading. tain repetitions of one specific unit from the previ- If the trouble source is one semantic unit (one ous turn and contain a copy of the preceding turn word or an idiomatic expression), a dictionary-like regardless the unit boundaries. The latter may be definition (synonyms + examples) is often selected placed between open class and restricted class re- to provide a repair. For longer messages or longer pair initiations. Such types of repetitions have not parts of longer messages, a strategy of splitting the been previously described in the academic litera- message into smaller semantic units and a separate ture and may be typical for non-native speakers. explanation of each unit can be chosen. Paraphras- (6) The communication medium influences re- ing is also one of the strategies used by the native pair initiation types and formats. In particular, re- speakers to explain longer messages. pair initiations eliciting a repetition of the trouble Example 3.3 shows how a machine translation source are uncommon in chat. Misreadings are

398 possible, but they are made visible through mis- repair initiation recognition routine needs to be ac- productions in repetition-based repair initiations. tivated after every user’s turn. Two essential prob- (7) The non-native speakers’ identity influences lems must be solved by a computer program in or- the format of candidate understandings which dif- der to react to a repair initiation properly: fer from those in native speaker talk. (1) Recognition of a repair initiation, (8) Repair initiation is one option to deal with (2) Extraction of the trouble source. trouble in comprehension. Other options include A repair proper needs to be generated after that. dictionary look-up and the "let-it-pass" strategy. Regarding repair carry-outs, it was found that: 5.1 Recognition of repair initiations (1) Explanations of the meaning through syn- Each class of repair initiations implies a specific onyms or paraphrases, translations and demonstra- form of referencing the trouble source. We con- tions are common forms of repair carry-outs. sider the following types of referencing for mod- (2) Repair design is linked to expectation of elling of the OISRL-sequences: what is known to the repair recipient. Conse- 1. Repeat-based initiations: reuse (a 1:1-copy of quently, repairs are designed for the language the trouble source), recycle the trouble source learners targeting difficulties in linguistic matters. (rewriting it in a slightly different way), (3) Repair carry-outs may be immediate and delayed. Consequently, references to trouble source 2. Demonstratives-based initiations: using may be realised by the same resources as for re- demonstrative determiners and pronouns. pair initiations. However, there are dependencies between types of trouble source and participants’ 3. Open-class initiations: referencing by a state- selection of resources for referencing the trouble ment of non-understanding in the immediate source. For instance, abbreviations are usually re- position. The adjacent position of the repair peated. initiation references the whole preceding turn (4) Split-repeat is a type of a reference to the as a trouble turn. Therefore we refer to this trouble source which did not appear in repair type of referencing as reference by position. other-initiations but was found in the correspond- Each class of repair initiations references trouble ing self-repair carry-outs. This way of referencing of a particular size: either it is the whole preceding corresponds to self-repairs where native speakers message (open-class and demonstratives-based re- only explained a few words from a longer turn or pair initiations) or it is only a part of it (repeat- longer part of a turn marked as a trouble source. based and recycle-based initiations). Therefore, The trouble source was split in tokens, and only to- we consider three cases of trouble sources: sin- kens that were supposed to cause the trouble were gle word (part of a longer message or a one-word explained. message), part of a message (PoM) of two or more Repair carry-out is the preferred and the most words and a whole message consisting of two or frequent response to a repair initiation but other more words. forms of responses are also possible, for instance Signalling trouble involves symbolic and/or lex- a new repair initiation to deal with difficulties in icalised means and a specific format designed ei- identification of the trouble and responses which ther to mark something as unclear or to compare do not address the trouble. Finally, repair initia- the trouble source with the own version of under- tion and carry-out formats need to be "translated" standing. We call this signalling format. into patterns and then into computational models The architecture of the repair initiation (RI) for of repair to make the findings applicable for com- OISR can be formalised as follows. Depending putational purposes. L on the time, different formats for the repair initiation may be used: 5 Computational model of OISRL RI = TIME RIF ormat × In order to "serve computational interests" (Sche- Time may be immediate or delayed: TIME = gloff, 1996), the following needs to be taken into immediate, delayed . A repair initiation for- { } account for the purpose of modelling. Because re- mat is a combination of a reference to the trouble pair initiations may occur everywhere, each user’s source and a selected signalling format: utterance may be a repair initiation. Therefore, a RIF ormat = REF SignalF ormat ×

399 The referencing types are repeat-based Delayed self-repairs need to update the focus of repeat(x), based on demonstratives Dem and the talk, and therefore, a repeat-based reference reference by position AP . Signalling format may makes more sense than other types of referencing. mark something in the trouble-turn as unclear In practice, the function explain(x) needs to unclear(x) or present a candidate understanding be implemented differently for different types of equals(x, y). The trouble source x and the trouble source. The quality of the response is candidate understanding y may be a single word, highly dependent on the linguistic resources avail- an idiomatic expression, part of a message or a able for the generation of the explanations. We complete turn (utterance). discuss various practical issues in the next section. REF = repeat(x), AP, Dem { } SignalF ormat = unclear(x), equals(x, y) 5.3 Model validation { } x, y word, idiom, P oM, utterance The purpose of this section is to validate the prac- ∈ { } This repair recognition procedure is also ex- tical applicability of the abstract model described pected to differentiate between ordinary questions in the preceding section. Because language under- related to the subject of the ongoing talk and repair standing and generation capabilities of each dia- initiations. It works because ordinary questions logue system determines the possibilities for im- are not formatted as unclear(x) or equals(x, y). plementation of the OISRL model, we took the If a complete turn is recognised as a trouble simplest form of such a system, namely an AIML- source and this turn is a longer message, further based chatbot (Bush, 2006). AIML (Artificial In- filters may be applied to identify more precisely, telligence Markup Language) covers the language which of the parts of the longer message may understanding and generation task (Droßmann, cause a problem with comprehension. This may 2005) in form of pattern-template pairs shown be- be influenced by the learner model, but also by the low. If the chatbot finds an input that matches to system’s capabilities to generate a repair proper. WIE GEHTS, the utterance stored in the template Section 5.3 will address this problem and provide tag will be delivered to the user as a response. examples of possible filters. WIE GEHTS Repair carry-outs can contain a lexical reference Example 5.1 illustrates how a chatbot can benefit to the trouble source, such as repeat-based and from patterns extracted from the dataset to come demonstratives-based references, or point to it just closer to the behaviour of a language expert. by the adjacent position to the repair initiation. A confirmation or a disconfirmation is an appro- Example 5.1. A sub-dialogue with the chatbot: priate type of self-repair carry-out after a repair other-initiated self-repair where the chatbot is the other-initiation presenting candidate understand- trouble-speaker. 1 User wie gehts? ings equals(x, y). All other self-repair carry-outs how are you? are expected to provide an explanation of the unit 2 Bot Gut, und selbst? Alles paletti? that is marked as problematic explain(x). Be- I’m fine, and you? Everything okay? 3 User paletti? cause different options are available for referenc- 4 Bot umgangssprachlich alles gut, alles in Ordnung, ing trouble source in immediate and delayed repair alles okay. colloquial everything good, everything fine, ev- carry-outs, time needs to be taken into account in erything okay. the abstract description: The bot uses a colloquial expression in turn 2 RCO = TIME RCOF ormat × which is not clear for the user. The user initiates TIME = immediate, delayed . { } the repair in turn 3. The bot recognises turn 3 as A self-repair carry-out is a product of a ref- a repair initiation and extracts the trouble source: erence to the trouble source and the function the repeated word paletti and the corresponding RCOF , which it is expected to perform: confirm- idiomatic expression alles paletti. Bot’s response ing/disconfirming answer or an explanation. in turn 4 is a repair carry-out generated from a lin- RCOF ormat = REF RCOF guistic database. × REF = repeat(x), AP, Det, splitRepeat(x) The work of the repair manager is organised in { } RCOF = explain(x), conf(equals(x, y)) two steps determined by the model. Every user’s { }

400 input that requires an explanation of a single en- planation processor returns and the tity (word, idiom) is redirected to the category that pre-stored Response-1 is sent to the user. If the implements this function. The implementation trouble source is found but its meaning is not of ProgramD includes so called processors stored in the database, the explanation processor to process specific AIML tags. A new AIML returns . A predefined tag has been introduced for the purpose of this Response-2 is then sent to the user. Finally, if the work: . An additional pro- explanation processor finds the trouble source in cessor named explanation processor has the database and at least one meaning of it is de- been implemented to generate a response. scribed, an explanation will be rendered. Five ad- The model for the recognition of repair initia- ditional categories not shown here are responsible tions described in Section 5.1 is used for the im- for rendering of the explanation and process mean- plementation in form of the rules describing re- ings, examples and notes. pair initiation formats. For instance, to recognise known beforehand in AIML-based chatbots, it is Every user’s input that corresponds to an in- possible to list all idioms to make their recognition quiry "does x mean y?" is redirected to the AIML easier. For this test implementation, a short list of category implementing meaning checks. An ad- idiomatic expressions and their parts was created. ditional tag has been added The explanation processor would first check, if the to carry out the repair of this type. The han- trouble source may be an idiom (comparing with dling of the meaning checks works in a similar the list and own preceding turns). If so, the entire way as the explanations described above. The pro- expression will be set as the trouble source. gram has been extended by a meaning check AIML provides a possibility to forward inputs processor to process this tag in the following with the same or similar meanings to a particular way. To generate a response to a candidate under- category handling responses to this meaning. Int standing, the chatbot needs to answer the question this way, all recognised repair initiations with the if x means the same as y? This is an instance of the meaning unclear(x) are redirected to the category textual entailment problem. If x is a single word, with the pattern: an idiom, a collocation or a proverb, the system ICH VERSTEHE * NICHT can check the list of the synonyms of the corre- where * is the matching token for the trouble sponding entry in the linguistic database. If x and source x. y are listed as synonyms, a confirming answer will The following template is responsible for the be generated. Otherwise, the system will explain generation of repair carry-outs for all such trou- the meaning of x. ble sources. The tag allows process- Only simple versions for each of paraphrasing ing of an input without without immediate output. and word-by-word explanation (split-reuse) were The explanation processor searches for the trou- implemented. A word-by-word explanation only ble source in the linguistic database which con- makes sense for words that could be difficult for tains only meanings, examples and notes about the learner. We use a list of 100 and 1000 most fre- 1 usage for German nouns, verbs, adjectives and quently used German words to filter those words adverbs. The database was automatically gener- that are supposed to be well known to everybody. ated from Wiktionary. If the trouble source can- The remaining words are explained separately. not be found in the linguistic database, the ex- 1http://wortschatz.uni-leipzig.de/html/wliste.html

401 6 Results mentable with a simple language understanding technology. The separation between resources for The new model of other-initiated self-repair when signalling trouble and resources for referencing the machine is the trouble-speaker allows recog- trouble source allows creating a rule-based gram- nising learner repair initiations and extracting the mar which can be implemented in dialogue sys- trouble source based on a description of language- tems with different levels of complexity. specific and medium-specific resources for repair initiation. The model is created on a necessary With regard to the analysis of causes of trou- level of abstraction to be applicable for text chat bles in understanding introduced in (Schlangen, interaction in languages other than German. This 2004), mainly problems on the level of meaning assumption builds on (Dingemanse et al., 2014)’s and understanding were subject of learner’s repair finding that similar repair initiation formats exist initiations. Consequently, the modelling was ap- across languages. Therefore, when provided a set proached in this work with the assumption that of language-specific devices for repair initiation, it the required kind of clarification is mainly deter- can be implemented for other languages. The ex- mined by the user model targeting language learn- traction of the trouble source is based on abstract ers. Similarly to the (Schlangen, 2004)’s approach features like repetition of parts of the trouble-turn to map the variance in form to a small number of and adjacent position. These features are language readings, repair initiations in this work are mapped independent. either to a content question What does X mean? or The problem of the trouble source extraction to a polar question Does X mean Y? where X is the is related to referring expression recognition or trouble source and Y is the candidate understand- reference resolution described in NLP textbooks ing. In this way, the two approaches to modelling (Martin and Jurafsky, 2009, Ch. 21), which is repair initiations are similar. addressed in a large number of scientific publica- Models of repair covering repair initiations pro- tions (Dahan et al., 2002; Iida et al., 2010). Usu- posed in (Purver, 2004) and (Schlangen, 2004) ally only noun phrases or their pronominalised and extended in follow-up work (Purver, 2006; alternatives are considered for reference resolu- Ginzburg et al., 2007; Ginzburg, 2012) were moti- tion in NLP. These are usually definite and indefi- vated by Conversation Analysis research. How- nite noun phrases, pronouns, demonstratives and ever, other approaches for modelling were pre- names. The analysis of repair initiations shows ferred because of the insufficient operationalisa- that verbs or parts of utterances may be used to tion of CA findings for computational modelling. refer to the trouble source. The presented model As an implication, the factors influencing the in- implicitly includes a local discourse model which teraction that have been identified as important "contains representations of entities which have in CA studies and building a system did not be- been referred to in the discourse" (Martin and Ju- come part of the baseline models in (Purver, 2004) rafsky, 2009, p. 730). The local discourse model and (Schlangen, 2004). Such factors include re- in repair sequences only conserns possible repre- pair, turn taking, membership categorisation, ad- sentations of the trouble source. jacency pairs and preference organisation. In con- Compared to the model of clarification requests trast to the previous models of repair (Purver, proposed in (Purver, 2004), the model introduced 2004; Schlangen, 2004) this work analyses repair in this work has the following advantages. First, initiations in a system of interconnected factors the inconsistencies form CA perspective found in in conversation. More specifically, the proposed (Purver, 2004)’s classification do not exist in the model of repair initiations takes turn taking and se- model presented in this work because of a close quential organisation of interaction explicitly into cross-disciplinary connection with CA. The model account by distinguishing between immediate and for repair initiations presented here strictly differ- delayed repair initiations and respective options entiates next-turn repair other-initiations from all for trouble source extraction. In addition, the new other types of repair and describes only these re- model takes virtual adjacency in chat into account. pair initiations. Second, (Purver, 2004) introduced It explicitly differentiates repair initiated by the the model for clarification requests in a strong con- user from repair initiated by the system taking the nection to the HPSG formalism. In contrast, the sequential organisation into account. Finally, the model presented in this work is already imple- preference organisation and recipient design were

402 taken into account by the user model. Based on the guage learners allows an implementation in a empirical findings, the user model assumes that CommICALL system that helps to practice con- language learners will request a special kind of versation. In this way, this research advances clarification. state-of-the-art in ICALL and strengthens mul- While recognition of repair initiations and trou- tidisciplinary connections to related disciplines, ble source extraction can be implemented us- such as Conversation Analysis and NLP. Other ing the simplest type of language understand- types of tutorial dialogues where a clarification of ing, namely, pattern-based language understand- the terminology may be necessary would also ben- ing, most repair carry-outs require more sophisti- efit from the presented model. cated linguistic capabilities. Definitions provide an explanation of the trou- 7 Conclusions ble source. Existing online dictionaries such as This article describes typical interactional re- Wiktionary or Wikipedia may be used to create sources employed for repair in native/non-native linguistic knowledge bases. Because one term speaker chat with the purpose of computation may have multiple meanings, a linking to the cor- modelling of repair for a conversational agent in rect meaning may be required. This problem is re- a CommICALL application. The study shows lated to lexical ambiguity resolution also known as that CA methods provide a valuable set of tools meaning resolution (Small et al., 1987) and is part for computational modelling of rare phenomena of a larger area of computational lexical semantics in talk from a small number of examples. To be (Martin and Jurafsky, 2009, Ch. 20). successful, such approaches require datasets repli- Paraphrases provide a reformulation of the cating the speech exchange systems that are envi- trouble source. A lot of efforts have been put in sioned in the communication with the agent. In automatic paraphrase generation and recognition. particular, this research showed that native/non- Several recent publications are (Metzler et al., native speaker chat data can be used for computa- 2011; Regneri and Wang, 2012; Marton, 2013). tional models of dialogues in a CommICALL ap- Synonyms provide usually a short reformulation plication. of the trouble source. Existing language resources such as WordNet (Fellbaum, 2010) and GermaNet Acknowledgements (Hamp et al., 1997) can be used for finding synonyms. Multiple meanings of a word may need to This research has been carried out at the Uni- be resolved. versity of Luxembourg with the great help and Translations may be generated by using exist- support of my scientific advisors Stephan Buse- ing machine translation systems (Avramidis et al., mann (DFKI GmbH), Christoph Schommer, Gu- 2015; Burchardt et al., 2014). Open source drun Ziegler (MultiLearn Institute), Leon van der statistical machine translation systems such as Torre and Charles Max. Moses2 make experimental implementations fea- sible. Commercial machine translation API can be References integrated into the dialogue manager, for instance Google Translate API3. Eleftherios Avramidis, Maja Popovic, and Aljoscha Burchardt. 2015. DFKI’s experimental hybrid MT Demonstrations include hyperlinks to websites system for WMT 2015. In Proceedings of the 10th containing relevant information examples of an Workshop on Statistical Machine Translation. ACL, object referenced by the trouble source. For pages 66–73. semi-automatically created databases of linguistic Aljoscha Burchardt, Arle Richard Lommel, Georg knowledge, such information may be included into Rehm, Felix Sasaki, Josef van Genabith, and Hans examples. Wikipedia articles sometimes also con- Uszkoreit. 2014. Language technology drives qual- tain links to example websites and pictures, which ity translation. MultiLingual 143:33–39. may be used as examples of concepts described in Noel Bush. 2006. Program D. the article. http://www.aitools.org/Program_D. Explicit handling of repairs targeted for lan- Okko Buß and David Schlangen. 2011. DIUM – An 2http://www.statmt.org/moses/ Incremental Dialogue Manager That Can Produce 3https://cloud.google.com/translate/docs Self-Corrections. In SEMDIAL.

403 Delphine Dahan, Michael K Tanenhaus, and Craig G Sviatlana Höhn. 2015. deL1L2IM: Corpus of Chambers. 2002. Accent and reference resolution in long-term instant messaging NS-NNS conversa- spoken-language comprehension. Journal of Mem- tions. ELRA http://islrn.org/resources/339-799- ory and Language 47(2):292–314. 085-669-8/.

Sviatlana Danilava, Stephan Busemann, Christoph Yuri Hosoda. 2006. Repair and relevance of differen- Schommer, and Gudrun Ziegler. 2013. Towards tial language expertise in second language conversa- Computational Models for a Long-term Interaction tions. Applied Linguistics 27(1):25–50. with an Artificial Conversational Companion. In Proc. of ICAART’13. Ryu Iida, Shumpei Kobayashi, and Takenobu Toku- naga. 2010. Incorporating extra-linguistic infor- Mark Dingemanse, Joe Blythe, and Tyko Dirksmeyer. mation into reference resolution in collaborative 2014. Formats for other-initiation of repair across task dialogue. In Proceedings of the 48th Annual languages: An exercise in pragmatic typology. Stud- Meeting of the Association for Computational Lin- ies in Languag 3(81):5–43. guistics. Association for Computational Linguistics, pages 1259–1267. Christian Droßmann. 2005. German AIML set. http://www.drossmann.de/wordpress/alicebot/. Cui Jian, Desislava Zhekova, Hui Shi, and John Bate- man. 2010. Deep Reasoning in Clarification Dia- Myroslava O Dzikovska, Charles B Callaway, Elaine logues with Mobile Robots. In Proceedings of the Farrow, Johanna D Moore, Natalie Steinhauser, and 19th European Conference on Artificial Intelligence. Gwendolyn Campbell. 2009. Dealing with interpre- IOS Press, pages 177–182. tation errors in tutorial dialogue. In Proceedings of the SIGDIAL 2009 Conference: The 10th annual Geert-Jan M. Kruijff, Michael Brenner, and Nick meeting of the special interest group on discourse Hawes. 2008. Continual planning for cross- and dialogue. Association for Computational Lin- modal situated clarification in human-robot interac- guistics, pages 38–45. tion. In The 17th IEEE International Symposium on Robot and Human Interactive Communication. Christiane Fellbaum. 2010. Princeton university: IEEE, pages 592–597. About WordNet. http://wordnet.princeton.edu. WordNet. Anthony J. Liddicoat. 2011. An Introduction to Con- Christine Fredriksson. 2012. About collaboration, in- versation Analysis. Continuum. teraction, and the negotiation of meaning in syn- chronous written chats in l2-german. In Linda Roy Lyster, Kazuya Saito, and Masatoshi Sato. 2013. Oral corrective feedback in sec- Bradley and Sylvie Thouësny, editors, CALL: Us- Language teaching. ing, Learning, Knowing, EUROCALL Conference, ond language classrooms. 46. Gothenburg, Sweden, 22-25 August 2012, Proceed- Numa Markee. 2000. Conversation Analysis. Mah- ings. Research-publishing.net, pages 88–92. wah, N.J.: Lawrence Erlbaum. Rod Gardner and Johannes Wagner. 2004. Second Lan- guage Conversations: Studies of Communication in James H Martin and Daniel Jurafsky. 2009. Speech Everyday Settings. A&C Black - Verlag. and language processing: an introduction to natural language processing, computational linguistics and Raphaela Gehle, Karola Pitsch, and Sebastian Wrede. speech resognition. Pearson International Edition, 2014. Signaling trouble in robot-to-group inter- second edition edition. action. emerging visitor dynamics with a museum guide robot. In Proceedings of the second inter- Yuval Marton. 2013. Distributional phrasal paraphrase national conference on Human-agent interaction. generation for statistical machine translation. ACM ACM, pages 361–368. Transactions on Intelligent Systems and Technology (TIST) 4(3):39. Jonathan Ginzburg. 2012. The interactive stance. Ox- ford University Press. Raveesh Meena, José Lopes Gabriel Skantze, and Joakim Gustafson. 2015. Automatic detection of Jonathan Ginzburg, Raquel Fernández, and David miscommunication in spoken dialogue systems. In Schlangen. 2007. Unifying self-and other-repair. 16th Annual Meeting of the Special Interest Group In Proceeding of DECALOG, the 11th International on Discourse and Dialogue. pages 354 – 363. Workshop on the Semantics and Pragmatics of Dia- logue (SemDial07). Donald Metzler, Eduard Hovy, and Chunliang Zhang. 2011. An empirical evaluation of data-driven para- Birgit Hamp, Helmut Feldweg, et al. 1997. Germanet- phrase generation techniques. In Proceedings of the a lexical-semantic net for german. In Proceedings 49th Annual Meeting of the Association for Com- of ACL workshop Automatic Information Extraction putational Linguistics: Human Language Technolo- and Building of Lexical Semantic Resources for NLP gies: short papers-Volume 2. Association for Com- Applications. Citeseer, pages 9–15. putational Linguistics, pages 546–551.

404 Kenneth A Petersen. 2010. Implicit Corrective Feed- back in Computer-Guided Interaction: Does Mode Matter?. Ph.D. thesis, Georgetown University. Matthew Purver. 2004. The theory and use of clari- ﬁcation requests in dialogue. Ph.D. thesis, King’s College, University of London. Matthew Purver. 2006. CLARIE: Handling clariﬁca- tion requests in a dialogue system. Research on Lan- guage and Computation 4(2-3):259–288.

Michaela Regneri and Rui Wang. 2012. Using discourse information for paraphrase extraction. In Proceedings of the 2012 Joint Conference on Empir- ical Methods in Natural Language Processing and Computational Natural Language Learning. Asso- ciation for Computational Linguistics, pages 916– 927. Yuan Ma Satomi Kawaguchi. 2012. Corrective feedback, negotiation of meaning and grammar develop- ment: Learner-learner and learner-native speaker interaction in esl. Open Journal of Modern Linguistics 2:57–70. Emanuel A Schegloff. 1996. Issues of relevance for discourse analysis: Contingency in action, interaction and co-participant context. In Computational and conversational discourse: Burning Issues – An Interdisciplinary Account, Springer-Verlag Berlin Heidelberg, pages 3–35. Emmanuel A. Schegloff. 2000. When ’others’ initiate repair. Applied Linguistic 21(2):205–243. David Schlangen. 2004. Causes and Strategies for Re- questing Clariﬁcation in Dialogue. In 5th Workshop of the ACL SIG on Discourse and Dialogue. Elizabeth Ellen Shriberg. 1994. Preliminaries to a The- ory of Speech Disﬂuencies. Phd, University of Cali- fornia at Berkeley. Steven Small, Garrison Cottrell, and Michael Tanen- haus. 1987. Lexical ambiguity resolution. Morgan Kaufman Publishers, Inc., Los Altos, CA. Evangeline Marlos Varonis and Susan Gass. 1985. Non-native/non-native conversations: A model for negotiation of meaning. Applied Linguistics 6(1):71–90. Sabrina Wilske. 2014. Form and Meaning in Dialog-Based Computer-Assisted Language Learn- ing. Ph.D. thesis, University of Saarland. Simon Zwarts, Mark Johnson, and Robert Dale. 2010. Detecting Speech Repairs Incrementally Using a Noisy Channel Approach. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010). August, pages 1371– 1378.

405