An Ontology-Based Dialogue Management System for Banking and Finance Dialogue Systems Duygu Altinok 4Com Innovation Center Berlin, Germany [email protected] Abstract Keeping the dialogue state in dialogue systems is a notoriously difficult task. We introduce an ontology-based dialogue manager (OntoDM), a dialogue manager that keeps the state of the conversation, provides a basis for anaphora resolution and drives the conversation via domain ontologies. The banking and finance area promises great potential for disambiguating the context via a rich set of products and specificity of proper nouns, named entities and verbs. We used ontologies both as a knowledge base and a basis for the dialogue manager; the knowledge base component and dialogue manager components coalesce in a sense. Domain knowledge is used to track Entities of Interest, i.e. nodes (classes) of the ontology which happen to be products and services. In this way we also introduced conversation memory and attention in a sense. We finely blended linguistic methods, domain-driven keyword ranking and ​ ​ domain ontologies to create ways of domain-driven conversation. Proposed framework is used in our in-house German language banking and finance chatbots. General challenges of German language processing and finance-banking domain chatbot language models and lexicons are also introduced. This work is still in progress, hence no success metrics have been introduced yet. Keywords: ontology, knowledge base, finance ontology, dialogue management, ontology based dialogue management, ontology based conversation, chatbot, virtual personal assistant, banking and finance virtual assistant, banking and finance chatbot umlaut-to-plain vowel replaced words are also a part of 1. Challenges with German Language chatbot language models. German language processing is inherently challenging in Morphologically rich languages have received general, independent of what the specific NLP task is. The considerable attention from many researchers. Many main challenge is high variability in word forms due to technical papers have been published to highlight the inflections and compound words. inherent technical difficulties in statistical methods e.g. Finance domain lexicons include many compound words MT, ASR-TTS, language models, text classification; just like other technical domains in German. Here are practical solutions are offered in (Mikolov et al., 2016). some examples from our in-house banking and finance We overcome the challenges of rich German morphology using DEMorphy, an open source German morphological lexicon: 1 analyzer and recognizer . Throughout our work, all Verfügungsberechtigung power to draw from an account lemmatizing and morphological analysis tasks are done by Sparkonto savings account DEMorphy. Girokonto checking account Steuernummer tax ID 2. Introduction Zahlungsverkehr payment transactions Keeping dialogue state in conversational interfaces is a Zahlungsverkehrsraum payment transactions area notoriously difficult task. Dialogue systems, also known Onlinebanking online banking as chatbots, virtual assistants and conversational interfaces Hypothekendarlehen mortgage are already used in a broad set of applications, from psychological support2 to HR, customer care and Nouns, adjectives and verbs can be inflected according to entertainment. gender, number and person. Rich word forms can pose a Dialogue systems can be classified into goal-driven challenge language understanding components. In this systems (e.g. flight booking, restaurant reservation) vs paper, we focus on dialogue management. However, one open-domain systems (e.g. psychological support, should keep in mind that input to dialogue management language learning and medical aid). As dialogue systems components are provided by natural language has gained attention, research interest in training natural understanding components. conversation systems from large volumes of user data has Another practical issue in everyday written language is grown. Goal-driven systems admit slot-filling and hand the umlaut (mutated vowels). Everyday informal written ​ crafted rules, which is reliable but restrictive in the text includes umlauts replaced by their plain counterparts conversation (basically the user has to choose one of i.e. “Madchen, uber, schon” rather than “Mädchen, über, available options). Open domain conversational systems, schön” etc. Especially in conversational interfaces, usage based on generative probabilistic models attracted of umlauts reduce significantly due to English layout keyboards or just being lax about punctuation while 1 https://github.com/DuyguA/DEMorphy ​ typing quickly on a smartphone. In our opinion, 2 https://www.wysa.io ​ attention from many researchers, due these limits for highly specific domains such as medicine, law and goal-oriented systems (Serban et al., 2015; Li et al., online-shopping. 2017). Banking and finance conversations, either seeking financial advice or inquiring banking products; aim to One problem with conversation is maintaining the get information rather than to accomplish a goal. Hence dialogue state. This comprises of what the user said and throughout this work, conversations are not goal-oriented how the chatbot answered, what we’re talking about and but rather domain-driven. Users usually do not aim to ​ which pieces of information are relevant to generating the achieve a well-defined goal, i.e. book a table at a current answer. Kumar et al. (2017) introduced neural restaurant or book a flight. The financial chat is mostly networks with memory and attention (DMN). Done up to about getting information about rates, prices, investment here DMN includes episodic memory and an attention instruments and sometimes about picking a suitable option module plus to a recurrent encoder decoder. DMN first i.e. purchase advice. Purchase rarely happens immediately ​ ​ ​ computes question representation. Then the question after the advice. The banking chat can include both asking representation triggers the attention process. Finally the for information about account types, credit card types, memory module can reason the answer from all relevant their yearly fees and rates i.e. the banking products; or information. making money transfer, asking for account balance, However, purely statistical approach has some drawbacks: viewing account activity i.e. achieving a certain goal. We ● statistical frameworks need huge training sets. will address these issues in next chapters. ​ Especially frameworks with many statistical components such as DMN, have a great number 3.2 Ontology-based Dialogues of parameters and are vulnerable to sparseness Ontology-based conversating is indeed a way of problems. domain-driven conversation. We used ontologies in our ● Anaphora resolution is implicit. The anaphora work for two purposes: resolutions go into neural network as implicit ● to store the knowledge parameters, there’s no direct easy way to see how ● to navigate through the domain the resolutions worked. Answers come through at The knowledge base component takes part in many least two distinct statistical layers, one encoder dialogue systems. After the NLU component turns queries and one decoder at least. Thus there is no easy into a logical form, next step is to interrogate the way to understand why a specific answer is knowledge base for the answer generation. In banking and generated and how the anaphora resolution finance domain ontologies, one potentially needs to store contributed to the generation. ● a range of banking products: credit, credit card, This paper addresses the dialog management. We will debit card… describe domain-driven ways to ● attributes of these products: general conditions, ● keep the conversation memory, both the user and rates and fees the bot side ● range of banking services: ATM, money ● make the anaphora resolution transfer... ● generate knowledge-based answers ● attributes of the banking services: ATM points, ● possibly contribute to what to say next ​ branch addresses... ● integrate linguistic features into the context For instance, in following conversations (Fig.1 & 2) it is NLU and answer generation modules will not be not possible to generate an answer without knowing the ​ ​ considered in detail in this paper. The focus is on how product. The knowledge base module provides ontologies can be used to generate natural conversations. information to the answer generation module. However we will present the outputs and presentations for clarity. The goal is here to improve quality of conversations via domain knowledge. This work is still in User: Was kostet die Bestellung im Internet? (How much an online progress, hence we were not able to include performance purchase costs?) Bot: Es entstehen keine Kosten für Sie.. (It costs you nothing.) metrics yet, given the difficult nature of evaluation of dialogue systems in general. Figure 1: Dialogue Example 3. Proposed Framework User: Haben Sie einen Geldautomaten in Berlin-Tiergarten? (Do you have an ATM in Berlin-Tiergarten?) 3.1 The Domain Bot: Ja, …….. (Yes, ….) This paper describes the methodology that is used in our Figure 2: Dialogue Example in-house banking and finance chatbots. We chose the banking and finance domain due to rich set of products In our work the ontology also drives the conversation: and specificity of proper nouns, named entities and verbs; keeps the context, provides a basis to the anaphora ​ ​ high potential for disambiguate the context and drive the resolution and possibly produces