A Prolog Datamodel for State Chart XML

Stefan Radomski Dirk Schnelle-Walka Stephan Radeck-Arneth TU Darmstadt TU Darmstadt TU Darmstadt Telekooperation Group Telekooperation Group Telekooperation Group Hochschulstrasse 10 Hochschulstrasse 10 Hochschulstrasse 10 [email protected] [email protected] [email protected] .tu-darmstadt.de .tu-darmstadt.de .tu-darmstadt.de

Abstract log control, such as error correction or even sensor fusion/fission. SCXML was proposed as one description As one proposed XML dialect for control doc- language for dialog control in the W3C uments, State Chart XML (SCXML) is given the Multimodal Architecture but lacks the fa- responsibility to model an applications dialog be- cilities required for grounding and rea- havior. SCXML as such is a markup language to soning. This prohibits the application of express Harel state charts (Harel and Politi, 1998) many dialog modeling techniques for mul- with nested and parallel machine configurations. timodal applications following this W3C The transitions between configurations are trig- standard. By extending SCXML with a gered by events delivered into the interpreter ei- Prolog datamodel and scripting language, ther from external components or raised by the we enable those techniques to be em- interpreter itself. Whenever an event arrives, the ployed again. Thereby bridging the gap SCXML interpreter can perform actions described between respective dialog modeling re- as executable content. This includes invoking or search and a standardized architecture to sending events to external components, processing access and coordinate modalities. data or updating the datamodel via an embedded scripting language. 1 Introduction SCXML has been proven to be suitable to de- Deploying multimodal applications has long been couple the control flow and presentation layer an activity of custom solutions, each with their in dialog management (Wilcock, 2007). It has own access to modalities, approaches to sensor fu- been used in several applications to express dialog sion and fission and techniques for dialog mod- states (Brusk et al., 2007) or to easily incorporate eling. With the advent of the W3C MMI archi- external information (Siguenza¨ Izquierdo et al., tecture (Bondell et al., 2012), the W3C proposed 2011). However, SCXML seems to be suited only a standardized approach to ensure interoperabil- to implement finite state or frame-based/form- ity among its constituting components (Schnelle- filling dialogue management approaches. Appli- Walka et al., 2013; Dahl, 2013). cations using theses dialog techniques are often- The architecture proposed by the W3C decom- times inflexible as they lack grounding and rea- poses a multimodal application into a nested struc- soning. In this regard, Fodor and Huerta (2006) ture of interaction managers for dialog control and demand that dialog managers should feature: (i) a modality components for in- and output. An ap- formal logic foundation, (ii) an interference en- plication is conceived as a set of control docu- gine, (iii) general purpose planners and (iv) knowl- ments expressed in SCXML (Barnett et al., 2012) edge representation and expressiveness. or CCXML (Auburn et al., 2011) for the interac- Most of these requirements are addressed by tion managers and a set of presentation documents employing Prolog. Embedding it as a scripting with modality-specific markup for the modality language into SCXML allows multimodal applica- components. A topmost root controller document tions in the W3C MMI Architecture to employ the describes the global dialog and instantiates modal- more elaborate dialog management techniques, re- ity components as required. Each modality com- sulting in more natural and flexible interaction. In ponent can, in turn, again be an interaction man- this paper we describe our integration of Prolog ager, handling more fine granular concerns of dia- as an embedded scripting language in an SCXML

127 Proceedings of the SIGDIAL 2013 Conference, pages 127–131, Metz, France, 22-24 August 2013. c 2013 Association for Computational Linguistics datamodel. All of the work described here is im- % element(child, plemented as part of our uSCXML interpreter1 by % [father=bob, name=jim], [])]). embedding the SWI Prolog implementation. { name:"The Bobsons", members: [’bob’,’martha’,’jim’,’john’] 2 The Prolog Datamodel } Datamodels in SCXML are more than simple % household({ % name:’The Bobsons’, repositories for storing data. With the exception % members:[bob, martha, jim, john]}). of the null datamodel, they provide access to Listing 1: Assignments and their results in Prolog. embedded scripting languages. The datamodels already specified by the SCXML draft are the If given, the id or location attribute iden- null, the xpath and the ecmascript data- tifies the predicate for which the content is to be model. Prolog itself is a declarative language for asserted as fact, otherwise the content is assumed logic programming in which facts and rules are to be a dot-separated list of prolog queries or ex- used to answer queries. The result of a query is pressions. The content might also be loaded per either a boolean value or the set of valid assign- URL in the element’s src attribute. In the con- ments for the queries variables. text of SCXML, it is important to support XML In the following sections, we will describe our and JSON data as shown in the last two examples. integration of Prolog as a datamodel in SCXML. Not only enables this an application developer to The structure of the description loosely follows the load data from existing XML and JSON files, it is existing descriptions for datamodels already found also important to support these representations for in the SCXML draft. incoming events as we will see in the next section. There is no standardized representation for 2.1 Assignments XML DOMs or JSON data in Prolog. We prag- In an SCXML document, there are two elements matically settled upon the structure returned by the which will assign values to variables in the data- SWI-Prolog SGML parser and the JSON converter model. These are for initial assignments as de-facto standards respectively. and itself. In Prolog, variable assign- With the Prolog datamodel, having an id ment is only available in the scope of a query. To or location attribute at assignment elements realize variable assignment nevertheless, we in- seems superfluous. We do keep them as the troduce the variables as predicates, with their as- SCXML draft specifies these as required at- signed data as facts. Listing 1 exemplifies some tributes. assignments followed by their resulting Prolog facts. 2.2 Structure of Events Whenever an event is received by the SCXML in- bob, jim. bob, john. terpreter, it has to be transformed into a suitable % father(bob, jim). representation in order to operate on its various % father(bob, john). fields and content as defined by the SCXML draft. We choose to represent an event as the single pred- mother(martha, jim). mother(martha, john). icate event/1 with its facts as compound terms reflecting the event’s fields as shown in listing 2. % mother(martha, jim). % mother(martha, john). event(name(’foo’)). event(type(’external’)). event(sendid(’s1.bar’)). retract(father(bob, jim)). event(origin(’http://host/path/basichttp’)). assert(father(steve, jim)). event(origintype(’http://www.w3.org/TR/scxml /#BasicHTTPEventProcessor’)). % father(bob, john). event(invokeid(’’)). % father(steve, jim). event(data(...)). event(param(...)). event(raw(...)). Listing 2: Example facts for event/1. % childs([ % element(child, This representation enables access to the events % [father=bob, name=john], []), individual fields by simple queries such as 1https://github.com/tklab-tud/uscxml event(name(X)), which will resolve X to the

128 event’s name foo. Whenever the interpreter is Deﬁning two predicates for ioprocessors is sim- about to process a new event, all old facts about ply a matter of convenience as their short names event/1 are retracted and reasserted with regard (e.g. basichttp or scxml) are suited as functors to the new event. for compound terms, where their canonic names The event’s data ﬁeld may contain a space nor- are not. Therefore ioprocessors/1 will only malized string as an atomic term, an XML DOM contain the short names, and ioprocessors/2 or, optionally, data from a JSON structure. The contains both. This allows us to send events, e.g structure of JSON and XML DOMs is the same as with the basichttp ioprocessor via: with assignments in listing 1. The