Computational Creativity Infrastructure for Online Software Composition: A Conceptual Blending Use Case Martin Znidarˇ siˇ cˇ1, Am´ılcar Cardoso2, Pablo Gervas´ 4, Pedro Martins2, Raquel Hervas´ 4, Ana Oliveira Alves2, Hugo Gonc¸alo Oliveira2, Ping Xiao3, Simo Linkola3, Hannu Toivonen3, Janez Kranjc1, Nada Lavracˇ1 1Jozefˇ Stefan Institute, Ljubljana, Slovenia 2CISUC, DEI, University of Coimbra, Coimbra, Portugal 3Department of Computer Science and HIIT, University of Helsinki, Finland 4Universidad Complutense de Madrid, Spain Abstract used in text mining tasks. Text mining has numerous open source algorithms and natural language processing (NLP) Computational Creativity is a subfield of Artificial Intel- ligence research, studying how to engineer software that software libraries available (such as NLTK (Bird 2006) and exhibits behaviors which would reasonably be deemed scikit-learn (Pedregosa et al. 2011)). However, even text creative. This paper shows how composition of soft- mining and NLP experiments are still difficult to reproduce, ware solutions in this field can effectively be supported including the difficulty of systematic comparison of algo- through a Computational Creativity (CC) infrastructure rithms. To this end, a number of attempts have been made that supports user-friendly development of CC software to develop easy-to-use workflow management systems, al- components and workflows, their sharing, execution lowing users to compose complex processing pipelines in a and reuse. The infrastructure allows CC researchers modular visual programming manner. to build workflows that can be executed online and be reused by others with a single click on the workflow web address. Moreover, it allows building of proce- Related work As regards the work related to the platform dures composed of software developed by different re- 1 searchers from different laboratories, leading to novel presented in this paper, we first mention myGrid which is ways of software composition for computational pur- used primarily for bioinformatics research, having in mind poses that were not expected in advance. This capabil- experiment replication. It is currently probably the most ad- ity is illustrated on a workflow that involves blending of vanced workflow management system, although, due to its texts from different domains, blending of correspond- complexity, not very easy to use. The most important part ing images, poetry generation from texts as well as con- of myGrid is Taverna, which is conceived as a suite of tools struction of narratives. The paper concludes by present- used to design and execute scientific workflows. A multi- ing plans for future work. lingual Internet service platform Language Grid2, which is based on a service-oriented architecture and supports a web- Introduction oriented version of the pipeline architecture typically em- ployed by NLP tools, is open source, but it is quite complex Computational creativity (CC) systems use as their basic 3 ingredients different types of resources, including musical, to install and use. The ARGO platform is a more recent pictorial and textual, to name a few. This paper focuses on development, which enables workflows to have interactive infrastructure support to CC systems that base their creativ- components, where the execution of the workflow pauses to ity on textual resources. Such CC systems include poetry receive input from the user, but ARGO is not open source generation, metaphor creation, generation of narratives, cre- and does not have sophisticated utilities for cataloguing the ation of fictional ideas and conceptual blending, which all available web services or workflows, nor a system of access represent CC tasks which request manipulation of text re- permissions. 4 sources that are provided as inputs. Our recently developed platform ClowdFlows (Kranjc, Infrastructures supporting text-based creative systems are Podpecan,ˇ and Lavracˇ 2012) is web-based thus requiring scarce. Ideally, a text-based CC system would automatically no local installation, is simple to use and install, and avail- build creative artefacts from the given text resources, which able as open source under the MIT Licence. While Clowd- 5 the end user would then inspect and potentially adapt to their Flows is mainly devoted to data mining, its fork TextFlows needs. An attempt in this direction is the FloWr system is focused on text mining and NLP workflows. A fork for automated flowchart construction, optimisation and al- platform for facilitation and reuse of computational creativ- teration (Charnley, Colton, and Llano 2014). While getting software to write CC code directly is a long-term research 1http://www.mygrid.org.uk/ goal, that line of research is—with the exception of FloWr— 2http://langrid.org/ still in its infancy stage. A substantially more mature area 3http://argo.nactem.ac.uk/ of research concerns the development of infrastructures sup- 4http://clowdflows.org porting modular development, sharing and execution of code 5http://textflows.org ity software is called ConCreTeFlows6. It is an indepen- and workflow components. Workflow components of sev- dent platform with a specific backend that is being continu- eral types allow graphical user interaction during run-time, ously adapted to computational creativity tasks and tools. As and visualization of results by implementing views in any forks of ClowdFlows, TextFlows and ConCreTeFlows bene- format that can be rendered in a web browser. Below we ex- fit from its service-oriented architecture, which allows users plain the concept of workflows in more detail and describe to utilize web-services as workflow components. The distin- the basic concepts of ConCreTeFlows. guishing feature of these platforms is the ease of sharing and The workflow model is the main component of the Con- publicizing workflows, together with an ever growing roster CreTeFlows platform and consists of an abstract represen- of reusable workflow components and entire workflows. As tation of workflows and workflow components. The graph- completed workflows, data, and results can be made public ical user interface for constructing workflows follows a vi- by the author, the platform can serve as an easy-to-access sual programming paradigm which simplifies the represen- integration platform for data mining, text mining or com- tation of complex procedures into a spatial arrangement of putational creativity processes. Each public workflow is as- building blocks. The basic unit component in a ConCreTe- signed a unique URL that can be accessed by anyone to ei- Flows workflow is a processing component, which is graph- ther replicate the experiment, or use the workflow as a tem- ically represented as a widget. Considering its inputs and plate to design new similar workflows. parameters every such component performs a task and stores the results on its outputs. Different processing components are linked via connections through which data is transferred In this paper we present ConCreTeFlows Contributions from a widget’s output to another’s input. An alternative and illustrate its use in a specific use case of conceptual widget input for a widget are parameters, which the user blending (introduction to blending theory is provided on enters into widget’s text fields. The graphical user inter- page 3). This example employs multiple software compo- face implements an easy-to-use way of arranging widgets nents that are being developed by various members of the on a canvas to form a graphical representation of a complex computational creativity community. The presented com- procedure. Construction of new workflows thus requires no position of software aims to conduct conceptual blending expertise, apart from knowing (usually from widget docu- conceptually, textually and visually. Given two descriptions mentation) the inputs and outputs of the widgets to ensure of arbitrary concepts in natural language, the presented ap- their compatibility. Incorporation of new software compo- proach provides conceptual graph representations of both nents, on the other hand, requires basic programming skills concepts and their blend, a textual description of the blended in Python or SOAP web-service development in any pro- concept and even a set of possible visual blends. gramming language. The paper is structured as follows. The first section ConCreTeFlows implements its own workflow execution presents ConCreTeFlows as a special purpose workflow engine. Currently there are no ways to reuse the workflows management platform aimed at supporting computational using third party software. We plan to implement special creativity tasks. In the next section is the core of this pa- widgets that will define inputs and outputs for REST API per. It provides a description of the use case and the basics endpoints which will allow execution of workflows on vari- of its theoretical foundations, followed by presentation of all able inputs by any third party software. the important methods and software components that are ap- The ConCreTeFlows graphical user interface is shown in plied for its purpose. Last part of this section is devoted to Figure 1. On the top of the graphical user interface is a tool- critical discussion and ongoing work on the presented com- bar where workflows can be saved, deleted, and executed. ponents. The paper concludes with a brief summary and Underneath on the left is the widget repository, which is a plans for further work. list of available widgets grouped by their functionality. Click on a widget in the repository adds it to the workflow con- Software Infrastructure struction canvas on the right. A console for displaying suc- This section briefly describes the main components of the cess and error messages is located on the bottom. ConCreTeFlows. It is a special purpose workflow manage- Workflows in ConCreTeFlows are processed and stored ment platform, aimed at supporting (primarily text-based) on remote servers from where they can be accessed from computational creativity tasks. anywhere, requiring only an internet connection. By default Like ClowdFlows, ConCreTeFlows can also be used in each workflow can only be accessed by its author, although a browser, while the processing is performed in a cloud one may also chose to make it publicly available. ConCreTe- of computing nodes. The backend of ConCreTeFlows Flows generates a specific URL for each workflow that has uses Django7, which is an open source web framework. been saved as public. The users can then simply share their The graphical user interface is implemented in HTML and workflows by publishing the URL. Whenever a public work- JavaScript, using jQuery8 and jQuery-UI9 libraries. Con- flow is accessed by another user, a copy of the workflow is CreTeFlows is easily extensible by adding new packages created on the fly and added to his private workflow reposi- tory. The workflow is copied together with widgets’ param- 6http://concreteflows.ijs.si eter settings, as well as all the data, in order to ensure the 7https://www.djangoproject.com experiments can be repeated. In this way the user is able 8http://jquery.com to tailor the workflow to his needs without modifying the 9http://jqueryui.com original workflow. In most computational approaches to CB, the input and blended spaces are represented as computational versions of Conceptual Maps (Novak 1998), i.e., graphs where nodes are concepts and arcs are relations between them (see Fig- ure 3).
see horse bird purpose 2 mane qty ppty run long eye run
pw leg ability fly ability beak snout qty bird pw pw pw qty pw horse pw pw 4 pw 2 sound sound pw pw purpose qty purpose qty neigh ride chirp
hoof wing cargo pet human Figure 1: A screenshot of the ConCreTeFlows graphical user interface opened in the Mozilla Firefox Web browser, pre- Figure 3: Concept maps of horse and bird. senting a motivational CC use case. Graph representations of concept blends are useful for au- tomated analysis and further processing, but are not very Conceptual Blending Online suitable and appealing for human perception of the blended The elements of the conceptual blending (CB) theory (Fau- spaces. To improve on this aspect of conceptual blending, connier and Turner 2002) are an inspiration to many algo- we have developed methodologies and algorithms for visual rithms and methodologies in the field of computational cre- blending and for textual representation of concept graphs. ativity (Veale and O’Donoghue 2000; Pereira 2005; Thagard Using these new techniques, we designed a CB process that and Stewart 2010; Schorlemmer et al. 2014). A key ele- results in conceptual blends that are described in natural lan- ment in the theory is the mental space, a partial and tem- guage and enriched with visual representations. The process porary structure of knowledge built for the purpose of local is sketched in Figure 4, where the boxes represent the main understanding and action (Fauconnier 1994). To describe (software) components and the arrows indicate the flow of the CB process, the theory makes use of a network of four data from inputs to outputs. mental spaces (Figure 2). Two of these correspond to the Each of the main process components (that is, each box input spaces, i.e., the content that will be blended. The pro- from the sketch in Figure 4) is implemented as an inde- cess starts by finding a partial mapping between elements of pendent software solution and represented as a widget or a these two spaces that are perceived as similar or analogous group of widgets in ConCreTeFlows. in some respect. A third mental space, called generic, encap- In the following, we describe the process components and sulates the conceptual structure shared by the input spaces, their implementations in detail, with a presentation of the generalising and possibly enriching them. This space pro- whole workflow and some exemplary results at the end. vides guidance to the next step of the process, where ele- ments from each of the input spaces are selectively projected Construction of Conceptual Networks into a new mental space, called the blend space. Further The Concept Network Builder component from Figure 4 ac- stages of the process elaborate and complete the blend. cepts a textual description of a concept in natural language and on its basis produces a conceptual graph.The set of pos- sible concepts and relations in the resulting graph is open and not limited to a particular fixed set (such as relations in ConceptNet10) or linguistic characteristic. We decided to represent also relations as concepts, which allows treating a particular entity as both a concept or relation, depending on the context. For example, the concept of eating can be used to relate the concepts of cows and grass, but it can also be a concept related through is a with animal activity. The software component that creates these conceptual