Image schemas in as a method for formal concept invention

Maria M. Hedblom Otto-von-Guericke University, Magdeburg http//:www.mariamhedblom.com

Supervisors: Oliver Kutz1, Till Mossakowski2 The ‘image schema’ is thought to be the abstracted spatial pattern from the embodied experience learnt in early infancy. Introduction Classic examples are CONTAINMENT,SUPPORT and PATH. In many fields artificial agents surpass humans, e.g. memory These mental structures offer a foundation to ground cogni- capacity, speed and accuracy of calculation. However, there tive phenomena, such as language, understanding, and rea- is at least one field in which humans still are superior: cre- soning, by providing a connection between the bodily expe- ative ability. Research investigating the creative side of AI is rienced relationships of physical objects in time and space called computational and it has some success with with the internal conceptual world of an agent. results such as artificial agents that paint (e.g. The Painting The cognitive benefit of image schemas lies in their gen- Fool (Colton, 2012)) and generate stories (Gervas,´ 2009). eralised nature, which enables analogical transfer of knowl- One of the hardest problems yet to be solved in computa- edge or expectations onto unknown situations. E.g., if the tional creativity and artificial general intelligence, is concept image schema of SUPPORT has been learnt through percep- invention and concept grounding. Here both what this means tual exposure of ‘plates on tables’, an infant can infer that on a cognitive level and how this is to be implemented are table-like objects such as ‘desks’ can SUPPORT ‘books’ as issues that needs to be addressed. well. As the environment becomes increasingly complex for the infant, this information transfer becomes a fundamental Research Foundation part of cognition and concept understanding. This extends to abstract language as well, where spatial Concept invention through and embodied language often is used for , e.g. “to The cognitive processes underlying concept invention are fall into a deep depression” (CONTAINMENT) and “the rise still largely unexplored ground, although promising theories to power” (VERTICALITY). have been developed within the last few decades. One of One important aspect of image schemas is their ability these is the embodied mind theory (Shapiro, 2011). It pro- to be combined with one another (and other building blocks poses that human cognition is grounded in our bodily ex- such as visual attributes) to explain increasingly complex sit- perience with the environment, and that these experiences uations. For example, consider how PATH can be combined are at the heart of how we structure our concepts, including with SUPPORT or CONTAINMENT to capture the essential abstract ones. meaning of the concept ‘transportation’. Looking at experimental studies in linguistic neuro- science (e.g. Tettamanti et al. (2005)) there are strong in- Conceptual blending dications that physical experiences are essential for concep- Conceptual blending (CB) is a theory of concept invention tual understanding. Given the role of bodily experiences in Fauconnier (1997). It argues the foundation of novel concept concept meaning, computational concept invention could be generation to occur as a product from combining already approached within this general framework. existing knowledge. By merging two, or more, conceptual Image schemas spaces, a blended conceptual space results. This blend con- tains information from both input spaces and has emergent The theory of image schemas aims to make concrete the ab- properties due to its own unique composition. The classic stract embodied experience Lakoff (1987); Johnson (1987). example is the blend of a ‘houseboat’, containing merged information from the input spaces ‘house’ and ‘boat’. 1The KRDB Research Centre, Faculty of Computer Science, Free University of Bozen-Bolzano, 39100 Bozen-Bolzano, Italy. One of the most important aspects of blending is not only http://www.inf.unibz.it/ okutz/about.html the merging of the input spaces, but also the search for com- 2Institute for Knowledge and Language Engineering, Fac- mon structure, represented as a generic space. The common ulty of Computer Science, Otto-von-Guericke University structure of the input spaces plays a vital role in rendering Madgeburg, 39106 Magdeburg, Germany. http://theo.cs.uni- the newly constructed concept meaningful, as it ensures that magdeburg.de/Staff/Till+Mossakowski.html the blended space also contains the structure found in the generic space. In order to integrate the image schemas into computa- Some ideas of how to formalise CB to make them tional concept invention, these abstracted patterns need to amenable to computational techniques can be seen in Kutz be translated into a formal language readable by a computa- et al. (2014). tional conceptual blending system.

Problem Formalisation Related Research Research idea Defining the nature of image schemas One problem for concept invention through CB is the gen- Within the research field a central idea is how image eration of a ‘sensible’ blend. In a completely automatised schemas can be combined with one another to generate more system, there is currently no simple way to distinguish the specific and complex image schemas (Kuhn, 2007; Oakley, blends that a human would consider meaningful from those 2010). Therefore, an alternative way to perceive the IN and that lack cognitive value. This captures the biggest problem OUT image schema is to view them as combinations of the for research. This problem grows two image schemas CONTAINMENT and PATH, rather than exponentially in relation to the size of the input spaces. In part of the CONTAINMENT schema. real life scenarios, the amount of information in the input One attempt to hierarchically structure the range from spaces can be vast, making successful computational con- simple to more complex and dynamic image schemas is that cept invention a problem. of Mandler and Pagan´ Canovas´ (2014). They build their sug- The framework of embodied cognition and more specif- gestion on empirical data from studies on cognitive develop- ically the notion of image schemas, provide a meaningful ment. In their approach the umbrella term ‘image schemas’ stepping stone to guide concept invention and conceptual is divided into three different levels: spatial primitives, im- blending. Image schemas can not only constitute the struc- age schemas and conceptual integrations. ture in the generic space but also provide heuristics for the blending process. As image schemas represent the concep- Related work on formalising image schemas tual building blocks, this dissertation hypothesize that guid- Prominent work in this field is the work by Kuhn (2007) ing CB with image schemas would result in cognitively more who argue that image schemas capture abstractions in order relevant information being transferred into the blend. to model affordances and formalises them using the formal programming language Haskell. Research goals and relevance Walton and Worboys (2009) build further on Kuhn’s work The main purpose of my research is to investigate the how by visualising and formalising the connections between dif- human concept invention takes place and to demonstrate ferent image schemas using bigraphs. Their method demon- how this understanding can be translated into a formal sys- strates how more complex dynamic image schemas such as tem of computational concept invention. BLOCKAGE could be generated using sequences of bigraph The primary goals are; 1. investigate image schemas as reaction rules on top of simpler static image schemas. conceptual building blocks from a cognitive and linguistic Bennett and Cialone (2014) performed a study in which point of view, 2. how to formalise the image schemas, and 3. eight different kinds of CONTAINMENT were distinguished provide a method to integrate them into a conceptual blend- from natural language using a method of sense clustering. ing system to test their role in concept invention. Up to date, The CONTAINMENT versions were formalised using the little research has been conducted on image schemas in con- GUM ontology and the RCC-8 topological relations in FOL. ceptual blending. The possible impact of the dissertation for the research field of computational concept invention has Related work on computational conceptual therefore great potential. blending The research is conducted under the believed that AI Conceptual blending has similarities with the more investi- and computational creativity systems could use this method gated phenomenon of analogical thinking. engines to generate novel concepts, blended ideas and possibly such as the structure mapping engine (SME) (Gentner, 1983) near a solution for computational understanding of image- and heuristic-driven theory projection (HDPT) (Schmidt et schematic conceptual metaphors and abstract concepts. al., 2014) search for common structure in the source and tar- get domain and use this information to perform information Research obstacles transfers. The undefined image schemas: Partly due to the interdis- Some formal approaches to conceptual blending is the ciplinary interest in image schemas, current image schema work by Goguen and Harrell (2010) and Mossakowski, research is inconsistent regarding terminology, definitions Maeder, and Luttich¨ (2007) who introduced Heterogeneous and borders between different image schemas, making fur- Tool Set HETS. ther research challenging. Preliminary method and result Formal representation: Also, the abstract nature of these Support for the role image schemas play in concept inven- cognitive phenomena is not only difficult to define, but also tion will be sought after through a set of linguistic experi- difficult to translate into a formal language. ments inspired by the work presented above. While most of the experimental set-up is still under development, current Hedblom, M. M.; Kutz, O.; and Neuhaus, F. 2015b. Im- work include: 1. Formally investigating the universality of age schemas as families of theories. In Besold, T. R.; image schema through data mining and corpus analysis. 2. Kuhnberger,¨ K.-U.; Schorlemmer, M.; and Smaill, A., Linguistic experiments with human test-subjects to investi- eds., Proceedings of the Workshop “Computational Cre- gate the cognitive aspects of image schemas in concept in- ativity, Concept Invention, and General Intelligence” vention. 2015, volume 2, 19–33. Regarding the formalisation of image schemas the cur- Hedblom, M. M.; Kutz, O.; and Neuhaus, F. 2016. Image rent approach can be seen in Hedblom, Kutz, and Neuhaus schemas in computational conceptual blending. Cognitive (2015a,b), where a case study on PATH was made. In the Systems Research. accepted for publication. papers image schemas were argued to be families of theo- ries rather than individual theories and presented in graphs Johnson, M. 1987. The Body in the Mind. The Bodily Basis of increasing complexity. of Meaning, Imagination, and Reasoning. The University Image schemas in CB was investigated in Hedblom, Kutz, of Chicago Press. and Neuhaus (2014, 2016). Here the first suggestions on how Kuhn, W. 2007. An Image-Schematic Account of Spa- image schemas could be used in the generic space in formal tial Categories. In Winter, S.; Duckham, M.; Kulik, L.; conceptual blending, were presented. The papers demon- and Kuipers, B., eds., Spatial Information Theory, volume strate how concepts such as “mothership” can be seen as an 4736 of Lecture Notes in Computer Science. Springer. image schema blend of the two input spaces ‘mother’ and 152–168. ‘ship’, with the guidance of the image schema CONTAIN- Kutz, O.; Bateman, J.; Neuhaus, F.; Mossakowski, T.; and MENT. Bhatt, M. 2014. E pluribus unum: Formalisation, Use- These results and ideas will be further developed by inves- Cases, and Computational Support for Conceptual Blend- tigating the combinatorial power of image schemas through ing. In Besold, T. R.; Schorlemmer, M.; and Smaill, A., integration into HDTP, and in a later stage also into a con- eds., Computational Creativity Research: Towards Cre- ceptual blending system. ative Machines, Thinking Machines. Atlantis/Springer. References Lakoff, G. 1987. Women, Fire, and Dangerous Things. What Categories Reveal about the Mind. The University Bennett, B., and Cialone, C. 2014. Corpus Guided Sense of Chicago Press. Cluster Analysis: a methodology for ontology develop- ment (with examples from the spatial domain). In Gar- Mandler, J. M., and Pagan´ Canovas,´ C. 2014. On defining bacz, P., and Kutz, O., eds., 8th International Conference image schemas. Language and Cognition 0:1–23. on Formal Ontology in Information Systems (FOIS), vol- Mossakowski, T.; Maeder, C.; and Luttich,¨ K. 2007. The ume 267 of Frontiers in Artificial Intelligence and Appli- Heterogeneous Tool Set. In Grumberg, O., and Huth, M., cations, 213–226. IOS Press. eds., Tools and Algorithms for the Construction and Anal- Colton, S. 2012. The painting fool: Stories from building an ysis of Systems, volume 4424 of Lecture Notes in Com- automated painter. In J. McCormack, M. d., ed., Comput- puter Science, 519–522. Springer. ers and Creativity. Springer-verlag. 551–596. Oakley, T. 2010. Image schema. In Geeraerts, D., and Fauconnier, G. 1997. Mappings in Thought and Language. Cuyckens, H., eds., The Oxford Handbook of Cognitive Cambridge: Cambridge University Press. Linguistics. Oxford: Oxford University Press. 214–235. Gentner, D. 1983. Structure mapping: A theoretical frame- Schmidt, M.; Krumnack, U.; Gust, H.; and Khnberger, K.-U. work for analogy. Cognitive Science 7(2):155–170. 2014. Heuristic-Driven Theory Projection: An Overview. In Prade, H., and Richard, G., eds., Computational Ap- Gervas,´ P. 2009. to storytelling and creativity. 49–62. proaches to Analogical Reasoning: Current Trends, Com- Goguen, J. A., and Harrell, D. F. 2010. Style: A Com- putational Intelligence 548. Springer-Verlag. putational and Conceptual Blending-Based Approach. In Shapiro, L. 2011. Embodied Cognition. New problems of Argamon, S., and Dubnov, S., eds., The Structure of Style: philosophy. London and New York: Routledge. Algorithmic Approaches to Understanding Manner and Meaning. Berlin: Springer. 147–170. Tettamanti, M.; Buccino, G.; Saccuman, M. C.; Gallese, V.; Danna, M.; Scifo, P.; Fazio, F.; Rizzolatti, G.; Cappa, Hedblom, M.; Kutz, O.; and Neuhaus, F. 2014. On the cog- S. F.; and Perani, D. 2005. Listening to action-related nitive and logical role of image schemas in computational sentences activates fronto-parietal motor circuits. Journal conceptual blending. In Lieto, A.; Radicioni, D. P.; and of cognitive neuroscience 17(2):273–281. Cruciani, M., eds., Proceedings of the Second Interna- tional Workshop on Artificial Intelligence and Cognition Walton, L., and Worboys, M. 2009. An algebraic approach to image schemas for geographic space. In Proceedings of (AIC 2014), 110–121. the 9th International Conference on Spatial Information Hedblom, M. M.; Kutz, O.; and Neuhaus, F. 2015a. Choos- Theory (COSIT), 357–370. ing the Right Path: Image Schema Theory as a Foundation for Concept Invention. Journal of Artificial General Intel- ligence 6(1):22–54. in print.