Logic, Ontology and Planning: The Robot’s Knowledge Lectures 3-4 From Conceptual Blending to Computational Concept Invention

Stefano Borgo & Oliver Kutz ESSLLI 2018, Sofia, Bulgaria

Laboratory for Applied Ontology (LOA),

ISTC-CNR, Trento (IT)

CORE Conceptual and Cognitive Modelling Group KRDB Research Centre for Knowledge and Data Free University of Bozen-Bolzano, Italy

. KRDB

3 Today's topic: Invention

• Artificial agents that act autonomously in a changing environment need to - adapt to constant change - find workable solutions to unexpected scenarios

• Fully autonomous robots require elements of human creativity to be successful • Key elements are: - Concept invention: combine previously learned/acquired ideas into novel ideas - Use invention to re-purpose available tools for new tasks Concept Invention: A highly interdisciplinary endeavour ...

• From Conceptual Blending - Cognitive / Embodied - Metaphor Theory / Analogies - Image Theory • to Computational Concept Invention - Computational Creativity (CC) - Knowledge Engineering / Ontologies - Category Theory / Non-Classical Logic / Computational Logic Concept Invention: A highly interdisciplinary endeavour

• Part 1: - what is conceptual blending? • Part 2: - an abstract framework and representation language • Part 3: - cognitive modelling and computational problems ‣ computing generic spaces via generalisation ‣ image schemas as generic spaces Part 1: Conceptual Blending Conceptual Blending ...

• Mark Turner (2014): a hypothetical explanation for the ‘human spark’: • The ‘lionman’, approximately 32.000 years old, blends the concepts of ‘human’ and ‘lion’. • The period of its creation marks the end of an apparent deadlock of human cultural development, ... • and the beginning of rapid cultural evolution (hypothesis: expansion of working memory). Conceptual Blending

• developed in the early 1990s by Gilles Fauconnier and Mark Turner • intended to understand and model the process of concept invention • much studied within cognitive and linguistics • Conceptual Blending concerns blending of two thematically rather different conceptual spaces yielding a new conceptual space with - emergent structure, selectively combining parts of the given spaces - whilst respecting common structural properties. Summarised by Fauconnier & Turner (2003):

• inputs have different organising frames • blend has an organising frame that receives projections • blend has emergent structure on its own • inputs offer the possibility of rich clashes • blends offer challenges to the imagination • resulting blends can turn out to be highly imaginative

A foldable toothbrush is not an analogy! Conceptual Blending: Example

Note: The diagram is upside-down (motivated by the formal model) Conceptual Blending: Example Conceptual Blending: Houseboat Conceptual Blending: Boathouse Blending Signs and Forests: Input 1

• Signs:

a piece of paper, wood or metal that has writing or a picture on it that gives you information, instructions, a warning

(Oxford Advanced Learner’s Dictionary) Blending Signs and Forests: Input 2

• Forests

complex eco- logical systems in which trees are the dominant life form

(Encyclopaedia Britannica) Blending Signs and Forests: Blend 1

Signs in Forests Blending Signs and Forests: Blend 2

Forestsigns Blending Signs and Forests: Blend 3

Signforests "Optimality Principles" (Heuristics): What makes a good blend?

• Integration: The blend must constitute a tightly integrated scene that can be manipulated as a unit. • Pattern Completion: complete elements in the blend . . . • Maximization of Vital Relations: change, identity, time, space, cause-effect, part-whole, . . . • Unpacking: The blend alone must enable the perceiver to unpack the blend to reconstruct the inputs, the cross- space mapping, the generic space, and the network of connections between all these spaces • Relevance: ... Graphical Representation of a Formal DOL Specification Part 2: Abstract Framework and Representation Language Blending: Formal Model

(C + BK) consistent with B (B + BK) entail R

Evaluation consistency consequence C R requirements requirements • Creating blends in Ontohub / DOL B • usage of background Input theory 1 I1 Blendoid I2 Input theory 2 ontologies image schemas as I1* I2* • base ontologies Generalised input theory 1 Generalised input theory 2 • evaluation features Base Ontology - constraints - requirements

BK

Rich Background Knowledge Blending with DOL, Hets, Ontohub

• Formal (meta)-language: DOL - describes structured ontologies / models / specifications - supports specification of blending diagrams - specifies requirements for evaluation • Heterogenous reasoning: Hets - proof support for structured ontologies/theories - computation of colimits • Repository for heterogeneous theories: Ontohub - supports a variety of logical languages for ontology, mathematics, music - supports ontology evaluation techniques T. Mossakowski et al. / Three Semantics for DOL 3

DOL intends to cover all state-of-the-art basic ontology languages, and to provide a meta level on top of these. This meta level allows for the representation of logically heterogeneous ontologies in the sense that DOL ontologies may comprise of modules written in ontology languages with different underlying logics. Moreover, the DOL meta level constructs allow for links between ontologies such as relative interpretations or conservative extensions. DOL intends to be an extensible framework for ontology languages. It is intended that any ontology language and any logic whose conformance with DOL has been es- tablished can be used with DOL. In particular, we are interested in establishing the con- formance of a number of widely used ontology languages and logics as a part of the standard; these include (ordered by increasing complexity) propositional logic (Prop), OWL [1] (with its profiles EL, RL and QL [23]), standard first-order logic with equality (FOL DOL=) and Common Logic (CL) [5]5, as well as translations between these ontology languages. Such translations have been developed in previous research; see e.g. [21]. Note in particular that, among these ontology languages, Common Logic is the most expressive- one,relate and (on is therefore the meta-level) a target language ontologies for translations that are from written all the in other lan- guages. Thus,diff whenerent reasoning formalisms. about heterogeneous distributed ontologies, one can prima facie translate all participating ontologies to Common Logic. However, note the differ- • E.g. prove that some OWL version of the ence to translating all ontologies to Common Logic in the first place: when, e.g., an OWL foundational ontology DOLCE is logically entailed ontology has been translated to Common Logic, it is no longer easily amenable to de- cidable or even tractableby the reasoning (reference) procedures first-order that OWL version;tools support. Therefore, DOL leaves all- ontologiesre-use ontology in their original modules formalisation even if they to take have advantage been of the optimised automated reasonersformulated for thatin a particular different language, formalism; and DOL tools should only translate them on demand. In summary,- re-use the ontology functionality tools of the likeDOL theoremframework provers can be capturedand module by the follow- ing “equation”:extractors4 along translations between formalisms. EdNote:4

DOL = meta(Prop,EL,QL,RL,OWL,FOL=,CL,... Prop OWL,Prop FOL,EL OWL,OWL FOL,FOL CL,...) ! ! ! ! ! where the meta( ) operation introduces the above mentioned meta-level constructs on · top of the basic ontology languages using the built-in logic translations.

3. An Introductory Example

We use an example from mereology, which lends itself well to heterogeneous formal- isation. While mereological relations such as parthood are frequently used in ontolo- gies, many of these ontologies are formalised in languages that are not fully capable of defining the mereological notions. For example, mereological relations are used in large biomedical ontologies, which are implemented in the EL profile of OWL for efficiency. OWL is partly capable of defining these relations, whereas more complete definitions require first-order or even second-order logic [14]. 5Listing 1 starts with a Prop formalisation of the taxonomy of the categories over EdNote:5

5In the future, we might add other logics such as IKL [10]. 4EDNOTE: CL: TODO review “equation more confusing than useful” 5EDNOTE: @Oliver and then Till: describe informally what the “overall ontology” in this example means Logic Graph supported by DOL

bRDF

OBOOWL RDF

EL++ DL-Lite DL-RL subinstitution UML-CD R Prop (OWL 2 EL) (OWL 2 QL) (OWL 2 RL) theoroidal subinstitution

Schema.org SROIQ RDFS simultaneously exact and (OWL 2 DL) model-expansive comorphisms

OBO 1.4 model-expansive comorphisms DDLOWL

OWL-Full

DFOL FOL= ECoOWL grey: no fixed expressivity

green: decidable ontology languages F-logic ECoFOL CL- yellow: semi-decidable CL EER orange: some second-order constructs

red: full second-order logic ms= FOL CASL

HOL DOL: Core Syntax Constructs

An ontology O can be, among other cases not covered here, one of the following: - a basic ontology ⟨Σ,∆⟩ written in some ontology language. For simplicity, we assume that it is directly given by a signature Σ and a set of axioms ∆ - a translation “O with logic ρ” of an ontology O along an ontology language translation ρ; - an extension O then CS? ⟨Σ, ∆⟩ of an ontology O by another basic ontology ⟨Σ,∆⟩; the extension can be marked as a conservative extension (CS; model-conservative or consequence-conservative) - a reference to an ontology existing on the Web, DOL ontology example: Mereology

Taxonomy Propositional

Parthood EL OWL profile SNOMED, Ligthweight Domain SpecProp NCI Thesaurus Ontology Proper Parthood OWL 2 DL

SpecOWL Expressive Galen Domain Parthood / SpecFOL1 Role Inclusions DL beyond OWL Ontology FOL Mereology SpecFOL2 First-order logic

BFO, Founda- Common Logic / GFO, tional Second-order Dolce Ontology fusion Second-order logic

SpecHOL

Re-using parts of mereology in biomedical ontologies 4 T. Mossakowski et al. / Three Semantics for DOL which DOLCE [19] defines mereological relations. Although Prop is rarely regarded as an ontology language, this logic is quite popular for formal modelling since consistency and logical consequence can be quite efficiently decided using SAT solvers. In particular, for early detection of modelling errors in an ontology design (especially when using a large number of classes), initial consistency checks and satisfiability of classes can be delegated to a SAT solver for debugging. We specify a similar ontology in OWL (alternatively we could import the Prop on- tology). As this ontology declares classes for all categories that are declared as propo- sitional variables in the Prop ontology satisfying the same disjointness and subsump- tion relationships, the OWL ontology interprets the Prop ontology (we here assume the Prop OWL ontology language translation from Definition 9 below6). In the OWL on- ! tology, we additionally introduce parthood properties, which clearly goes beyond the ex- pressivity of Prop. OWL still has quite good automatic reasoning support, but therefore also has its limits: OWL is not capable of defining e.g. the antisymmetry of the isPartOf relation. Overcoming this restriction, we give a full definition of several mereological re- lations in a CL ontology, which imports and extends the OWL ontology (which is implic- itly translated using the default OWL CL translation). CL extends FOL with second- ! order style modelling, of which we use the possibility to quantify over predicates here. This allows us to concisely express the restriction of the variables x, y, and z to the same taxonomicExample: category (perdurants,Mereology abstract regions, in DOL etc.), and it allows us to define a notion of second-order fusion [11]. 6 EdNote:6

Listing 1: A heterogeneous ontology for mereology [14] prefix = prefix owl = prefix log = prefix trans = distributed-ontology Mereology logic log:Propositional %% syntax used: similar to OWL Manchester [12] ontology Taxonomy = %% DOLCE’s basic taxonomic information about mereology props PT %[ Particular ]%, PD %[ Perdurant ]%,T%[ TimeInterval ]%, S %[ SpaceRegion ]%, AR %[ AbstractRegion ]% .S T AR PD PT %% PT is the top concept .S_ T _ _ %% PD,! S, T, AR are pairwise disjoint .T^ AR ! ? %% ... ^ ! ? end logic log:OWL %% syntax: OWL Manchester serialization [12] ontology BasicParthood = %% Parthood in OWL DL, as far as expressible Class: ParticularCategory SubClassOf: PT %% other class declarations omitted DisjointUnionOf: S, T, AR, PD %% pairwise disjointness more compact ObjectProperty: isPartOf Characteristics: Transitive ObjectProperty: isProperPartOf Characteristics: Transitive, Asymmetric SubPropertyOf: isPartOf Class: Atom EquivalentTo: inverse isProperPartOf only owl:Nothing

6As Prop OWL is a non-default translation, it has to be stated explicitly; cf. the end of Section 5 for an ! explanation. 6EDNOTE: CL: TODO check “end” keywords (review 1) 4 T. Mossakowski et al. / Three Semantics for DOL which DOLCE [19] defines mereological relations. Although Prop is rarely regarded as an ontology language, this logic is quite popular for formal modelling since consistency and logical consequence can be quite efficiently decided using SAT solvers. In particular, for early detection of modelling errors in an ontology design (especially when using a large number of classes), initial consistency checks and satisfiability of classes can be delegated to a SAT solver for debugging. We specify a similar ontology in OWL (alternatively we could import the Prop on- tology). As this ontology declares classes for all categories that are declared as propo- sitional variables in the Prop ontology satisfying the same disjointness and subsump- tion relationships, the OWL ontology interprets the Prop ontology (we here assume the Prop OWL ontology language translation from Definition 9 below6). In the OWL on- ! tology, we additionally introduce parthood properties, which clearly goes beyond the ex- pressivity of Prop. OWL still has quite good automatic reasoning support, but therefore also has its limits: OWL is not capable of defining e.g. the antisymmetry of the isPartOf relation. Overcoming this restriction, we give a full definition of several mereological re- lations in a CL ontology, which imports and extends the OWL ontology (which is implic- itly translated using the default OWL CL translation). CL extends FOL with second- ! order style modelling, of which we use the possibility to quantify over predicates here. This allows us to concisely express the restriction of the variables x, y, and z to the same taxonomic category (perdurants, abstract regions, etc.), and it allows us to define a notion of second-order fusion [11]. 6 EdNote:6

Listing 1: A heterogeneous ontology for mereology [14] prefix = prefix owl = prefix log = prefix trans = distributed-ontology Mereology logic log:Propositional %% syntax used: similar to OWL Manchester [12] ontology Taxonomy = %% DOLCE’s basic taxonomic information about mereology propsExample:PT %[ Particular Mereology ]%, PD %[ Perdurant in DOL ]%,T%[ TimeInterval ]%, S %[ SpaceRegion ]%, AR %[ AbstractRegion ]% .S T AR PD PT %% PT is the top concept .S_ T _ _ %% PD,! S, T, AR are pairwise disjoint .T^ AR ! ? %% ... ^ ! ? end logic log:OWL %% syntax: OWL Manchester serialization [12] ontology BasicParthood = %% Parthood in OWL DL, as far as expressible Class: ParticularCategory SubClassOf: PT %% other class declarations omitted DisjointUnionOf: S, T, AR, PD %% pairwise disjointness more compact ObjectProperty: isPartOf Characteristics: Transitive ObjectProperty: isProperPartOf Characteristics: Transitive, Asymmetric SubPropertyOf: isPartOfT. Mossakowski et al. / Three Semantics for DOL 5 Class: Atom EquivalentTo: inverse isProperPartOf only owl:Nothing end %% an atom has no proper parts 6As Prop OWL is a non-default translation, it has to be stated explicitly; cf. the end of Section 5 for an ! explanation.interpretation TaxonomyToParthood : Taxonomy with logic PropToOWL to BasicParthood 6EDNOTE: CL: TODO check “end” keywords (review 1) logic log:CommonLogic %% syntax: CLIF dialect of Common Logic [5, Annex A] ontology ClassicalExtensionalParthood = BasicParthood then { %% import OWL ontology from above, translate it to CL .(forall (X) (if (or (= X S) (= X T) (= X AR) (= X PD)) (forall (x y z) (if (and (X x) (X y) (X z)) (and %% now list all the axioms (if (and (isPartOf x y) (isPartOf y x)) (= x y)) %% antisymmetry (iff (overlaps x y) (exists (pt) (and (isPartOf pt x) (isPartOf pt y)))) (iff (isAtomicPartOf x y) (and (isPartOf x y) (Atom x))) (iff (sum z x y) (forall (w) (iff (overlaps w z) (and (overlaps w x) (overlaps w y))))) (exists (s) (sum s x y)) %% existence of the sum ))))) .(forall (Set a) (iff (fusion Set a) %% definition of fusion (forall (b) (iff (overlaps b a) (exists (c) (and (Set c) (overlaps c a))))))) }

4. Syntax of DOL

The meta level on top of basic ontologies is specified with its syntax (this section) and semantics (Section 6). Note that the DOL language (and its semantics) as introduced here only covers the core meta logical constructs that will be part of the full DOL language.7

4.1. Distributed Ontologies

A distributed ontology consists of at least one (possibly heterogeneous) ontology, plus, optionally, interpretations between its participating ontologies. More specifically, a distributed ontology consists of a name, followed by a list of LOGIC-SECTIONs. A LOGIC-SECTION selects a specific ontology language (actually the underlying logic) that is used to interpret the subsequent DIST-ONTO-ITEMs. A DIST-ONTO-ITEM is either an ontology definition (ONTO-DEFN), or an interpretation between ontologies (INTPR-DEFN). Alternatively, a distributed ontology can also be the verbatim inclusion of an ontology written in a specific ontology language, using that language’s structuring construct instead of DOL’s (ONTO-IN-SPECIFIC-LANGUAGE).

DIST-ONTO-DEFN ::= distributed-ontology DIST-ONTO-NAME LOGIC-SECTION* | ONTO-IN-SPECIFIC-LANGUAGE %% logic-specific LOGIC-SECTION ::= logic LOGIC-REF DIST-ONTO-ITEM* DIST-ONTO-ITEM ::= ONTO-DEFN | INTPR-DEFN

4.2. Heterogeneous Ontologies

An ontology O can be, among other cases not covered here, one of the following:

7Features not covered here for conciseness include e.g. heterogeneous alignments and heterogeneous least upper bounds of ontology collections (i.e. heterogeneous colimits of logical theories). But see [16] for a con- crete use of these constructs. Example: MereologyT. Mossakowski et al. / Three in SemanticsDOL for DOL 5 end %% an atom has no proper parts interpretation TaxonomyToParthood : Taxonomy with logic PropToOWL to BasicParthood logic log:CommonLogic %% syntax: CLIF dialect of Common Logic [5, Annex A] ontology ClassicalExtensionalParthood = BasicParthood then { %% import OWL ontology from above, translate it to CL .(forall (X) (if (or (= X S) (= X T) (= X AR) (= X PD)) (forall (x y z) (if (and (X x) (X y) (X z)) (and %% now list all the axioms (if (and (isPartOf x y) (isPartOf y x)) (= x y)) %% antisymmetry (iff (overlaps x y) (exists (pt) (and (isPartOf pt x) (isPartOf pt y)))) (iff (isAtomicPartOf x y) (and (isPartOf x y) (Atom x))) (iff (sum z x y) (forall (w) (iff (overlaps w z) (and (overlaps w x) (overlaps w y))))) (exists (s) (sum s x y)) %% existence of the sum ))))) .(forall (Set a) (iff (fusion Set a) %% definition of fusion (forall (b) (iff (overlaps b a) (exists (c) (and (Set c) (overlaps c a))))))) }

4. Syntax of DOL

The meta level on top of basic ontologies is specified with its syntax (this section) and semantics (Section 6). Note that the DOL language (and its semantics) as introduced here only covers the core meta logical constructs that will be part of the full DOL language.7

4.1. Distributed Ontologies

A distributed ontology consists of at least one (possibly heterogeneous) ontology, plus, optionally, interpretations between its participating ontologies. More specifically, a distributed ontology consists of a name, followed by a list of LOGIC-SECTIONs. A LOGIC-SECTION selects a specific ontology language (actually the underlying logic) that is used to interpret the subsequent DIST-ONTO-ITEMs. A DIST-ONTO-ITEM is either an ontology definition (ONTO-DEFN), or an interpretation between ontologies (INTPR-DEFN). Alternatively, a distributed ontology can also be the verbatim inclusion of an ontology written in a specific ontology language, using that language’s structuring construct instead of DOL’s (ONTO-IN-SPECIFIC-LANGUAGE).

DIST-ONTO-DEFN ::= distributed-ontology DIST-ONTO-NAME LOGIC-SECTION* | ONTO-IN-SPECIFIC-LANGUAGE %% logic-specific LOGIC-SECTION ::= logic LOGIC-REF DIST-ONTO-ITEM* DIST-ONTO-ITEM ::= ONTO-DEFN | INTPR-DEFN

4.2. Heterogeneous Ontologies

An ontology O can be, among other cases not covered here, one of the following:

7Features not covered here for conciseness include e.g. heterogeneous alignments and heterogeneous least upper bounds of ontology collections (i.e. heterogeneous colimits of logical theories). But see [16] for a con- crete use of these constructs. DOL Roadmap

• DOL enables interoperability between heterogeneous ontologies, with support for all major ontology languages and their serialisations; • It has built-in support for major structuring and modularity concepts, as well as a various links and mappings between ontologies. • New ontology languages and translations can be added by establishing conformance with DOL. • The Ontohub repository will allow to host such heterogeneous ontologies and should fully support meta- theoretical linking • Finalisation of DOL standardisation in OMG in 2018. DEMO of ontohub/conceptportal Further resources for DOL/Hets/Ontohub

• Hets website: http://hets.eu

• Ontohub platform: https://ontohub.org

• DOL website at OMG: https://www.omg.org/spec/DOL/About-DOL/

• ESSLLI 2016 course on DOL by Kutz & Mossakowski http://esslli2016.unibz.it/?page_id=171 Part 3: Cognitive Modelling and Computational Approaches Goal: Computationally Generate Concepts COINVENT's Model for CCB

Challenges: • How to represent the blending process?

• What do we keep from the input spaces?

• How to find the right base space + morphisms? Creating EL++ Concepts via Conceptual Blending Computational Model of Conceptual Blending: Amalgamation AMALGAMS as Blends

Generic Space

coloured European car GenInput GenInput red European sedan coloured German car

red French sedan blue German minivan Input Input red German sedan Blend An Example in EL++

• The Description Logic EL++ is a sub-Boolean logic optimised for polynomial reasoning to classify very large taxonomies • The basic constructs are - conjunction of concept descriptions - existential restriction on roles • An expression of the form

is interpreted as the set of all objects a that are related to some object b with the relation 'hasBodyPart' and such that b belongs to the concept 'Legs'. Generalising EL++ Concepts: Why? Generalisation operators Generalising an EL++ Concept Generalisation Operator: basic definition

• The least general generalisations for atomic names and roles are defined in the 'upcover' (we just show it for concepts):

• Then we can define the EL generalisation operator by Generalisations and Generic Space Generalisations and Generic Space (cont'd) Implementation of Generalisation in Answer Set Programming: Overview Blends in EL++ Goal: Computationally Generate Concepts Hypothesis

How to find the right base ontology for blending?

Hypothesis • Image schemas may form a conceptual skeleton of bases spaces Image schemas?

• Mark Johnson (1987) describes them as - ‘‘. . .a recurring, dynamic pattern of our perceptual interactions and motor programs that gives coherence and structure to our experience”

• Todd Oakley (2007) defines an image schema as - ‘‘...a condensed re-description of perceptual experience for the purpose of mapping spatial structure onto conceptual structure” Image schemas: Lakoff & Johnson 1987

• Spatial Motion Group - Containment - Path - Source-Path-Goal - Blockage • Force Group - Counterforce - Link • Balance Group - Axis Balance - Point Balance … Image Schema Days Image schemas, blending, ontologies, and symbol grounding

• Motivation: image schemas ground the search for meaningful concepts in human cognition and embodiment • Image schemas provide candidates (the conceptual skeleton) for (parts of) the generic space in blending • Image schema formalisations provide an approach to generalisation and abstraction in blending

• Core problem: - What are appropriate formal/logical approaches to representing and reasoning with image schemas? What have these things in common?

• Space ship • North Korea • The universe • Marriage • Bank account Simile

• This space ship • prison • North Korea • leaky pot • The universe is like a • treasure chest • Their marriage • bottomless pit • My bank account • balloon Simile ('Objects')

• This space ship • prison • North Korea • leaky pot • The universe is like a • treasure chest • Their marriage • bottomless pit • My bank account • balloon

If the concepts on the left are so dissimilar, why can they be meaningfully compared to the same things? Simile ('Objects')

• prison • leaky pot is like a • treasure chest • bottomless pit • balloon Container Simile ('Events')

• The story • a roller coaster ride • Watching the • a Prussian military football game parade is like • Their marriage • a marathon • Bob’s career • escaping a maze • Democracy in Italy • stroll in the park Simile ('Events')

• a roller coaster ride • a Prussian military parade is like • a marathon • escaping a maze • stroll in the park Source-Path-Goal What are Image Schemas (Logically)?

• What is the ontology of image schemas • What are the primitive notions - spatial primitives - spatial schemas - time / simulation - physics / forces • Understanding time and/or space led to specialised logics of time and space, and of spatio-temporal combined reasoning • Is the logic of image schemas a particular kind of spatio- temporal logic? • Or do we require a new kind of logic? The image schema family of path, loop, and revolving

movement on a path movement in loops

add source add target add center S T o add target add source

S T revolving around

extending an image x D schema axiomatically adding spatial primitives S = T S = T to an image schema closed path, source closed path, with additional extending by spatial and target coincide distinguished point primitives and axioms Event structure / patterns: Image schema of caused movement

Path Time 1 Path schema

Time 2 Contact Contact schema

Time 3A Caused_movement Time 3B Bouncing Open problems for us

• Analyse the ontology of image schemas further • Identify different levels of logical expressivity, cognitively adequate for various phenomena • Develop the computational side of using image schema families for generalisation / base space discovery in blending • Develop the logical and computational side of combination and multi-modality for image schemas • Many spatio-temporal logics have been devised. Do image schemas necessitate a novel combination, i.e.:

Do we need a new Logic of Image Schemas? Image Schema Logic ISL

The image schema logic • Two-object family ISL combines The Two-Object Family: an excerpt from the extended image schema family of relationships between two objects The Region Connection - Object Calculus RCC8

add object Attraction Verticality (Force) - Cardinal directions Contact - A simple modelling of add above add force 'force' Above-Support Force-Support Qualitative Trajectory - Support Calculus QTC - Linear temporal logic Models of Space

• Models of space - 2d vs 3d - continuous vs discrete - geometry - topology • Language: talk about - distances - regions - orientations - reachability, direction Containment as RCC8 Containment as RCC8?

Eight variations of containment as discussed in Bennett and Cialone Models of time

• Flow - past to future - future to past - upwards / downwards - forward / backward • Structure - linear - forward branching - backward branching - discrete / dense Languages for time

• Ontology of time - time points - time intervals - events • Quantification - FOL-style quantification over time points/intervals etc - Modal-logic style of implicit quantification ‣ always in the future / some time in the past ‣ derived from temporal natural language • Cognitive adequacy of languages? Allen’s Interval calculus Allen’s Interval calculus

• Ontology of time - time points - time intervals - events • Quantification - FOL-style quantification over time points/intervals etc - Modal-logic style of implicit qunatification ‣ always in the future / some time in the past ‣ derived from temporal natural language • Cognitive adequacy of languages? Image Schema Logic ISL: Two-object family

Contact: the image schema family of relationships between several objects

Object

Contact Attraction Verticality (Force)

add verticaltiy add force

Support-lite1 Support-lite2

add force

Glue-Link Support

add object add force

Chain- Link Abstract- Link Computing Generic Spaces: Summary

• Two basic approaches: - Identification approach: use the idea of formalised image schema families (ontology patterns) to identify them in an input space via theory interpretation - Generalisation approach: generalise the input spaces to a common core via generalisation operators, and prioritise image- schematic structure. Adding Agency to ISL (FOIS 2018) STIT Theory

• no ontology of actions: i.e. categories of actions or events • modalities for - 'A sees to it that phi' : Does_A phi - historical possibility (Diamond phi) - necessity, something is settled (Box phi) - Some time in the future (F phi) - Always in the future (G phi) Folk physics

• A typical principle of folk physics is “what goes up must come down.” If e stands for the Earth, and s is the sky, it can be formalised as:

• Such a statement, rather than being an axiom of the ISL logic, can be seen as an axiomatic constraint for the naive physics theory involved in the use of image schemas. • This allows us to avoid encoding physics into the semantics directly but rather rely on a simulation shared with the robot Levels of agency: Ball

• The billiard ball is an object and not an agent proper. • In STIT it is therefore simply modelled in a way that the choice set is empty. • That is, at every moment w, the billiard ball has one unique choice, which consists in selecting all the histories going through w. It cannot interfere with the course of nature. Levels of agency: Mouse

• An object/agent a is truly agentive for a proposition φ when:

• Agent a may never exercise its power to decide whether φ or ¬φ is eventually true, but there is a history and a moment where it does.

• In fact, if the billiard ball and the mouse are moving towards each other, only the mouse can avoid contact: Summary

• Conceptual blending provides a rich cognitively motivated theory for computational concept invention • Image Schema Theory is essential for understanding the dynamics of concept invention • Current and future work includes: - Rich spatial-temporal logics for image schemas - Integrating social choice theory and argumentation - Building image-schematic semantics into VR environments based on physics simulations - Use of image schemas in common sense encodings for robotic AI Thank you for your attention! Some Relevant Publications

• M M Hedblom, O Kutz, F Neuhaus: "Choosing the Right Path: Image Schema Theory as a Foundation for Concept Invention", Journal of Artificial General Intelligence 6 (1): 22-54, De Gruyter, 2015. • O Kutz, J Bateman, T Mossakowski, F Neuhaus, M Bhatt: "E pluribus unum: Formalisation, Use-Cases, and Computational Support for Conceptual Blending", in T. R. Besold et al., editors, Computational Creativity Research: Towards Creative Machines, Atlantis/Springer, Thinking Machines, 2015. • M M Hedblom, O Kutz, F Neuhaus: "Image schemas in computational conceptual blending", Cognitive Systems Research 39, 42-57, Elsevier, 2016. • T R Besold, M M Hedblom, O Kutz: “A narrative in three acts: Using combinations of image schemas to model events”, Biologically Inspired Cognitive Architectures, Elsevier, 2016. • R Confalonieri, M Eppe, M Schorlemmer, O Kutz, R Penaloza, E. Plaza: "Upward Refinement Operators for Conceptual Blending in the Description Logic EL++", Annals of Mathematics and Artificial Intelligence (AMAI), Springer, 2016. • M Codescu, E Kuksa, O Kutz, T Mossakowski, and F Neuhaus: "Ontohub: A semantic repository for heterogeneous ontologies", Journal of Applied Ontology, IOS Press, 2017 • M M Hedblom, O Kutz, T Mossakowski, F Neuhaus: "Between Contact and Support: Introducing a logic for image schemas and directed movement", 16th International Conference of the Italian Association for Artificial Intelligence (AI*IA 2017)}, Bari, Italy, Springer, 2017. • D Porello, N Troquard, R Confalonieri, P Galliani, O Kutz, R Peñaloza: "Repairing Socially Aggregated Ontologies Using Axiom Weakening", 20th International Conference on Principles and Practice of Multi-Agent Systems (PRIMA 2017)}, Nice, France, Springer, 2017. • M Eppe, E Maclean, R Confalonieri, O Kutz, M Schorlemmer, E Plaza, K-U Kühnberger: “A Computational Framework for Conceptual Blending”, Artificial Intelligence, 2017. • O. Kutz, N. Troquard, M. M. Hedblom, and D. Porello: “The Mouse and the Ball: Towards a cognitively-based and ontologically grounded logic of agency”, FOIS 2018, Cape Town, South Africa, IOS Press, 2017.