Lecture Notes in Artificial Intelligence 10871

Subseries of Lecture Notes in Computer Science

LNAI Series Editors Randy Goebel University of Alberta, Edmonton, Canada Yuzuru Tanaka Hokkaido University, Sapporo, Japan Wolfgang Wahlster DFKI and Saarland University, Saarbrücken, Germany

LNAI Founding Series Editor Joerg Siekmann DFKI and Saarland University, Saarbrücken, Germany More about this series at http://www.springer.com/series/1244 Peter Chapman • Gem Stapleton Amirouche Moktefi • Sarah Perez-Kriz Francesco Bellucci (Eds.)

Diagrammatic Representation and Inference 10th International Conference, 2018 Edinburgh, UK, June 18–22, 2018 Proceedings

123 Editors Peter Chapman Sarah Perez-Kriz Edinburgh Napier University Mitre Corporation Edinburgh McLean, VA UK USA Gem Stapleton Francesco Bellucci University of Brighton University of Bologna Brighton Bologna UK Italy Amirouche Moktefi Tallinn University of Technology Tallinn Estonia

ISSN 0302-9743 ISSN 1611-3349 (electronic) Lecture Notes in Artificial Intelligence ISBN 978-3-319-91375-9 ISBN 978-3-319-91376-6 (eBook) https://doi.org/10.1007/978-3-319-91376-6

Library of Congress Control Number: 2018942344

LNCS Sublibrary: SL7 – Artificial Intelligence

© Springer International Publishing AG, part of Springer Nature 2018, corrected publication 2019 Chapter “How Cross-Representational Signaling Affects Learning from Text and Picture: An Eye-Tracking Study” is Open Access. This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/). For further details see license informa- tion in the chapter. This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of , recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published and institutional affiliations.

Printed on acid-free paper

This Springer imprint is published by the registered company Springer International Publishing AG part of Springer Nature The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Preface

The 10th International Conference on the Theory and Application of Diagrams (Dia- grams 2018) was hosted by Edinburgh Napier University during June 18–22, 2018. For the first time, Diagrams was co-located with the 23rd International Conference on Conceptual Structures (ICCS), which stimulated discussion between researchers of both communities. Submissions to Diagrams 2018 were solicited in the form of long papers, short papers, posters and, for the first time in the conference’s history, Abstracts. The Abstracts category was introduced to encourage participation from authors working in fields where publications at conferences are not as prestigious as those appearing in journals. The peer-review process involved all papers and abstracts receiving at least three reviews (at least two for posters) by members of the Program Committee or a nomi- nated sub reviewer. After reviews were received, authors had the opportunity to submit a rebuttal. The reviews and rebuttals led to a lively discussion involving the Program Committee and the conference chairs to ensure the highest-quality submissions, cov- ering a broad range of topics, were accepted for the conference. We would like to thank the Program Committee members and the additional reviewers for their considerable contributions. The robust review process, in which they were so engaged, is a crucial part of delivering a major conference. In total, 124 submissions were received. Of these, 26 were accepted as long papers. A further 28 were accepted as short papers, 20 as Abstracts, and 20 for poster presentation. Diagrams 2018 sought to expand the research community, and introduced two special submission tracks on Philosophy and Psychology after noting their underrep- resentation in past editions of the conference. This action was extremely successful with 20, out of the total 124, submissions made to the Psychology track and 54 submissions to the Philosophy track. In addition, 28 Abstracts were submitted across the main track and the two special tracks, with strong representation from both philosophers and psychologists. Alongside the technical program, Diagrams 2018 included a Graduate Symposium, the Set and Reasoning Workshop, and six tutorials covering a diverse set of topics on diagrams. There were two keynote speakers. Professor Ahti-Veikko Pietarinen, from Tallinn University of Technology, gave an excellent keynote talk on philosophical aspects of diagrams. Professor Keith Stenning, from the University of Edinburgh, gave a joint keynote with ICCS covering research on diagrams from a cognitive perspective. There are, of course, many people to whom we are indebted for their considerable assistance in making Diagrams 2018 a success. We thank Sarah Perez-Kriz, for her role as Abstracts Chair, and Renata de Freitas, for her role as Workshops and Tutorials Chair. We are grateful to Andrew Blake for organizing the Graduate Symposium, which supported the next generation of researchers. Atsushi Shimojima did an VI Preface outstanding job as Publicity Chair, no doubt reflected by the number of submissions. Francesco Bellucci took on the demanding role of Proceedings Chair, for which we are immensely grateful. We are grateful to the NSF for providing significant funding, which supported graduate students from US-based institutions with their participation. Our institutions, Edinburgh Napier University, the University of Brighton, and Tallinn University of Technology, also provided support for our participation, for which we are grateful. Amirouche Moktefi was further supported by the ERC project “Abduction in the Age of Uncertainty” (PUT 1305, Principal Investigator: Prof. Ahti-Veikko Pietarinen). Lastly, we thank the Diagrams Steering Committee for their continual support, advice, and encouragement.

June 2018 Peter Chapman Gem Stapleton Amirouche Moktefi Organization

Program Committee

Francesco Bellucci University of Bologna, Italy Lisa Best University of New Brunswick, Canada Mireille Betrancourt University of Geneva, Switzerland Alan Blackwell University of Cambridge, UK Jean-Michel Boucheix LEAD-CNRS, University of Burgundy, France Richard Burns West Chester University, USA Jim Burton University of Brighton, UK Jean-Yves Béziau University of Brazil, Brazil Jessica Carter University of Southern Denmark, Denmark B. Chandrasekaran The Ohio State University, USA Peter Chapman Edinburgh Napier University, UK Peter Cheng University of Sussex, UK Jean-Marie Chevalier Collège de France, France Daniele Chiffi Tallinn University of Technology, Estonia Lopamudra Choudhury Jadavpur University, India Peter Coppin OCAD University, Canada James Corter Columbia University, USA Frithjof Dau SAP, Germany Jim Davies Carleton University, Canada Silvia De Toffoli Stanford University, USA Aidan Delaney University of Brighton, UK Lorenz Demey K.U. Leuven, Belgium Maria Giulia Dondero Fonds National de la Recherche Scientifique/Université de Liège, Belgium Tim Dwyer Monash University, Australia George Englebretsen Bishop’s University, Canada Isobel Falconer University of St. Andrews, UK Jacques Fleuriot The University of Edinburgh, UK Janet Folina Macalester College, USA Valeria Giardino Archives Henri Poincaré, Nancy, France Emily Grosholz The Pennsylvania State University, USA Mary Hegarty University of California, Santa Barbara, USA John Howse University of Brighton, UK Mateja Jamnik University of Cambridge, UK Mikkel Willum University of Copenhagen, Denmark Johansen Yasuhiro Katagiri Future University Hakodate, Japan Sarah Kriz The MITRE Corporation, USA VIII Organization

John Kulvicki Dartmouth College, USA Brendan Larvor University of Hertfordshire, UK John Lee The University of Edinburgh, UK Catherine Legg Deakin University, Australia Javier Legris Universidad de Buenos Aires, Argentina Jens Lemanski FernUniversität in Hagen, Germany Christina Ljungberg University of Zurich, Switzerland Danielle Macbeth Haverford College, USA Emmanuel Manalo Kyoto University, Japan Kim Marriott Monash University, Australia Luana Micallef Helsinki Institute for Information Technology HIIT, Finland Amirouche Moktefi Tallinn University of Technology, Estonia Hari Narayanan Auburn University, USA Marco Panza CNRS, France Ahti-Veikko Pietarinen Tallinn University of Technology, Estonia Beryl Plimmer The University of Auckland, New Zealand Helen Purchase University of Glasgow, UK Peter Rodgers University of Kent, UK Yuri Sato University of Brighton, UK Dirk Schlimm McGill University, Canada Stephanie Schwartz Millersville University, USA Atsushi Shimojima Doshisha University, Japan Sun-Joo Shin Yale University, USA Jurgis Skilters CCSS, Germany Hans Smessaert K.U. Leuven, Belgium Gem Stapleton University of Brighton, UK Ben Steichen California State Polytechnic University, Pomona, USA Frederick Stjernfelt Aalborg University, Denmark Andrew Stull University of California Santa Barbara, USA Takeshi Sugio Doshisha University, Japan Ryo Takemura Nihon University, Japan Yuri Uesaka The University of Tokyo, Japan Jean Paul van Bendegem Vrije Universiteit Brussel, Belgium Agnes Veszelszki Corvinus University of Budapest, Hungary Petrucio Viana Federal Fluminense University, Brazil

Additional Reviewers

Bach, Benjamin Linker, Sven Blake, Andrew Morris, Imogen Chakraborty, Mihir Nakagawa, Masanori Chang, Chunlei Shams, Zohreh Hullman, Jessica Sim, Kevin Klein, Karsten Smeltzer, Karl Contents

Keynote Contributions

Diagrams and Nonmonotonic : What Is the Cognitive Relation? ...... 3 Keith Stenning

The Beauty of Graphs ...... 9 Ahti-Veikko Pietarinen

Tutorials

Were “Super-Turing” Diagrammatic Reasoning Mechanisms Ancient Products of Biological Evolution? ...... 15 Aaron Sloman

Using Verbal Protocols to Support Design ...... 20 Thora Tenbrink

Peirce on Diagrammatic Reasoning and Semeiotic...... 24 Javier Legris and Cassiano Terra Rodrigues

Picturing Quantum Processes: A First Course on Quantum Theory and Diagrammatic Reasoning ...... 28 Bob Coecke and Aleks Kissinger

Carroll Diagrams: Design and Manipulation ...... 32 Amirouche Moktefi

Generating and Drawing Euler Diagrams

Generating Effective Euler Diagrams ...... 39 Almas Baimagambetov, John Howse, Gem Stapleton, and Aidan Delaney

Variational Pictures ...... 55 Martin Erwig and Karl Smeltzer

Edge Label Placement in Layered Graph Drawing...... 71 Christoph Daniel Schulze, Nis Wechselberg, and Reinhard von Hanxleden

Generation of Kolam-Designs Based on Contextual Array P Systems ...... 79 Ibrahim Venkat, Thamburaj Robinson, K. G. Subramanian, and Philippe de Wilde X Contents

Diagrams in Mathematics

Visual Algebraic Proofs for Unknot Detection ...... 89 Andrew Fish, Alexei Lisitsa, and Alexei Vernitski

A Typology of Mathematical Diagrams ...... 105 Mikkel Willum Johansen, Morten Misfeldt, and Josefine Lomholt Pallavicini

The Classificatory Function of Diagrams: Two Examples from Mathematics ...... 120 Christophe Eckes and Valeria Giardino

Mathematical Pictures ...... 137 Axel Arturo Barceló Aspeitia

Impact and Prevalence of Diagrammatic Supports in Mathematics Classrooms ...... 148 Bryan J. Matlen, Lindsey E. Richland, Ellen C. Klostermann, and Emily Lyons

What Sort of Information-Processing Machinery Could Ancient Geometers Have Used? ...... 164 Aaron Sloman

Interpreting Diagrammatic Reasoning – Between Empiricism and Realism . . . 172 Michał Sochański

Diagram Design, Principles and Classification

Picturing Science: Design Patterns in Graphical Abstracts ...... 183 Jessica Hullman and Benjamin Bach

A Framework for Analyzing and Designing Diagrams and ...... 201 Yuri Engelhardt and Clive Richards

A Classification of ...... 210 Helen C. Purchase, Katherine Isaacs, Thomas Bueti, Ben Hastings, Aadam Kassam, Allen Kim, and Steffan van Hoesen

Diagrammatic Maps of the New York Subway: An Historical Perspective . . . 219 Peter B. Lloyd

Sonifying Napoleon’s March by Identifying Auditory Correlates of the Graphic-Linguistic Distinction ...... 228 Peter W. Coppin and Richard C. Windeyer Contents XI

Towards a Typology of Diagrams in Linguistics ...... 236 Hans Smessaert and Lorenz Demey

Reasoning with Diagrams

Accessible Reasoning with Diagrams: From Cognition to Automation ...... 247 Zohreh Shams, Yuri Sato, Mateja Jamnik, and Gem Stapleton

Using Diagrams to About Biological Mechanisms ...... 264 William Bechtel, Adele Abrahamsen, and Benjamin Sheredos

A Survey and Evaluation of Diagrams for Navya-Nyāya...... 280 Jim Burton, Lopamudra Choudhury, and Mihir Chakraborty

Operations on Single Feature Indicator Systems ...... 296 Atsushi Shimojima and Dave Barker-Plummer

The Observational Advantages of Euler Diagrams with Existential Import . . . 313 Gem Stapleton, Atsushi Shimojima, and Mateja Jamnik

Observational Advantages: A Philosophical Discussion ...... 330 Francesco Bellucci

‘Diagrams’: A Hybrid Visual Information Representation and Reasoning Paradigm Towards Video Analysis ...... 336 Chayanika Deka Nath and Shyamanta M. Hazarika

Diagrammatic Definitions of Causal Claims ...... 346 Dean McHugh

Arcform...... 355 Benjamin Brink Allsopp

Euler and Venn Diagrams

Euler Diagrams Through the Looking Glass: From Extent to Intent ...... 365 Gem Stapleton, Amirouche Moktefi, John Howse, and Jim Burton

Rigor and the Context-Dependence of Diagrams: The Case of Euler Diagrams...... 382 David Waszek

Investigating Diagrammatic Reasoning with Deep Neural Networks ...... 390 Duo Wang, Mateja Jamnik, and Pietro Liò

Sequent Calculus for Euler Diagrams...... 399 Sven Linker XII Contents

Empirical Studies and Cognition

Metro Colour-Coding: Effect on Usability in Route Tracing ...... 411 Peter B. Lloyd, Peter Rodgers, and Maxwell J. Roberts

The Effect of Graphical Format and Instruction on the Interpretation of Three-Variable Bar and Line Graphs ...... 429 Nadia Ali and David Peebles

Read It This Way: Scaffolding Comprehension for Unconventional Statistical Graphs ...... 441 Amy Rae Fox and James Hollan

Rediscovering Isotype from a Cognitive Perspective ...... 458 Günther Schreder, Nicole Hynek, Florian Windhager, and Eva Mayr

Perceptual Processing and the Comprehension of Relational Information in Dynamic Diagrams ...... 470 Richard Lowe, Jean-Michel Boucheix, and Marilyne Menant

Visualizing Conversational Structure: Effects of Conversation-Analytical Knowledge and Social Media Experience...... 484 Leonie Bosveld-de Smet and Anne-Roos Bakker

Comparing and Contrasting Within Diagrams: An Effective Study Strategy ...... 492 Jennifer G. Cromley and Kyle Mara

The Role of Top-Down Knowledge in Spatial Cueing Using Hierarchical Diagrams ...... 500 Takeshi Sugio

Exploring the Relationship Between Visual Context and Affect in Diagram Interpretation ...... 509 Matthew Wood and Susan Stocklmayer

Visualising Latent Semantic Spaces for Sense-Making of Natural Language Text ...... 517 Ana Šemrov, Alan F. Blackwell, and Advait Sarkar

How Users Transform Node-Link Diagrams to Matrices and Vice Versa . . . . 526 Simone Kriglstein, Margit Pohl, and Johanna Doppler Haider

Peirce and Existential Graphs

Peirce and Proof: A View from the Trees...... 537 Dave Beisecker Contents XIII

A Weakening of Alpha Graphs: Quasi-Boolean Algebras ...... 549 Minghui Ma and Ahti-Veikko Pietarinen

Assertive and Existential Graphs: A Comparison...... 565 Ahti-Veikko Pietarinen and Daniele Chiffi

On the Transformation Rules of Erasure and Insertion in the Beta Part of Peirce’s Existential Graphs ...... 582 Shigeyuki Atarashi

Existential Graphs as a Basis for Structural Reasoning...... 590 Javier Legris

Multiple Readings of Existential Graphs ...... 598 Takashi Sasaki

Logic and Diagrams

Towards a Proof Theory for Heterogeneous Logic Combining Sentences and Diagrams ...... 607 Ryo Takemura

Iconic Logic and Ideal Diagrams: The Wittgensteinian Approach ...... 624 Timm Lampert

Aristotelian and Duality Relations Beyond the Square of Opposition ...... 640 Lorenz Demey and Hans Smessaert

Syllogistic with Jigsaw Puzzle Diagrams ...... 657 José Martín Castro-Manzano

Diagrammatically Formalising Constraints of a Privacy Ontology ...... 672 Shan Chen and Aidan Delaney

On Diagrams and General Model Checkers ...... 680 Sheila R. M. Veloso, Paulo A. S. Veloso, Mario R. F. Benevides, and Isaque M. S. Lima

Depicting the Redundancy of Fourth Figure Using Venn-Peirce Framework . . . 689 Sumanta Sarathi Sharma

Posters

Towards Diagram-Based Editing of Ontologies...... 699 Peter Chapman and William Roberts

Diagrams Including Increase Stock-Flow Performance...... 704 Friederike Brockhaus and Peter Sedlmeier XIV Contents

A Simple Decision Method for Syllogistic ...... 708 Frank Thomas Sautter and Gisele Dalva Secco

Wallis’s Use of Innovative Diagrams...... 712 Erika Rita Ortiz

Venn Diagram and Evaluation of Syllogisms with Negative Terms: A New Algorithm ...... 716 Mehdi Mirzapour and Christian Retoré

Making Sense of Schopenhauer’s Diagram of Good and Evil ...... 721 Jens Lemanski and Amirouche Moktefi

How Cross-Representational Signaling Affects Learning from Text and Picture: An Eye-Tracking Study ...... 725 Juliette C. Désiron, Mireille Bétrancourt, and Erica de Vries

Effect of Handedness on Mental Rotation ...... 729 Gunta Krumina, Jurgis Skilters, Annija Gulbe, and Vsevolod Lyakhovetskii

Euler Box Diagrams to Represent Independent and Non-independent Events ...... 734 James E. Corter

A Symmetry Metric for Graphs and Line Diagrams...... 739 Roman Klapaukh, Stuart Marshall, and David Pearce

Wrapping Layered Graphs ...... 743 Ulf Rüegg and Reinhard von Hanxleden

Diagrams, Musical Notation, and the Semiotics of Musical Composition . . . . 748 Vinícius Jonas de Aguiar

Calculus CL as Ontology Editor and Inference Engine...... 752 Jens Lemanski

Moded Diagrams for Moded Syllogisms ...... 757 José Martín Castro-Manzano and José Roberto Pacheco-Montes

A Review of Murner’s Cards for Syllogistic...... 761 José Martín Castro-Manzano, Jorge Medina-Delgadillo, and Juan Manuel Campos Benítez

Towards Executable Representations of Social Machines ...... 765 Dave Murray-Rust, Alan Davoust, Petros Papapanagiotou, Areti Manataki, Max Van Kleek, Nigel Shadbolt, and Dave Robertson Contents XV

Transforming Storyboards into Diagrammatic Models ...... 770 Elena-Teodora Miron, Christian Muck, Dimitris Karagiannis, and David Götzinger

Logical Reasoning with Object Diagrams in a UML and OCL Tool ...... 774 Khanh-Hoang Doan and Martin Gogolla

Scanning the Invisible: Framing Diagrammatic Cognition in Experimental Particle Physics ...... 779 Javier Anta

Teaching Argument Diagrams to a Student Who Is Blind ...... 783 Marc Champagne

Correction to: How Cross-Representational Signaling Affects Learning from Text and Picture: An Eye-Tracking Study...... C1 Juliette C. Désiron, Mireille Bétrancourt, and Erica de Vries

The Role of Diagrams in Contemporary Mathematics: Tools for Discovery? ...... 787 Jessica Carter

Unveiling Darwin’s Theory of Evolution Through the Epistemological Study of His Diagram ...... 791 Fabien Ferri

Can a Picture Prove a Theorem? Using Empirical Methods to Investigate Diagrammatic Proofs in Mathematics...... 796 Josephine Relaford-Doyle and Rafael Núñez

Otto Neurath’s Isotype and C. K. Ogden’s Basic English ...... 800 James McElvenny

Diagrammatic Approaches in Computational Musicology: Some Theoretical and Philosophical Aspects...... 803 Moreno Andreatta, Carlos Agon, Alexandre Popoff, and Andrée Ehresmann

Fruitful Over-Determination in Knot Diagrams ...... 807 Silvia De Toffoli

The Epistemology of Mathematical Necessity ...... 810 Catherine Legg

The Central Role of Diagrams in Algebraic Topology ...... 814 Emily R. Grosholz XVI Contents

Can Spontaneous Diagram Use be Promoted in Math Word Problem Solving? ...... 817 Hiroaki Ayabe and Emmanuel Manalo

Cognitive Control as an Underpinning of Relational Reasoning from Diagrams ...... 821 Leonidas A. A. Doumas, Robert G. Morrison, and Lindsey Engle Richland

Intersemiotic Translation: Transcreation and Diagrams ...... 825 Letícia Vitra and João Queiroz

Author Index ...... 829