DOCSLIB.ORG

Improving Knowledge Development and Exchange Via Transformative Pictogram Design

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Improving Knowledge Development and Exchange Via Transformative Pictogram Design 2020 24th International Conference Information Visualisation (IV) Visual Design Thinking for Public Education: Improving knowledge development and exchange via transformative pictogram design 1st Nana Wang 2nd Leah Burns Sichuan University Aalto Univeristy China Finland 0000-0003-3902-7393 leah.burns@aalto.fi Abstract—How might the exchange and development of knowl- edge improve through a critical examination of pictogram- based methods of information visualization? In this article, we investigate the International System of TYpographic Picture Education(ISOTYPE), an influential model of pictorial diagram design theory and practice. Given ISOTYPE’s continuing impact on information design, we use it as a critical case study to assess the potential and limitations of pictorial diagrams(PD) for knowl- edge development and exchange. The goal of this study is not to evaluate artistic quality, style, or designer talent; rather, our focus is on analysis of the learning functions of pictograms and their potential for knowledge transmission and supporting reasoning behaviour among diverse audiences. We explore the different features of a pictogram, and how these features might support knowledge development through diagrammatic reasoning(DR). Three key questions are posed: 1. What are the advantages and disadvantages of pictograms for promoting learning and reasoning behaviour versus more abstract or representational visual information design? 2. What are the criteria for choosing the visual characteristics of pictograms? 3. How can the visual characteristics of pictograms be evaluated and revised base on the goal of supporting learning and reasoning behaviour? Index Terms Fig. 1. ISOTYPE, Atlas, Gesellschaft und Wirtschaft,1930, reproduced from —ISOTYPE; Pictorial diagram(PD); Public ed- Osterreichisches¨ Gesellschafts - und Wirtschaftsmuseum ucation; Informal learning; Knowledge visualization(KV); dia- grammatic reasoning(DR) I. INTRODUCTION ”chart junk” [7] in information Visualization(info VIS) [4] [6], The International System of TYpographic Picture Educa- due to these pictograms hide the exact statistic number, and tion(ISOTYPE) considered as one of the classic pictogram these decorations of varies seems invalid to do tell the viewers language systems and which aimed for the public education, anything new. [7] [8] originated from the interaction of Austrian socialist Otto In this article, we argue that the pictorial diagram (PD) Neurath’s educational ambitions with the context of social- has been underestimated and neglected their contribution to democratic Vienna in the 1920s [1] [2] [3]. ISOTYPE is public education area in the KV relevant researches. ISOTYPE considered as one of the pioneers for the modern visualization adopting pictorial diagrams represent complex concept are still method even decades later. Like shown in Fig.1, ISOTYPE worthy to dig in. Especially when more and more wicked adopted pictogram to exhibit knowledge from different do- problems [9] addressed need inter-professional collaboration mains based on quantitative information, pictograms play a key or that seek to support various forms of public education and role in their works. The core founders, Otto Neurath and Marie engagement. Like COVID-19, it is difficult to get through the Neurath cooperated with German artist Gerd Arntz created epidemic without cooperation from the whole society. How- about 2000 pictograms as a “visual dictionary.” ever, just base on the line chart of infection cases update, like Otto Neurath strongly highlighted the primary responsibility shown in Fig.2, is impossible to gain a holistic view about the of ISOTYPE is for supporting learning. However, conversely, situation. What’s worse, the side effect even can provoke the ISOTYPE has been scrutinised, analysed and critiqued as turbulence and panic emotion, raise intense situation. Pictorial 978-1-7281-9134-8/20/$31.00 ©2020 IEEE 502 DOI 10.1109/IV51561.2020.00087 an informal learning environment [16], learning is intrinsically motivated and reflects the learning individuals themselves, like learning ability, educational background, preferences, as so on [15] [17]. In museums, there are different groups of visitors, and even within one group, such as a school group, one still needs to consider different levels of education. Uncertainties about learning ability, experience, and learning background cause a bottleneck in trying to provide an accurate learner portrait, and the learner-centred approach lacks proof and accuracy. Besides, museum pedagogy prefers to describe learning in the museum as free-choice learning [16], in terms of the learning process starts from they want to, rather than because they have to. For supporting learning behaviours, ISOTYPE Fig. 2. New COVID-19 cases reported by day and by Race/Ethnicity provides a countable and meaningful pictorial language to in the U.S,https://www.cdc.gov/coronavirus/2019-ncov/cases-updates/cases- communicate. By counting and comparing the pictograms, in-us.html people without typical education background can grasp the answer of some wicked problems, some examples include the apparatus in a diagram not only provides an accessible way management of food systems, health care, and education. Ad- to approach complex concept but also leading an integrated dressing these concerns requires the construction of complex knowledge transmission process, rather than a simplification adaptive systems [18]. of complex concepts in ways that limit possible responses. II. UNDERSTANDING THE VISUALIZATION OF PD IN THE C. PD and the advantages PUBLIC EDUCATION SETTING A. ISOTYPE as data-based knowledge representation method The linguistic turn in semiotics in the 1970s provided a instead of info VIS style new theoretical framework for the generation of meaning [19]. Pictogram is one of the most appealing keywords in existing This change also affected the study of knowledge transfer ISOTYPE researches, a collective agreement that the purpose in pedagogy [20]. The new theoretical trend highlights the of ISOTYPE is not to make data decorative [4] [5] [11]. function of interpretation by the receiver. It states that the However, majority researches still considered ISOTYPE as acquisition of knowledge is a dynamic process, emphasizing data-base info VIS style, in terms of their investigation focused the process of the receiver’s continuous logic construction, on the cognitive meanings of the pictogram, and examine the and application of knowledge or beliefs in the process of functionality of simplifying cognition [5] [10] [11] [12]. knowledge acquisition [21]. Info VIS reports the facts, While KV aims to transfer However, it is difficult for the public education agency to insights, experiences, and attitudes, which are invisible, in- evaluate the learning width and depth from a diverse audience. tangible, resistant to quantification, and difficult to externalise PD not only provide readable images but also by utilizing [13]. As the main founders of ISOTYPE, Neuraths considered visual attributes provide multiple levels of reasoning process ISOTYPE to be “more than sum of its pictograms”, by instead of a single target for all audience. Diagrammatic using their words, these pictograms just the “bricks” in ISO- reasoning (DR) is a reasoning method by using a diagram to TYPE(architecture), [1] . The most extraordinary contribution externalize the tacit logical procedures, ‘greatly facilitates the of ISOTYPE, is based on data, using pictograms to provide solution of problems of logic’ [22]. As visual inquire method, several learning possibilities, learn from the information (facts) pictogram plays the role in guiding in the DR process, bridges and discovery the knowledge(reasons). the invisible logic connections, visual connections undertake B. Informal learning setting and public education the semantic meanings in the reasoning process. ISOTYPE PDs, mainly work for externalizing logic connections by using Exhibitions and museums are appealing keywords when we visual apparatus, conducting reasoning and inquiring step by scope on ISOTYPE. The majority of diagrams came across step [23]. with audiences in the museum or other informal learning Moreover, for supporting learning behaviours, ISOTYPE settings, like a department store or park [3]. For example, PDs provides a countable and meaningful pictorial language the exhibition named Around Rembrandt, which ISOTYPE to communicate, by counting and comparing the pictograms, curated in 1935, held in department stores in Amsterdam, people without typical education background can grasp the Rotterdam, and Hague. By using PDs, exhibited all the relevant answer base on the diagrammatic framework, provide a more knowledge around Rembrandt, illustrated vivid pictures about accessible approach with the public to get in touch with typical artist real life back to the old time. Compare with school, in knowledge as they wished. 503 III. THE VALUE OF PD FOR PUBLIC EDUCATION Knowledge transmission and learning support both can be defined as the functions of KV. However, these synonyms re- flect different design approaches: learner-oriented and sender- oriented processes. In KV, human-centeredness is seen in that communities of learners determining the meaning instead of the senders of the visualization [24], generally emphasising support learning [25] [26]. However, in public education contexts, the huge amount of learners and their diverse back- grounds lead to learners being uncertain of their analysis. It becomes difficult to apply
Load more

Recommended publications
  • Maintaining Spatial Relations in an Incremental Diagrammatic Reasoner
    Maintaining Spatial Relations in an Incremental Diagrammatic Reasoner Ronald W. Ferguson, Joseph L. Bokor, Rudolph L. Mappus IV and Adam Feldman College of Computing Georgia Institute of Technology 801 Atlantic Avenue Atlanta, GA 30332 {rwf, jlbokor, cmappus, storm} @cc.gatech.edu Abstract gram. This work extends the GeoRep diagrammatic rea- soner (Ferguson & Forbus, 2000), which is described in Because diagrams are often created incrementally, a qualitative section 3. After describing GeoRep, we discuss how Ge o- diagrammatic reasoning system must dynamically manage a po- Rep was modified to allow incremental processing, and tentially large set of spatial interpretations. This paper describes cover a number of implementation issues: how to handle an architecture for handling spatial relations in an incremental, nonmonotonic diagrammatic reasoning system. The architecture composite objects, the interface between low-level and represents jointly exhaustive and pairwise disjoint (JEPD) spatial high-level reasoning, and a modified default assumption relation sets as nodes in a dependency network. Examples of these mechanism. We also describe extensions to a user interface spatial relation sets are interval relations, relative orientation rela- allowing a user to create diagrams and update the infer- tions, and connectivity relations. The network caches dependen- ences of the reasoner. We then discuss future challenges cies between low-level spatial relations, allowing those relations for this architecture. to be easily assumed or retracted as visual elements are added or removed from a diagram. We also describe how the system sup- ports high-level reasoning, including support for creating default 2. Related Work assumptions. Finally, we show how this system was integrated In qualitative spatial reasoning, researchers have explored with an existing drawing program and discuss its possible use in how to process qualitative spatial vocabularies incremen- diagrammatic and geographic reasoning.
    [Show full text]
  • Knowledge Representation in Bicategories of Relations
    Knowledge Representation in Bicategories of Relations Evan Patterson Department of Statistics, Stanford University Abstract We introduce the relational ontology log, or relational olog, a knowledge representation system based on the category of sets and relations. It is inspired by Spivak and Kent’s olog, a recent categorical framework for knowledge representation. Relational ologs interpolate between ologs and description logic, the dominant formalism for knowledge representation today. In this paper, we investigate relational ologs both for their own sake and to gain insight into the relationship between the algebraic and logical approaches to knowledge representation. On a practical level, we show by example that relational ologs have a friendly and intuitive—yet fully precise—graphical syntax, derived from the string diagrams of monoidal categories. We explain several other useful features of relational ologs not possessed by most description logics, such as a type system and a rich, flexible notion of instance data. In a more theoretical vein, we draw on categorical logic to show how relational ologs can be translated to and from logical theories in a fragment of first-order logic. Although we make extensive use of categorical language, this paper is designed to be self-contained and has considerable expository content. The only prerequisites are knowledge of first-order logic and the rudiments of category theory. 1. Introduction arXiv:1706.00526v2 [cs.AI] 1 Nov 2017 The representation of human knowledge in computable form is among the oldest and most fundamental problems of artificial intelligence. Several recent trends are stimulating continued research in the field of knowledge representation (KR).
    [Show full text]
  • What's in a Diagram?
    What’s in a Diagram? On the Classification of Symbols, Figures and Diagrams Mikkel Willum Johansen Abstract In this paper I analyze the cognitive function of symbols, figures and diagrams. The analysis shows that although all three representational forms serve to externalize mental content, they do so in radically different ways, and conse- quently they have qualitatively different functions in mathematical cognition. Symbols represent by convention and allow mental computations to be replaced by epistemic actions. Figures and diagrams both serve as material anchors for con- ceptual structures. However, figures do so by having a direct likeness to the objects they represent, whereas diagrams have a metaphorical likeness. Thus, I claim that diagrams can be seen as material anchors for conceptual mappings. This classi- fication of diagrams is of theoretical importance as it sheds light on the functional role played by conceptual mappings in the production of new mathematical knowledge. 1 Introduction After the formalistic ban on figures, a renewed interest in the visual representation used in mathematics has grown during the last few decades (see e.g. [11–13, 24, 27, 28, 31]). It is clear that modern mathematics relies heavily on the use of several different types of representations. Using a rough classification, modern mathe- maticians use: written words, symbols, figures and diagrams. But why do math- ematicians use different representational forms and not only, say, symbols or written words? In this paper I will try to answer this question by analyzing the cognitive function of the different representational forms used in mathematics. Especially, I will focus on the somewhat mysterious category of diagrams and M.
    [Show full text]
  • Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering
    This PDF is available from The National Academies Press at http://www.nap.edu/catalog.php?record_id=13362 Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering ISBN Susan R. Singer, Natalie R. Nielsen, and Heidi A. Schweingruber, Editors; 978-0-309-25411-3 Committee on the Status, Contributions, and Future Directions of Discipline-Based Education Research; Board on Science Education; 282 pages Division of Behavioral and Social Sciences and Education; National 6 x 9 PAPERBACK (2012) Research Council Visit the National Academies Press online and register for... Instant access to free PDF downloads of titles from the NATIONAL ACADEMY OF SCIENCES NATIONAL ACADEMY OF ENGINEERING INSTITUTE OF MEDICINE NATIONAL RESEARCH COUNCIL 10% off print titles Special offers and discounts Distribution, posting, or copying of this PDF is strictly prohibited without written permission of the National Academies Press. Unless otherwise indicated, all materials in this PDF are copyrighted by the National Academy of Sciences. Request reprint permission for this book Copyright © National Academy of Sciences. All rights reserved. Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering DISCIPLINE!BASED EDUCATION RESEARCH Understanding and Improving Learning in Undergraduate Science and Engineering Committee on the Status, Contributions, and Future Directions of Discipline-Based Education Research Board on Science Education Division of Behavioral
    [Show full text]
  • When Everything Goes Wrong Make a Diagram Focusing on Visualising Reasoning and Trying to Reclaim the Power of Logical Thinking and Good Argumentation
    https://doi.org/10.25145/b.2COcommunicating.2020.006 34 COmmunicating COmplexity When Everything goes wrong Make a Diagram focusing on visualising reasoning and trying to reclaim the power of logical thinking and good argumentation. It might feel a small action, but diagrams are actually really powerful. Maria Rosaria Digregorio De Montfort University, Leicester Media School, Faculty of Technology, 2 The Power of Diagrams The Gateway, Leicester, LE1 9BH, United Kingdom focusing on visualising reasoning and trying to reclaim the power of logical thinking [email protected] and good argumentation. It might feel a small action, but diagrams are actually reallyVisual inferences. In math and geometry, visual configurations are used as proper powerful.inferences to demonstrate the validity of reasoning – for instance in the case of the Pythagoras’s Theorem and the binomial theorem. Despite written words or Abstract. Looking at historical examples, this research explores the power of notations can also be used to visually represent theorems, the diagram is able to diagrams and visual reasoning. It focuses on their ability to make knowledge show at glance the reasons behind the rule (Perondi, 2012). more accessible, train critical thinking and trigger a form of intellectual 2 The Power of Diagrams resistance in a post-truth world; it encourages a deeper integration between diagrams and writing, and the diffusion of information design approaches Visual inferences. In math and geometry, visual configurations are used as proper outside the design realm. inferences to demonstrate the validity of reasoning – for instance in the case of the Keywords: post-truth / diagrammatic reasoning / non-linear writing / visual Pythagoras’s Theorem and the binomial theorem.
    [Show full text]
  • Do We Really Reason About a Picture As the Referent?
    Do We Really Reason about a Picture as the Referent? Atsushi Shimojima1,2 and Takugo Fukaya 1School of Knowledge Science, Japan Advanced Institute of Science and Technology 1-1 Asahi-dai, Tatsunokuchi, Nomi-gun, Ishikawa 923-1292, Japan 2ATR Media Information Science Laboratories 2-2-2 Hikari-dai, Keihanna Science City, Kyoto 619-0288, Japan [email protected] [email protected] Abstract the previous case, however, the hypothesized operation is not an operation on the hinge picture, but an operation A significant portion of the previous accounts of infer- on the physical hinge the picture depicts. You may refer ential utilities of graphical representations (e.g., Sloman, 1971; Larkin & Simon, 1987) implicitly relies on the exis- back to the hinge picture now and then, but what your in- tence of what may be called inferences through hypothet- ference is about is the movement of the upper leg of the ical drawing. However, conclusive detections of them by hinge, not the movement of the upper line of the hinge means of standard performance measures have turned out picture. You are reasoning about the picture’s referent, to be difficult (Schwartz, 1995). This paper attempts to fill rather than the picture itself. the gap and provide positive evidence to their existence on the basis of eye-tracking data of subjects who worked Well, the concept of “reasoning about the picture it- with external diagrams in transitive inferential tasks. self” is thus clear, but is it real? Are we really engaged in that type of inferences in some cases? If so, how do Schwartz (1995) cites an intriguing example to dis- we tell when we are? tinguish what he calls “reasoning about a picture as the Schwartz (1995) himself assumed the existence of that referent” from “reasoning about the picture’s referent.” type of inferences, and went on to investigate what fea- Suppose you are given the picture of a hinge in Figure tures of pictorial representations encourage it.
    [Show full text]
  • Visualization for Constructing and Sharing Geo-Scientific Concepts
    Colloquium Visualization for constructing and sharing geo-scientific concepts Alan M. MacEachren*, Mark Gahegan, and William Pike GeoVISTA Center, Department of Geography, Pennsylvania State University, 302 Walker, University Park, PA 16802 Representations of scientific knowledge must reflect the dynamic their instantiation in data. These visual representations can nature of knowledge construction and the evolving networks of provide insight into the similarities and differences among relations between scientific concepts. In this article, we describe scientific concepts held by a community of researchers. More- initial work toward dynamic, visual methods and tools that sup- over, visualization can serve as a vehicle through which groups port the construction, communication, revision, and application of of researchers share and refine concepts and even negotiate scientific knowledge. Specifically, we focus on tools to capture and common conceptualizations. Our approach integrates geovisu- explore the concepts that underlie collaborative science activities, alization for data exploration and hypothesis generation, col- with examples drawn from the domain of human–environment laborative tools that facilitate structured discourse among re- interaction. These tools help individual researchers describe the searchers, and electronic notebooks that store records of process of knowledge construction while enabling teams of col- individual and group investigation. By detecting and displaying laborators to synthesize common concepts. Our visualization ap- similarity and structure in the data, methods, perspectives, and proach links geographic visualization techniques with concept- analysis procedures used by scientists, we are able to synthesize mapping tools and allows the knowledge structures that result to visual depictions of the core concepts involved in a domain at be shared through a Web portal that helps scientists work collec- several levels of abstraction.
    [Show full text]
  • The Mystery of Deduction and Diagrammatic Aspects of Representation
    Rev.Phil.Psych. (2015) 6:49–67 DOI 10.1007/s13164-014-0212-5 The Mystery of Deduction and Diagrammatic Aspects of Representation Sun-Joo Shin Published online: 20 January 2015 © Springer Science+Business Media Dordrecht 2015 Abstract Deduction is decisive but nonetheless mysterious, as I argue in the intro- duction. I identify the mystery of deduction as surprise-effect and demonstration- difficulty. The first section delves into how the mystery of deduction is connected with the representation of information and lays the groundwork for our further discussions of various kinds of representation. The second and third sections, respec- tively, present a case study for the comparison between symbolic and diagrammatic representation systems in terms of how two aspects of the mystery of deduction – surprise-effect and demonstration-difficulty – are handled. The fourth section illustrates several well-known examples to show how diagrammatic representation suggests more clues to the mystery of deduction than symbolic representation and suggests some conjectures and further work. When we have a choice to get from given premises to a conclusion either by deduc- tion or by induction, the choice is obvious: deduction is the way to go. With a correctly carried out deduction, the truth of the conclusion is guaranteed by the truth of the premises. Hence, after accepting premises to be true and checking each deduc- tive step, we embrace the truth of the conclusion. It is one sure way to secure the certainty of the truth of a proposition. This is the strength of deduction. Hence, math- ematics, the discipline where deduction plays a crucial role, has enjoyed its special status in the world of knowledge.
    [Show full text]
  • Diagrammatic Reasoning in Support of Situation Understanding and Planning
    DIAGRAMMATIC REASONING IN SUPPORT OF SITUATION UNDERSTANDING AND PLANNING B. Chandrasekaran, John R. Josephson, Bonny Banerjee and Unmesh Kurup Department of Computer & Information Science The Ohio State University Columbus, OH 43210 Robert Winkler U. S. Army Research Lab 2800 Powder Mill Rd Adelphi, MD 20783 ABSTRACT Visual representations consisting of terrain maps with an overlay of diagrammatic elements are ubiquitous in Army situation understanding, planning and plan monitoring. The Objective Force requirements of responsiveness, agility and versatility call for digitized graphical decision support interfaces that automate or otherwise help in various reasoning tasks. The main purpose of the paper is to describe the issues involved in building a diagrammatic reasoning system, specifications for a diagrammatic representation formalism, and an architecture for problem solving with diagrams. We have begun implementation of an application that infers maneuvers from data, obtained from an exercise at the Figure 1. A COA diagram using Army standard National Training Center, about locations and motions of symbology. Blue and Red forces. We present algorithms used in the initial stages of an implementation. conceptual knowledge to make a complex series of inferences to help them predict, evaluate and plan. The 1. INTRODUCTION whole process is so natural that we often fail to appreciate the complexity of the cognitive activities involved. Visual representations with overlay of diagrammatic Nevertheless, understanding and formalizing the elements are ubiquitous in military decision-making. processes involved in such apparently effortless reasoning Commanders represent their situation understanding and is necessary if we wish to provide effective decision intended maneuvers and plans, and monitor the action, on support systems for a commander and his staff.
    [Show full text]
  • Diagrammatic Reasoning Meets Medical Risk Communication
    Diagrammatic Reasoning Meets Medical Risk Communication Angela Brunstein ([email protected]) Department of S.A.P.E. - Psychology, American University in Cairo, AUC Ave, PO Box 74, New Cairo 11835, Egypt Joerg Brunstein ([email protected]) Department of S.A.P.E. - Psychology, American University in Cairo, AUC Ave, PO Box 74, New Cairo 11835, Egypt Ali Marzuk ([email protected]) Royal College of Surgeons in Ireland – Bahrain, PO Box 15503, Adliya, Bahrain Abstract 2006). Also medical students perform better for displays on accumulation problems than non-medical students for some, Informed consent for medical procedures requires that patients understand risks associated with diagnostic and but not all medical scenarios (Brunstein, Gonzalez, & treatment options. Similar to performance for diagrammatic Kanter, 2010). reasoning and system dynamics, patients, physicians and In this study, we aimed to combine the lessons learned medical students are reported to perform poorly on from diagrammatic reasoning for understanding a treatment understanding medical risk-related information. At the same scenario on ventricular fibrillation with medical students time, different presentation formats seem to support different and non-medical undergraduates. kind of conclusions across domains. In this research, we investigated different formats of presenting risk information For decision whether or not to undergo surgery to get an related to a treatment scenario with 22 medical and 50 non- implantable cardioverter defibrillator (ICD) after surviving a medical students. As expected, medical students performed heart attack, patients need to evaluate the risk of having better than non-medical students for all versions of the another heart attack (i.e., severity of disease) and how likely problem, while non-medical students could partially an ICD can save their life during that heart attack (i.e., compensate missing medical knowledge with displays that effectiveness of treatment).
    [Show full text]
  • Diagrammatic Reasoning in Separation Logic
    Diagrammatic Reasoning in Separation Logic M. Ridsdale1, M. Jamnik1, N. Benton2, and J. Berdine2 1 University of Cambridge 2 Microsoft Research Cambridge Abstract. A new method of reasoning about simple imperative pro- grams in separation logic is proposed. Rather than proving program specifications symbolically, the hope is to model more closely human diagrammatic reasoning, and to perform automated diagrammatic rea- soning in separation logic. 1 Introduction Separation logic is used for reasoning about low-level imperative programs that manipulate pointer data structures. It enables the writing of concise proofs of correctness of the specifications of simple programs, and such proofs have been successfully automated. When reasoning informally about separation logic, it is often useful to draw diagrams representing program states, with memory locations represented by boxes, pointers represented by arrows, etc. We aim to formalise these diagrams and implement an automated theorem prover (ATP) which makes use of this formalism. This proposal outlines a promising direction for research. The ideas on diagrammatic reasoning are drawn from [1], which is on diagrammatic theorem proving in arithmetic; we also draw on ideas from [2], which is on symbolic automated theorem proving in separation logic. An example of the kind of y:=nil; while x!=nil do x (k:=[x+1]; [x+1]:=y; y:=x; x:=k) α α k α α k := [x+1] 4. y 1 2 3 4 α α α α [x+1]:= y 1. x 1 2 3 4 Initial state nil nil nil y α α k α α y := x α α α α k := [x+1] 5.
    [Show full text]
  • Arxiv:2001.11224V1 [Cs.CL] 30 Jan 2020 the Perspective of Multimodality Research
    Introducing the diagrammatic mode Tuomo Hiippala1 and John A. Bateman2 1 University of Helsinki, Helsinki, Finland 2 Bremen University, Bremen, Germany [email protected] [email protected] Abstract. In this article, we propose a multimodal perspective to di- agrammatic representations by sketching a description of what may be tentatively termed the diagrammatic mode. We consider diagrammatic representations in the light of contemporary multimodality theory and explicate what enables diagrammatic representations to integrate natu- ral language, various forms of graphics, diagrammatic elements such as arrows, lines and other expressive resources into coherent organisations. We illustrate the proposed approach using two recent diagram corpora and show how a multimodal approach supports the empirical analysis of diagrammatic representations, especially in identifying diagrammatic constituents and describing their interrelations. Keywords: Diagrams · Semiotics · Multimodality · Corpora · Annota- tion. 1 Introduction Multimodality research is an emerging field of study which examines how com- munication builds on appropriate combinations of multiple modes of expression, such as natural language, illustrations, drawings, photography, gestures, layout and many more. Modern multimodality theory has developed a battery of the- oretical concepts to support more strongly empirical analysis of such complex communicative situations and artefacts. Several core concepts, including semi- otic mode [4,18], medium [8,5] and genre [3,12], theorise how individual modes of expression are structured and what precisely enables them to combine and co-operate with each other. Although diagrams are often acknowledged to draw on multiple modes of expression [26], they have rarely been approached from arXiv:2001.11224v1 [cs.CL] 30 Jan 2020 the perspective of multimodality research.
    [Show full text]