The Foundation of Computational Visualistics
Total Page:16
File Type:pdf, Size:1020Kb
Variations and Application Conditions Of the Data Type »Image« The Foundation of Computational Visualistics Habilitationsschrift zur Erlangung der Venia legendi für Computervisualistik (computational visualistics) angenommen durch die Fakultät für Informatik der Otto-von-Guericke-Universität Magdeburg von: Dr. rer. nat. Jörg R.J. Schirra (Dipl.-Inform.) geb. am 03. August 1960 in Illingen (Saarland) Gutachter: Prof. Dr. Thomas Strothotte Prof. Dr. Wolfgang Wahlster Prof. Dr. Arno Ros Prof. Dr. Jerome A. Feldman Magdeburg, den 11. Mai 2005 THE FOUNDATION OF COMPUTATIONAL VISUALISTICS i A NOTE AHEAD This book is a map. It maps the landscapes of the country of digital images, or, as it was lately renamed, the realms of Computational Visualistics. Like any picture, a map – and hence this book – is a context builder: it allows the readers to ex- plore different paths in an abstract region, to connect many landmarks on several ways, and to establish their own distinctions of figures and backgrounds according to their proper interests. However, a text is bound to its linear progression of propositions woven into the digital fabric of argumentation that only mimics the true spatial quality of images. As an extended path, read- ing this text snakes through the map in the effort to systematically cover all of its regions: the map only appears in the reader’s mind. Not all of the details present may be integrated on first view. After all: a real map presents all its details simultaneously, but only those details are ac- tually “read” that are relevant for the reader’s present intentions. The map reveals its contents not on a single glance. That is to say: this text is explicitly written in order to be read more than one time. ii VARIATIONS AND APPLICATION CONDITIONS OF THE DATA TYPE »IMAGE« Concerning the sources of the ideas described: it is a valuable academic tradition to mention all means used, well observed by the bibliography at the end of this book. There is however a prob- lem in the strict application of this principle depending on the enormously extended medial ac- cess to the thoughts of others. It has actually become impossible to explicitly quote or even re- member everything that has contributed ideas to an ambitious academic work: apart from the classical form of scientific papers, books, talks, discussions, dialogues, and (long ago) lectures, which usually can be traced back easily, there have been documentaries in television, features in radio, articles in newspapers, fictional films and novels, comics and advertisements transmit- ting views effective in this book; from visits to exhibitions to web-browsing, many other kinds of mediated communication have provided arguments to the present text without the author being able to remember them in detail. Although I am not able to trace back all the “underground” elements not originated by the au- thor, and to list their sources: without them, this work would not have been possible. Magdeburg, October 2003 THE FOUNDATION OF COMPUTATIONAL VISUALISTICS iii Table of Content 1 IMAGES IN COMPUTER SCIENCE: CLARIFICATIONS REQUIRED 1 1.1 The Age of the Images 1 1.2 Toward Information Society 2 1.3 Images and Computers: The Digital Picture 3 1.4 Requirements for a Modern Computer Scientist 4 1.5 Determining the Goal 5 2 COMPUTATIONAL VISUALISTICS: SEEN FROM ITS ROOTS 7 2.1 Computer Science: Subject and Methodology 7 2.2 Visualistics and the Many Sciences of Pictures / Images 10 2.3 Computational Visualistics and the Data Type »Image« 12 3 PRELIMINARY CLARIFICATIONS FROM VISUALISTICS 15 3.1 Pictures on the Border: Overlooking a Wide Kingdom 16 3.2 A Synthetic Proposal: Images as ”Perceptoid“ Signs 22 3.2.1 »Sign« as Genus Proximum for Pictures 22 3.2.2 »Perceptoid« as Differentia Specifica for Pictorial Signs 25 3.2.3 A Note on “Natural Images”, “Indices”, and “Icons” 27 3.3 Image and Object 29 3.3.1 The Naïve Approach to Resemblance 30 3.3.2 The Act-Theoretic Basis of the Concept »Resemblance« 31 3.3.3 Perception, Deception, and Primary Object Constitution 32 3.4 Image and Language 34 3.4.1 Assertions, Identity, and Contexts 34 3.4.2 Communication Among Pre-Object Creatures 36 3.4.3 Context Builders and Referential Anchoring 38 3.4.4 Secondary Object Constitution: Sortal Concepts & Geometry 40 3.4.5 Pictures as Context Builders: Resemblance Once More 43 3.5 Image and Image User 46 3.5.1 Reflection Modes of Dealing with Pictures 46 3.5.2 The Game of Picture Making 49 3.5.3 Who Is Communicating with Whom? 51 3.5.4 Indirect Resemblances & Rhetoric Derivations 53 3.5.5 Reflective Communication & Pictures of Art 59 3.6 Conclusions for Computational Visualistics 60 iv VARIATIONS AND APPLICATION CONDITIONS OF THE DATA TYPE »IMAGE« 4 THE GENERIC DATA TYPE »IMAGE«: GENERAL ASPECTS 63 4.1 The Organizational Principle of the Discussion 63 4.2 Syntactic Aspects 65 4.2.1 Pictorial Resolution and the Identity of Images 65 4.2.1.1 Density, Continuity, and Decidability 67 4.2.1.2 Syntactic Types of Pictures in Computers 67 4.2.2 Remarks on Compositionality 70 4.2.2.1 Composition of One Picture: Pictorial “Morphology” 71 4.2.2.2 Compositions With Pictures: Pictorial “Text Grammars” 74 4.2.3 Some Notes on Formalizing Color 76 4.2.4 Syntactic Transformations and Image Processing 77 4.2.5 The Limitations of Pictorial Syntax 79 4.3 Semantic Aspects 80 4.3.1 Computer Graphics, Spatial Objects, and Perspective 81 4.3.1.1 Sortal Objects and Geometric Models 82 4.3.1.2 Excursion into the Theory of Rational Argumentation 83 4.3.1.3 Reasoning with Spatial Objects 86 4.3.1.4 A Perspective on Perspectives 88 4.3.2 Two Levels of Computer Vision: An Example 90 4.3.2.1 Constructing Visual Gestalts – Or Finding Pixemes 90 4.3.2.2 Instantiating Object Schemata 94 4.3.2.3 Determining Configurations 96 4.3.2.4 Computer Vision and “Picture Understanding” 97 4.3.2.5 Reference Semantics and Pictorial Reference 99 4.3.3 Embedding Semantics in Pragmatics 102 4.4 Pragmatic Aspects 103 4.4.1 Interactive Systems as a New Type of Media 104 4.4.1.1 Media of Class IV 106 4.4.1.2 The Selection Problems: Content 108 4.4.1.3 The Selection Problems: Form 110 4.4.1.4 Combined Selection Problems for Choosing a Picture 113 4.4.2 Anticipating the Unknown Beholders 114 4.4.2.1 Remarks on the Purposes of Picture Uses 114 4.4.2.2 User Modeling for Pictures 118 4.4.2.3 Adaptation to the Pragmatics of Context-Building 123 4.4.3 Authenticity and Media of Class IV 125 4.4.3.1 Beholder Models and Authenticity 126 4.4.3.2 Authenticity as a Technical Problem: Syntactic Approaches 127 4.4.4 Information Visualization and the Rhetorics of Structural Pictures 130 4.4.4.1 On Source Domains and Target Domains 130 4.4.4.2 Finding Appropriate Visualization Parameters: An Overview 133 4.4.4.3 Interactive Visualizations 135 4.4.5 Remarks on the Pragmatics of Computer Art 136 4.4.5.1 Reflective Pictures and the Reflective Mode of Reception 137 4.4.5.2 Computer Art – Art with the Computer 138 4.4.5.3 Interactivity in Computer Art 141 THE FOUNDATION OF COMPUTATIONAL VISUALISTICS v 5 CASE STUDIES: USING THE DATA TYPE »IMAGE« 151 5.1 Semantic Requests to Image Databases in IRIS 151 5.1.1 Image Retrieval for Information Systems 152 5.1.2 Results and Queries 155 5.2 Rhetorically Enriched Pictures 156 5.2.1 Descibing Style Parameters 158 5.2.2 The Heuristics of Predicative Naturalism 159 5.2.3 Example Application of the Heuristics 160 5.3 A Border Line Case: Immersion 161 5.3.1 Virtual Architectur: The Atmosphere Projekt 163 5.3.2 Types of Use of Virtual Architecture 169 5.3.3 The Virtual Institute of Image Science 174 5.3.4 Conclusions 179 5.4 Another Border Line Case: Mental Images 179 5.4.1 An Example Task: Understanding Reports From Absent Spatial Events 180 5.4.2 On the Cognitive Function of Mental Images 182 5.4.3 Building a Computer Model 184 5.4.4 Conclusions: The Data Type »Image« and Explaining Mental Images 188 6 CONCLUSIONS – PERSPECTIVES 191 6.1 The Components of »Image« as Basis of Computational Visualistics 191 6.2 Computational Visualistics in Education 193 6.2.1 An Example: Structure of Computational Visualistics in Magdeburg 194 6.2.2 Mental Imagery as a Preview Criterion for Study Success 194 6.2.3 An Empirical Investigation 196 6.3 The Future of an Institutional Computational Visualistics 197 APPENDICES 201 A References 201 B List of Figures 210 C List of Tables 214 D Overview in German 215 vi VARIATIONS AND APPLICATION CONDITIONS OF THE DATA TYPE »IMAGE« THE FOUNDATION OF COMPUTATIONAL VISUALISTICS 1 1 Images in Computer Science: Clarifications Required 1.1 The Age of the Images Images take a rather prominent place in contemporary life in the western societies. Together with language, they have been connected to human culture from the very beginning. Recently – that is, after several millennia of written word’s dominance – their part is increasing again remarkably. Can’t we even characterize the 20th century as the century of pictures? Photography and film have reached a heyday barely anticipated when they were invented at the end of the 19th century.