‘Modality Relativity’ The Influence of Sign Language and Spoken Language on Conceptual Categorization Von der Philosophischen Fakultät der Rheinisch-Westfälischen Technischen Hochschule Aachen zur Erlangung des akademischen Grades einer Doktorin der Philosophie genehmigte Dissertation vorgelegt von Klaudia Grote aus Surwold Berichter: Prof. Dr. phil. Ludwig Jäger Universitätsprofessor Dr. rer. nat. Klaus Willmes-von Hinckeldey Tag der mündlichen Prüfung: 30. April 2013 „Diese Dissertation ist auf den Internetseiten der Hochschulbibliothek verfügbar.“ Acknowledgment First and foremost I owe my gratitude to all deaf people who have made this dissertation possible and because of whom my research and work experience has been one that I will cherish forever. My deepest gratitude is to my advisors, Prof. Dr. Ludwig Jäger and Prof. Dr. Klaus Willmes-von Hinckeldey for their academic guidance, scientific expertise, and intellectual counsel throughout the maturation process of this thesis. I am thankful to have had the opportunity to work in the interdisciplinary research project 'Mediality of linguistic Signs' at the 'Center for Cultural Research' of the University of Cologne and would like to thank the members of the group, Erika Linz, Luise Springer, Gisela Fehrmann, Cornelia Epping-Jäger, Wiebke Iversen, Horst Sieprath and Isa Werth. I have been amazingly fortunate having had advisors and colleagues who gave me the opportunity to develop my own ideas and, at the same time, the support to overcome many difficulties and finish the empirical studies. In addition, many other people have taken the time to discuss with me the contents of the thesis and the ideas developed in it. I would especially like to thank Prof. Walter Huber, Prof. Claudia Becker, Prof. Jill Morford, Prof. Karen Emmorey, Prof. Jan-Peter de Ruiter, Dr. Ulla Louis-Nouvertné and Alice Wiedemann as I am indebted to them for sharing their knowledge and expertise, and I have benefited greatly from it. This thesis would not have been possible without the help and friendship of the many deaf people whom I have had the good fortune of meeting since working at the RWTH University. I am especially grateful to Horst Sieprath, who’s pedagogical and didactic skills paved my way into the world of signing. I am immensely grateful to the deaf and cross-modal bilingual individuals who took the time to participate in the studies. My thanks also go to Stefan Dörpinghaus for technical support. Finally, my family and friends have encouraged and supported me, and I thank them for standing by me. They have helped me stay sane through all these years. Their support and care helped me overcome setbacks and continue the writing of this dissertation. I greatly value their friendship and I deeply appreciate their belief in me. Abstract This dissertation investigates the effects of language modality on conceptual categorization. It is hypothesized that spoken and signed languages have an influence on the processing and structure of conceptual categories. Two aspects of language modality in which spoken and signed languages show major differences are Simultaneity and Iconicity. Compared to the audio-vocal modality of spoken languages, the visual-gestural modality of signed languages is well suited for simultaneously producing a large amount of information and moreover, shows a much greater disposition for iconic signs. The simultaneous articulation in signed languages is only possible for concepts which are syntagmatically related (noun related to an attribute, action etc.) and the so-called polycomponential signs are iconic. Paradigmatically related concepts (noun related to a superordinate, coordinate, subordinate etc.) are expressed in a linear sequence as in spoken languages and can be, but are not necessarily, iconic. Because of this difference between the languages in two modalities they are particularly interesting for the study of the underlying semantic concepts. At the focus of this thesis are especially the questions whether simultaneity and iconicity lead to differences in discourse and communication and as a consequence to different semantic relations between concepts. It is hypothesized that the preference for simultaneous structures in the visual-gestural modality lead to stronger relations between syntagmatically related concepts compared to paradigmatically related ones in the visual-gestural modality. Thus, the general objective of this thesis is to investigate the issue of ‘Modality Relativity’ in terms of the old controversy of ‘Linguistic Relativity’, i.e. whether language can affect thought. A series of experiments will be reported testing whether the two aspects of simultaneity and iconicity affect the semantic relations of concepts in any significant ways and thus, have an influence on the formation of conceptual knowledge. The first part of the thesis addresses the theoretical background. An overview of static and dynamic theories of concepts and categorization is given. The issue of the organization of concepts in the general cognitive system and the question whether concepts are acquired or innate is discussed. In the following the influence of language on perceptual and conceptual categorization will be examined. The Whorfian Linguistic Relativity Principle will be outlined and the basic arguments will be presented for discursive clarity. Subsequently, objections against Whorf’s ideas will be addressed and discussed in some detail. In the next paragraph the focus is on experimental studies about Linguistic Relativity within spoken language cultures and how new evidence changed the theoretical climate and discussion about Linguistic Relativity. The issue of Language Modality, i.e. the condition in which linguistic signs are produced and perceived, will be introduced. Commonalities and differences between German Sign Language (GSL) and German Spoken Language (GSpL) and some non-linguistic studies, which reveal some evidence for an impact of language modality on cognition, are discussed. Finally, in the fourth paragraph, the influence of simultaneity and iconicity of signed languages on conceptual and perceptual categorization is examined. The theoretical part of the thesis ends with the formulation of research questions relating to iconicity and simultaneity. In the empirical part of the thesis, several experiments will be presented, with the aim of contributing and broadening the present research on linguistic relativity. The experiments focus on cross-linguistic differences between German Sign Language (GSL) users, German Spoken Language (GSpL) users and cross- modal bilinguals (GSL and GSpL). Different language comprehension tasks, i.e. verification task, triad- comparison-task and memory-recognition task, are utilized, in order to study the influence of language modality on conceptual structures. Reaction times and Number of Choices, both in linguistic and non- linguistic tasks, will be used as dependent variables. The studies combine a comparison of structural diversities between sign language and spoken language related to language-modality with a demonstration of highly distinctive patterns of concept structure. Based on the results of the empirical studies, the last chapter finally will summarize and discuss the influence of simultaneity and iconicity on conceptualization processes. It will be argued that the studies do provide some evidence for the hypothesis that the modality of the language one uses might qualitatively alter the structure of concepts. The empirical research presented in this thesis aims at expanding the scope of inquiry and discussion on the linguistic relativity proposal by including the aspect of language modality. Zusammenfassung ‚Mediale Relativität‘? Der Einfluss von Gebärdensprache und Lautsprache auf konzeptuelle Kategorisierung In dieser Arbeit wird der Einfluss der Sprachmodalität auf konzeptuelle Kategorisierung untersucht. Dabei wurde angenommen, dass die beiden Sprachmodalitäten - Lautsprache und Gebärdensprache - einen Einfluss auf die Verarbeitung und Struktur von konzeptuellen Kategorien haben. Zwei Aspekte der Sprachmodalität, in denen gesprochene und gebärdete Sprache große Unterschiede aufweisen, sind Simultaneität und Ikonizität. Im Gegensatz zur auditiv-vokalen Modalität der Lautsprache finden wir in der visuell-gestischen Modalität der Gebärdensprachen eine wesentlich ausgeprägtere Präferenz für Simultaneität sowie eine größere Anzahl von ikonischen Gebärdenzeichen. Allerdings ist Simultaneität, d.h. die 'Verschmelzung' verschiedener konzeptueller Einheiten, nur für Konzepte möglich, die in einer syntagmatischen Beziehung zueinander stehen. Diese sogenannten polymorphematischen Gebärden sind immer auch ikonisch. Paradigmatisch verbundene Konzepte können in der Gebärdensprache – ähnlich wie in der Lautsprache – ausschließlich sequentiell artikuliert werden und können, müssen aber nicht notwendigerweise, ikonisch sein. Aufgrund dieser Unterschiede in den zwei genannten Sprachmodalitäten soll untersucht werden, ob Simultaneität und Ikonizität einen Einfluss auf die Stärke von Relationen im konzeptuellen System haben. Es wird angenommen, dass die Prädisposition und Präferenz für simultane Strukturen in der gestisch-visuellen Modalität insbesondere zu stärkeren Relationen zwischen Konzepten führen, die in syntagmatischer Relation zueinander stehen. Das Ziel dieser Dissertation besteht demzufolge darin, den Aspekt der ‚Medialen Relativität‘ angelehnt an die These der ‚Linguistischen Relativität‘, zu untersuchen und damit