Part II Order in Peirce's Sign Relations and Derived Categories
Part II
Order in Peirce’s Sign Relations and Derived Categories
214 215
Chapter 4
Sign Relations and Categories
D’una citt`a non godi le sette o le settantasette meraviglie, ma la riposte che d`a a una tua domanda. Italo Calving, Le Citt´a Invisibility
I have neither desires nor fears, the Khan declared, and my dreams are composed either by my mind or by chance. Cities also believe they are the work of the mind or of chance, but neither the one nor the other suffices to hold up their walls. You take delight not in a city’s seven or seventy wonders, but in the answer it gives a question of yours. Or the question it asks you, forcing you to answer, like Themes through the mouth of the Sphinx. Italo Calving, Invisible Cities 1
1Italo Calving, Invisible Cities, Harvest Books, William Weaver (Translator), 1986, added italics 216
4.1 HCI and Communication
The relatively recent awareness for a mutual influence between Informatics and com- munications has attracted the attention of some people across HCI and Informatics to disciplines that study language, communication, and interaction. In the remainder of this dissertation I increasingly narrow down the thesis’ scope to such a niche. Across the HCI literature, several authors have already stressed the importance of com- munication in Informatics and in HCI. I quote some authors who expressed related thoughts in Table 4.1. However it should not be forgotten that linguistics played a key role both in the introduction of computer languages and in its formalizations. The presence of a broad understanding of communication in Informatics slowly drifted from a key role in the sixties to the periphery of the discipline. It has remained in its outskirts until recently. During the 1960s, the focus on syntax given in courses such as programming languages and compiler construction were complemented by topics with broader foci on languages and cognition in information retrieval and artificial intelligence. With the development of Informatics, once programming languages and compilers acquired a certain autonomy, the semantic and the pragmatic aspects of language were slowly abandoned by the involved community. The trajectory of phenomena related to communications, language, and interaction has followed the same pattern discussed here in Informatics and HCI. See Table 4.2 for a brief trajectory of language and communication related topics across the ACM curricula recommendations. Noam Chomsky’s contribution to the formalization of grammars has been key to the theoretical understanding of computers since the sixties. In 1956 he was working at the MIT Department of Modern Languages and at the Research Laboratory of Electronics. I have chosen Noam Chomsky’s work to discuss how narrow are the concepts of languages in Informatics for two reasons. Firstly, Chomsky’s theoretical scaffold has been partially discarded, and only a slice of his work in known in com- puter science. Secondly, in language studies, his whole work in linguistics has even 217
1993 Tom Dayton et al. Skills Needed by User-Centered Design Practitioners in Real Software Development Environments (Dayton et al., 1993, p 17) [S]ocial, organizational, and communication skills are at least important as formal, technical knowl- edge, and are even necessary to ensure that the formal, technical knowledge is used effectively. 1993 Judith S. Olson et al. Computer Supported Cooperative Work: Research issues for the 90s (Olson et al., 1993, p 93) Research in CSCW contains everything in HCI plus: There is a need for theories and models of the users. With group work, theories must additionally encompass the conversations among the participants, the roles they adopt, and the organizational setting which guides many group work actions implicitly, and the cultural practices. 1994 John Seely Brown and Paul Duguid Borderline Issues: Social and Material Aspects of Design (Brown and Duguid, 1994, p 5) We take design to be fundamentally a communicative process. 1994 Terry Winograd Designing the Designer (Winograd, 1994) Brown and Duguid’s Article [Brown and Duguid (1994)] reflects a growing awareness that HCI has a social and a communicative dimension, which is equally central to the field as the psychological and cognitive issues that have long been at its core. 1994 Gary Strong et al. New Directions in Human Computer Interaction (Strong et al., 1994, pp 26) HCI research should probably confront human-human communication as the first order approximation of information transfer and study that. As development then works to make HCI more seamless with human-human interaction, success will be assured because interaction will then be transparent in the hands of the user, allowing the user to focus complete attention on the tasks at hand. 1996 Jean B. Gasen HCI Education: Past, Present, and Future? (Gasen, 1996, p 25-26) The emphasis on the social contexts of computing will become more important as everyone considers ways in which technology is metamorphosing communication at a global level. 1996 Brad Myers et al. Strategic Directions in Human-Computer Interaction (Myers et al., 1996) Human-computer interaction (HCI) is the study of how people design, implement, and use interactive systems and how computers affect individuals, organizations, and society. This encompasses not only ease of use but also new interaction techniques for supporting user tasks, providing better access to information, and creating more powerful forms of communication.
Table 4.1: HCI and the study of communication 218
The historical relation between computing and communication can be traced across the several ACM Curricula recommendations.
Briefly, in Curriculum 68 there were proposed courses on: (a) Formal Languages and Syntactic Analysis (b) Information Organization and Retrieval in which semiformal languages and models for the representation of structured information, as well as several methods for natural language analysis and evaluation were presented. (c) Computer Graphics, which were aimed at the study of handling graphic information in computers and its applications (d) Artificial Intelligence and Heuristic Programming, which introduced students to non-arithmetic applications of computing machines and issues related to cognition and intelligence. There were several other courses in common with engineering, and Large Scale Information Processing Systems certainly intersected with Information Systems.
In Curriculum’78, natural language processing and non-arithmetic issues were diluted into a topic of Artificial Intelligence, large scale information systems were narrowed to file-processing and database management, computer graphics disappeared, and an advanced course on computers and society was proposed, but seldom implemented. Formal languages remained in an elective course on Automata, Computability, and Formal Languages.
In Curricula 1991, formal languages topics were listed within the areas of Algorithms and Program- ming Languages. Human-Computer Communication, basically interfaces and computer graphics, was introduced. Restricted topics in information retrieval, natural language, and social and ethical issues were listed as sub-topics of Database and Information Retrieval, and Artificial Intelligence. The social and professional context was included but not clearly emphasized.
In Curricula 2001, draft version of March 2000, formal languages are an item in automata theory within the area of Algorithms and complexity. Human-Computer Interaction and Computer graphics will be included; natural language processing is a topic of Intelligent Systems; Collaboration and groupware technology is a topic of net-centric Computing; Social and Professional Issues are clearly emphasized; and Databases turns into Information Management.
In 2001 the committee subdivided the curricula in five strands, emphasizing the relations among four branches of Informatics: Computer Engineering, Software Engineering, Computer Science, and Information Systems.
Table 4.2: Communication in computer science curricula. See Chapter 1 for detailed references. 219 been criticized for being excessively narrow. Within the framework introduced here, Chomsky’s formalization of syntactical structures are quasi-independent linguistic strata (levels). Indeed, as the first sentences of “Syntactic Structures”, Chomsky wrote:
1957 Noam Chomsky Syntactic Structures (Chomsky, 1957, Preface) This study deals with syntactic structure both in the broad sense (as opposed to semantics) and the narrow sense (as opposed to phonemics and morphology). It forms part of an attempt to construct a formalized general theory of linguistic structure and to explore the foundations of such a theory.
In that book, Chomsky dedicated a specific chapter to syntax and semantics. In it, he clearly discusses the relation of the syntactical level of language to the phonologic, morphologic and semantic ones. Chomsky clearly stated that only a broader theory of language, one that included the theory of linguistic form and theory of linguistic use, would be capable of addressing the correspondence between formal and semantic traces (Chomsky, 1957, 9.3). Indeed, Chomsky also stated that to understand a sentence, knowledge about each linguistic level was not enough, because notions such as sense and reference, which belonged to the domain of semantics, were necessary (Chomsky, 1980, section 9.4, p 113) and Chomsky (1957). The assumed independence of Chomsky’s levels facilitated the segmentation of language studies into several strata and its appropriation by Informatics. For exam- ple, computer scientists appropriated the notion of syntax, and with some exceptions, discarded all the other parts of language studies necessary to understand effective communication.2 In Figure 4.1 I have briefly sketched a conceptual diagram in which the concepts used by several authors in the theory of computing, logic, linguistics, language studies,
2Behind this brief comment on linguistics and computer science resides my deep criticisms about the boundaries that have been set in Informatics in what concerns language understanding in a broad sense. 220 and semiotics are plotted. The diagram is only illustrative, because the ambiguity, and the uses and misuses of terms such as syntactics, semantics, and pragmatics, does not enable a direct comparison between them and across authors. 3
signs and people objects actions
Computing Finite- Push- Linear Turing state down bounded machines automata automata automata Wegner plus Interactive machines Frege sinn (sense) bedeutung (reference) Chomsky competence performance syntactics semantics syntax and semantic phonemics relations (3) (2) (1) (0) and regular context context recursively morphology free sentitive enumerable Foley lexical syntactic semantic conceptual conversation design design design (user model) protocol Norman semantic distance articulatory distance Gibson’s affordances Peirce firstness secondness thirdness Saussure langue (language) parole (speech) subject object Morris syntactics semantics pragmatics Austin stating performative locution (state of mind)) illocution (promise) perlocution (attempt)
internal relations external relations
Figure 4.1: Examples of language scales and hierarchies
However, it is possible to note that despite the differences and contradictions,4 the
3The conflation of signs, people, and computers to single columns intend to facilitate this brief comparison among different scopes. It flattens out their differences. For example, Figure 4.1 depicts people’s cognitive systems in the same columns as computing processing architectures, a decision with which only some approaches in cognitive science would agree. Independence or connected- ness between columns is also not depicted. More dimensions are needed to distinguish different approaches, and that is what is proposed in this thesis with 3D models. 4It is not uncommon to use the term semantics of computer languages to denote the objects inside the computer itself. This use is closer to what Chomsky called the phonemic and morphological level, because they imply the substrate languages are expressed in. The separation between hardware and software probably reinforces this perspective. 221 realm of computing has the most restricted understanding of language of all. Within the theoretical framework adopted in informatics, (see Figure 4.1’s dark gray dashed rectangle) there is no space for functionality (Foley’s level of semantic design), for usage (Chomsky’s distinction of competence and performance and Austin’s concepts of illocutionary and perlocutionary acts), for actual interaction (Norman’s articu- latory distance and Gibson’s concept of affordance), and for the usual meanings of semantics and pragmatics in linguistics, semiotics, and language studies in general. Interactive machines do not represent the standard view in Informatics, but rep- resent a example of a model that opens abstract machines to interact with the world.5 The historical consolidation of Informatics diverged in terms of communication. It is undeniable that the influence of Informatics on communications has increased. It is also undeniable the great depth that theoretical models of machines and grammars have achieved, and the widespread uses of computer languages. However, in terms of breadth, Informatics falls short in terms of theoretical frameworks of language, communication, and interaction in the broad sense (e.g. semantics, performance, and pragmatics). Paul Adler and Terry Winograd, for example, addressed the inseparability of com- puter based systems design from communication in the broad sense (as encompassing semantics and pragmatics). Adler and Winograd wrote:
1992 Paul S. Adler and Terry Winograd The Usability Challenge (Adler and Winograd, 1992, p 3) To break free with the prevalent myths and go beyond current practice, we must articulate new criteria that are appropriate to the tasks of modern computer-based system design and the interwoven tasks of work design. [. . .]
5Models such as the Universal Turing machine, widely used in computer science, and the basis of algorithms, do not interact with the world. Peter Wegner, the same author who wrote about the three cultures of computing in the early seventies, has recently proposed extending the concept of abstract machine by adding a notion of interaction. Wegner called such an extended model interaction machines. See Wegner (1995, 1997, 1998); Wegner and Goldin (1999) and Chapter 3. Indeed, computer scientists have been reductionist by calling Turing Machines universal. 222
Usability [. . .] assumes a communications dimension. The technology itself, even when it is not intended as a communication product, serves as a communication medium between user and user and between designer and user. A realistic characterization of work – even routine work – is that it is essentially entwined with communicative actions generated to deal with the novel situations that continually arise and with the need to interpret the intentions embodied in the machines.
In the second part of this thesis I actually address design in informatics as com- munication. In Chapter 3 I address various models of communication, interaction, organizational change, and design processes in the light of the cultural ecology of HCI, as discussed until now. I use these models to discuss, for example, the limita- tions of Norman’s gulfs of execution and evaluation in terms of pragmatics, and of Moore’s “chasm” model of technology acceptance. Some of these models make use of terms such as semantics and pragmatics, but there is no consistency across the different uses given to them. In this Chapter I explore the theoretical framework that later contributed to the consolidation of the terms syntactics, semantics, and pragmatics, showing that the confusions and contradictions are not a peculiarity of HCI and Informatics. Indeed, they have been a constant in semiotics since its inception in the nineteenth century. In the fourth chapter, I make extensive use of frameworks developed in engineering and Informatics to systematize Charles Sanders Peirce’s several concepts of sign and associated categories.
4.2 Sign Relations
It is undeniable that literacy and numeracy have both been present to varying degrees throughout the development of the intricate cultural matrix that structure some current cultures, in particular the dominant ones. The cultural disciplinary ecology of Semiotics is probably as intricate as the one of Informatics, and in my understanding they are inseparable. However, the communities that study informatics and semiotics have diverged as if these two realms did not intersect. 223
The broad perspective envisioned in the previous chapters has been important to situate and characterize informatics in relation to itself, and to other disciplines. My discussion of informatics, and computer science in particular, suggests it has been restricted to syntactic views, rather than broader ones. I started this thesis with an eye on the broader bridge between semiotics and com- puter science. At several points I have pointed to possible links between informatics and semiotics. In any work the goal, the horizon, the whole context are necessary, but not sufficient. I will conclude this thesis with a much narrower topic in semiotics, as I mentioned before. In particular, I address a small topic in semiotics with the aid of frameworks developed in the computing sciences and engineering. The focus is on partial orders in Peirce’s sign relations and associated categories. I do not address in detail what a sign is, what its components stand for, or what practices can be enriched with semiotics. Therefore, it does not encompass even the sign relation in its full breadth. The sign relation, in its many possible definitions, has been the key element across the cultural ecology of semiotics. For example, dyadic, triadic, and cultural mediations have been deeply entwined within the developed identities of Semiology, Semiotics, and Cultural Historical Activity Theory. I grouped in Figure 4.2 some diagrams developed in three semiotics schools in order to illustrate the diversity that permeates the field and the associated frameworks that structure the respective sign relations. The lines of Figure 4.2 are organized according to the legacy of these three schools, which sustained and further developed the work of Ferdinand de Saussure (1857–1913), Charles Sanders Santiago Peirce (1839-1914), and Lev Semenovich Vygotsky (1896-1934), respectively. Figure 4.2(a) illustrates Saussure’s concept of sign re-read by (Thibault, 1997, p 182) in the light of a social perspective. As I have discussed in the preceding chapter, structuralism differentiated structure from use in order to discard use. It discarded the individual from the subject of linguistics. Paul Thibault rescued the speaker-listener, giving a triadic structure to Saussure’s relation between sound and thought (Thibault, 1997, p 182). Figure 4.2(b) illustrates a diagram developed by Louis Hjelmslev Hjelmslev 224
(1975), who stratified Saussure’s model. In this stratification, both expression and content are correlated with a scale that starts on purport (amorphous matter), passes through substance, and reaches form (N¨oth, 1995, pp 64-73). Indeed, the main contribution of this chapter is a systematization of similar re- sources used by Peirce to elaborate his several structures of sign relations and their corresponding categories. As illustrated in Figure 4.2(c), Peirce’s most known sign relation6 is triadic and it is composed of representamen, object and interpretant. In Peirce’s own words:
A Representamen is a subject of a triadic relation to a second, called its Object, for a third, called its Interpretant, this triadic relation being such that the Rep- resentamen determines the interpretant to stand in some triadic relation to the same object for some interpretant (Peirce, 1931-1935, 1.541).
In 1904, Peirce defined the cenopythagorean categories of Firstness, Secondness, and Thirdness in respect to their mode of being (Peirce, 1958, 8.328) in a letter to Lady Welby as:
Firstness is the mode of being of that is such as it is, positively and without reference to anything else. Secondness is the mode of being of that is such as it is, in bringing a second but regardless of any third. Thirdness is the mode of being of that which is such as it is, in bringing a second and a third into relation to each other. I call these three ideas the cenopythagorean categories (Peirce, 1958, 8.238).
Peirce correlated the triadic sign relation (R,O,I) with the cenopythagorean cat- egories (I,II,III) to derive classes of signs. He also explored hexadic and decadic sign relations, which I explore later in the final sections of this chapter. Before I focus on the derivation of classes of signs and the several graphic formalisms proposed to visualize it, I proceed with a contextualization of the sign within semiotics, within Peirce’s systematic philosophy, and within computer semiotics. Figure 4.2(d) illustrates one version of the widely disseminated semiotic triangle introduced by Charles Kay Ogden and Ivor Armstrong Richards. As N¨oth (1995, p
6See Marty (1997) for seventy-six related definitions of the sign found across Peirce’s work. 225
(a) Saussure succession diachronic (b) Hjelmslev Content-purport semiotically amorphous matter langue speaker-listener semiotically formed
synchronic (interpreter) pure content-substance simultaneity value S content-form S regular I I forms G G signifier signified N expression-form N ‘sound’ significant signifié ‘thought’ typical re-constructed re-constructed patterns semiotically formed as phonic substance as thought-substance expression-substance semiotically amorphous matter of masse tree expression-purport parlante
(c) Peirce (d) Ogden &