Psychological Review Copyright 1985 by Ihe American Psychological Association, Inc. 1985, Vol. 92, No. 3, 350-371 0033-295X/85/S00.75 So You Think Gestures Are Nonverbal? David McNeiil University of Chicago In this article I argue that gestures and speech are parts of the same psychological structure and share a computational stage. The argument is based on the very close temporal, semantic, pragmatic, pathological, and developmental parallels between speech and referential and discourse-oriented gestures. Most of the article consists of a description of these parallels. A concept that unites outer speech and gesture is the hypothesis of inner speech. Many cognitive psychologists hold that pany acts of speaking. In such gestures the overt acts of linguistic production are the hands function as symbols that are closely result of internal "computations." My aim in connected to the speech channel in terms of this article is to make the following point both time and semantic and pragmatic func- concerning gestures: Gestures share with tion. In the idiom of my title, such gestures speech a computational stage; they are, ac- are verbal. They are the overt products of the cordingly, parts of the same psychological same internal processes that produce the structure. The metaphor of a shared com- other overt product, speech. putational stage captures the processing as- Because there are such close connections pects of speech: that sentences and gestures between gesture and overt speech, gestures develop internally together as psychological offer themselves as a second channel of ob- performances. The metaphor of a common servation of the psychological activities that psychological structure captures the idea that take place during speech production—the speech and gesture respond to the same forces first channel being overt speech itself. The at the same times. channels of gesture and speech are close, yet Taking into account concurrent gestures different. Combining a spoken sentence and suggests that in the dynamic situation under- its concurrent gesture into a single observation lying sentence generation two opposite kinds gives two simultaneous views of the same of thinking, imagistic and syntactic, are co- process, an effect comparable to triangulation ordinated. The types of gestures that provide in vision. In this article I consider some of this insight are the referential and discourse- the implications that can be drawn from the oriented gestures that spontaneously accom- systematic comparison of speech and gesture. The statement that gestures and speech are This article was written while I was a fellow at the parts of the same psychological structure is Netherlands Institute for Advanced Study, Wassenaar, contrary to the idea of body language, that The Netherlands. Financial support for my research was is, a separate system of body movement and provided by the National Science Foundation (linguistics postural signals that is thought to obey its section), the Spencer Foundation, and the University of own laws and convey its own typically affective Chicago. The gesture research was carried out with the help of the sharp eyes of Catherine Greeno, Mitsuko and unconscious meanings. It is also contrary Iriye, Elena Levy, Laura Fedelty, and Debra Stephens. to the assumptions of many linguistic analyses I have greatly benefited from comments at various that hold that language structures should be times and in various forms from friends and colleagues, analyzed only in terms of speech sounds plus among them E. Bates, J. Bybee, J. Cosnier, G. B. Flores d'Arcais, S. Frey, M. Johnson, D. Kelter, A. Kendon, J. grammar. We tend to consider linguistic what Lawler, W. J. M. Levelt, L. Mackenzie, N. B. McNeiil, we can write down, and nonlinguistic, every- N. Quinn, M. Silverstein, and W. Washabaugh. The title thing else; but this division is a cultural owes something to the late Glenn Gould. artifact, an arbitrary limitation derived from Requests for reprints should be sent to David McNeiil, a particular historical evolution. Both body Committee on Cognition and Communication, Depart- ment of Behavioral Sciences, University of Chicago, 5848 language and customary linguistic analyses South University Avenue, Chicago, Illinois 60637. follow a narrow approach, in which connected 350 SO YOU THINK GESTURES ARE NONVERBAL? 351 parts of a single psychological structure are is described here. Emblems are gestures that studied separately. Both begin by radically have a specific social code of their own, have separating gestures from speech, and from conventional paraphrases or names (Kendon, this starting point they go in opposite direc- 1983), are learned as separate symbols, and tions. That gesture and speech are parts of a can be used if they were spoken words; in single psychological structure has been a view many uses they are, in fact, unspoken words. held, however, by a number of psychologists An example is the okay sign or the psycho- and psychiatrists (Argyle, 1975; Barroso, logically peculiar sign (the first finger pointing Freedman, Grand, & van Meel (1978); Con- at the temple while the hand moves in a don & Ogston, 1971; Cosnier, 1982; Ekman small circle). These kinds of gesture are in- & Friesen, 1969;Kendon, 1972, 1980, 1983— terpretable in the absence of speech, and this to mention a few), but not by linguists (for a is one of their chief functions. The types of brief but quite definite opinion, see Chomsky, gesture described in this article, in contrast, in Rieber, 1983). In this article 1 present new are not interpretable in the absence of speech, evidence that supports the broader view, that are individual and spontaneous, are outside the whole of gesture and speech can be of any special social code that regulates them, encompassed in a unified conception, with and show interesting cross-cultural similarities gesture as part of the psychology of speaking, (again, in contrast to emblems, which, as along with, and not fundamentally different Morris et al. show, are highly specific to from, speech itself. particular linguistic and national groups and Acts of speaking are often accompanied in even subgroups). our culture by movements of the arms and In this article I first explain and present hands that are termed gestures (Kendon, arguments for the statement that gestures are 1972, uses the term gesticulation). These are manual symbols. Second, I exemplify the movements that (with a class of exceptions similarities that point to gesture and speech to be described) occur only during speech, sharing a computational stage. Third, I con- are synchronized with linguistic units, are sider and reply to counterarguments; and last, parallel in semantic and pragmatic function I present conclusions regarding the psycho- to the synchronized linguistic units, perform logical process of speech production based text functions like speech, dissolve like speech on considerations of gesture and speech data. in aphasia, and develop together with speech in children. Because of these similarities, a Gestures Are Manual Symbols strong case can be made for regarding gestures and speech as parts of a common psycholog- To demonstrate the symbolic character of ical structure. gestures, I will present examples of gestures The similarities hold for the kinds of spon- produced by 5 adult female subjects who taneous and semiconscious gesture that can were narrating the same event from a cartoon be seen accompanying much conversational story (they had been shown the cartoon just and narrative speech. Such gestures play pic- previously and were now telling the story to torial and discourse-related roles (Efron, 1941) a listener—each subject separately). The point and have been variously called "pictograms," I wish to make is that, like conventional "ideograms," "illustrators," and "batons" liguistic symbols, there is great commonality (Ekman & Friesen, 1969). among subjects. Despite individual variation There is another better known, more ste- in the detailed manner of performance, every reotypic type of gesture that should not be gesture includes upward movement. This confused with these. The French and Italians commonality is produced, not by a shared are acknowledged as master gesticulators, but code or language that exists specifically for the Italianate gesture (a type called the em- gestures, but by each speaker separately cre- blem by Ekman & Friesen, 1969, and docu- ating her own manual symbol of upward mented throughout Europe by Morris, Collett, movement. The concurrent sentences also Marsh, & O'Shaughnessy, 1979—see Ken- conveyed the idea of upward movement, and don's, 1981, review of the latter) is not the this fact shows that semantic parallels existed referential and discourse-oriented gesture that between gestures and concurrent speech. As 352 DAVID McNEILL they performed their gestures the subjects cluding their intended meanings as well as were saying1 their memories. Whether these concepts are categories, propositions, prototypical exam- he tries going up inside the drainpipe ples, contextualized individual ideas, or something else, is left unresolved, and undis- ("hand rises and"! cussed. [points upwardj The gestures above are symbols in Saus- he tries climbing up the drainspout of the sure's (1916/1959) sense. Even though they building are not conventional or arbitrary, these ges- tures are analyzable as paired signifiers and fhand rises "I signifieds. The physical upward movement of |_and starts to point upwardj the gesture itself is the signifier,