The Psycholinguistics of Signed and Spoken Languages: How Biology Affects Processing Karen Emmorey
Total Page:16
File Type:pdf, Size:1020Kb
43-Gaskell-Chap43 3/11/07 10:53 AM Page 703 CHAPTER 43 The psycholinguistics of signed and spoken languages: how biology affects processing Karen Emmorey inguistic research over the last few decades consists of dynamic and constantly changing has revealed substantial similarities forms rather than static symbols. Further, nei- Lbetween the structure of signed and spo- ther sign language nor spoken language comes ken languages (for reviews see Emmorey, 2002; pre-segmented into words and sentences for the Sandler and Lillo-Martin, 2006). These similari- perceiver. The production of writing, although ties provide a strong basis for cross-modality performed by the hand, differs substantially comparisons, and also bring to light linguistic from sign language production because writing universals that hold for all human languages. In derives its structure from a separate system (the addition, however, biology-based distinctions orthography of a spoken language). In contrast between sign and speech are important, and can to written language, sign and speech are both be exploited to discover how the input–output primary language systems, acquired during systems of language impact online language pro- infancy and early childhood without formal cessing and affect the neurocognitive underpin- instruction. nings of language comprehension and production. For example, do the distinct perceptual and pro- ductive systems of signed and spoken languages 43.1 Sign perception and exert differing constraints on the nature of lin- guistic processing? Recent investigations have visual processing suggested that the modality in which a language Although non-signers may interpret the visual is expressed can impact the psychological mech- signed signal simply as a collection of rapid hand anisms required to decode and produce the lin- and arm motions, signers quickly extract com- guistic signal. This chapter explores what aspects plex meaning from the incoming visual signal. of language processing appear to be universal to Similarly, speakers extract meaning from a rap- all human languages and what aspects are idly changing acoustic stream, if they know the affected by the particular characteristics of audi- language. Listeners and viewers are able to auto- tion vs. vision, or by the differing constraints on matically parse an incoming auditory or visual manual versus oral articulation. linguistic signal by virtue of stored internal rep- Sign language processing is appropriately resentations. Speech perception involves seg- compared to speech processing, rather than to mentation of speech sounds into phonemic units. reading, because unlike written text, which can be For signed languages, a first question is whether characterized as “visual language,”sign language signs actually exhibit sublexical linguistic structure 43-Gaskell-Chap43 3/11/07 10:53 AM Page 704 704 · CHAPTER 43 The psycholinguistics of signed and spoken languages that could be used by a parser to segment visual (Brentari, 1998; Corina and Sandler, 1993; Wilbur, signed input. Is it possible to have a phonology 1993). The syllable is a unit of structure that is that is not based on sound? below the level of the word but above the level of the segment, and is required to explain phono- 43.1.1 Phonology in a language logical form and patterning within a word. Although sign phonologists disagree about pre- without sound cisely how sign syllables should be characterized, Several decades of linguistic research has shown there is general agreement that a sign syllable that signed languages, like spoken languages, have must contain a movement of some type. In ASL, a level of structure in which meaningless elements several phonological constraints have been are combined in rule-governed ways to create identified that must refer to the syllable. For meaningful forms (e.g. Stokoe, 1960; Battison, example, only certain movement sequences are 1978; Sandler, 1986; Brentari, 1998). For spoken allowed in bisyllabic (two-movement) signs: languages, these elements are oral gestures that circle + straight movements are permitted, but create sounds. For signed languages, manual and straight + circle movements are not (Uyechi, facial gestural units are combined to create 1994). Although a straight + circle movement distinct signs. The discovery that sign languages sequence is ill-formed as a single sign, it is well- exhibit phonological structure was ground- formed when it occurs in a phrase. Thus, the con- breaking because it demonstrated that signs are straint on movement sequences needs to refer to not holistic pantomimes lacking internal organ- a level smaller than the word (the constraint does ization. Furthermore, this discovery showed that not hold across word boundaries), but larger human languages universally develop a level of than the segment. Within Sandler’s (1986) Hand meaningless linguistic structure and a system Tier model, signed segments consist of Movements that organizes this structure. and Locations, somewhat akin to Vowels and Briefly, signs are composed of three basic Consonants. phonological parameters: hand configuration, However, syllables in signed language differ location (place of articulation), and movement. from syllables in spoken language because there Orientation of the hand/arm is another con- is little evidence for internal structure within the trasting parameter, but many theories represent signed syllable. Spoken syllables can be divided orientation as a sub-component of hand config- into onsets (usually, the first consonant or con- uration or movement, rather than as a basic sonant cluster) and rhymes (the vowel and final phonological element. Figure 43.1 provides consonants). Such internal structure does not illustrations of minimal pairs from LIS (Lingua appear to be present for sign syllables, although Italiana dei Segni, Italian Sign Language). The some linguists have argued for weight distinc- top part of the figure illustrates two LIS signs tions, i.e. “heavy” vs. “light” syllables, based on that differ only in hand configuration. Not all differences in movement types. Because of the sign languages share the same hand configura- lack of internal structure, there do not appear to tion inventory. For example, the “t” hand config- be processes such as resyllabification in sign lan- uration in American Sign Language (the thumb guages (e.g. a segment from one syllable becomes is inserted between the index and middle fingers part of another syllable). These facts are impor- of a fist) is not found in European sign languages. tant, given the emphasis that speech production Chinese Sign Language contains a hand config- models place on syllabification as a separate uration formed with an open hand with all fin- processing stage (e.g. Levelt et al., 1999). gers extended except for the ring finger, which is Syllabification processes and/or the use of bent—this hand configuration does not occur a syllabary may be specific to phonological in American Sign Language (ASL). In addition, encoding for speech production. The syllable signs can differ according to where they are likely serves as an organizational unit for speech, made on the body or face. Figure 43.1B illus- providing a structural frame for multisegmental trates two LIS signs that differ only their place of words. For example, MacNeilage (1998) argues articulation, and these different locations do not that the oscillation of the mandible creates a add meaning to the signs. Signs can also differ frame around which syllable production can be minimally in orientation, as illustrated in Figure organized. Meier (2000; 2002) points out that 43.1C. Finally, movement is another contrasting signing differs dramatically from speaking category that distinguishes minimally between because signing does not involve a single, pre- signs, as shown in Figure 43.1D. dominant oscillator, akin to the mandible. Rather, In addition to segment-like units, syllables have signs can have movement that is restricted to just also been argued to exist in signed languages about any joint of the arm. Thus, sign production 43-Gaskell-Chap43 3/11/07 10:53 AM Page 705 Sign perception and visual processing · 705 (a) BICICLETTA (“bicycle”) CAMBIARE (“change”) (b) AFFITTO (“rent”) STIPENDIO (“salary”) (c) PAPA (“father”) UOMO (“man”) (d) FORMAGGIO (“cheese”) SASSO (“stone”) Figure 43.1 Examples of minimal pairs in Lingua Italiana dei Signi, LIS (Italian Sign Language) (A) signs that contrast in hand configuration; (B) signs that contrast in place of articulation (location); (C) signs that contrast in orientation; (D) signs that contrast in movement. Illustrations from V. Volterra (ed.), La lingua italiana dei segni. Bologna: Il Mulino, 1987 (new edn 2004) Copyright © Virginia Volterra and Elena Radutzky. Reprinted with permission. 43-Gaskell-Chap43 3/11/07 10:53 AM Page 706 706 · CHAPTER 43 The psycholinguistics of signed and spoken languages may not be constrained to fit within a frame place of articulation (Emmorey et al., 2003; Baker imposed by a single articulator. Further, multi- et al., 2005). This result is consistent with previ- segmental signs (e.g. signs with more than three ous studies which found that deaf and hearing segments) are relatively rare, regardless of how participants exhibit similar perceptual groupings signed segments are defined (see Brentari, 1998). and confusability matrices for hand configura- In contrast, syllabification processes for speech tion and for place of articulation (Lane et al., production may serve a critical framing