Facial Expressions and Vocalizations of Rhesus Macaques (Macaca Mulatta) Author(S): Sarah R

Single and Multichannel Signal Composition: Facial Expressions and Vocalizations of Rhesus Macaques (Macaca mulatta) Author(s): Sarah R. Partan Reviewed work(s): Source: Behaviour, Vol. 139, No. 8 (Aug., 2002), pp. 993-1027 Published by: BRILL Stable URL: http://www.jstor.org/stable/4535968 . Accessed: 28/07/2012 15:18

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at . http://www.jstor.org/page/info/about/policies/terms.jsp

. JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected].

BRILL is collaborating with JSTOR to digitize, preserve and extend access to Behaviour.

http://www.jstor.org SINGLE AND MULTICHANNEL SIGNAL COMPOSITION: FACIAL EXPRESSIONS AND VOCALIZATIONS OF RHESUS MACAQUES (MACACA MULATTA)

SARAH R. PARTAN12) (Animal BehaviorGroup, University of California,Davis)

(Acc. 7-IV-2002)

Summary A methodfor simultaneouslyexamining visual and vocal componentsof expressivebehavior is described, compiled from video recordings of social behavior of a free-ranginggroup of rhesus macaques on Cayo Santiago, Puerto Rico. I developed a catalog of expressive movements, and chronicled detailed informationon visual and vocal components of 1215 individual behaviors. Two thirds of the events recorded were silent, supportingthe idea

1)Department of Psychology, Universityof SouthFlorida St. Petersburg,1400 South Seventh Avenue, St. Petersburg,FL 33701, USA. e-mail: [email protected] 2) This work is based on my dissertationresearch in Peter Marler'slaboratory. I would like to give my heartfelt thanks to Dr. Marler for his encouragement,support, and advice, as well as for his comments on several drafts of the manuscript.I am gratefulto Marc Hauser for encouraging me to work on rhesus monkeys, and to William Mason for helping me to understandrhesus behavior during pilot observationsat the CaliforniaRegional Primate Research Center.This project was developed during discussions with Drs. Marler,Mason, Hauser,Christopher Evans and Joseph Macedonia.I would like to thank Charles Snowdon, CorreighGreene, Jill Soha, Katya Partan,and two anonymousreviewers for comments on the currentmanuscript; Dr. Mason, John Endler,and Arlene Alvaradofor commentson prior versions; and J.A.R.A.M. van Hooff for a suggestion on terminology.Neal Willits and Tim Allis provided key statistical advice; Renee Allen and Marc Fourriergave indispensable assistance in the field, and Jeannine Logan, Rebecca Wylie, and Virginia Price in the laboratory;Bill Patrickand Todd Hughes helped produce Figure 4. Finally, I would like to thank John Berard,Matt Kessler, the University of PuertoRico, and the CaribbeanPrimate ResearchCenter for providing access to the field site and the genealogical data, and Edgar Davila for introducingme to the monkeys.This study was fundedby grantsfrom the National Science Foundation,the L.S.B. Leakey Foundation,the Animal BehaviorSociety, and Sigma xi. ( KoninklijkeBrill NV, Leiden, 2002 Behaviour139, 993-1027 Also availableonline - 994 PARTAN that visual behaviors are primary for short distance communication in these macaques. Clustersof expressive components detected by PrincipalComponent Analysis and Multiple CorrespondenceAnalyses correspondedto threatening,submissive, and affiliativebehaviors describedpreviously, providing quantitative support both for these previous descriptionsand for the suggestion that these threepoles of behaviorare importantin daily social interaction. Silent expressions involved a greater variety of mouth positions than did vocalizations, which were produced with stereotyped mouth shapes. Other components of the face, not involved with articulation,were nonethelessassociated with particularvocalizations: specific associationswere found among barks,ears retracted,and head lowered on the one hand, and pant-threats,ears forward, and head raised on the other. Screams and squeaks were highly stereotyped,combined with prototypicalgrimace mouthpositions, crouchingand retreating. Girney vocalizations were accompanied by lipsmacking. Grunts were unaccompaniedby other expressivecomponents, evoking the suggestion thatthey may be predominantlyneutral in valence.

Introduction Although communicationinvolves the use of multiple sensory channels, most researchersfocus on one sensory channel at a time, in isolation from the full repertoireof the animal. The combination of channels, however, can have importantramifications for signal meaning and efficacy (Marler, 1965; Partan& Marler, 1999; Rowe, 1999). Human facial expression and visual articulatorymovements, for example,play a role in speech perception (McGurk & MacDonald, 1976; Massaro, 1998). In birds, odor cues from prey can interact with visual stimuli of particularcolors to produce food aversionsthat do not occur without the odor (Rowe & Guilford, 1996), and odor plays an importantrole in combinationwith other cues during sexual behaviorin baboons (Goldfoot, 1982). Signal componentsin multichannel displays can be redundant(e.g. Conner,1987), or each componentmay play an independentrole (e.g. Fusani et al., 1997). It is difficult, however, to determinethe functionalrole of each componentof multicomponentsignals, as Green (1975, p. 87) mentioned for Japanese macaques: "...their vocal behavior is inextricably tied to simultaneous olfactory, tactile, and visual signals, hence considerationsolely of evoked responses cannot disentangle the roles of the concurrentsignals available by different sensory moda- lities .. Before parsingthe role of each componentin communication,one needs to determinewhich particularcomponents are combinedsimultaneously into multimodalsignals duringsignal production.Although researchers studying SIGNAL COMPOSITION 995 the receptiveside of communicationhave developed sophisticatedmethods for measuringthe perceptionof multichannelsignals (e.g. visual influences on human speech perception,Massaro, 1998), there are to date few established methodsfor quantifyingnatural multichannel signal production. This study describes how naturalvisual signals of facial expression and body postureare associatedwith vocalizationsof rhesus macaques(Macaca mulatta),bridging the separateanalyses of visual and vocal signals of this species (e.g. primarilyvisual analysesby Hinde & Rowell, 1962; van Hooff, 1962; Maxim, 1982, 1985; Zeller, 1986, 1996; Maestripieri& Wallen, 1997; and primarily vocal analyses by Rowell, 1962; Gouzoules et al., 1984; Hauser, 1991, 1996; Hauser& Marler, 1993). Although some authorshave described visual and vocal signals together (e.g. Altmann, 1962; Rowell & Hinde, 1962; Lindberg, 1971; Mason, 1985; Kalin et al., 1992), none have reportedthe frequencieswith which particularvisual componentsare associated with particularvocal components.This is an importantfirst step towards an understandingof the multimodalnature of communication.A similar approachhas been taken by Adams & Schoel (1982) in a study of stumptailmacaque (Macaca arctoides) communication:vocalizations were analyzed separatelybut in parallelwith the simultaneousfacial and postural componentsof behavior.My work also builds on the work of Hauseret al. (1993), who suggestedthat each class of rhesusvocalization is accompanied by an unique articulatorygesture. Here I not only examine mouth position, but also include facial expressionsand body posturesuninvolved with actual phonationthat nonethelesspredictably accompany each vocalization. One goal is to explorethe variabilityin facial expressivecomponents used during communication.I constructeda catalog of expressive movements of the face and head, grouping behaviors into 'morphological'categories (classifiedby body part:eye, ear,head, etc.; cf. Reynolds, 1976). Withineach category,I definedlogical, mutuallyexclusive states for each body part(e.g. the ear categoryincludes earsforward, retracted,orflapped back andforth). This categorizationscheme is more detailed than previous rhesus catalogs with regard to facial movements (see Zeller, 1986, 1996, for a detailed look at the face of other Macaca species) but may not be as detailed with regardto gross body movements.Reynolds (1976) thoroughlydocumented and cross-referencedterms used by early rhesus monkey observers, and I attempthere to use many of the terms of my predecessorsin the hope of achieving marginalconsistency and common understanding.My terms for 996 PARTAN the vocalizationsfollow most closely those used by Rowell & Hinde (1962) and Hauseret al. (1993). The behavior of the animals was recordedaction by action in sequence. An 'action' was identifiedas a discrete unit of behavior,which may include multiple simultaneous components. This designation was based on my impressionof a behavioral'unit.' Behavior,although it occurs in continuous streams, can be described as being made up of sequences of more or less discrete actions. The identificationof these action units can, however, be difficult. I separatedthe behavioral stream into discrete actions based partiallyon physical changes in the body (e.g. the head moving from raised to lowered), and partially on timing considerations (if two components of behavior occurred simultaneously, such as head lowered and bark, I consideredthem to be componentsof the same action, whereas if there was a time lag between them, they were consideredto be two separateactions). Green (1975) also emphasized dynamic transitionsin patternsas an aid to classification. Drummond (1981) provides an extensive and interesting discussion of how we categorize and describe behavior. He notes that the stream of behavior can be segmented at many different levels, each appropriateto a differenttype of analysis. A common definition of a 'unit' is as a reliably recurringphenomenon (Drummond, 1981, p. 13-14). Ethologists may have to use intuition to determine where the breaks in behavior occur, but our intuitionis based on tractabledata such as regularityin patternsor bouts of behavior(Drummond, 1981; see also Marler& Hamilton,1966, Chapter20). The ultimatetest of our intuitionis to ask the animals themselves how they categorizetheir own signals (see Marler,1982). Describing real, complex behavior,involving multiple simultaneoussig- nal components,is a challenge. To simplify, I included only the visual and acoustic channels. In addition,I typically analyzed only two classes of behavior simultaneously:vocalizations, along with one class of visual behaviors (e.g. eye gaze, or mouth position). My rationalewas to documentthe bimodal visual and vocal behaviors of these animals. Detailed quantitative analyses of the compositions of facial and vocal expressions are presented in an effort to describehow visual and auditorysignal componentsare combined in this populationof rhesus monkeys. For furthersuggestions of how to analyze complex multicomponentbehaviors, see van Hooff (1982), Brad- bury& Vehrencamp(1998), and Deputte (2000). SIGNALCOMPOSITION 997

TABLE 1. Demographicsof main study animals

Study Subjects GroupY GroupV Adults (6 yrs +) Females 19 13 Males1) 12 8 Subadults(4-5 yrs) Females 8 4 Males 8 3 Juveniles (1-3 yrs) Females 15 6 Males 14 6 Infants(<1 yr) Females 11 6 Males 9 3 Total 96 49

1)The numberof adult males was approximatesince males occasionally transferredamong groups duringthe study.

Methods

Subjectsand study site

The subjectswere free-rangingmale and female rhesus macaques,Macaca mulatta,on Cayo Santiago, a 15.2-hectareisland off the southeast coast of Puerto Rico (Rawlins & Kessler, 1986). The monkeys ranged freely and formed social groups, were provisionedwith food, and had no naturalpredators. All of the animals were trappedas yearlings and given tattoos and ear notches for identification.At the time of the study, there were six social groups on the island, ranging in size from approximately49 to 354 animals per group. The study was conducted on two groups: group 'Y' and group 'V' (see Table 1 for age demographicsof the groups). Twenty-twoadult females and 13 adult males were the primaryfocal subjects. Occasionalsamples were taken from animals in neighboringgroups.

Videotaperecording methods

All data were collected on audio and videotape for subsequentanalysis. Adult animalswere filmed during 30-minute focal-animal follows. The primaryfocal animals had at least four samples each. Two simultaneousvideo recordings of the animals were taken: one camera filmed a close-up shot of the facial expressions of the focal animal, and the other took a wide-angle view that also encompassed the surroundinganimals. I collected 270 hours of video tape footage on VHS & SVHS tape, summing from both cameras, along with supplementaryaudio cassette tape recordings, during March and September-Decemberof 1994. Close-up videos used a PanasonicAG460 movie cameraand wide-angle videos used a PanasonicAG455. Both cameraswere mountedon Bogen 3179 tripods.A SennheiserMKH 816 shotgunmicrophone with windcoverwas strappedto the close-up video cameraand fed into the externalaudio inputfor betteraudio recordingson the videotape.I filmed only when the focal animal was interactingwith other monkeys at relatively close range, so signals 998 PARTAN propagatedby solitary monkeys, such as branch-shakingand food-calling (coos), were not common. Signals concernedwith mating were infrequentbecause I collected data primarily duringthe nonbreedingseason. (Publishedrecords, such as Rawlins & Kessler, 1985, indicate that the Cayo Santiago breeding season was from approximatelyJuly to November in the 1970's to early 80's. However,the season has shifted forwardslowly over the years, currently beginningin May and mostly over by September[M. Gerald,pers. comm., December2001].)

Data collectionfrom videotape

Of the 270 hours recorded on both cameras, I watched 230 hours of tape (approximately 130 hours of real time). In many instances, the two videos had to be watched one at a time because they displayed different views of the same scene, and included different players. While watching the tapes, I sampled for social interactionsusing the 'behavior-sampling' method (Martin& Bateson, 1993, p. 87). Specifically,I sampledfor any interactioninvolving vocal or visual communicationbetween two or more individuals.This broad sampling rule includedall social interactions,with the exceptionof groominginteractions already underway at the time the camerastarted to film, since they usually did not include vocal or visual signals. If two animalsbeing filmed began to groom duringthe sampleperiod, however,this behavior was included. Once an interactionwas located and data collection began, data were collected continu- ously until either (a) the individualsceased interacting,or (b) one or both ran off screen and out of view of both cameras. Since I was recordingin the field, with trees, bushes, and in- terveningtopography, many interactionsin the databaseended prematurely(i.e. the animals went out of sight before they were done interacting).However, this is not problematicfor the purposes of this study, since I am interestedhere in the structureof visual and auditory signals, ratherthan the outcomes of the interactions. I described 402 social interactions (both agonistic and affiliative), watching the interactions in detail, often proceeding frame by frame to identify the components used in the signals. The individualactions that each animal performed(which often included multiple simultaneouscomponents, as defined in the introduction)were logged sequentially,totaling to 1215 actions. The median numberof actions per interactionwas 2 (the range was 1-26; the mean was 3). This means that most interactionswere short, with one animal emitting a display and the other a response. Focal animals were all adults, but younger animals were included when they interactedwith focal subjects. Although the primaryfocal animals each had 4-10 focal follows, the total numberof actions performedby each focal animal ranged from 5 to 65 (dataon the numberof times each individualcontributed available on requestto the author). The databasecontained one entryfor each action, with the following fields: the time of the event (video frame); the identities of the actor and recipient(s)involved, including their sex and age; and a set of eight componentsof the signaler'sbehavior. These included:vocalization type, mouth position, eye gaze, ear position, head position, tail position, body posture, and overall movement.If a particularbody partwas out of view, as could happenif the animal's face was on the camera screen but its tail was not, I left blank the correspondingcell in the database.I did not include observationswith blank cells in my analyses. For each of the eight components of the signal, I compiled a list of possible expressive positions or states (Table 2). This list provides a detailed repertoire of rhesus monkey communicativebehavior, focused largelyon movementsof the face and head. I avoidedusing SIGNALCOMPOSITION 999

TABLE 2. Repertoire of communicativebehaviors of rhesus macaques: expressivecomponents of theface, voice and body

Vocal behavior: No vocalization silence Bark loud, voiced, harsh(broadband) sound Pant-threat softer,more breathyharsh sound usually emittedin a rapid bout of threeunits Scream loud sound, can be harshor tonal (includes noisy, archedand tonal screams) Squeak shortpunctuated harsh (pulsed) scream Bark/Scream mixed call that startswith barkand ends with scream Grunt quiet, harshsound usually not repeated Girney frequencymodulated, quiet sound, may be narrow-or broadband Coo tonal (narrow-band)sound Gecker staccatoharsh sounds, usually from infants Copulationcall shortharsh scream-like calls given by some males during copulation Othervocalization vocalizationwhich cannotbe put in any of the above classes Mouth position: Neutral mouth is closed, lips closed, loose Open mouth lower jaw droppedso lips form 'o' shape, upperteeth covered Grimace lips retractedhorizontally to expose teeth,jaws can be togetheror apart Tense mouth mouthis closed, lip cornersdrawn back to form straightline Lipsmack(LS) lips moved repeatedlytogether and apart;may be audible LS with tongue same as above but with tongue rhythmicallymoving in and protrusion out of mouth Chin-up1) mouthis closed, lips togetherand often pursed,chin is angled up Chin-up& LS same as above, but with lips making small, rapidmovements Teeth-chatter mouthrapidly opens & closes; teeth exposed; may be audible Toothchomp jaw opens and closes but lips remainclosed; may be audible Puckeredlips lips drawnforward together, cheeks furrowed Yawn mouthopens widely in stereotypedgaping movement Bite mouthopened to bite other monkey Mouth matches mouthopens only enough to emit a sound, and only for the vocalization durationof the sound, then closes

Eye gaze2): Neutral eyes relaxed,may be half shut, not looking in any particular direction 1000 PARTAN

TABLE 2. (Continued)

Stare direct, prolonged,unwavering look at specific individual Look at look in directionof specific individual Look away look in directionobviously other than thatof specific individual Look between look rapidlybetween two or more otherindividuals Look around look rapidlyin several directions,often at distant individual(s)/group(s) Ear position: Neutral ears relaxed, in medium position (about45 degrees from head) Ears forward ears pointing perpendicularlyout from head Ears back ears retractedtightly to scalp Ear flap ears move quickly backwardsand forwardsmore than once Eyebrow position: Neutral eyebrows relaxed Eyebrowsraised eyebrows raised and remainup Eyebrowslowered eyebrows lowered and remaindown Eyebrow flash eyebrows move quickly up and down Head position3): Neutral head relaxed Head raised head held up in high position Head lowered head held in low position, neck usually angled forward Headjerk head moved quickly and abruptlyup and down Head bob head moved slowly and smoothly up and down, often repeated Tail position: Neutral tail held in relaxed, low position Tail vertical tail held up perpendicularto body, straightfrom base to tip Tail up looped tail held straightup from base, but tip is curledin tight loop Tail up crooked tail held straightup from base, but tip is bent over Tail 45 degrees tail held up at 45 degrees from base, tip often slightly bent Tail parallel tail held straightout parallelto body Tail wave tail held low, waving Body positions & movements: Neutral posturerelaxed Crouch posturewith belly low, arms and legs bent Lunge suddenand quick movementof upperor entirebody towards recipient Mount mount other monkey in typical posture Lean away lean whole body away from anothermonkey Slap ground quick hit to ground,usually one hand only SIGNAL COMPOSITION 1001

TABLE 2. (Continued)

Dip-turn stereotypedmovement of approachinganother individual very closely, then suddenlybreaking approach by bending arms, turningquickly aroundand moving away (head usually swings aroundlast, so that actoris looking at recipientuntil last moment) Branchshake bounce up and down on branchor other object, holding on with feet so that the substratemoves as well Freeze body suddenlybecomes still Rough grab use hand to grabhold of other monkey Swat use handlarmto hit at other monkey Gentle touch use hand to gently touch othermonkey Piloerection hair is raised on body Fight fast-pacedagonistic interactioninvolving chasing, tumbling, biting, barking,screaming all at once Present-sexual presentown body for mountingusually by moving tail aside and presentingrear end to other monkey Present-for-grooming presentown body for groomingusually by moving neck or side or back towardsother monkey,looking away, and becoming still Groom hands systematicallycombing throughother monkey's hair Overall movement: No change remainstationary Approach move towardsother monkey Retreat move rapidlyaway from other monkey Chase run afterother monkey Go in several directions move back and forth Pass by pass anotherwithout stopping Leave leave anotherwith whom it had been associating(not rapidly) Follow walk close behind anothermonkey, same direction

1)Identical to 'muzzle-up' (Partan,1998). 2) Eye gaze directionwas determinedby head orientationas well as eye gaze. 3) Includeshead movementsin the vertical,rather than horizontal,domain. names with functional connotationsto describe the elements of the expressions, with the exception of pant-threat.I kept the 'threat' as a part of this name to be consistent with the previousliterature (e.g. Rowell, 1962; Rowell & Hinde, 1962; Hauseret al., 1993). All expressive components were defined as a departurefrom a neutral state (based on Sade, 1973). A neutral expression was defined as the countenance of a relaxed monkey (Fig. la). Neutralpositions of each body partwere recordedalong with expressive positions (see Table 2). Expressive positions are mutually exclusive within categories. For example, when an individual's head is raised, it cannot simultaneouslybe lowered; or if an animal is barking,it cannot be also cooing. The only categoryfor which internalmutual exclusivity does not hold is the 'body' category,because it is possible, for instance,for an animaltofreeze and crouch at the same time. In these cases I enteredthe most pronouncedbehavior, putting 1002 PARTAN

(t) ~~~~~~(b)

Fig. I. (a) Sub-adult rhesus male (X85) with neutral facial expression. Photo by author. (b) Adult female (E76) with silent stare and open-mouth. Digitized from videotape using Adobe Premieresoftware and Panasonicediting stations. the secondary behavior in an 'other' category. I chose to lump all body postures into this one category because I wanted to focus specifically on facial expressions and vocalizations. Body posturewas of secondaryinterest here, althoughcertainly of great importancein social behavior. Eye gaze direction is difficult to assess on videotape. I have followed Altmann (1962) in his distinction of stare from look at, and I have included three other active states of the eyes (see Table 2): look betweeni,look arounid(perhaps similar to Altmann'sunit #35, 'looks apprehensively'),and look away (similar to Altmann'sunit #34, 'avoids staringat'). In each of the latterthree cases the eyes move quickly between, among, or away from social targets in the environment,respectively. In contrast,eyes nieutrailwere relaxed eyes, often half shut or gazing off into space, not moving quickly. In cases where I could not assess eye direction from the videotape, I left the correspondingdata cell blank. Head position was troublesometo categorizebecause the head often moves in conjunction with other behavior, such as yawning (Deputte, 1994). Movements of the head are also incorporatedinto gaze behavior. I decided to prioritize mouth and eye behavior, since I was primarily interested in facial expression. Therefore yawn was categorized as a mouth behavior,and all eye gaze directions involving both head and eye movements were categorized as eye behaviors (e.g. look arounidand look between). Head movements that occurredin the vertical domain (raised, lowered, bobbed,jerked) were included in the head position category. My use of the term head bob refers to a slow, rhythmic up and down motion, whereas head jerk is a sudden sharp movement, not repeated. I follow Hinde & Rowell (1962) and Reynolds (1976) in their use of the term 'jerk' for the latter behavior (departingfrom Altmann, 1962, and Drickamer,1975, in their use of the term 'bob' to refer to quick head movementsduring threat). Vocalizationswere classified by ear from the audio trackof the videotape,after categories were established via spectrographic analysis and verified by comparison to published SIGNALCOMPOSITION 1003 spectrograms(Rowell & Hinde, 1962; Hauser & Marler, 1993) and discussions with other researchers(P. Marler, W. Mason, & M. Hauser, pers. comm.). Although rhesus screams have been subdividedinto five categories by Gouzoules et al. (e.g. 1984), I could reliably distinguish only two types by ear. One type, which I labeled scream, included 'noisy', 'tonal', and 'arched' screams (Gouzoules et al., 1984); the second type, which I called squeak,included scream-like vocalizations that were very shortin duration('pulsed' screams, Gouzoules et al., 1984). If a vocalizationhad severalsyllables (as was common with screams and pant-threats),I entered it only once into the database,to avoid inflating the numberof vocalizations. Pilot observationsof mouth positions during vocal behavior indicated that some vocalizations were producedwith a utilitarianor purely articulatoryuse of the mouth:the mouth opened only to the degree and durationnecessary for sound emission. Other vocalizations, however,were accompaniedby elaboratemouth movements that exceeded eitherthe duration of the call or the degree of opening necessaryto producethe sound. I distinguishedbetween these possibilities by includinga categorycalled mouthmatches vocalization (Table 2), along with the expressive mouth positions. This allowed me to set aside articulatorymovements for particularanalyses (noted below). Note that all vocalizations are 'simultaneouslymulti- modal' in that each occurs with a particularmouth position, providingboth visual and vocal stimuli, regardlessof whetherthe mouth is open as long as or longer than the call. A finer- grained temporalanalysis might consider the latter cases to include unimodal signaling of mouthopening, followed by multimodalsignaling duringthe call, followed again by the unimodal mouthposition afterthe call has ended. However,here the behavioralunit includedthe vocalizationin the context of the visual signals that encompassedit; any action that included a vocalizationwas considered 'multimodal'.

Data analysis All data analysis was done using SAS software.Any entries (observations)with missing data for the behaviorbeing analyzedwere omittedprior to analysis. This reducedthe sample size differentiallyfor the differentanalyses, dependingon which behaviorswere being assessed. I carriedout three main analyses. First I examined the entire data set to determinewhich components co-occurred, using Principal Component Analysis (PCA) to examine overall relationships among the variables. Since PCA requires numeric data, I transformedthe nominal categories into a series of indicatorvariables which read '1' or '0' depending on whetherthe particularbehavior was presentor absent(this methodis also used and described by Deputte, 2000). I used all eight behavioralclasses (vocalizations,eye, ear, head position, etc.). When I transformedthe data, there were a total of 57 columns, because the 8 classes included 57 unique behavioralcomponents or elements that were included in the analysis. (All behavioralcomponents in Table 2 were included in the PCA except articulatorymouth matches vocalizationmovements and some other rarecomponents.) The raw data matrixfor the PCA analysis consisted of the 57 columns correspondingto each behavioralcomponent, and 1215 rows correspondingto each event. The analysis thereforeproduced a correlation matrixwith 57 columns and 57 rows, correlatingeach of the behavioralelements with each other element. This matrix was then analyzed to produce 57 principalcomponents, the top six of which had eigenvalueslarge enough (> 1.9) to warrantdiscussion. The second analysis was a comparisonof the silent and vocal behaviorsof the animals, based on the frequenciesof occurrenceof each expressivecomponent during silent and vocal behavioras a whole. 1004 PARTAN

In the third set of analyses I examined in detail the structureof the bimodal, visual/vocal expressions.I conductedMultiple Correspondence Analyses (MCA's)to explore the distribu- tion of componentstwo or threeclasses of behaviorat a time, and presentedthe resultsgraph- ically. The frequency data were taken from columns in the main database,with behavioral componentsacross the columns, and one composite action per row. A correspondenceanalysis, like a Principal ComponentAnalysis, detects trends in the data by establishing which components are associated (see van der Heijden et al., 1990, for detailed explanation).The analysis then defines 'dimensions' to describe the strongestassociations. Like the PCA, the correspondenceanalysis has many dimensions, but only the two most importantones are plotted in the graphs. Strong associations between variablesare shown in two ways on the MCA graphs:by distance from the origin, and by trajectory.If two behavioralcomponents plot out on the same trajectory,they are closely associated,and the fartherthey are away from the origin, the strongerthe association. (Diagonal lines drawnon the graphsmake it easier to see which variables are associated.) Note that for the correspondenceanalysis of mouth positions and vocalizations,I excluded the articulatorymouth matches vocalizationcategory. Finally, I documentedwhich visual componentswere associated with each vocalization, using frequencymatrices and histograms. For all analyses, behavior was pooled across individuals. Although I am aware of the potential dangers of pooling data for parametrictests (Machlis et al., 1985; Leger & Didrichsons, 1994), my data are nominal frequencies that cannot be averaged. I cannot, for example, find the mean of 10 open mouth threats and 5 fear grimaces to come up with an 'average expression' for a given individual who produced these 15 expressions. FurthermoreI collected data on myriad components of expression, with the individuals involved in many differentpossible dyadic combinations,making the concept of an 'average' even less appropriate.Instead, the focus of analysis is on the frequencieswith which multiple expressivecomponents are combined, across all cases of expression.

Results

Overall relationshipsamong the behaviors

Principal Component Analysis yielded six top components (with largest eigenvalues; Table 3). The clusters revealed by the PCA correspond to behavioral suites that have been reportedpreviously in a more anecdotal fashion.

Component1 The most striking association in the data was among open mouth, stare, ears forward, and head lowered (Fig. lb depicts the open mouthand stare). This suite of behaviorscorresponds to descriptionsof silent threatfrom the literature.Grimace did not occur with this cluster. SIGNALCOMPOSITION 1005

TABLE 3. Principal ComponentAnalysis

Component FactorWeight Variable Coefficient Interpretation PC 1 3.67 stare 0.38 Silent threat head low 0.31 open mouth 0.28 ears forward 0.28 [grimace] -0.23 PC 2 2.75 bark 0.31 Vocal fight look-around 0.27 scream 0.23 move around 0.23 PC 3 2.59 tail up 0.34 Approach approach 0.31 PC 4 2.32 girney 0.28 Affiliation lipsmack 0.24 ears back 0.24 tail wave 0.23 [grimace] -0.22 PC 5 2.08 look-at 0.38 Aggressive approach lunge 0.35 approach 0.32 PC 6 1.94 look-at 0.35 Submission grimace 0.31 /Male solicitation retreat 0.24 ears back 0.21 chin-up 0.21 [bark] -0.22

Top six components.All variableswith Icoefficientl>0.2 are listed for each component.A negative coefficient indicates that the variablewas negatively associated.

Component 2 The second principalcomponent involved bark, scream, looking around, and moving in several directions.This is a typical suite of behaviorsinvolved in agonistic interactions.

Component 3 Steady, walking approach with tail raised. Typical behavior of dominant males (Hinde, 1966; Lindburg, 1971). 1006 PARTAN

TABLE4. Comparisonof visual componentsassociated withsilent and vocal expressions

SILENT VOCAL N Row % Column % N Row % Column % Total Mouth: Neutral 249 98.03 44.15 5 1.97 2.02 254 Open Mouth 108 63.53 19.15 62 36.47 25.00 170 Grimace 88 58.28 15.60 63 41.72 25.40 151 Lipsmack 69 82.14 12.23 15 17.86 6.05 84 Chin-up 16 100.00 2.84 0 0.00 0.00 16 Yawn 9 100.00 1.60 0 0.00 0.00 9 Bite 9 100.00 1.60 0 0.00 0.00 9 ToothChomp 8 100.00 1.42 0 0.00 0.00 8 TongueProtrusion 8 88.89 1.42 1 11.11 0.40 9 Matches Sound n/a 0.00 0.00 102 100.00 41.13 102 ColumnTotal 564 69.46 100.00 248 30.54 100.00 812 Eye: Look-at 252 73.68 38.83 90 26.32 33.21 342 Stare 161 73.18 24.81 59 26.82 21.77 220 Neutral 99 89.19 15.25 12 10.81 4.43 111 Look-between 64 65.98 9.86 33 34.02 12.18 97 Look Away 40 83.33 6.16 8 16.67 2.95 48 Look-around 33 32.35 5.08 69 67.65 25.46 102 ColumnTotal 649 70.54 100.00 271 29.46 100.00 920 Ear: Neutral 458 72.01 73.63 178 27.99 68.46 636 Back 100 66.67 16.08 50 33.33 19.23 150 Forward 55 75.34 8.84 18 24.66 6.92 73 Flapped 9 39.13 1.45 14 60.87 5.38 23 ColumnTotal 622 70.52 100.00 260 29.48 100.00 882 Head: Neutral 575 69.70 81.91 250 30.30 88.34 825 Lowered 82 80.39 11.68 20 19.61 7.07 102 Jerked 20 80.00 2.85 5 20.00 1.77 25 Raised 13 68.42 1.85 6 31.58 2.12 19 Bobbed 12 85.71 1.71 2 14.29 0.71 14 Column Total 702 71.27 100.00 283 28.73 100.00 985 Body: Neutral 515 69.69 71.63 224 30.31 74.92 739 Lunge 45 59.21 6.26 31 40.79 10.37 76 Lean Away 30 90.91 4.17 3 9.09 1.00 33 Crouch 22 50.00 3.06 22 50.00 7.36 44 SIGNALCOMPOSITION 1007

TABLE 4. (Continued)

SILENT VOCAL N Row % Column % N Row % Column % Total Freeze 22 91.67 3.06 2 8.33 0.67 24 Slap Ground 17 89.47 2.36 2 10.53 0.67 19 Gentle Touch 16 94.12 2.23 1 5.88 0.33 17 Rough Grab 15 100.00 2.09 0 0.00 0.00 15 Present 11 100.00 1.53 0 0.00 0.00 11 Swat 10 76.92 1.39 3 23.08 1.00 13 Shift Position 9 90.00 1.25 1 10.00 0.33 10 Dip-turn 5 100.00 0.70 0 0.00 0.00 5 Fight 2 25.00 0.28 6 75.00 2.01 8 Piloerection 0 0.00 0.00 4 100.00 1.34 4 ColumnTotal 719 70.63 100.00 299 29.37 100.00 1018 Movement: None 411 71.98 54.15 160 28.02 51.95 571 Approach 172 78.54 22.66 47 21.46 15.26 219 Retreat 98 63.64 12.91 56 36.36 18.18 154 Pass By 25 100.00 3.29 0 0.00 0.00 25 Chase 22 53.66 2.90 19 46.34 6.17 41 Back & Forth 15 37.50 1.98 25 62.50 8.12 40 Follow 11 91.67 1.45 1 8.33 0.32 12 Leave 5 100.00 0.66 0 0.00 0.00 5 ColumnTotal 759 71.13 100.00 308 28.87 100.00 1067

There were a total of 760 silent behaviorsand 356 vocal ones, but the totals for each section above are smaller,owing to dataunavailable because a particularbody partcould not be seen in certaincases. Component4 Girney, lipsmack,ears back, tail wave. This is typical affiliativebehavior. Grimacedoes not occur with these behavioralelements.

Component5 Lunge, chase, look at. This is typical of an aggressiveapproach.

Component6 Retreat, grimace, look at, ears back, chin-up. The first two behaviors are typical of submission. The last three are components of a solicitation behaviordescribed by Altmann(1962, p. 378), in which a male approacheda female with a stereotypedposture, head tilted back, chin up, and lips pursed 1008 PARTAN

ii .

_ i I ...... i,,~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .

_ ...... ,, _~~~~~~~~~...... U _.,...... ,,,t, ,

I'l"...... _ .: | ~~ ~~~~~~~~~d ....F;},... ,~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~......

Fig 2. Images from videotape of vocalizing animals. Spectrograms were collected from videotape at the same frame as the picture, using a Kay Digital Sona-Graphmodel 7800. Horizontalbars mark I-kHz intervals (1-8 kHz). (a) Adult male barking, with open mouth and ears back. (b) Adult female (845) giving a pant-threatvocalization. (c) Sub-adultfemale (X70) giving a broadband('noisy') scream. (d) Adult male (C78) grunting. (e) Sub-adult male girneying and waving his tail. Although this girney contains primarilybroadband components, girneys can also include narrow-bandsounds (e.g. see spectrogramin Kalin et al., 1992). Figure 2a reprinted with permission from: Partan, S. & Marler, P.: Communication goes multimodal- Science 283(5406), p. 1272-1273; copyright 1999 American Associa- tion for the Advancementof Science. and sometimes rapidly smacking.He typically approachedvery close to the female, almost touched his face to hers, and then immediately tured away with a stereotyped 'dip' (arms bend quickly down), and walked away. Only males were seen performingthe chin-up and dip-turn; when a recipientwas identified, it was female (Partan, unpublished data).

Comparison of silent and vocal expressions

Four of the top six Principal Components were silent, and two were vocal. To compare silent and vocal behavior, I explored which visual components were SIGNAL COMPOSITION 1009 associated with vocalizations. The presence or absence of a vocalization could be reliably determinedfor 1116 out of the 1215 total events recorded. Of these 1116 events, 68% (760) were silent and 32% (356) included vocalizations. Silent and vocal expressionswere differentiallyaccompanied by the various visual signal components (Table 4). The 'Vocal' column in Table4 lumps the six majorvocalizations, each of which occurred20 times or more: barks, pant-threats, screams, squeaks, grunts, and girneys (see Fig. 2 for spectrograms).Omitted from the table are vocalizationsthat were recordedfewer thanten times (bark-screammixes, coos, geckers,copulation calls, harmonicarches, and unclassifiablecalls). The 'row %' column in Table 4 compares the silent to vocal behavior. Silent expressionsinvolved more visual componentsthan vocal expressions. This can be demonstratedby a quick scan over the 'row %' columns to see the disparityin the numberof behaviors at 100% for the silent group comparedto the vocal. The mouth was most often in a neutralposition duringsilence, but many other mouth positions were used. All possible eye positions were recorded during silent expressions,the most common being looking at the recipient, followed by staring and neutral eye position. All ear and head positions were also recorded during silent expressions, with the neutral position predominantin both cases. Head bobs were predominantlysilent. Many body and arm postures occurred during silent expressions, although most of the time the body was neutral.Presents, grabs, and dip-turnswere always silent; lean away, touch,freeze, and slap ground usually so. During most silent expressionsthe animalwas stationary.Pass by and lunge were always silent;follows usually were as well. Expressions including vocalizations incorporatedfewer visual components than did silent expressions overall. Mouth positions during the six majorvocalizations included open mouths,grimaces, lipsmacks,and mouth matchingthe vocalization.There were no vocalizationsduring yawns, bites, tooth chomps,or chin-upmovements. Lookingaround occurred more duringvocalizations than during silent behavior;looking away, however,predominantly occurred during silent behavior.Ear-flapping, lunging, crouching, fighting, andpiloerection occurred pro- portionatelymore often duringvocal than silent behaviors,as did retreating, chasing, and movingback andforth. 1010 PARTAN

Structureof multimodalsignals: Visualbehaviors associated with each vocalization

I used Multiple CorrespondenceAnalysis to examine in greaterdetail how behaviorswere associatedwhen consideringnot the entire data set, as in the above PCA analysis, butjust two of the eight majorcategories at a time (e.g. vocalization and mouth position). Associations among vocalizations and mouthpositions are shown in Fig. 3, using only 'expressive'mouth positions (all positions except mouthmatches vocalization). The componentssplit into three groups, each of which contains behavioral elements devoted to one of three functions described previously in the literature:threat, submission, and affiliation (Altmann, 1962; Hinde & Rowell, 1962; Rowell & Hinde, 1962; Lindburg,1971). Threatincluded bark,pant-threat, bark-scream, and

, : * 4 ,J:A.....' ! . 146,

* , .. ..;: .,:.; . .:.fs..e.j... t:..'* ".'.

Fig. 3. Multiple correspondence analysis of vocalizations (circles) and mouth positions (triangles). Associations among variables are shown by common trajectory from the origin. Aggressive components clustered on upper left; submissive ones on right; neutral and friendly ones on lower left. Diagonal lines are included to facilitate grouping of the variables. SIGNALCOMPOSITION 1011

* 1 3-U

11..~~N"

1: ..._^K 1. ...

Fig. 4. Multiple correspondenceanalysis of vocalizations (circles), mouth positions (triangles), and eye gaze (squares). open mouth; submission included scream, squeak, and grimace; affiliation includedno vocalization,girney, grunt, coo, lipsmack,and chin-up. To test whether these three clusters still appearedwhen combined with a visual component uninvolved with phonation,I added eye gaze direction to the vocal and mouth data and ran anotherMCA (Fig. 4). The same three clustersoccurred. Stare fell into the aggressive cluster;look aroundand look betweenfell with submission;look at and neutral eye with affiliation. Along with the above associations predictedfrom the literature,the next MCA analyses revealed several new and unexpected associations. These depict a difference in visual accompaniments of the two vocalizations considered to be aggressive, bark and pant-threat. Associations among vocalizations and ear position are shown in Fig. 5a. Threateningbehaviors are on the right,and nonthreatening(affiliative and submissive)behaviors are on the left. The nonthreateningvocalizations were loosely associated with _,, . . : . , . .. _ . i .

1012 PARTAN

I _-''...' . i: s' -1' ''' ",;-'*'', . '*.';:*/.' "ji '-' .:': "*' 'y v l:****:'.. * v *t . ;*'-* * -,w .'' **ri *'<'.' } *::'*. -.w !; :y_, * .+i9 ^ .:' . ****:* ' . -:* .' '-: ' ' ' ^* ^ !* *'*^ '^ * * :" i . e! ! |l , . l i j i ^ j .EX '"' * * * .-- C» AN' ,,i .:.*'_'; . ^ *''i, *:,n:, ' : i. ' f: ' '-'

"¢* ' !".' .., ,- i-I,

...... ,' . l . .','' .'.e · t.;. A ..t ......

' '

: : ^ '- .^ .'*.l ^ i -4, i: ' '.;*':;;- . --.'.', .'s1V;

_0 - F- · ;-- ---;·~' · . -~;~r . CL- .rcu·* 1.L-~~;-i,l;i:ii~P I~Li·~+ - .. . .. 1 ......

' (a) ...... -*.' ji:^'i"t ii; > '

Fig. 5. Multiple correspondenceanalysis of (a) vocalizations (circles) and ear positions (triangles), and (b) vocalizations (circles) and head positions (triangles). In both graphs, aggressive behaviors fell to the right of the origin, and submissive and affiliative ones fell primarilyto the left.

neutral ear position. The interesting finding is that the threat vocalization bark was associated with ears back, while pant-threat was with ears forward. Figure 5b shows the associations among vocalizations and head position. Affiliation and submission are on the left, weakly associated with neutral head position. On the right, pant-threat vocalizations were associated with head raised or forward, whereas barks were with head lowered. , . ._ F-.s :

SIGNAL COMPOSITION 1013

. . -.& -0-00- . . . - . - -

.~.·

^.^ .- . ': :..., .:.'.:. ::'-

*.,... : .. e ... .,., ..-" : , - - ' . . . ." , '. "'...... 7 . :......

'i l[ ~.~.'.. '".: ·, . :~. -~ .. .. -1 .711.M.1U :, .-. -- - t - : : il - ...::...... uy .*., .- ,:.[ : , -:...:..-:,-'...:.::..: ''" . ,, ,:, ',',, T,. ;...... :¥ · . ,. ,: ~..'...4.. . .'"~: ...' .S,i:-:t."'"....; .. ,',.:.:..:''..

* .' 'jitr~~~~~~~~~~ :

·WqL' .:. Ti- i· -i IT;· - · ,~ i · -. ·· .-I- .. · I -· .. ·- - I i- ·.l :i-· 7 · l : ..'-'^; -I, *...... ;.. VE Ak': · .. --I...f.L','. ... '..:':.'..'.. :".. (b.)

Fig. 5. (Continued).

I systematicallydocumented the visual componentsthat occurredsimul- taneouslywith each of the six majortypes of vocal behaviors(Fig. 6). These are presentedbelow, organizedaccording to the visual components.

Mouthposition Mouth position was tightly linked to vocalization type (Fig. 6a). Barks and pant-threats were accompanied by rounded open mouths, screams and squeaks were accompanied by grimaces, and girneys (and, to a lesser extent, grunts) were accompaniedby lipsmacks.Each vocalization also had a significant proportionof mouth shapes that simply matched the sound, 1014 PARTAN synchronizedin time, opening and closing without any furtherexpressive posture. For grunts, almost 80% involved no mouth expression other than opening slightly for the grunt. The lowest matching scores were found among screams and, especially, squeaks, which often occurredin the midst of prolonged silent grimaces. These vocalizationswere still simultaneously multimodal:the squeaksco-occurred in time with grimaces.

Eye gaze All six vocalizationswere accompaniedby substantialproportions of looking directlyat the recipientand of lookingaround (Fig. 6b). The girney involved a higher percent of looking at the recipient (67% of all girneys), and a lower proportionof lookingaround (only 17%). Barks andpant-threats were accompaniedby a high proportionof staring (37% of barks, and 50% of pant-threats).Screams and squeaks had the highest proportionsof looking between the recipient and a third party (13% of screams; 30% of squeaks) and of looking away from the recipient(5% of screams; 15%of squeaks).

Ear position Ear position (Fig. 6c) was predominantly(>80%) neutral for screams, squeaks, girneys, and grunts. Barks and pant-threatsdiffered dramatically. Ears were neutral in only 51% of barks and 40% of pant-threats.During barks, the most common nonneutralposition was retracted(28%), but they could also be juttingforward(10%) orflapped back and forth(11%). In pant- threats the ears were most oftenforward (35%) but also could be retracted (15%) orflapped (10%).

Head position Head position (Fig. 6d) was always neutralfor screams,squeaks, and grunts. During girneys, the head was predominantly(92%) neutral, but in the remainderof the cases showed a distinctive bobbing movement not seen during any other vocalizations. Barks and pant-threatswere most variable in head position. Barks were accompanied16% of the time by head lowered, 3% by headjerked, and 2% by head raised. Pant-threatswere accompanied more often thanbarks by head raised (20%) and headjerked (10%),and less often by head lowered (10%). SIGNAL COMPOSITION 1015

Body posture A wide array of body postures was observed (Fig. 6e). Grunts, girneys, and pant-threats were given at least 95% of the time with neutral body position. The remaining girneys were accompaniedby reaching out and gently touching the recipient. The remainingpant-threats involved body crouches. Barks, screams, and squeaks were relatively active. Barks were accompanied 16% by lunges; 4% by crouching, and 3% by piloerection. Screams were given 16% with crouching, 12% with lunging, 7% with tumbling duringfighting, 4% with swatting, and 2% with leaning away. Squeakswere emitted 7% with lunging or crouching,and 4% with leaning away,freezing, or shiftingposition.

Overallmovement Movement (Fig. 6f) showed similar patternsto body posture, except for girneys. 40% of girneys were stationary;52% involved approaching the recipient. Grunts and pant-threatswere 90% stationary;for the remainder, the actor approached during grunts, and either approachedor chased the recipientduring pant-threats. Barks were heardduring chases or approaches 13%each, duringretreats 5%, and6% while moving backandforth. Screams and squeaks were commonly accompaniedby retreats (42% of screams; 36% of squeaks).They also were heardduring approaches (12% of screams; 16%of squeaks)and while moving back andforth (14% of screams; 10%of squeaks).

Discussion

This study provides quantitativesupport for two areas of literature.First, the data supportthe idea of three 'poles' of behaviorimportant during social communication:aggression, submission, and affiliation.Deputte (2000, p. 118) calls this the "3 A's concept (affiliation, aggression, and avoid- ance)," and Mason (1985) also singles out these three motivationalstates. In the presentstudy this was shown most clearly in the MultipleCorrespon- dence Analyses of the associationsamong mouthposition, vocalization,and eye gaze (see Figs 3 & 4). Previous work identified behavior involved in these three domainsin rhesus (i.e. open mouthsand barksbeing aggressive; 1016 PARTAN

(a) 100% [ eta t U~~~~~~~~~~~~~~~~~~~~~~OTongueprotr. 13Lipsmack

10%80% *~~~~~~~~~~~~~~~~~~~~0Grimace 12Ope~~~~~~~~~~~~~~~Qno mouth E Matches Sound

60%

40%

20%

BARK(102) PANT(20) SCREAM(62) SQUEAK(25) GIRNEY(20) GRUNT (19)

(b)K (111)OPANT (20) SCREAM(59)SQUEAK(27)GIRNEY(23)GRUNT(20)Neutral 12LookAway 0f% - 13Look-between Fig. 6 Visual expressive components associated with each major Stare 1ULook-around * Look-ut 60%

40%

20%

0% BARK(1 13) PANT (25) SCREAM(67) SQUEAK(27) GIRNEY(24) BRUNT (25)

70%

60%

50%

40%

30%

20%

10%

BARK(It ) PANT(20) SCREAM(09) SQUEAK(27) GIRNEY(23) GRUNT(20)

Fig. 6. Visual expressive components associated with each major vocalization. (a) Mouth position; (b) eye gaze direction; (c) ear position; (d) head position; (e) body posture; (f) overall movement. The N's are based on the numberof times that the vocalization was SIGNAL COMPOSITION 1017

(d) 100% Nur 90% OBobbed

80% B~~~~~~~~~~~~~~~~~~~~Jerked U Raised

70% *Lowered

60%

50%

40% 30%{

20% --

10%I

BARK (110) PANT(20) SCREAM(75) SQUEAK(29) GIRNEY(24) GRUNT (20)

700 O Neutral (e) 80% ( 90% Shift position MuGentle Touch 80% El~~~~~~~~~~~~~~~~~~~~UFreeze EoSlapGround

70% 12LeanAway MSwat 13Piloerection 60% U Fight U3Crouch 50% U Lunge

40%

30%

20%-

10%

0% BARK (120) PANT(20) SCREAM(86) SQUEAK(29) GIRNEY(24) GRUNT (20)

7\ 100% (1) ~~~~~~~~~~~~~~~~~~ONone 90% * Follow 13Chaue eO% HEBach& FoSth

Retreat N 70%60% U~~~~~~~~~~~~~~~~~~~~~~~~1Approach

00%

BARK (120) PANT(20) SCREAM(92) SQUEAK(31) GIRNEY(25) GRUNT (20)

Fig. 6. (Continued)recorded in conjunctionwith any position of the mouth, eyes, etc. The N's differ slightly among panels owing to cases lost hecause the particularvisual component could not he seen. 1018 PARTAN screams, squeaks, and grimaces submissive; and girneys and lipsmacksaf- filiative; Altmann, 1962; Hinde & Rowell, 1962; van Hooff, 1962, 1967; Rowell & Hinde, 1962; Marler, 1965; Lindburg, 1971; Drickamer, 1975; Redican, 1975; Mason, 1985; Boccia, 1986; Kalin et al., 1992; Maestrip- ieri, 1997; Maestripieri& Wallen, 1997; Partan, 1998). Van Hooff (1973) providedquantitative evidence for five behavioralstates importantin chimpanzees, including the three discussed here as well as play and excitement. He pointed out that the determinationof these groupingsdepends partially on the degree of lumpingand splittingof the initial behavioralelements that make up the catalog. Adams & Schoel (1982) found six motivationalstates in male stumptailmacaques, including the three discussed here, and defense (which I included with submission),sexual behavior,and display. Second, these data provide quantitativesupport for the previous descrip- tions of behavioralclusters in rhesus monkeys.For example, the top components of the PrincipalComponent Analysis, each of which involved a suite of behaviorsthat clusteredtogether, corresponded to behaviorsreported (usually in a nonquantitativefashion) to have been observedtogether.

Comparisonof silent and vocal expressions

Silent visual expressions dominated the communication signals used by these monkeys at the close distances at which they were observed. This agrees with Rowell's (1962) suggestion that the visual channel is primary for rhesus macaques, and supports the argumentthat, owing to selective pressures imposed by the environment,terrestrial primates such as rhesus macaques rely more heavily on vision while arborealones rely more on audition(Altmann, 1967; Redican, 1975). A higher proportionof behaviors in this study were vocal (31.9%),however, than was trueof Altmann's(1962, 1965, 1967) study,in which only 5.1% of behaviorsincluded vocalizations. These figures are difficult to compare, particularlybecause Altmann split some categories of visual behaviors more finely than I did, increasing the proportionof visual to vocal signals in his study relativeto mine. The silent expressionsreported here were highly complex; in fact, silent expressionsas a whole incorporateda widerrange of visual componentsthan did expressionsaccompanied by vocalizations(see Table4). This may partly be due to physical constraintsplaced on the mouthwhen an animalvocalizes, inhibiting the concurrenceof vocalizations with certain mouth movements SIGNAL COMPOSITION 1019 such as yawning, biting, tooth chomping, or pursing or compressing the lips. However, these particularmouth movements do not entirely preclude vocalizing:yawns have been observedto coincide occasionallywith barksin grey-cheekedmangabeys, for example (Deputte, 1994). In additionto mouthpositions, some body posturesnever occurredwhile an animal vocalized, but did occur during silent behavior(see Table4). All occasions when an individualpresented its body to another,whether for grooming, occasional sexual interactions,or to indicate submission, were silent. Also silent were chin-up and dip-turn, which Hinde and Rowell referred to as 'dancing' (see Fig. 8b & c in Hinde & Rowell, 1962, p. 17). They did not mention any vocalizationsoccurring during this behavior. Altmann also described this behavior in detail, although he did not name it. He considered it to be an 'extreme form' of 'smacks lips at' (Altmann, 1962, p. 378), and it did not occur in combination with any vocal units. This behavior may be analogous to the 'stylized trot' described by Green (1975, p. 65) for Japanesemacaques. The facial componentmay be similar to the 'LEN' observedin pigtail macaques(Lips protruded-Eyebrowsraised- Neck forward;Jensen & Gordon, 1970, p. 268) and the 'pucker'described by Maestripieri& Wallen (1997). The males I studied were silent during the chin-up and dip-turnbehavior regardlessof whether or not they were lipsmacking. Hauser (1993) has reportedthat male rhesus monkeys who vocalize duringcopulation, perhaps analogous, are morelikely to be attacked by other males thanthose who copulate silently.

Visualsignals associated with vocalizations

Multimodalsignals are an importantpart of the rhesus macaquerepertoire: in this studythey made up just over 30% of all behaviorsin the database.For example, barkand pant-threatvocalizations were accompaniedby particular visual components(open mouth,staring, ears neutral or back, head neutral or lowered,body neutralor lunging, and approaching,chasing or remaining stationary)as suggested in the literature(Altmann, 1962; Hinde & Rowell, 1962; van Hooff, 1962, 1967; Rowell & Hinde, 1962; Redican, 1975; de Waal et al., 1976; Boccia, 1986). Novel findings include a contrastin the frequenciesof the visual accompanimentsof the two aggressive vocalizations. Pant-threatswere more often accompaniedby staring, ears in the forward position, and head raised 1020 PARTAN or forward than were barks. Ear position during threathas been described by others to be retracted(e.g. Drickamer, 1975), but the two threatening vocalizations have not been distinguishedpreviously in terms of these associated visual signals. Staring or looking at anotherindividual may be a method of indexing with the eyes, indicatingto whom a vocalization is directed (Itani, 1963; Altmann, 1967; Mitchell, 1972; Argyle & Cook, 1976; Green & Marler, 1979). Ears in the forwardposition may serve a similar function, indicating the intended addressee (Partan& Marler, 2002). The correlationsbetween pant-threatand indexical eye and ear positions suggest that pant-threatsare typically directed at one or more targetedindividuals, whereas barks are propagatedmore generally. Barks were more often accompanied by looking around and looking between other individuals than were pant-threats. 'Broadcast' signals are not directed to any recipient in particularbut are presented generally for all to see or hear. Other exam- ples of broadcastsignals in rhesusinclude branchshaking, yawning, and tail raising. In the stumptailmacaque, broadcast signals such as branchshaking (called 'bouncing') and patrollingwere also associated with barks (Adams & Schoel, 1982). The visual accompanimentsof the two submissivevocalizations, screams and squeaks,differed. Both were generallyaccompanied by grimacing,looking at, around, or between two other animals, neutral ears, heads, and bod- ies, and retreatingor remainingstationary (Fig. 6), in agreementwith previous studies (Altmann, 1962; Hinde & Rowell, 1962; van Hooff, 1962, 1967; Rowell & Hinde, 1962; de Waal et al., 1976; Maestripieri& Wallen, 1997). Not previously documented,however, are that the ears were never held forward or flapped during submissive vocalizations, and that screams and squeaks differed in the proportionsof their visual accompaniments. Squeaks (shorterthan screams) occurredwith prolonged, stereotypicalgri- macing mouth positions, and were more often accompaniedby looking between the recipientand a thirdanimal than were screams. Squeaksnever oc- curredduring actual fighting, however; they were more likely to occurduring breaks in fighting or after the fight was over. Since looking between can be used for recruitment,squeaks may have been used more often than screams when soliciting supportfrom allies presentat the scene. Two acoustically distinct vocalizations,girney and grunt, were the only ones to be sometimes accompaniedby the affiliativebehaviors of lipsmacking and, in the case of girneys, head bobbing (Fig. 6). Vocalizing animals SIGNAL COMPOSITION 1021 usually looked at the recipient, had neutral ear, head, and body positions, and either approached(in the case of girneys) or remainedstationary. The ears, as with submissivebehaviors, were never heldforward orflapped during these two vocalizations.Girneys and grunts differed in the proportionsof their visual accompaniments.Girneys were most often given with lipsmacking, an affiliativesignal (Hinde & Rowell, 1962; Mason, 1985; Maestripieri & Wallen, 1997). Gruntswere much more likely than girneys to be given without any mouth expressionat all. Gruntswere also more likely than girneys to be accompaniedby neutralpositions of the eyes, ears,head, andbody, and grunts were usually not accompaniedby movementin any particulardi- rection. Adams & Schoel (1982) found that stumptailmacaque grunts were likewise not significantlyassociated with any particularacts or postures. Rowell & Hinde (1962) suggested that rhesus girneys (or 'girns') are affiliative, and Kalin et al. (1992) provided empirical evidence for this in infant monkeys. Hauser et al., (1993) classified both girneys and grunts as affiliative.Hauser & Marler (1993) stated that grunts occur in multiple contexts, including both affiliativeand food-related.Kaldor (1996) lumped grunts with girneys in her analyses and empiricallyfound them togetherto be affiliative. My data are consistent with the suggestion that girneys are affiliative,but I have little evidence that grunts have any emotionalvalence whatsoever.

Mouthposition

Mouthposition differs from the other visual expressionsdiscussed in that it is intricatelytied with vocal production.Although vocalizationsand mouth shapes must to some extent be associated, there is disagreement about the order of events on an evolutionarytimescale. Darwin (1872, p. 91) mentioned this as an "obscure point, namely, whether the sounds which are produced under various states of the mind determine the shape of the mouth, or whether its shape is not determinedby independentcauses, and the sound thus modified".Rowell (1962) suggested that vocalizations are secondaryto visual signals, postures, and breathingpatterns of rhesus monkeys. Andrew (1963) agreedthat vocalizationsare dependenton mouth shapes, althoughhe suggestedthat the mouth shapes originatedprimarily as protectiveresponses toward noxious environmentalstimuli. Ohala (1984), however, indicated that the vocalizations were primary:he suggested that 1022 PARTAN mouth shape originatedin order to produce specific types of sound. Ohala based his ideas on Morton's (1977) 'motivational-structuralrules,' which predictthat an aggressiveanimal will attemptto appearlarge by using sounds of lower frequencies,and submissiveanimals will try to appearsmall, using higher-pitchedsounds (but see Hauser et al. 1993). Ohala suggested that large size could also be communicatedby a large resonantchamber, such as that createdby the roundedmouth of a macaquedisplaying an open-mouth threat.Conversely, smallness could be conveyed by high resonance, which would occur if the monkey pulled its lips back, shorteningthe length of the resonance chamber.Ohala suggested that throughritualization, the mouth shapes, originally in service of vocalizations, became independentand are now communicativeon their own. I distinguished between articulatoryand expressive movements of the mouth. I found that the most common mouth shapes accompanyingeach vocalization either matched the vocalization exactly (i.e. the mouth was opened only to the degree necessary to produce the sound) or were of a shape perhaps constrainedby the requisite sound production(grimace for screams and squeaks;rounded open mouthfor barks;neutral or lipsmacking for girneys). This providesquantitative support for the illustrationsin Hauser et al. (1993) of typical articulatorygestures accompanying eight main vocalizations of rhesus macaques. These data suggest that the mouth and the voice are not emancipatedduring bimodal (vocal) production.During unimodal, silent (visual) behavior,the mouth took on a variety of postures and movementsnot observed duringvocalizations, although the silent open mouths,grimaces, and lipsmackswere similarin gross structureto the vocal ones. I compared the proportionof each vocalization that was accompanied by matchingmouth shapes (see Fig. 6a). Gruntshad the highest scores for mouthpositions exactly matchingthe vocalization,indicating that there was no particularsilent expressiveshape of the mouththat accompanied grunting. Hauser et al. (1993) found, similarly,that grunts were producedwith little separationof the lips. I found thatscreams and squeaks,in contrastto grunts, had low scores for mouthmatching, indicating that these vocalizationswere often accompaniedby extended mouth expressions, in this case grimaces. Girneys also had low scores for mouth matching,because they were often producedsimultaneously with extensive lipsmacking. SIGNAL COMPOSITION 1023

Matching mouth positions may carry no new information,being purely redundantwith the vocalization, whereas an extended expressive mouth position may provide additional information.For example, an expressive mouth position such as a prolonged, exaggeratedgrimace might reflect a higher intensity response than a more utilitarian(smaller, briefer) grimace that simply matcheda scream (see Maestripieri,1997; Partan,1998). Marler (1992) discussed the idea that certain components of a signal may reflect intensityof response while othersreflect referential information.

Conclusion The detailed associations among visual and vocal components of rhesus monkey expressions demonstratea method for quantifyingthe structureof multichannelsignals. The results provide new insights into signal structure and quantitativesupport for previous descriptionsof rhesus behaviorfound in the literature.That particularvocalizations are not always paired with the exact same visual expressions reflects the fact that all behavior is probabilistic (as Altmann, 1965, discussed for sequences of behavior). However,some vocal expressionswere consistentlypaired with predictable visual components;this was more often the case for screams and squeaks thanfor barksand pant-threats. Silent expressionswere the most variablein terms of the numbersof associatedvisual components.

References Adams, D.B. & Schoel, M.W. (1982). A statisticalanalysis of the social behaviorof the male stumptailmacaque (Macaca arctoides).- Am. J. Primatol.2, p. 249-273. Altmann,S.A. (1962). A field study of the sociobiology of rhesus monkeys,Macaca mulatta. - Ann. N.Y. Acad. Sci. 102, p. 338-435. - (1965). Sociobiology of rhesus monkeys. II: Stochasticsof social communication. J. theoret.Biol. 8, p. 490-522. - (1967). The structureof primatesocial communication.- In: Social Communication Among Primates (S.A. Altmann, ed.). The University of Chicago Press, Chicago, p. 325-362. Andrew, R.J. (1963). The origin and evolution of the calls and facial expressions of the primates.-Behaviour 20, p. 1-109. Argyle, M. & Cook, M. (1976). Gaze and mutual gaze. - CambridgeUniversity Press, Cambridge. Boccia, M.L. (1986). Groomingsite preferencesas a form of tactile communicationand their role in the social relations of rhesus monkeys. - In: Currentperspectives in primate social dynamics(D.M. Taub& F.A. King, eds). VanNostrand Reinhold Co., New York. 1024 PARTAN

Bradbury,J.W. & Vehrencamp,S.L. (1998). Principlesof animalcommunication. Sinauer Associates, Inc., Sunderland,Massachusetts. Conner,W.E. (1987). Ultrasound:Its role in the courtshipof the arctiidmoth, Cynia tenera. - Experientia43, p. 1029-1031. Darwin, C. 1872/1897. The expression of the emotions in man and animals. - Second edition. D. Appleton & Co., New York. Deputte, B.L. (1994). Ethological study of yawning in primates.I. Quantitativeanalysis and study of causation in two species of Old World monkeys (Cercocebus albigena and Macacafascicularis). - Ethology 98, p. 221-245. - (2000). Primate socialization revisited: Theoretical and practical issues in social ontogeny.- Adv. Study Behavior29, p. 99-157. Drickamer,L.C. (1975). Quantitativeobservation of behavior in free-rangingMacaca mulatta: Methodology and aggression. -Behaviour 55, p. 209-236. Drummond,H. (1981). The natureand descriptionof behaviorpatterns. - In: Perspectives in ethology, Vol. 4 (P.P.G.Bateson & P.H. Klopfer,eds). Plenum, New York,p. 1-33. Fusani, L., Hutchison, R.E. & Hutchison, J.B. (1997). Vocal-posturalco-ordination of a sexually dimorphicdisplay in a monomorphicspecies: The Barbarydove. - Behaviour 134, p. 321-335. Goldfoot, D.A. (1982). Multiplechannels of sexual communicationin rhesus monkeys:Role of olfactory cues. - In: Primatecommunication (C.T. Snowdon, C.H. Brown & M.R. Petersen,eds). CambridgeUniversity Press, Cambridge,p. 413-428. Gouzoules, S., Gouzoules, H. & Marler, P. (1984). Rhesus monkey (Macaca mulatta) screams:Representational signaling in the recruitmentof agonistic aid. - Anim. Behav. 32, p. 182-193. Green, S. (1975). Variationof vocal pattern with social situation in the Japanese monkey (Macaca fuscata): A field study. - In: Primate behavior (L.A. Rosenblum, ed.). Academic Press, New York,p. 1-102. - & Marler, P. (1979). The analysis of animal communication.- In: Handbook of behavioralneurobiology, Vol. 3; Social behaviorand communication(P. Marler& J.G. Vandenbergh,eds). Plenum Press, New York. Hauser, M.D. (1991). Sources of acoustic variationin rhesus macaque (Macaca mulatta) vocalizations.- Ethology 89, p. 29-46. - (1993). Rhesus monkey copulation calls: Honest signals for female choice? - Proc. R. Soc. Lond. B 254, p. 93-96. - (1996). Vocal communicationin macaques:Causes of variation.- In: Evolutionary ecology and behaviorof macaques(J.E. Fa & D. Lindburg,eds). CambridgeUniversity Press, Cambridge. - , Evans, C.S. & Marler,P. (1993). The role of articulationin the productionof rhesus monkey,Macaca mulatta,vocalizations. - Anim. Behav. 45, 423-433. - & Marler,P. (1993). Food-associatedcalls in rhesus macaques (Macaca mulatta):I. Socioecological factors.- Behav. Ecol. 4, p. 194-205. van der Heijden,P.G., de Vries, H. & van Hooff, J.A.R.A.M.(1990). Correspondenceanalysis of transitionmatrices, with special attentionto missing entriesand asymmetry.- Anim. Behav. 40, p. 49-64. Hinde, R.A. (1966). Ritualizationand social communicationin rhesus monkeys. - Phil. Trans.Roy. Soc. London 251, p. 285-294. SIGNALCOMPOSITION 1025

& Rowell, T.E. (1962). Communicationby posturesand facial expressionsin the rhesus monkey (Macaca mulatta).- Proc. Zool. Soc. Lond. 138, p. 1-21. van Hooff, J.A.R.A.M. (1962). Facial expressions in higher primates.- Symp. Zool. Soc. Lond. 8, p. 97-125. (1967). The facial displays of the Catarrhinemonkeys and apes. - In:Primate ethology (D. Morris,ed.). Aldine Pub. Co., Chicago, p. 7-68. - (1973). A structuralanalysis of the social behaviour of a semi-captive group of chimpanzees.- In: Social communicationand movement:Studies of interactionand expression in man and chimpanzee (M.V. Cranach& I. Vine, eds). Academic Press, London, p. 75-162. ( 1982). Categoriesand sequences of behavior:Methods of descriptionand analysis.- In: Handbookof methods in nonverbalbehavior research(K.R. Scherer & P. Ekman, eds). CambridgeUniversity Press, Cambridge,p. 362-439. Itani,J. (1963). Vocal communicationof the wild Japanesemonkey. - Primates4, p. 11-66. Jensen, G.D. & Gordon,B.N. (1970). Sequencesof mother-infantbehavior following a facial communicativegesture of pigtail monkeys.- Biol. Psychiatry2, p. 267-272. Kaldor,E.L. (1996). The meaning and social function of grunt and girney vocalizations in Macaca mulatta.- Msc thesis, Universityof California,Los Angeles. Kalin, N.H., Shelton, S.E. & Snowdon, C.T. (1992). Affiliativevocalizations in infantrhesus macaques(Macaca mulatta).- J. Comp. Psychol. 106, p. 254-261. Leger,D.W. & Didrichsons,I.A. (1994). An assessmentof datapooling and some alternatives. - Anim. Behav. 48, p. 823-832. Lindburg,D.G. (1971). The rhesus monkey in North India: An ecological and behavioral study. In: Primatebehavior: Developments in field research(L.A. Rosenblum,ed.). Academic Press, New York,p. 1-106. Machlis, L., Dodd, P.W.D. & Fentress, J.C. (1985). The pooling fallacy: Problems arising when individualscontribute more than one observationto the data set. - Z. Tierpsy- chol. 68, p. 201-214. Maestripieri, D. (1997). Gestural communication in macaques: Usage and meaning of nonvocal signals. Evol. Communic. 1, p. 193-222. & Wallen, K. (1997). Affiliative and submissive communicationin rhesus macaques. - Primates38, p. 127-138. Marler, P. (1965). Communicationin monkeys and apes. - In: Primate behavior: Field studies of monkeys and apes (I. DeVore, ed.). Holt, Rinehardtand Winston,New York, p. 544-584. - (1982). Avian and primate communication:The problem of natural categories. Neurosci. & Biobehav.Rev. 6, p. 87-94. - (1992). Functions of arousal and emotion in primatecommunication: a semiotic approach.- In: Topics in primatology(T. Nishida, W.C. McGrew,P. Marler,M. Pickford & F.B. deWaal,eds). Univ. of Tokyo Press, Tokyo, p. 225-233. - & Hamilton,W.J. III (1966). Mechanismof animal behavior.- John Wiley & Sons, NY. Martin, P. & Bateson, P. (1993). Measuring behaviour. - Cambridge University Press, Cambridge,UK. Mason, W.A. (1985). Experientialinfluences on the developmentof expressive behaviorsin rhesus monkeys. - In: The developmentof expressivebehavior: Biology-environment interactions(G. Zivin, ed.). Academic Press, New York,p. 117-152. 1026 PARTAN

Massaro, D.W. (1998). Perceiving talking faces: From speech perception to a behavioral principle. The MIT Press, Cambridge,Massachusetts. Maxim, P.E. (1982). Contexts and messages in macaque social communication.- Am. J. Primatol.2, p. 63-85. - (1985). Multidimensionalscaling of macaquesocial interaction.- Am. J. Primatol.8, p. 279-288. McGurk,H. & Macdonald,J. (1976). Hearinglips and seeing voices. - Nature264, p. 746- 748. Mitchell, G. (1972). Looking behaviorin the rhesus monkey. J. Phenomenol.Psychol. 3, p. 53-67. Morton, E.S. (1977). On the occurrence and significance of motivation-structuralrules in some bird and mammalsounds. - Am. Nat. 111, p. 855-869. Ohala,J.J. (1984). An ethological perspectiveon common cross-languageutilization of Fo of voice. - Phonetica40, p. 1- 18. Partan,S.R. (1998). Multimodalcommunication: The integrationof visual and vocal signals by rhesus macaques.- Dissertation,University of California. - & Marler,P. (1999). Communicationgoes multimodal.- Science 283, p. 1272-1273. - & (2002). The Umweltand its relevanceto animal communication.Introduction to special issue. -J. Comp. Psychol. 116, p. 116-119. Rawlins, R.G. & Kessler, M.J. (1985). Climate and seasonal reproductionin the Cayo Santiago macaques.- Am. J. Primatol.9, p. 87-99. - & - (1986). The history of the Cayo Santiago colony. - In: The Cayo Santiago macaques: History, behavior and biology (R.G. Rawlins & M.J. Kessler, eds). State Universityof New YorkPress, Albany,p. 13-45. Redican, W.K. (1975). Facial expressions in nonhumanprimates. - In: Primatebehavior: Development in field and laboratoryresearch (L.A. Rosenblum,ed.). Academic Press, New York,p. 103-194. Reynolds, V. (1976). The origins of a behavioralvocabulary: the case of the rhesus monkey. J. Theory Social Behav. 6, p. 105-142. Rowe, C. (1999). Receiver psychology and the evolution of multicomponentsignals. Anim. Behav. 58, p. 921-931. - & Guilford, T. (1996). Hidden colour aversions in domestic chicks triggered by pyrazineodours of insect warningdisplays. - Nature383, p. 520-522. Rowell, T.E. (1962). Agonistic noises of the rhesus monkey (Macaca mulatta). - Symp. Zool. Soc. Lond. 8, p. 91-96. - & Hinde, R.A. (1962). Vocal communicationby the rhesus monkey (Macaca mulatta). - Proc. Zool. Soc. Lond. 138, p. 279-294. Sade, D.S. (1973). An ethogramfor rhesus monkeys: I. Antitheticalcontrasts in posture and movement.- Am. J. Phys. Anthrop.38, p. 537-542. de Waal, F.B.M., van Hooff, J.A.R.A.M. & Netto, W.J. (1976). An ethological analysis of types of agonistic interactionin a captivegroup of Java-monkeys(Macacafascicularis). - Primates17, p. 257-290. Zeller, A.C. (1986). Comparisonof componentpatterns in threateningand friendly gestures in Macaca sylvanusof Gibraltar.- In: Currentperspectives in primatesocial dynamics (D.M. Taub& F.A. King, eds). Van NostrandReinhold Co., New York,p. 487-504. SIGNALCOMPOSITION 1027

- - (1996). The inter-playof organisationand facial communicationin the macaques.- In: Evolutionand ecology of macaquesocieties (J.E. Fa & D.G. Lindburg,eds). Cambridge UniversityPress, Cambridge.