Modality Effects in the Coding Reproduction of Rhythms
Total Page:16
File Type:pdf, Size:1020Kb
Memory & Cognition 1989. 17 (4), 373-383 Modality effects in the coding and reproduction of rhythms ARTHUR M. GLENBERG, STUART MANN, LISA ALTMAN, TIM FORMAN, and SEAN PROCISE University of Wisconsin, Madison, Wisconsin The temporal coding assumption is that time of presentation is coded more accurately for audi tory events than for visual events. This assumption has been used to explain the modality effect, in which recall of recent auditory events is superior to recall of recent visual events. We tested the temporal coding assumption by examining the coding and reproduction of quintessentially temporal stimuli-rhythms. The rhythms were produced by sequences of short and long auditory stimuli or short and long visual stimuli; in either case, the task was to reproduce the temporal sequence. The results from four experiments demonstrated reproduction of auditory rhythms superior to that of visual rhythms. We conclude that speech-based explanations of modality ef fects cannot accommodate these findings, whereas the findings are consistent with explanations based on the temporal coding assumption. In classic information processing models of memory Glavanov, and Murdock (1981) presented stimuli in dif (e.g., Atkinson & Shiffrin, 1968), the input modality of ferent spatial locations. Auditory stimuli were recalled bet information plays little role after the information is coded ter than were visual stimuli when recall was by temporal in short-term store. In fact, early work on the modality position, but this difference was eliminated when recall effect (e.g., Crowder & Morton, 1969) seemed to con was by spatial location. Glenberg and Swanson (1986) firm this limited role: Auditory information is recalled manipulated the duration of the intervals separating suc better than is visual information, but only at the end of cessive to-be-remembered items on a list. This manipu a list of items, supposedly indicating recall from a sen lation greatly affected recall of auditory items (enhanc sory memory or a short-term memory. In the interven ing the recency effect), but had little effect on the recall ing years, we have found, however, that modality plays of visual items. Glenberg and Fernandez (1988) discov a role throughout the cognitive system. For example, ered modality effects in an order judgment task. After a J. M. Gardiner and Gregg (1979) have demonstrated long list of stimuli was presented, subjects judged if the items lasting modality effects that are robust in the face of in in a re-presented pair were in the correct order. They terference. Glenberg and Fernandez (1988) reported mo found an auditory superiority both for the stimuli pre dality effects in position judgments and in frequency es sented at the end ofthe list and for those presented at the timations. In this article, we report a new type of modality beginning of the list. Furthermore, the auditory superi effect, one that offers support for a general interpretation ority increased with duration of the interitem intervals. ofmodality effects in memory. The new effect is that the Thus, across these paradigms, the temporal coding as coding and reproduction of rhythmic stimuli are more ac sumption is supported by the fact that there was greater curate for auditory rhythms than for visual rhythms. This sensitivity to temporal manipulations of auditory events supports the interpretation that modality effects reflect the than there was to temporal manipulations of visual events fact that coding of temporal information for auditory (but see Crowder and Greene, 1987, for a dissenting events is superior to that for visual events. opinion). Glenberg and Swanson (1986) proposed an account of These demonstrations of effects consistent with the tem modality effects based on what we will call the temporal poral coding assumption rely on relatively long-term codingassumption: Coding of time of presentation is more memory. Even in the immediate recall of a short list of accurate for auditory events than for visual events. Evi items, the modality effect is enhanced by serial recall, dence for this claim can be found in a variety of paradigms which delays recall ofthe end-of-list items where the lar using a variety ofmanipulations. For example, Metcalfe, gest modality effect is found. The temporal coding as sumption leads us to expect, however, effects much closer This research was funded by National Science Foundation Grant BNS to the perception of the events. That is, it proposes that 8416300 and University of Wisconsin Graduate School Grant 890200 time ofoccurrence is coded (not just remembered) more awarded to Arthur Glenberg. We thank Michael Davis, Lisa Knudson, accurately for auditory than for visual stimuli. This ex Nancy Lightfoot, and Rebecca Mauer for their help in designing these pectation led us to look for modality effects in a domain experiments. Requests for reprints may be sent to Arthur G1enberg, Department of Psychology, University of Wisconsin, Madison, WI that is quintessentially temporal-the perception and re 53706. production of rhythms. 373 Copyright 1989 Psychonomic Society, Inc. 374 GLENBERG, MANN, ALTMAN, FORMAN, AND PROCISE A rhythm is produced by a sequence of stimuli that can the duration is perceived more accurately than is the du be identicalexcept for their temporal durations. Thus, per ration of the visual stimulus). ceiving a rhythm requires coding of temporal informa We tested these ideas by manipulating the background. tion and not much more. Our expectation is that rhythms In the auditory-noise-background condition, the stimuli produced by a sequence of long- and short-duration au (both auditory and visual sequences) were presented while ditory stimuli will be easier to perceive and reproduce than white noise was being played. The white noise should will identical rhythms produced by a sequence of long decrease the change from background for the auditory and short-duration visual stimuli.' Inthe General Discus stimulus and thereby reduce performance on the auditory sion section, we present the implications of our findings sequences. In the visual-noise-background condition, the for explanations of the modality effect. stimuli were presented while visual noise (flickering ran Inour rhythm task, subjects were presented with a se dom dot patterns) appeared on the display screen. This quence of n (n = 5-8) stimuli. Each stimulus was pre background should selectively reduce performance of the sented for either a long duration (750 msec) or a short visual stimuli. Finally, in the no-background condition, duration (250 msec), and each was followed by an empty, the stimuli were presented in a quiet, dark environment. short (250 msec) interstimulus interval. For an auditory Our reasoning did not depend on equating the auditory sequence, the stimuli were long- and short-duration tones. and visual backgrounds or changes in the backgrounds. For a visual sequence, the stimuli were asterisks that were Instead, we intended to determine if changing the back presented on a display terminal for a long or short dura ground in modality selectively affects coding and repro tion. Immediately following the last stimulus, subjects duction of rhythms presented in that modality. reproduced the rhythm, using button presses to indicate the sequence of long- and short-duration stimuli. Method This task is not ideal for studying the perception and Subjects. The subjects were 20 students enrolled in introductory coding of time of presentation; there is still a memory psychology courses at the University of Wisconsin-Madison. They component. The memory component is minimized, how received credit toward a research requirement for participating in the experiment. ever, in that there are only two stimuli (long and short), Materials. The subjects were seated approximately 55 ern in front and the retention interval is very short. of a visual display terminal and two loudspeakers. All subjects wore In overview, we conducted four experiments. Experi a limited-vision mask that restricted vision to a 6x 12 em area of ments 1 and 2 demonstrated the basic finding that per the display screen. This mask was used to eliminate ambient visual formance is better for auditory rhythms than for visual stimulation in the no-background condition, but all subjects wore rhythms. Inaddition, these experiments ruled out expla the mask throughout the experiment. The auditory stimulus was a tone of complex composition.? The nations of the effect based on modality differences in the intensity of the tone at the ear was approximately 77 dB(A). The detectability of onsets and offsets of auditory and visual tone was presented for either 250 msec (short) or 750 msec (long). stimuli. In Experiment 3, we explored the possibility that During the block of trials corresponding to the auditory-noise the effect is due to response requirements and that it may background condition, white noise was played continuously at an reflect differential experience in coding auditory and intensity of approximately 75 dB(A). visual rhythms. Experiment 4 demonstrated a correspon The visual stimulus was a single asterisk (produced using an Apple II's Turtlegraphics font) centered in the visible area of the dence between the rhythm modality effect and the mo display screen. In the visual-noise-background condition, a flick dality effect found in recall tasks. ering random dot pattern appeared on the display screen, except for a .5 x 1.2 cm rectangle in which the asterisk appeared. The ran dom dot pattern appeared after the ready signal (see below) and EXPERIMENT 1 was presented for 500 msec before the first stimulus and through out the presentation of the stimuli. Experiment 1 was designed to determine if auditory su Each sequence began with a ready signal consisting of a warning periority in the rhythm task is due to perceived change tone, a visual message indicating whether the following sequence from background. OUT experience in the task indicated would consist of tones or asterisks, andthe number of tones or aster that the auditory stimulus is more alerting than is the visual isks (five, six, or seven).