Memory & Cognition 2002, 30 (7), 1096-1105 Presentation format and its effect on working memory PAULA GOOLKASIAN and PAUL W. FOOS University of North Carolina, Charlotte, North Carolina In three experiments, we examined the separate cognitive demands of processing and storagein work- ing memory and looked at how effectivethe coordination was when items for storage varied in format/ modality. A sentence verification task involving arithmetic facts was combined with a span task in- volving two to six items presented in picture, printed word, or spoken word format. The first two ex- periments were the same, except for the added requirement of articulation of the math sentence in Ex- periment 2. Experiment 3 varied the length of the span item and compared recall with recognition performance. The results showed that both spoken words and pictures produced superior recall and recognition, as compared with printed words, and are consistent with Baddeley and Logie’s (1999) and Mayer’s (2001)models of working memory. Also, the differencesin processing performance across spans variedwith the difficulty of the task but showed the strongest support for the resource allocationmodel (Foos, 1995). There has been much empirical interest in the processing In terms of input modality, Penney (1989) suggests a of information from stimuli presented in varying formats. model of verbal information in which auditory and visual However, the early work concentrated on either picture– information (pictures or words) are processed in separate word differences (Goolkasian & Park, 1980; Kroll & Cor- streams. Several studies have shown that when the mode rigan, 1981; Pellegrino, Rosinski, Chiesi, & Siegel, 1977; of presentation has been auditory rather than visual, supe- Smith & Magee, 1980) or auditory–visual differences. rior recall for auditory items occurs, and researchers have More recently, this work has been broadened in order to referred to this advantageas themodalityeffect(e.g., Greene, considerthe effects of multimedia presentation techniques 1985; Greene, Elliott, & Smith, 1988). This superior per- (Mayer, 2001) that includedvideo and sound, togetherwith formance consists of at least two components: a sustained illustrationsand textualmaterial. The present experiments sensory memory for auditory input and another that is extended the latter work by examining working memory brief and can be eliminatedby presentationof a similar au- for material presented in different modalities(auditory vs. ditory item (the suffix effect; Kellogg, 2001; Nicholls & visual) and different formats (pictures vs. printed words) Jones, 2002). in the same modality (visual). The long-term modality effect resides in the processing Picture–word studies(Smith & Magee,1980) have shown of auditory input that takes place in working memory a naming advantage for word processing but a picture ad- (Gardiner, Gardiner, & Gregg, 1983). Auditory input of vantage when problem-solving tasks or text-processing words results in superior performance when compared tasks were used. When Potter (1976) studied short-term with visual input of the same words. This long-term effect memory for pictures,she found that pictureswere detected is not eliminated by the presentation of a suffix item and better than printed words. Short-term memory for visual is found for both serial and free recall (Greene & Crowder, patterns has been found to resemble verbal memory in its 1986).In the present work, we investigatedthe advantages susceptibilityto similarity effects and in its capacity limi- of auditory inputand compared it with two visual formats, tation to around three or four objects (Luck & Vogel, 1997; pictures and printed words. Moreover, an attempt was Wheeler & Treisman, 2002). There are well-recognized made to separate the effects that were due to sensory plus differences in encoding stimuli across these two formats working memory (Experiment 1) from those that were due (Theios & Amrhein, 1989), and it is generally acknowl- only to working memory (Experiment 2 and Experi- edgedthat comprehensionof instructionalmaterials is en- ment 3). It was expected that in all conditions, auditory hanced when illustrationsappear together with textual ma- inputwould result in significantlybetter performance than terial (Glenberg & Langston,1992;Larkin & Simon, 1987). would presentation of the same printed words and, when the sensory memory componentwas eliminated,would re- sult in the same high performance as that expected with We thank Colleen Farrell, Matthew Hurley, and David Thao for their picture presentation. assistance with data collection and data analysis. Correspondence should be addressed to P. Goolkasian, Department of Psychology, Uni- Recently, Mayer and his colleagues (Mayer, 2001; versity of North Carolina, Charlotte, NC 28223 (e-mail: pagoolka@ Mayer& Sims, 1994)developeda cognitivetheory of multi- email.uncc.edu). media learning that explained why learning is often facil- Copyright 2002 Psychonomic Society, Inc. 1096 PRESENTATION FORMAT 1097 itated by explanations that incorporate at least two for- processing accuracy would occur with increased demands mats. Mayer argued that there are two parallel channels, for storage, but fewer stored items would be recalled. visual/pictorial and auditory/verbal, for processing infor- We measured the storage versus processing tradeoff by mation through working memory. Although the paths for looking at the span effect and what would happen to recall, processing pictures and spoken words are direct and iso- RTs, and verification accuracy when participants kept in lated within a channel, processing of printed words was mind from two to six items presented in different presen- hypothesizedto involve both channels in a more complex tation format conditions.A large span effect on all of these manner.Accordingto Mayer,format/modalityeffects should measures would support the first version of the resource- be evident in both the storage and the processing compo- sharing model, whereas an absence of a span effect on ver- nents of the working memory task. ification accuracy would support the allocation guideline The present research extended prior efforts by examin- hypothesis. ing both format and modality effects in working memory In sum, the present experimentstested the hypothesisthat (Baddeley& Hitch, 1974; Baddeley& Logie,1999).Work- concrete nouns presented as pictures or as spoken words ing memory was thought to be a multicomponent system would result in better memory for those nouns, as compared in which a central executive controls and regulates work- with printedwords. Such a findingwould also supportMay- ing memory by coordinatinga visuospatialsketchpadand er’s (2001) predictions regarding the complexity of pro- a phonologicalloop.It was expectedthat format and modal- cessing printed words. In addition,presentationformat in- ity effects would be evident primarily in the operation of fluences on both processing and storage were examined. these two specialized stores, with picture formats repre- One version of the resource-sharing model predicts that as sented in the visuospatial store and auditory or linguistic span increases, performance on all measures will decline, information in the other. whereas the allocation guideline hypothesis predicts that In our experiments, we also examined the separate cog- processing accuracy will not decline as span is increased. nitive demands of processing and storage in working memory. The dual task was modeled on a complex oper- EXPERIMENT 1 ations span task used by La Pointe and Engle (1990). Our processing componentwas a sentence verificationtask in- Method volving arithmetic facts. Performance was recorded by Participants. The participants were 100 male and female students measuring reaction time (RT) and accuracy to each arith- from the University of North Carolina at Charlotte. They were vol- metic sentence. The storage component was a span task unteers who participated in the experiment to obtain extra credit points involvingtwo to six items presented for immediate recall. toward their psychology class grade. Twenty-four students partici- pated in Experiment 1, 23 in Experiment 2, and 53 in Experiment 3. Span items were presented in picture, printed word, or Stimulus materials. A complex operations span task was used. spoken word conditions. Although these conditions rep- The processing task required the participants to verify the accuracy resented variations in both format and modality,they were of a series of math facts presented in a sentence form. Following the considered in these experiments as levels of a presenta- participant’s keypress response, a concept appeared as a printed tion format variable. We were interested in the degree to word, a spoken word, or a picture, and the participants were required which presentationformat effects would influenceperfor- to recall from two to six of these when cued. mance in the dual task. Previous work (Goolkasian, 1996) The sentences for the processing task were questions adapted from the math task used by La Pointe and Engle (1990). Each sen- with a sentence verification task had shown a picture ad- tence contained two components: the multiplication or division of two vantage, but those format differences were attributed pri- integers and the addition or subtraction of an integer from the result. marily to encoding. Given the separate verbal and pictor- The numbers used in both components were integers between 1 and ial channels of processing through working memory