Original article doi: 10.1111/j.1365-2729.2009.00335.x The modality effect tested in children in a user-paced multimedia environmentjcal_335 132..142 M.J. Witteman*† & E. Segers* *Behavioural Science Institute, Radboud University Nijmegen, Nijmegen, The Netherlands †Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands Abstract The modality learning effect proposes that learning is enhanced when information is pre- sented in both the visual and the auditory domains (e.g. pictures and spoken information) compared with presenting information solely in the visual channel (e.g. pictures and written text). Most of the evidence for this effect comes from adults in a laboratory setting. Therefore, we tested the modality effect with 80 children in the highest grade of elementary school in a naturalistic setting. In a between-subjects design, the children either saw representational pictures with speech or representational pictures with text. Retention and transfer knowledge was tested at three moments: immediately after the intervention, one day after and after one week. The present study did not find any evidence for a modality effect in children when the lesson was learner-paced. Instead, we found a reversed modality effect directly after the intervention for retention. A reversed modality effect was also found for the transfer questions one day later. This effect was robust, even when controlling for individual differences. Keywords long term effects, individual differences, modality effect, multimedia learning, primary school, user-paced. adults, often on college students. It is not clear, however, Introduction whether these results can be generalized to children. The information age is in full swing, and digital media The goal of the present study is to test the modality influence more and more of our daily lives. Digital effect in children and look at the influence of learner and media has also found its way into classrooms, and is cognitive characteristics. becoming increasingly popular. The computer provides another dimension that books cannot offer: in addition to reading information, it is also possible to listen to Literature review spoken information. Both speech and text can be Already before the use of the computer became wide- accompanied by pictures. An important question is spread, research had looked at the effects of combining whether people learn more if information is presented auditory and visual stimuli (e.g. Penney 1975). An in two modalities (visual and auditory; pictures and influential theory in this field is the cognitive theory of speech) rather than one (visual; pictures and text). Most multimedia learning (Mayer 2001, 2003). This theory is research that addresses this question has focused on based on the dual coding theory by Paivio (1986) and Baddeley’s (2000) working memory model. The dual Accepted: 20 September 2009 coding theory poses that there are separate channels Correspondence: Marijt Witteman, Max Planck Institute for Psycholinguistics, P. O. Box 310, 6500AH Nijmegen, The Netherlands. for the processing of visual and verbal presentations Email: [email protected] and works under the assumption that the amount of 132 © 2010 Blackwell Publishing Ltd Journal of Computer Assisted Learning (2010), 26, 132–142 The modality effect in children 133 processing that can take place within each channel is peers in theAT-condition. The effect sizes indicated that extremely limited (Paivio 1986). these effects were substantial. Therefore, the authors Baddeley’s working memory model consists of four found evidence for the existence of the modality effect. distinct parts. Three parts, the phonological loop, visu- Another study that tested the modality effect was ospatial sketchpad and episodic buffer, are controlled conducted by Brunken et al. (2004). In two within- by a fourth part, a supervisor system called the central subjects studies, they tested two groups of 10 female executive. The phonological loop processes auditory students. The research focused on the influence of information, the visuospatial sketchpad takes care of the induced cognitive load on the modality effect. The par- visual and spatial information, and the episodic buffer ticipants completed a pre-test and then participated in integrates everything and adds time sequencing. The an intervention in which they learned about the human latter part is thought to have links to long-term memory. circulatory system or about Florence. There were two The cognitive theory of multimedia learning (Mayer conditions: visual (pictures and on-screen text) and 2001) poses that there are two different channels audiovisual (pictures and speech) Afterwards, their through which information can enter the brain: a visual knowledge was tested again. The participants in the channel and an auditory channel. In a multimedia pre- visual condition got significantly lower scores than the sentation, both channels are used. Therefore, informa- participants in the audio–visual condition: a modality tion can be divided between two channels. Presenting effect. This held for the primary (learning) task and the all the information to one channel will only create cog- secondary (distracting) task. nitive overload, as working memory capacity is limited. Other studies, however, have found conflicting Evidence for this phenomenon is also found in cognitive results with regard to the modality effect (e.g. Dean load theory (Paas et al. 2004). If there is cognitive over- et al. 1988; Mann et al. 2002). Tabbers et al. (2004) load, the brain cannot process all the incoming informa- tested 111 second year university students in a class- tion, resulting in inferior learning performance. room setting. The participants saw diagrams on a com- According to the modality effect, it is possible to both puter with either auditory or visual instructions. retain more information (retention benefits) and foster Afterwards, the participants completed retention and deeper learning (transfer benefits) when information is transfer tests. The results showed a reversed modality presented in two channels. This has been tested with effect, as the participants scored better on retention tests animations or pictures (visual domain) accompanied by when the diagrams were paired with visual cues. In the either text (also in the visual domain) or narration (audi- auditory condition, the participants got lower scores on tory domain). The effect is only present, however, when both retention and transfer tests. the two streams of information are complementary: pre- An important distinction between this study and senting additional information that is redundant, will those mentioned previously can be found in the pacing not aid learning (Mayer et al. 2001). of the information. Whereas the former was system- paced, Tabbers et al. (2004) allowed their participants to process the information at their own speed (learner- The modality effect tested with adults paced). The authors offered this distinction as an expla- The existence of the modality effect in computer- nation for the reversed modality effect found in their learning environments has been tested on several occa- study, suggesting that when learners can decide when to sions. Mayer and Moreno (1998), for example, start the spoken information, it is possible to process the conducted two experiments with college students. The pictorial information and the spoken information sepa- participants either saw an animation on a computer rately Therefore, the advantage of presenting informa- screen accompanied by auditory information (AN- tion in two channels disappears. Moreover, the authors condition), or an animation accompanied by text (AT- added that it was easier for the participants in the visual condition). Experiment 1 focused on meteorological condition to switch between the picture and the phenomena, while experiment 2 focused on car on-screen text because these were both present at the mechanics. The two experiments yielded the same basic screen throughout the intervention. For the audio files, conclusions: on both retention and transfer questions, however, it was impossible to switch back and forth the participants in the AN-condition outperformed their between the spoken information and the pictures on the © 2010 Blackwell Publishing Ltd 134 M.J. Witteman & E. Segers screen because the spoken information disappears modality effect, also in children (e.g. Mousavi et al. immediately after it is presented. 1995; Jeung et al. 1997). This effect was larger for studies that used high interactivity elements (with inter- activity defined as ‘the extent to which the learning task The modality effect tested with children requires the student to hold several related chunks of Less is known about the modality effect in children. to-be-learned information in working memory simulta- Only a few studies focused on children in primary neously in order to comprehend then learn the concept school, even though this group’s main focus during the or procedure’, p. 320), and for system-paced studies day is studying. By extending the research to children, it (compared with user-paced). Age did not have a signifi- is possible to learn more about the developmental cant effect. The modality effect was largest for students process of the modality effect. Moreover, it is possible that learned with virtual reality, followed by partici- to study the modality effect in a more heterogeneous pants in computer-based studies. The studies that used group as not all children in elementary