Learners' Choices and Beliefs About Self-Testing
Total Page:16
File Type:pdf, Size:1020Kb
Self-testing 1 Running Head: SELF-TESTING Learners’ choices and beliefs about self-testing Nate Kornell University of California, Los Angeles Lisa K. Son Barnard College In press at Memory. Please do not quote without permission. Address correspondence to: Nate Kornell UCLA Department of Psychology 1285 Franz Hall Los Angeles, CA 90095-1563 Phone: 310-825-7028 Fax: 310-206-5895 Email: [email protected] Self-testing 2 Abstract Students have to make scores of practical decisions when they study. We investigated the effectiveness of, and beliefs underlying, one such practical decision: the decision to test oneself while studying. Using a flashcards-like procedure, participants studied lists of word pairs. On the second of two study trials, participants either saw the entire pair again (pair mode) or saw the cue and attempted to generate the target (test mode). Participants were either asked to rate the effectiveness of each study mode (Experiment 1) or to choose between the two modes (Experiment 2). The results demonstrated a mismatch between metacognitive beliefs and study choices: Participants (incorrectly) judged that the pair mode resulted in the most learning, but chose the test mode most frequently. A post-experimental questionnaire suggested that self-testing was motivated by a desire to diagnose learning rather than a desire to improve learning. Keywords: Self-testing, testing effect, flashcards, judgments of learning Self-testing 3 Acknowledgments We thank Robert A. Bjork for his guidance and support and Bridgid Finn for her suggestions and help in conducting the experiments. Grant 29192G from the McDonnell Foundation to Robert A. Bjork supported this research. Self-testing 4 Learners’ choices and beliefs about self-testing Self-regulated learning (e.g., homework) requires students to make scores of decisions about how to study. Laboratory research has shown that these decisions are affected by many factors, including the difficulty of the materials (Metcalfe, 2002); the student’s goals (Thiede & Dunlosky, 1999); the amount of time pressure the student is under (Metcalfe & Kornell, 2003); a student’s age (Kuhn, 2002); and, of course, the type of decision to be made, whether it be strategic (e.g., should I make flashcards?), item- specific (e.g., do I need to review this chapter again?), or global (e.g., should I study tonight?). Many of the practical decisions that students face have not been addressed by experimental research, however (see, e.g., Kornell & Bjork, 2007). One such practical question, which is the focus of the present research, is whether or not to test oneself while studying. A large body of data suggests that self-testing enhances memory. This so-called testing effect refers to the finding that taking a test improves learning more than passively reading the same information (Glover, 1989; McDaniel & Fisher, 1991; Metcalfe, Kornell, & Son, 2007; Roediger & Karpicke, 2006b). The testing effect has been shown to occur when the test is followed by corrective feedback (e.g., Carrier & Pashler, 1992; Cull, 2000), and when it is not (Carpenter & DeLosh, 2005; Landauer & Bjork, 1978). Moreover, testing appears to be especially effective for promoting long-term learning (Hogan & Kintsch, 1971; Roediger & Karpicke, 2006a). In addition to enhancing memory directly, testing has a second benefit: It allows learners to accurately diagnose what they do and do not know. Making such diagnoses accurately can play an important role in guiding study decisions. Learners’ diagnoses of Self-testing 5 what they know are often assessed via Judgments of Learning (JOLs)—that is, ratings of how probable it is that one will remember an item on a future test. The correlation between the JOLs and actual memory increases when learners can test themselves as they study (e.g., Dunlosky & Nelson, 1992). To the degree that JOLs guide study decisions, accurate JOLs promote effective study decisions, including decisions about which items to study and how long to spend studying (Kornell & Metcalfe, 2006; Nelson, Dunlosky, Graf, & Narens, 1994), and even when to schedule study (Son, 2004). Beliefs Concerning Self-Testing Two recent studies suggest that people do not recognize the benefits of testing— instead, they seem convinced that presentations result in more learning than does testing (Roediger & Karpicke, 2006a; Agarwal, Karpicke, Kang, Roediger, & McDermott, 2008). In both studies, participants who read a text passage multiple times gave higher JOLs than participants who read the passage and then took one or more tests (Roediger & Karpicke, 2006a, used free recall tests; Agarwal et al., 2008, used cued recall tests). This evidence suggests that, at least in the context of reading text passages, participants thought re-reading was more effective than testing. There is evidence that in some situations people recognize the benefits of tests. In experiments on the generation effect—the finding that generating an answer, for example by unscrambling an anagram or filing in missing letters in a word, leads to better later recall than seeing the word presented whole (Hirshman & Bjork, 1988; Jacoby, 1978; Slamecka & Graf, 1978)—participants have been shown to give higher JOLs for words that they generated than for words that were presented (Begg, Vinski, Frankovich & Self-testing 6 Holgate, 1991; Mazzoni & Nelson, 1995)—or to give equal ratings on both types of trials (Maki, Foley, Kajer, & Willert, 1990). Thus there is evidence to suggest that in some situations, people believe they can learn more via re-reading than being tested. But do students choose to test themselves? To investigate this question, Son (2005) asked first grade students to study a set of cue- target synonym pairs (e.g. occupation - job). The participants were asked to judge how well they knew each pair, and then to decide, for each pair, whether to re-read the pair or test themselves. The first graders chose to test themselves on items that they felt they knew well, and preferred re-presentation on less well-known items, although they may have chosen tests on items that they thought could answer correctly, not as a study strategy but simply to impress their teachers. Choosing Self-Testing: A Pilot Experiment We wanted to follow up on the question of self-testing using a procedure that seemed to more strongly resemble real-world learning. Thus, we used a flashcard-like procedure. When learners use flashcards, which are a ubiquitous study tool (Kornell & Bjork, 2007; Kornell, in press), they generally cycle through a set of cards repeatedly, looking at the front of each card, testing themselves, and then looking at the back. Flashcard-like programs are becoming increasingly popular online. Asking participants to study computer-based flashcards seemed to be an appropriate experimental context in which to investigate self-testing. In the pilot study, the learning materials were English-Indonesian vocabulary (e.g., Left-Kiri, Hot-Panas, Late-Terlambat), which were presented for study on a computer. Participants were allowed to decide how they studied: They could choose to Self-testing 7 read the pairs intact (pair mode), or to have the cue presented alone before the target appeared (test mode). If a participant changed modes, which they could do at any time using buttons on the computer screen, the remaining items in the list would be presented in whichever mode was selected, unless the mode was changed again. There were three between-participant conditions: In addition to the condition in which participants could choose the study mode, there was an all-pair mode, in which there were no test trials, and an all-test mode, in which there were no pair trials. In the same way that students doing homework are under time pressure, our participants were given a time limit of 10 minutes to study, and they controlled the timing of presentations as they studied. The fixed time limit gave participants an incentive to manage their time wisely, because studying quickly translated into more study trials later (unlike most previous self-regulated study research; see Son & Metcalfe, 2000). There was a final test at the end of experiment. The results from the pilot study showed that when given the choice, participants began by studying in presentation mode but quickly switched to test mode. There was also a testing effect: On the final test, the recall accuracy of participants in the choice condition—who only tested themselves on an average of only 55% of the trials—matched that of participants in the all-test mode (mean performance = 63%), and surpassed that of participants in the all-pair mode (mean performance = 37%). Thus it appears that a mix of presentation trials and test trials were no less effective than constant test trials. Participants in the pilot experiment chose to test themselves The question addressed in the present research was why did they choose to do so? Tests enhance learning, as the pilot experiment demonstrated, but another reason to test oneself is as a way to diagnose one’s memory. In a survey of 472 UCLA undergraduates (Kornell & Self-testing 8 Bjork, 2007), for example, 91% reported testing themselves regularly while studying. Out of all respondents, 68% reported that they test themselves primarily to make a diagnosis of what they do and do not know. Thus students seem to be aware of the metacognitive benefits of testing (also see Dunlosky, Serra, Matvey, & Rawson, 2005). They seem less aware of the direct memory benefits of testing, however: only 18% reported testing themselves to enhance their learning directly. Thus the majority of students seem to view tests the way their teachers might: primarily as assessments, not learning tools. The Present Experiments In sum, previous research provides a puzzling picture of learners’ attitudes towards testing: People believe testing is less effective than re-reading (e.g., Roediger & Karpicke, 2006a), and yet, as our pilot results demonstrate, they choose self-testing preferentially over re-presentation nonetheless.