Research Report No. 2002-3 Knowing What You Know and What You Don’t: Further Research on Metacognitive Knowledge Monitoring Sigmund Tobias and Howard T. Everson www.collegeboard.com 07-1626.RD.CBRpt02-3Txt 4/11/07 1:38 PM Page i College Board Research Report No. 2002-3 Knowing What You Know and What You Don’t: Further Research on Metacognitive Knowledge Monitoring Sigmund Tobias and Howard T. Everson College Entrance Examination Board, New York, 2002 07-1626.RD.CBRpt02-3Txt 4/11/07 1:38 PM Page ii Acknowledgments This work was funded by the College Board. Earlier versions of this paper, and the studies reported herein, were presented in April 1999 at the annual meeting of Sigmund Tobias is a distinguished scholar in the the American Educational Research Association in Division of Psychological and Educational Services in Montreal. The findings and opinions expressed in this the Graduate School of Education at Fordham article are our own and do not reflect the positions or University at Lincoln Center. policies of the College Board. Howard Everson is Vice President for Academic The research reported here was conducted in collabora- Initiatives and Chief Research Scientist at the College tion with our colleagues: Lourdes Fajar and Katherine Board. Santos conducted Study I in partial fulfillment of the requirements for the Seminar in Educational Research Researchers are encouraged to freely express their at the City College of New York; Rhonda Romero con- professional judgment. Therefore, points of view or opin- ducted Study II in partial fulfillment of the requirements ions stated in College Board Reports do not necessarily for the Seminar in Educational Research at the City represent official College Board position or policy. College of New York; Edgar Feng conducted Study III in partial fulfillment of the requirements for the Seminar in Educational Research at the City College of New The College Board: Expanding College Opportunity York; Fred Dwamena conducted Study IV in partial ful- The College Board is a national nonprofit membership fillment of the requirements for the Seminar in association dedicated to preparing, inspiring, and connect- Educational Research at the City College of New York; ing students to college and opportunity. Founded in 1900, Study V was supported by a contract with the Battelle the association is composed of more than 4,200 schools, Corporation relying on resources made available by the colleges, universities, and other educational organizations. Navy Personnel Research and Development Center; Each year, the College Board serves over three million stu- Harold Ford conducted Study VII in partial fulfillment dents and their parents, 22,000 high schools, and 3,500 of the requirements for the qualifying examination at colleges, through major programs and services in college Fordham University’s Graduate School of Education; admissions, guidance, assessment, financial aid, enroll- Julie Nathan conducted Study X as her doctoral disser- ment, and teaching and learning. Among its best-known tation research at Fordham University’s Graduate programs are the SAT®, the PSAT/NMSQT®, and the School of Education; Hyacinth Njoku conducted Study Advanced Placement Program® (AP®). The College Board XI in partial fulfillment of the requirements for the is committed to the principles of equity and excellence, and qualifying examination at Fordham University’s that commitment is embodied in all of its programs, ser- Graduate School of Education. vices, activities, and concerns. We are grateful to both Patrick C. Kyllonen and Irving For further information, contact www.collegeboard.com Katz of Educational Testing Service for their helpful comments in earlier versions of this paper. Additional copies of this report (item #993815) may be obtained from College Board Publications, Box 886, New York, NY 10101-0886, 800 323-7155. The price is $15. Please include $4 for postage and handling. Copyright © 2002 by College Entrance Examination Board. All rights reserved. College Board, Advanced Placement Program, AP, SAT, and the acorn logo are registered trademarks of the College Entrance Examination Board. PSAT/NMSQT is a registered trademark jointly owned by both the College Entrance Examination Board and the National Merit Scholarship Corporation. Visit College Board on the Web: www.collegeboard.com. Printed in the United States of America. 07-1626.RD.CBRpt02-3Txt 4/11/07 1:38 PM Page iii Study V: The Impact of Knowledge Contents Monitoring, Word Difficulty, Dynamic Assessment, and Ability I. Introduction ................................................1 on Training Outcomes.............................7 The Importance of Participants and Procedures ....................7 Knowledge Monitoring..........................1 Materials..................................................8 II. Assessing Knowledge Monitoring ................1 Results and Discussion ............................8 Analysis of Monitoring Accuracy.............2 Study VI: Knowledge Monitoring III. Knowledge Monitoring Accuracy and Scholastic Aptitude .........................10 and the Developing Reader .......................3 Participants and Procedures ..................10 Study I: Knowledge Monitoring Accuracy and Reading in Bilingual Results and Discussion ..........................10 Elementary School Students.....................3 Study VII: Knowledge Monitoring, Participants and Procedures ....................3 Scholastic Aptitude, College Grades ......10 Results and Discussion ............................3 Participants and Procedures ..................10 Study II: Reading, Help Seeking, Results and Discussion ..........................10 and Knowledge Monitoring .....................3 Summary: Knowledge Monitoring Participants and Procedures ....................4 and Academic Ability ............................11 Results and Discussion ............................4 V. Knowledge Monitoring: Is It a Domain General or Specific Ability?.....................11 Summary: Knowledge Monitoring and Reading.............................................5 Study VIII: Math and Vocabulary KMAs, SAT® Tests, and Grades: Relationships ....11 Study III: Strategic Help Seeking in Mathematics ............................................5 Participants and Procedures ..................11 Participants and Procedures ....................5 Results and Discussion ..........................12 Results and Discussion ............................5 Study IX: Knowledge Monitoring, Reading Ability, and Prior Knowledge ....12 IV. Knowledge Monitoring and Ability .............6 Participants and Procedures ..................12 Study IV: Triarchic Intelligence and Knowledge Monitoring............................6 Results and Discussion ..........................12 Participants and Procedures ....................7 Other Relevant Studies ..............................13 Results and Discussion ............................7 VI. Self-Reports of Metacognition and Objective Measures of Knowledge Monitoring ..............................................14 07-1626.RD.CBRpt02-3Txt 4/11/07 1:38 PM Page iv Summary: Relationships of KMA Figures Scores and Self-Report Measures 1. A componential model of metacognition..........1 of Metacognition ...................................17 2. The interaction between knowledge monitoring ability Study X: An Investigation of the and help seeking reviews. .................................4 Impact of Anxiety on Knowledge Monitoring ............................................17 3. The relationship between knowledge monitoring and help seeking reviews in Study III. .............................5 Participants and Procedures ..................18 4. Relationship of KMA scores with Results and Discussion ..........................18 ASVAB performance.........................................9 5. Relationship between test anxiety Study XI: Cross-Cultural Perspective and help seeking. ............................................20 on Knowledge Monitoring.....................18 6. Math KMA means by type of problem Participants and Procedures ..................19 reviewed for Nigerian and American male students..................................................21 Results and Discussion ..........................19 VII. Metacognitive Knowledge Monitoring and Strategic Studying Among Secondary Students .....................19 VIII.General Discussion....................................21 IX. Suggestions for Future Research ................22 References .........................................................23 Tables 1. Two Prototypical KMA Item Score Patterns. ..................................................2 2. Correlations Among ASVAB Measures and KMA Scores by Word Difficulty. ...............8 3. Descriptive Statements for KMA Scores and GPA. ...............................................9 4. Correlations of Math and Verbal KMA Score and GPA......................................12 5. Descriptive Statistics, Correlations, and Beta Weights with Posttest Score. ............13 6. Correlations Between KMA and Learning in Training Course...........................14 7. Correlations of Verbal and Math KMA Scores, Metacognitive Self-Report Scales, Teacher Ratings, and SAT I-V and SAT I-M Scores. ..................16 8. Correlations of MSLQ, LASSI, KMA Scores, and GPA. ............................................16 07-1626.RD.CBRpt02-3Txt 4/11/07 1:38 PM Page 1 I. Introduction The Importance of Knowledge Monitoring For more than a decade our program of research has concentrated