Consequential Validity and Social Studies Education: An
Total Page:16
File Type:pdf, Size:1020Kb
CONSEQUENTIAL VALIDITY AND SOCIAL STUDIES EDUCATION: AN EXAMINATION OF STANDARDS, ASSESSMENT POLICIES, AND TEACHER PREPARATION A Dissertation Submitted to the Graduate Faculty of the North Dakota State University of Agriculture and Applied Science By Michael Scott Learn In Partial Fulfillment of the Requirements for the Degree of DOCTOR OF PHILOSOPHY Major Program: Education Option: Institutional Analysis April 2019 Fargo, North Dakota North Dakota State University Graduate School Title Consequential Validity and Social Studies Education: An Examination of Standards, Assessment Policies, and Teacher Preparation By Michael Learn The Supervisory Committee certifies that this disquisition complies with North Dakota State University’s regulations and meets the accepted standards for the degree of DOCTOR OF PHILOSOPHY SUPERVISORY COMMITTEE: Brent Hill Chair Claudette Peterson Justin Wageman Christopher Whitsel Approved: April 23, 2019 Chris Ray Date Department Chair ABSTRACT Educational quality is a way to influence the future of the American economy (Hanushek, 1986). Large-scale assessments are designed to determine quality in education by measuring student achievement. A connection exists between the standards, teachers, and assessments that form a system of accountability within education. State and national accountability policies place value in certain educational fields, thereby preferring some while excluding others. As a result, accountability systems influence the field of social studies in several unanticipated ways. Consequential validity suggests that assessments should include value implications and relevance (Messick, 1989). While assessment research examines the disciplines of mathematics, science, and language arts in a more holistic manner, the few social studies assessments are often divided among the field’s various disciplines. The purpose of social studies, and its development of standards, is firmly linked to the current state of disunity within the social studies field. These issues are reflected in teacher preparation policies as well as state assessment policies. Social studies advocates have proven that, in other subjects, teachers and instructional methods are influenced by assessment. Fortunately, the National Assessment of Educational Progress (NAEP) provides an opportunity to determine whether the exclusion of social studies within the state/national accountability system is impacting student achievement in social studies. The current systems make social studies uniquely positioned for studying the effects of large-scale assessment upon the field. Uses and interpretations of assessment data by researchers have been limited in social studies because the subject is not incorporated into most accountability policies. State policy governs educational standards, teacher licensure, and the extent of assessments upon students. For this study, the social studies NAEP assessment is iii divided into three separate tests (U.S. History, Geography, and Civics). By looking at data from the fourth-, eighth-, and twelfth-grade 2010 social studies NAEP tests, this study will investigate some of the unintended consequences of educational assessment culture. I will examine social studies through different lenses and apply the concept of consequential validity to social studies in order to understand the value of social studies within education. iv ACKNOWLEDGEMENTS As with any project of this magnitude, many people helped me get through this long arduous process. First, I want to acknowledge my very supportive committee. Without them, this project would never have reached fruition. Claudette Peterson, who always willingly gave me advice even when I felt lost, helped me sketch out my thoughts in more concrete ways. The two figures in Chapter 1 are thanks to you. Chris Whitsel, a needed shoulder to lean on, provided essential critiques even when he was on the other side of the world. Justin Wageman gave me insights into the nature of large-scale assessments that added nuance to this study. And finally, Brent Hill picked me up when I was at my bleakest and encouraged me to finish this study. When statistics proved challenging, he cheered me to continue onward. I owe you all a great deal of thanks. Along the way, numerous people at NDSU’s School of Education contributed in significant and meaningful ways. David Silkenat started me down this path. Though he left before I finished, his direction laid the foundation for Chapter 2 and its monograph. He also made me question my own understanding of social studies. Nate Wood tested my theoretical foundations and pushed me to question everything I knew about education and learning. Nate also explained to me that even negative results can be good research. I would like to also thank my classmates and colleagues who provided me feedback, thoughtful conversation, and inspiration. Of particular note, Christine Okurut-Ibore, Sheri Okland, Veronique Walters, Karla Thoennes, Janna M. Stoskopf, and Kim Bruemmer. I could not have completed this without the help of Lea Roberts, who was always willing to help me find a missing faculty member. Peggy Cossette, the glue that hold the Dean’s Office together, would smile when I came in with some new and difficult problem. She was always able to fix it. Thank you, Lea and Peggy. v Numerous people throughout NDSU helped me through this process. Heather Fuller provided insight into writing and the use of restricted data. Many people in the Modern Language Department helped me, including Stephen Disrud, Anneli Ryan, and Carol Bishop. You all provided a temporary job as well as a friendship. In the College of Arts and Humanities, John Cox gave me several opportunities to think about my own social studies teaching as did Linda Fricker, the best college advisor I know. Of course, I need to thank Angela Smith who saved me hours of nerve-wracking formatting. Thank you. The biggest thanks go to my family for their support and sympathy as I worked through this process. For my parents, Susan and Howard Learn, and my wife’s parents, Cheryl and Roby Barrett (as well as Carol Sedberry and Betty Hitchings), who very patiently waited for me to complete this work. Of note, Cheryl read through bits and pieces and provided me with tons of constructive feedback. I cannot say enough about my boys, Gus and Eli. I could not have done it without them. Even though I was not always available to them, they kept me sane throughout this process. Whether going to hockey practice or music, playing games, traveling to cross country meets, or attending some other activity; I definitely needed those distractions. Thank you. The final and most important person I need to acknowledge is my loving and adoring wife, Tracy Barrett, my best friend. Encouraging me, supporting me, listening to me, and believing in me, I could never have done this without her. This work is as much your work as mine. Thank you. I love you, too. vi PREFACE Since I started the program, I focused on topics related to social studies education. As a teacher, I have a passion for social studies. I want to see the field improve and not become irrelevant as so many studies indicate. I also have an interest in assessment and specifically the uses for large-scale assessment. When I applied to the program, I thought that I knew exactly what my dissertation would look like and what I would prove. I quickly realized that research generally does not prove anything; it simply provides evidence for future research. Additionally, I realized that your own preconceived conclusions often do not turn out the way you expect. I am still linking social studies education and assessment together, but I am more cognizant that my findings may turn out the exact opposite, or, worse yet, be statistically inconclusive. I plan to study the connection between standards, authentic pedagogy, and large-scale assessment within the field of social studies. This study changes a more traditional line of research by actually looking at what the social studies assessments can tell researchers. Much of this was written as part of an independent study with David Silkenat. When I started my doctorate, I thought I had a grasp of what social studies are. David asked me to look at both the cognitive notion of teaching social studies (Wineburg, VanSledright, Barton, etc.) as well as the National Standards for History debate in 1994. I started with the teaching aspect of social studies but did not grasp the context from which the researchers were writing. When I switched to the National Standards for History, I realized that I needed to understand the origins of social studies to better frame the debate. I started writing a narrative of why the National Standards for History failed. Nash, Crabtree, and Dunn (1997) gave the impression that the failure of the National Standards for History was not understandable. They decided it was simply that it happened at a bad political movement. Actually, this struggle over social studies vii curriculum is a continuation of conflicts from its origins. As a result, I was able to establish a framework for the current theories about standards in social studies education. But as with everything, I wanted to get back to my own interest in assessment. This dissertation started taking on a life of its own and I also needed to understand why testing mattered to social studies. Standards are only one aspect; large-scale assessments were also something I soon realized that I needed to understand. So, I soon went down that