Dynamic Measurement: a Theoretical–Psychometric Paradigm for Modern Educational Psychology
Total Page:16
File Type:pdf, Size:1020Kb
Educational Psychologist ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/hedp20 Dynamic measurement: A theoretical–psychometric paradigm for modern educational psychology Denis Dumas , Daniel McNeish & Jeffrey A. Greene To cite this article: Denis Dumas , Daniel McNeish & Jeffrey A. Greene (2020) Dynamic measurement: A theoretical–psychometric paradigm for modern educational psychology, Educational Psychologist, 55:2, 88-105, DOI: 10.1080/00461520.2020.1744150 To link to this article: https://doi.org/10.1080/00461520.2020.1744150 Published online: 03 Apr 2020. Submit your article to this journal Article views: 334 View related articles View Crossmark data Full Terms & Conditions of access and use can be found at https://www.tandfonline.com/action/journalInformation?journalCode=hedp20 EDUCATIONAL PSYCHOLOGIST 2020, VOL. 55, NO. 2, 88–105 https://doi.org/10.1080/00461520.2020.1744150 Dynamic measurement: A theoretical–psychometric paradigm for modern educational psychology Denis Dumasa, Daniel McNeishb, and Jeffrey A. Greenec aDepartment of Research Methods and Information Science, University of Denver; bDepartment of Psychology, Arizona State University; cLearning Sciences and Psychological Studies, University of North Carolina at Chapel Hill ABSTRACT Scholars have lamented that current methods of assessing student performance do not align with contemporary views of learning as situated within students, contexts, and time. Here, we intro- duce and describe one theoretical–psychometric paradigm—termed dynamic measurement— designed to provide a valid representation of the way students respond to school-based instruc- tion by estimating the learning potential of those students given their observed learning trajec- tory. We examine the century-long history of dynamic measurement, from its nascent theoretical beginnings, through its limited initial implementations, to the current development of a special- ized modeling framework that allows it to be applied to large-scale educational data. We illustrate how dynamic measurement models (DMMs) can realize the goals of modern theory and measure- ment using a large longitudinal dataset of mathematics assessment data (i.e., 11,368 students nested within 98 schools and measured at 6 time points). These historical review and methodo- logical demonstrations show the value of dynamic measurement, including better modeling of complex and contextually dependent influences on student learning. Modern educational psychology theory and research are Here, we review and describe one recently developed built upon the assumption that educational outcomes are methodological procedure—dynamic measurement models the result of dynamic interactions among intraindividual fac- (DMM; Dumas & McNeish, 2017; McNeish et al., 2020)—as tors (e.g., prior knowledge, motivation, emotion) and the one potential solution to the psychometric and resource contexts in which people develop, learn, are taught, and per- challenges that have otherwise hampered educational psy- form (Bronfenbrenner & Morris, 2006; National Academies chology’s aspirational goals of fully accounting for the inter- of Sciences, Engineering, and Medicine, 2018). Most actions of student individual differences, history, and researchers and educators espouse the beliefs that educators context that result in performance. DMMs provide research- should focus on student potential and that teaching should ers and educators a powerful, flexible, and scalable way to target students’ zone of proximal development, dynamically assess not just what students can do on a given day, in a adapting as students internalize instruction and improve given context, with a given assessment, but also how their history and context led to that performance. DMMs also their performance (Vygotsky, 1931/1997). These beliefs estimate what students could achieve in the future if their imply that student abilities change and grow over time, and current circumstances persist and, critically, provide some that different students can display differing trajectories of insight into how that future could be different if instruction growth in response to instruction, context, and history were optimized to their potential. (Alexander et al., 2009). By extension, judgments about stu- In this article, first we provide a century-spanning history ’ dents future potential based upon a single observation of of dynamic assessment, reviewing its potential to realize their ability, without information about student learning tra- educational psychology’s theoretical and aspirational goals, jectories or the contexts from which those learning trajecto- as well as the challenges that limited its realization. Then we ries emerged, may perpetuate social inequities (Haertel, introduce DMM, showing how this modern psychometric 2018; NRC & NAE, 2010). Yet, despite the acknowledgment and statistical technique can provide better support for that an individual student’s performance is a complex emul- inferences about learning and education from large-scale test sion of potential and opportunity, educational practice and data, including inferences about how former and current policy continue to rely upon static or oftentimes resource- conditions have shaped individual performance. Finally, we intense assessments that fail to properly account for the describe new directions for educational psychology theory, effects of context, and likely underestimate students’ capacity measurement, and advocacy that leverage DMMs to address to learn in the future. pressing questions in research (e.g., capturing interactions CONTACT Denis Dumas [email protected] Department of Research Methods and Information Science, University of Denver, Denver, CO 80210. ß 2020 Division 15, American Psychological Association EDUCATIONAL PSYCHOLOGIST 89 among individuals and context), psychometrics (e.g., model- Earliest attempts ing learning capacity as a function of multiple assessments Many of the progenitors of educational and psychological with non-linear trajectories), policy (e.g., shaping contexts to measurement and testing were aware that the inferences foster optimal growth), and critical thought (e.g., leveraging about learning derived from static testing methods held ser- multiple methods and theories to advance the goals of ious issues, and could result in problematic societal conse- social justice). quences when used in high-stakes educational decisions. For A few caveats are necessary before we begin our review. example, as early as 1909, Alfred Binet advocated for a pro- First, we do not seek to advance any particular existing the- cess-assessment paradigm in which students would be ory of learning within the educational psychology literature observed learning new material as a means of assessing their (e.g., constructivism, situated learning). Instead, we follow capacity for performance, although he never developed such Alexander et al. (2009) in conceptualizing learning pragmat- a procedure. Despite Binet’s later misgivings about static ically, as a phenomenon that occurs in ways that are testing, and perhaps because of the less resource intensive- described across each extant theory, and perhaps cannot be nature of static testing compared to the demands of specifically described across all learning instances by any dynamic assessment, the decade of the 1910s saw a massive existing theory. In doing this, we use the term learning to growth of static testing procedures. This was especially evi- mean a change in student performance in response to dent in North America where psychometric batteries devel- instruction within the school context. Questions of how oped by Thorndike et al. (i.e., the Alpha and Beta tests; learning occurs and how to facilitate it are beyond the scope Thorndike, 1919) were used to sort military recruits into of this article but can be explored in novel ways using positions during World War I. DMMs. Second, in this article we use the terms static and After World War I ended, scholars in North America dynamic in specific ways. We use the term static to refer to and Europe began to specifically critique the static testing assessment or measurement practice that targets abilities paradigm that had prevailed during the war. Chief among that are current at the time of testing (e.g., fourth grade these critics was W.E.B. DuBois. In his essay Race mathematics in fourth grade) and tend to do so without Intelligence, DuBois (1920/2013) laid out a specific argument emphasis on contextual factors that support the develop- for the critical need to incorporate information about the ment of those abilities. Using that terminology, static meas- dynamic process of learning into large-scale tests, and ures could be either summative or formative, depending on offered the prophecy that static testing—with its conflation the inferences made from their scores, and their application of current knowledge and future potential to learn—would in the educational setting. In contrast, we use the term always benefit individuals who had experienced the richest dynamic to refer to measurement that targets developing learning opportunities early in life. In addition, DuBois ’ constructs (e.g., fourth grade students future capacity to made the additional point that, if students who scored lower learn mathematics) through a procedure that couples testing on static tests were then sorted into contexts where they with instruction in order to observe student improvement, never received the instruction required