The Cla+ and the Two Cultures: Writing Assessment and Educational Testing
Total Page:16
File Type:pdf, Size:1020Kb
Purdue University Purdue e-Pubs Open Access Dissertations Theses and Dissertations January 2015 The LC A+ and the Two Cultures: Writing Assessment and Educational Testing Fredrik B. Deboer Purdue University Follow this and additional works at: https://docs.lib.purdue.edu/open_access_dissertations Recommended Citation Deboer, Fredrik B., "The LC A+ and the Two Cultures: Writing Assessment and Educational Testing" (2015). Open Access Dissertations. 1358. https://docs.lib.purdue.edu/open_access_dissertations/1358 This document has been made available through Purdue e-Pubs, a service of the Purdue University Libraries. Please contact [email protected] for additional information. Graduate School Form 30 Updated 1/15/2015 PURDUE UNIVERSITY GRADUATE SCHOOL Thesis/Dissertation Acceptance This is to certify that the thesis/dissertation prepared By Fredrik deBoer Entitled THE CLA+ AND THE TWO CULTURES: WRITING ASSESSMENT AND EDUCATIONAL TESTING For the degree of Doctor of Philosophy Is approved by the final examining committee: Richard Johnson-Sheehan Chair April Ginther Janet Alsup Bradley Dilger To the best of my knowledge and as understood by the student in the Thesis/Dissertation Agreement, Publication Delay, and Certification Disclaimer (Graduate School Form 32), this thesis/dissertation adheres to the provisions of Purdue University’s “Policy of Integrity in Research” and the use of copyright material. Approved by Major Professor(s): Richard Johnson-Sheehan Nancy Peterson 4/20/2015 Approved by: Head of the Departmental Graduate Program Date i THE CLA+ AND THE TWO CULTURES: WRITING ASSESSMENT AND EDUCATIONAL TESTING A Dissertation Submitted to the Faculty of Purdue University by Fredrik B deBoer In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy i May 2015 Purdue University West Lafayette, Indiana ii This dissertation is for my parents. : ii iii ACKNOWLEDGEMENTS I am deeply indebted to, and grateful for, my family and friends for their support throughout my six years of graduate education, especially Anika, John, Hendrik, Kerste, Aeris, and Aurora; for Susan and John Beers, who have helped me in my adult life more than I can say; for all of the faculty members who have guided and taught me, especially Richard Johnson-Sheehan and April Ginther; and for my cohort members Chris Dorn, Jeff Gerding, Charlotte Hyde, Christine Masters Jach, Beth Jones, Gracemarie Mike, Stacy Nall, Fernando Sanchez, Ellery Sills, Luke Redington, and Kyle Vealey, who have made this journey one of friendship, support, and joy. I would also like to thank my haters for making me famous. iii iv TABLE OF CONTENTS Page ABSTRACT ........................................................................................................................ v CHAPTER 1: INTRODUCTION ....................................................................................... 1 A Growing Movement for Change .................................................................................... 1 The Assessment Mandate ............................................................................................................. 2 The Role of Writing ...................................................................................................................... 4 Understanding the Present, Facing the Future ........................................................................... 6 Statement of the Problem ............................................................................................................. 8 Data & Methods .......................................................................................................................... 10 IRB .............................................................................................................................. 13 Textual/Archival .......................................................................................................................... 14 Interviews ............................................................................................................................... 16 Chapter Summaries ..................................................................................................................... 17 CHAPTER 2: THE HIGHER EDUCATION ASSESSMENT MOVEMENT................... 20 Truman, Eisenhower, Kennedy: Three Reports ...................................................................... 21 A Nation at Risk ........................................................................................................................... 27 Response From Accreditation Agencies 33 .................................................................................. The Spellings Commission 35 iv ........................................................................................................ The Obama Administration ........................................................................................................ 43 Conclusions ............................................................................................................................... 50 CHAPTER 3: HISTORY AND THEORY OF THE COLLEGIATE LEARNING ASSESSMENT ....................................................................................................................... 53 v Page Early Precursors........................................................................................................................... 43 The Old Standards: The GRE and Similar Entrance Exams ................................................. 57 The Council for Aid to Education ............................................................................................. 61 The Collegiate Learning Assessment ....................................................................................... 62 The Performance Task ................................................................................................................ 64 The Analytic Writing Section .................................................................................................... 66 From CLA to CLA+ ................................................................................................................... 67 Validity .............................................................................................................................. 72 Reliability ............................................................................................................................... 77 Criterion Sampling and Psychometric Assessment ................................................................ 79 The CLA and the SAT: Is Another Test Necessary? .............................................................. 83 The Slippery Measurement of Value Added ........................................................................... 88 Future Directions ......................................................................................................................... 91 CHAPTER 4: THE TWO CULTURES ................................................................................... 93 A Brief History of Practitioner Writing Assessment .............................................................. 93 Sources of Friction ................................................................................................................... 100 The Higher Education Assessment Movement and the Two Cultures .............................. 105 The Contested Role of Quantification in Writing Studies .................................................. 108 The Road Ahead: Reasons for Optimism? 117 ............................................................................ v CHAPTER 5: LOCAL CONTEXT, LOCAL CONTROVERSIES .................................. 120 Local Contexts .......................................................................................................................... 121 Previous Assessment: Accreditation ...................................................................................... 123 A Controversial Catalyst: the Administration of Mitch Daniels ........................................ 126 Perceived Needs and the Foundations of Excellence Plan ................................................. 129 Identified Issue: Administrative Redundancy ...................................................................... 134 Identified Issue: A Campus Divided ...................................................................................... 136 An Early Reform: the Core Curriculum ................................................................................ 137 The Initial Assessment Push ................................................................................................... 140 vi Page The Roots of Conflict .............................................................................................................. 144 Piloting ............................................................................................................................ 151 Initial Results ........................................................................................................................... 155 Internal Skepticism .................................................................................................................. 162 Faculty Resistance .................................................................................................................... 165 Was the CLA+ Preordained? .................................................................................................