The Silent Majority: an Examination of Nonresponse in College Student Surveys
Total Page:16
File Type:pdf, Size:1020Kb
University of Massachusetts Amherst ScholarWorks@UMass Amherst Open Access Dissertations 9-2012 The iS lent Majority: An Examination Of Nonresponse In College Student Surveys Ethan A. Kolek University of Massachusetts Amherst, [email protected] Follow this and additional works at: https://scholarworks.umass.edu/open_access_dissertations Part of the Education Commons Recommended Citation Kolek, Ethan A., "The iS lent Majority: An Examination Of Nonresponse In College Student Surveys" (2012). Open Access Dissertations. 613. https://doi.org/10.7275/wn37-0y87 https://scholarworks.umass.edu/open_access_dissertations/613 This Open Access Dissertation is brought to you for free and open access by ScholarWorks@UMass Amherst. It has been accepted for inclusion in Open Access Dissertations by an authorized administrator of ScholarWorks@UMass Amherst. For more information, please contact [email protected]. THE SILENT MAJORITY: AN EXAMINATION OF NONRESPONSE IN COLLEGE STUDENT SURVEYS A Dissertation Presented by ETHAN A. KOLEK Submitted to the Graduate School of the University of Massachusetts Amherst in partial fulfillment of the requirements for the degree of DOCTOR OF EDUCATION September 2012 School of Education Educational Policy and Leadership © Copyright by Ethan A. Kolek, 2012 All Rights Reserved THE SILENT MAJORITY: AN EXAMINATION OF NONRESPONSE IN COLLEGE STUDENT SURVEYS A Dissertation Presented by ETHAN A. KOLEK Approved as to style and content by: _______________________________________ Elizabeth Williams, Chair _______________________________________ Gary Malaney, Member _______________________________________ Gloria DiFulvio, Member ____________________________________ Christine B. McCormick, Dean School of Education ACKNOWLEDGMENTS I have many people to thank for their help, support, encouragement, interest, and love that have made it possible for me to write this dissertation. I have been fortunate to have had many teachers (both inside and outside of the classroom), family members, and friends who have shaped who I am and how I think. Miss Pretat, Mrs. Pawelczyk, David Brownell, Toby Gillis, Liz Rapoza, David Boyce, Priscilla Bates, and Jim Bean took an interest in me as an adolescent and encouraged my curiosity and creativity. I have thought of them, as well as many of my Star Island friends, as I have reflected on my dissertation journey. My parents, Carol and Drew, and their respective partners, Carlton and Sandy, have always supported me and have encouraged me throughout this process. My friendships with my brother, Adam, and good friends, Aric Russom and Dan Saunders, helped sustain me while I was in the midst of writing. I would like to thank my Center for Community Engagement colleagues for their encouragement and understanding, particularly my supervisor, Molly Mead, who created my part-time position which enabled me to complete this dissertation. My partner, Tamatha Gaumnitz, has been incredibly supportive throughout my time in graduate school and during the time I spent writing this dissertation. Tamatha was instrumental in the success of my focus group study, helping to recruit many participants and finding a place to hold the focus groups at the university. Tamatha, thank you for being so understanding when I curtailed nearly all social commitments, abandoned my domestic responsibilities, and locked myself in my office to finish this project. I have been very lucky to have your love and encouragement. iv I am grateful for the small, but vibrant, intellectual community in the higher education program, especially the members of our informal “Doc Reading Group,” which included Aaron Kuntz, Dan Saunders, Tom Fleenor, Raldy Laguilles, April Stroud, Shelly Perdomo, and Karen Howard. The members of my committee, Liz Williams, Gary Malaney, and Gloria Difulvio, have asked great questions and shared important perspectives. Ryan Wells graciously helped me learn logistic regression, which I employed in this project. A special thanks goes to Dan Saunders, who read and commented on several chapters and prodded me to finish my dissertation. Gary Malaney’s sarcasm and cynicism were reassuring when I first arrived at the University. I have valued his encouragement, advice and friendship. During the nearly six years that I spent in the Student Assessment, Research and Evaluation Office, I worked closely with my mentor, Liz Williams. I am grateful to her for that incredibly rich experience, through which I learned more than I could have imagined about research and higher education. At work, in the classroom, or while working on independent projects, Liz always challenged me to be a better writer, thinker, and researcher. For over a decade, Liz has been a wonderful supervisor, teacher, advisor, collaborator and friend. v ABSTRACT THE SILENT MAJORITY: AN EXAMINATION OF NONRESPONSE IN COLLEGE STUDENT SURVEYS SEPTEMBER 2012 ETHAN A. KOLEK, A.B., VASSAR COLLEGE M.Ed., UNIVERSITY OF MASSACHUSETTS AMHERST Ed.D, UNIVERSITY OF MASSACHUSETTS AMHERST Directed by: Adjunct Assistant Professor Elizabeth Williams Nonresponse is a growing problem in surveys of college students and the general population. At present, we have a limited understanding of survey nonresponse in college student populations and therefore the extent to which survey results may be biased. The purpose of this dissertation is to explore three facets of nonresponse in surveys of college students in order to strengthen our empirical and conceptual understanding of this phenomenon. This dissertation seeks to contribute to our understanding of who participates in surveys and who does not, how students experience the process of being asked to complete surveys, and whether or not students’ perspectives about surveys suggest that college student surveys should be conceptualized as organizational surveys. To begin to answer these questions, I conducted three studies – a secondary data analysis that examines student characteristics associated with the odds of completing a survey, a “survey on surveys” study that asks students about their experiences with surveys, and a series of focus groups to understand how students made vi sense of surveys at their institutions. Taken together, these findings provide a basis for a more developed and nuanced understanding of nonresponse in student surveys. vii TABLE OF CONTENTS Page ACKNOWLEDGMENTS ................................................................................................. iv ABSTRACT ....................................................................................................................... vi LIST OF TABLES .............................................................................................................xv CHAPTER 1: INTRODUCTION ...........................................................................................................1 Introduction: Survey Research and the Study of College Students .........................1 Survey Methodology ................................................................................................3 Response Rates in Decline .......................................................................................4 Purpose of the Study ................................................................................................5 Research Questions ..................................................................................................9 Significance of the Study .......................................................................................11 Definitions..............................................................................................................13 Surveys .......................................................................................................13 People: Populations, Samples, Respondents, and Response Rates ............14 Probability Sampling .................................................................................16 Survey Error ...............................................................................................16 Sampling Error ...............................................................................16 Coverage Error ...............................................................................17 Measurement Error ........................................................................17 Nonresponse Bias...........................................................................18 Summary ................................................................................................................20 2: LITERATURE REVIEW ..............................................................................................22 Introduction ............................................................................................................22 Section 1: Declining Response Rates ....................................................................23 Documenting Declines in Survey Response Rates in the General Population ............................................................................................23 Potential Reasons for Declining Response Rates ......................................26 Declining Response Rates Summary .........................................................27 Section 2: Nonresponse Bias .................................................................................28 viii Causal Nonresponse Models ......................................................................29 Separate Cause Model....................................................................29