Louisiana Tech University Louisiana Tech Digital Commons Doctoral Dissertations Graduate School Summer 2012 Response rate and response error in marketing research M. Yasemin Ocal Atinc Louisiana Tech University Follow this and additional works at: https://digitalcommons.latech.edu/dissertations Part of the Marketing Commons Recommended Citation Ocal Atinc, M. Yasemin, "" (2012). Dissertation. 339. https://digitalcommons.latech.edu/dissertations/339 This Dissertation is brought to you for free and open access by the Graduate School at Louisiana Tech Digital Commons. It has been accepted for inclusion in Doctoral Dissertations by an authorized administrator of Louisiana Tech Digital Commons. For more information, please contact [email protected]. RESPONSE RATE AND RESPONSE ERROR IN MARKETING RESEARCH by M. Yasemin Ocal Atinc, B.S., M.B.A. A Dissertation Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Business Administration COLLEGE OF BUSINESS LOUISIANA TECH UNIVERSITY August 2012 UMI Number: 3533094 All rights reserved INFORMATION TO ALL USERS The quality of this reproduction is dependent upon the quality of the copy submitted. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if material had to be removed, a note will indicate the deletion. UMI 3533094 Published by ProQuest LLC 2012. Copyright in the Dissertation held by the Author. Microform Edition © ProQuest LLC. All rights reserved. This work is protected against unauthorized copying under Title 17, United States Code. ProQuest LLC 789 East Eisenhower Parkway P.O. Box 1346 Ann Arbor, Ml 48106-1346 LOUISIANA TECH UNIVERSITY THE GRADUATE SCHOOL July 18, 2012 Date We hereby recommend that the dissertation prepared under our supervision , Yasemin Ocal Atinc by entitled Response Rate and Response Error in Marketing Research be accepted in partial fulfillment of the requirements for the Degree of Doctor of Business Administration isor of Dissertation Research Head of Department Department of Marketing and Analysis Department Recommendation concurred in: Advisory Committee Approved; Approved: Director of Graduate Studies Dean of the Graduate School Dean of the/Qollege GS Form 13a (6/07) ABSTRACT In this study, I investigate the perspectives of marketing researchers views about the two important concepts of survey research, response rate and response bias. I specifically aimed to answer the following questions: Research Question #1: Do marketing researchers separate the concepts of response rate from response bias? Research Question#2: How exactly should data quality be measured? Is it about sample representativeness, minimizing non- response bias or just solely Increasing the response rate? Research Question#3: What are researchers doing to assess and minimize response bias? Research Question#4: Do additional efforts put forth by survey researchers, such as reminder letters and incentives, for the purpose of increasing survey response rate affect additional sampling bias? Research Question#5: Are the common techniques used by survey researchers to increase response rate equally effective? In an attempt to answer the research questions stated, I have collected both primary and secondary data. The primary data was collected from the Academy of Marketing Science active members. Eight versions of an excerpt was prepared that is taken from an actual article that was accepted for publication recently. The experimental design adopted was 2x2x2 with each treatment trying to answer one of the research questions stated in this study. iii iv The first treatment was regarding the consideration of population. The subjects were manipulated with two versions of the excerpt of which one was with a Canadian sample and the second was with a so called North American sample. The second treatment was about the manipulation of the initial number of surveys sent out which as a result would change the response rate percentage. The two different versions included 500 vs. 5000 initial surveys sent out varying the response rate from 5.1% to 50.2%. The third treatment included the manipulation of Armstrong and Overton (1977) citation. The over use of this citation in the marketing literature has been noteworthy. In an attempt to investigate this matter further, this treatment was created. First version contained a sentence that stated that the early and the late respondents were compared and no significant differences were found as evidence of no response bias including the citation of Armstrong and Overton (1977). The second version of the excerpt included a table with the expected demographics regarding the population of interest. In addition to these, the subjects were also assigned to two different conditions where they were asked to evaluate the excerpt as an author or as a reviewer. In the invitation email respondents were asked to toss a coin or to click on a web link that would toss the coin for them and select the appropriate link that corresponds with their choice. In order to assess the popular techniques of enhancing response rate, I have divided the sample into several groups as the pre-notification, the reminder and the pre- notification and reminder groups and a control group that received neither the reminder V nor the pre-notification treatment. The results revealed that, according to my sample none of the techniques mentioned above, improve the response rate. The secondary data was collected from major outlets of the marketing science {Journal of Marketing- JM, Journal of Marketing Research-JMR, and Journal of the Academy of Marketing Science-}AMS) during the periods of 2005-2010. The final sample consisted of 68 JM, 23 JMR and 84 JAMS articles. In addition to these, I also randomly selected 31 rejected articles from the Journal of Business Research (JBR) archives. The results of the study revealed that, survey researchers do not clearly grasp the concepts of response rate and response bias. In addition, the results demonstrated that the data quality should be measured by the sample's representativeness of the population and the researcher's capability of decreasing the response and the non-response biases. Further, the techniques used to enhance response rate such as reminder and pre- notification letters as well as incentives are not as effective and are likely to introduce additional response bias to a study. The results also showed that the optimal data collection method researchers should consider adopting is the combination method. The study ends with the discussion of the implications of these results and possible future extensions of this study. APPROVAL FOR SCHOLARLY DISSEMINATION The author grants to the Prescott Memorial Library of Louisiana Tech University the right to reproduce, by appropriate methods, upon request, any or all portions of this Dissertation. It is understood that "proper request" consists of the agreement, on the part of the requesting party, that said reproduction is for his personal use and that subsequent reproduction will not occur without written approval of the author of this Dissertation. Further, any portions of the Dissertation used in books, papers, and other works must be appropriately referenced to this Dissertation. Finally, the author of this Dissertation reserves the right to publish freely, in the literature, at any time, any or all portions of this Dissertation. Author Date GS Form 14 (5/03) TABLE OF CONTENTS ABSTRACT iii LIST OF TABLES x LIST OF FIGURES xi ACKNOWLEDGMENTS xii CHAPTER 1 INTRODUCTION 1 Introduction 1 Motivation of the Study 2 Proposed Contributions and Objectives of the Study 5 CHAPTER 2 LITERATURE REVIEW 7 Telephone Surveys 7 Web Surveys 19 Face to Face Surveying 37 Mail Surveys 39 Social Exchange Theory and Monetary Incentives 43 Dissonance Theory 44 Self-Perception Theory 44 Theory of Commitment and Involvement 45 Theory of Planned Behavior 45 Elaboration Likelihood Model 47 vii viii Effects of Socio-Demographic Factors in Mail Survey Response Rate and Quality 48 Education 48 Age 48 Gender 49 Marital Status 49 Ethnicity/Race 50 Religion 50 Community Size/Geographical Region 50 Survey Response Rate and Survey Bias 51 What is Response Rate? 51 Response Error and Response Bias 53 Selection and Analysis of Articles for the Pilot Study 57 Results of the Pilot Study 58 Research Questions 60 General Research Questions 60 CHAPTER 3 METHODOLOGY 67 Article Coding 67 Coding Variables 68 Calculation of Response Rate 72 Experimental Design 72 Summary 73 CHAPTER 4 RESULTS 75 Data Collection: Primary Data 75 ix Treatment 1 76 Treatment 2 77 Treatment 3 78 Survey Administration 79 Study Response Rate 80 Secondary Data Collection 82 Answers to Research Questions 82 Primary Data Collection Results 84 Results from Secondary Data 91 Tests for Sample Population Congruency 91 Results from Primary Data 92 Primary Data Collection Results 94 Regression Results 98 Post-Hoc Results 107 CHAPTER 5 DISCUSSION AND CONCLUSION 117 Discussion 118 Future Research and Limitations 128 Conclusion 130 REFERENCES 132 APPENDIX A AUTHOR SURVEY FORM 148 APPENDIX B REVIEWER SURVEY FORM 156 APPENDIX C SURVEY LETTER 163 APPENDIX D HUMAN USE APPROVAL LETTER 188 LIST OF TABLES Table 2.1 Recruiting Methods of Web Surveys 36 Table 2.2 Response Rate Change 41 Table 2.3 Response Rate 59 Table 4.1 Overview of Responses 85 Table 4.2 Descriptive Statistics 86 Table 4.3 Descriptive Statistics 87 Table 4.4 Results of Regression Analysis 89 Table 4.5 Secondary Data Results 90 Table 4.6 Survey Completion Rates 95 Table 4.7 Descriptive Statistics and Pearson Correlations 100 Table 4.8 Regression Results 101 Table 4.9 Regression Results 102 Table 4.10 Regression Results 103 Table 4.11 Regression Results 104 Table 4.12 Multivariate GLM Results 109 Table 4.13 GLM Results 112 Table 4.14 Independent T-Tests (Author vs. Reviewer) 116 x LIST OF FIGURES Figure 1.1 Minimization of Bias with a Smaller Sample 3 Figure 2.1 Categories of Survey Errors 56 xi ACKNOWLEDGMENTS This dissertation would not have been possible without the support of many people.