Survey Burden and its Impact on Attitudes Toward the Survey Sponsor Jaki Stanley McCarthy and Daniel G. Beckler _______________________________________________________________________________________ The organization sponsoring a survey data collection may affect respondents’ willingness to respond. USDA’s National Agricultural Statistics Service (NASS) has begun asking agricultural establishments questions during ongoing surveys about respondents’ knowledge and attitudes about NASS. These questions were asked of both respondents and non-respondents to the surveys, and clearly showed a correlation between respondents’ knowledge and attitudes toward NASS and their willingness to cooperate when contacted by NASS. Past burden (number, length, complexity, frequency of contacts, etc.) imposed by NASS was also measured and found to have little correlation with survey cooperation. It was hypothesized that increased contacts may provide opportunities for public relations and help foster more positive attitudes towards the survey sponsor. If increases in burden create more positive opinions of the survey sponsor, this may offset expected declines in cooperativeness as burden increases. However, little relationship was found between respondents’ attitudes toward NASS and the past burden imposed on them. KEY WORDS: Respondent burden, accumulated burden, agricultural survey ______________________________________________________________________________________ 1 INTRODUCTION1 validation, authority, scarcity and liking. One method interviewers report using to gain Survey methodologists have long speculated cooperation is to tailor the interaction (and about what factors affect survey respondents’ consequently which compliance principles are willingness to cooperate when contacted. used) according to the particular respondent. Attributes of the interviewer, the respondent, the survey process or the external environment may all impact survey cooperation. (For an Snijkers, Hox, and de Leeuw (1999) studied extensive review of these factors in household the tactics that high performing survey surveys, see Groves and Couper, 1998.) In interviewers use to gain cooperation. Similar interview surveys, the interviewer - to Groves, et al. (1992) they found that respondent interaction is critical to gaining tailoring the interaction was important. cooperation. Interviewers are often free to However, they also found that mentioning introduce an interview in whatever way they Statistics Netherlands as the survey sponsor feel is suitable. The survey introduction may was rated as a highly effective means of include any number of different appeals securing cooperation. Successful interviewers intended to increase cooperation. Groves, also felt that the agency should pay more Cialdini, and Couper (1992) have argued that attention to public relations and thus, “the many fall into one of six principles of image of the agency is seen as a tool to work compliance: reciprocation, consistency, social with and attain a better response rate.” One of the critical components to a survey 1 Paper presented at the International Conference introduction is the identification of the on Establishment Surveys II, June 2000. Both authors are with the National Agricultural Statistics Service, Research sponsoring organization and explanation of and Development Division. Jaki Stanley McCarthy is the the survey purpose. It has generally been Head of the Data Quality Research Section. Daniel Beckler assumed that government or university works in that section. sponsorship of a survey increases cooperation. by the survey results (see Interagency Group However, to date, there is little empirical on Establishment Nonresponse, 1999 for an evidence to show what the relationship extensive discussion of issues relevant to between survey sponsorship and cooperation nonresponse in establishment surveys). is. The Census Bureau found there was not a significant correlation between reported Within the agricultural establishment knowledge of the Census and likelihood of population, research done in the late 1970's returning a Census mail form (Bates and indicated that farmers’ self reported Buckley, 1999). However, in an evaluation of participation in USDA surveys was related to response to the 1990 Census, respondents’ how well they felt the survey purpose had attitudes toward the Census Bureau’s handling been explained to them, whether they of data confidentiality and privacy were themselves used USDA reports, and their somewhat correlated to both self reported opinion of whether or not they thought Census returns (Fay, Bates and Moore, 1991) government reports were generally worthwhile and actual Census returns (Singer, Mathiowetz (Jones, Sheatsley and Stinchcombe, 1979). and Couper, 1993). Harris-Kojetin and This is similar to findings in household Tucker (1999) also found that in times of surveys, where survey cooperation has been more positive public opinion regarding the linked to respondents’ general attitudes government and government leaders, toward the usefulness of surveys (Frankel and cooperation rates on the Current Population Sharp, 1981). Survey, a major government survey, were higher. Aside from speculation about the influence of survey sponsorship on survey response, a While there may be little research to show commonly held belief is that increased burden positive effects of changing respondents’ is negatively correlated with survey attitudes about a survey sponsor, Federal cooperation. Federal Statistical Agencies are agencies do feel that public relations and currently striving to lessen the reporting publicity are important. Some agencies burden placed on respondents. The U.S. devote more to this than others, but the Office of Management and Budget has set a importance placed on this is evident in the government-wide goal of five percent yearly Census Bureau’s budget of $167 million in reduction in information collection burdens advertising designed to promote the 2000 (Paperwork Reduction Act of 1995). Burden Census. may be defined in a number of ways – length of the interview or questionnaire, number of Survey respondents representing contacts, difficulty in reporting the requested establishments may be somewhat different data, etc. The length of the survey from respondents representing themselves or questionnaire is often assumed to be positively their households. These differences may correlated with survey non-response. make attitudes toward the survey sponsor However, a literature review by Bogen (1996) more important than in general household found that while this claim was supported in surveys. For example, establishment survey some studies, other studies showed exactly the respondents, particularly large or unique ones, opposite and still others showed no may be contacted much more frequently by an relationship. Frankel and Sharp (1981) also organization than household respondents. found the length of a single completed survey They may also use or be more directly affected interview was related to expressed willingness to participate in later interviews. Respondents sponsoring agency to promote the survey or who participated in a 25 minute interview agency. This may in turn change respondents’ were more likely to agree to participate in a attitudes for the better with increased burden, future interview than respondents participating instead of simply making respondents less in a 75 minute interview. However, there was likely to cooperate. If this is the case, we little difference in actual cooperation between might expect declines in cooperation due to those who had the long or short initial increased burden to be offset with more interview when later contacted for the second cooperative attitudes due to a more positive interview. view of the survey agency and its mission. Perhaps the public relations work that may be USDA’s National Agricultural Statistics part of our survey contacts can mitigate Service (NASS) publishes official statistics negative effects of potential increases in based on data collected from farmers, ranchers survey burden. and other agribusiness operators in voluntary surveys. For example, the Quarterly This paper looks first at the accumulated Agricultural Survey collects data on inventory burden imposed on agricultural operations and production; the Farm Labor Survey selected as respondents in our surveys and at collects information on hours worked and the relationship between respondents’ wages; the Agricultural Resource attitudes about NASS as the survey sponsor Management Study collects information on and their survey cooperation. Based on prior production practices, chemical and pesticide NASS research, we expect little relationship use, and farm economics. A particular between burden and cooperation, but a strong agricultural establishment may be selected for correlation between respondents’ attitudes and any or all of these surveys, both within a cooperation. In addition, this paper looks at single year and over multiple years. the relationship between the burden imposed by NASS and the attitudes held by In NASS surveys, there is little evidence that respondents (and non-respondents). Our accumulated burden contributes to later non- hypothesis is that since burden does not response. Reasons given for refusing to appear to be negatively correlated with participate in NASS surveys are more often cooperation, as burden increases, respondents’ “too busy/lack of time,” or privacy concerns attitudes become more positive to offset than