An Incentives Experiment in the U.S. Consumer Expenditure Quarterly Survey December 2006 David E. McGrath BAE Systems Information Technology / Bureau of Labor Statistics Abstract1 1.2 The U.S. Consumer Expenditure Program Response rates to the Consumer Expenditures Quarterly The U.S. Consumer Expenditure Survey program provides (CEQ) Survey fell from 86 percent in 1990 to 76 percent by continuous information on the expenditures of American 2004. To reduce the current downward trend in CEQ response consumers for use in economic research and in support of rates, the Bureau of Labor Statistics introduced an incentives revisions to the Consumer Price Index. BLS sponsors the experiment in November, 2005. The goal was to increase collection of expenditures in two separate surveys. First, the response rates by offering respondents a pre-paid monetary Consumer Expenditures Diary survey is designed to collect incentive that is unconditional on response to the survey. We small, detailed expenditures that would be difficult for mailed debit-card incentives along with the survey’s advance respondents to recall during an interview. The diary is a self- letter prior to contacting the potential survey respondent for administered survey where respondents record all household the Wave 1 interview. The experimental design contrasts a expenditures for two one-week periods. Second, the control group receiving no incentive with groups that receive Consumer Expenditures Quarterly (CEQ) survey is a detailed either $20 or $40 debit cards. In this paper, we show the series of five interviews designed to collect less frequent design of the incentives experiment, provide preliminary purchases. Interviewers from the Census Bureau collect the results on the effects of the incentives on response rates, and data for both surveys. Results from the two surveys are discuss our assessment of the debit cards. integrated to create published expenditures estimates. The research described in this paper uses data from only the CEQ survey. Keywords : Incentives, Response rates, Experiment, Debit Cards The Census Bureau conducts about 40,000 CEQ interviews across the nation each year. Each selected household 2 is interviewed five times over one year. The Wave 1 interview 1. Introduction is primarily a bounding interview, designed to limit telescoping errors in the Wave 2 interview. Data from the first 1.1 Overview interview do not contribute to expenditures estimates. Waves 2-5 of the survey ask respondents about detailed expenditures Response rates to the Consumer Expenditures Quarterly for most non-food purchases , such as housing, furniture, (CEQ) Survey fell from 86 percent in 1990 to 76 percent automobile, and vacation expenses. The survey is (AAPOR RR1) by 2004. To reduce the current downward administered by computer assisted personal interview (CAPI), trend in CEQ response rates, the Bureau of Labor Statistics either in person or over the telephone (not centralized). An (BLS) introduced an incentives experiment beginning in average CEQ interview takes approximately one hour to November, 2005. This paper shows the design of the complete. incentives experiment, provides preliminary results on the effects of the incentives on response rates, and discusses our assessment of the debit cards. Section 1 introduces the 1.3 Incentives Literature Consumer Expenditures Survey program and discusses the general incentives literature. Section 2 describes the design of Providing potential survey respondents with incentives the CEQ incentives experiment, and the specific literature we generally increases response rates (Church, 1993; Singer, used to guide the design decisions. In Section 3, we present 1999). Incentives can be cash or non-cash gifts and are preliminary results including the effects of the incentives on typically pre -paid to all potential respondents, or promised to response rates , possible effects of the incentives on potential respondents and paid upon compliance with the nonresponse bias, and an assessment of the debit cards, survey. Most early research on incentives occurred in mail including respondents’ experiences cashing the cards and the surveys (Church, 1993; Shettle, 1999). However, Singer’s rates at which respondents cash the cards. Section 4 shows (2002) meta-analysis on the effects of incentives in some effects of the incentives on the interview process using interviewer administered surveys (face-to-face and phone) contact history data. Section 5 provides a brief summary of showed that the incentive effects illustrated in the mail survey findings to date, and Section 6 discusses future data analysis literature also apply to interviewer-administered surveys. and areas for future research. Three important effects are that cash incentives perform better than non-cash incentives (e.g., gifts), pre-paid incentives are 2 The CEQ collects data from consumer units, which include people living in 1 The author thanks Karen Goldenberg, David Swanson, and Moon Jung Cho a household related by blood or marriage, or unrelated people who share of the BLS for helpful comments. Any opinions expressed in this paper are household expenditures. Each household consists of one or more consumer those of the author and do not constitute policy of the Bureau of Labor units. For most housing units, the household and consumer unit are the same. Statistics or BAE Systems Information Technology. For this paper, we use the terms interchangeably. better than promised payment for survey cooperation, and compared with cash. The POS cards had the highest response response rates improve with increasing amounts of money. rate, but only 36 percent of households used their cards. Also, 18 percent of the POS card recipients called with questions or Despite these known effects, government surveys have complaints related to either the PIN or activation process.5 traditionally not offered respondents incentives to participate in surveys. This has begun to change. Beginning in 1996, the If respondents don’t use the debit cards, the intended effect of Census Bureau conducted a series of incentive experiments on the cash incentive may be reduced. A non-trivial percentage the Survey of Income and Program Participation (SIPP). This of card recipients assume the card is a scam, view cashing the research supports current literature, showing that incentives card as an inconvenience, or have technical problems with the work similarly for large, interview-administered government card. (See Section 3.5 for respondent’s experiences with the surveys. In the 1996 SIPP panel, James (1997) found that a debit cards in our incentive experiment.) If respondents don’t $20 pre -paid incentive significantly lowered SIPP perceive the debit card as a legitimate gift, they may not nonresponse rates in Waves 1-3 compared with a $10 reciprocate by participating in the survey. incentive and a no-incentive control group.3 Despite these concerns, we selected debit cards for the CEQ incentive experiment because the Census Bureau has methods in place to use debit cards and has successfully used them on 2. Design of the Incentives Experiment other surveys.6 In addition, the cards are less susceptible to theft, and they cost less because a large percentage of respondents never use them.7 Because the CEQ is an interviewer administered survey, most of the decisions regarding the CEQ incentives experiment are based on results from the Singer meta-analysis . For cost 2.3 Use Pre-Paid and Unconditional Incentive estimates and expectations related to response rates, we relied on results from the SIPP experiments because of the The incentives literature consistently demonstrates that pre- similarities between the SIPP and CEQ (large, burdensome paid incentives that are unconditional upon survey response government surveys, face to face mo de, Census interviewers). most effectively increase response rates (Singer, 2002; Berk, 1987). To prepay the incentives, we mailed the debit cards The following sections discuss key design decisions we made with the Wave 1 advance letter. We added the following while planning the CEQ incentive experiment. paragraph to the current CEQ advance letter: “Please accept the enclosed Debit card as a token of our 2.1 Distribute Incentive Only in Wave 1 appreciation for considering this important survey. See the enclosed envelope for instructions about using the card, which We made this decision for budget reasons, but the SIPP you may use immediately.” research also provided evidence that distributing incentives only in Wave 1 of a panel survey may have lasting effects. To make the incentive as easy to use as possible, we printed During the initial SIPP incentives experiment, the Census the personal identification number (PIN) and the debit card Bureau provided incentives only in Wave 1. Expanding on the amount directly on the debit card. James analysis of the 1996 SIPP experiment, Mack et al. (1998) reported that the $20 Wave 1 SIPP incentive held nonresponse lower through Wave 6, two years out. Based on 2.4 Amount of the Incentive this experience, we decided to distribute the CEQ incentive As with all incentive experiments, the goal is to find the least only with the initial interview (Wave 1).4 However, if later expensive incentive that achieves the desired effects, mainly analysis of Waves 4 and 5 CEQ response rates shows a increasing response rates. Although academic and commer- diminishing incentive effect, we will need to assess whether cial survey research centers often provide $1 and $5 this design is adequate. incentives, we felt that we needed larger incentives for two reasons. First, although small incentives generally work well, larger incentives work even better (Rodgers, 2002; James, 2.2 Use a Monetary Incentive (Debit Card) 1997). Second, our decision to use debit cards constrained us Singer’s (2002) meta-analysis demonstrates that monetary to incentives in multiples of $20, because many ATM incentives are the preferred incentive. However, the monetary incentives tested in these studies are usually cash, not a cash 5 Mitchell’s study oversampled households in poverty, which may affect the substitute such as debit cards.