When Push Comes to Shove a Polling Industry Call to Arms

When Push Comes to Shove A polling industry call to arms

By Karl G. Feld

s the smoke from the 2000 ing the survey need not be identified. AAPC’s “high percentage of voters” as elections continues to clear, Push pollsters generally provide no any sample numbering over 1,000. we begin to sort fact from fic- name or a false name, a characteristic A tion to learn how to do it all better next noted by all three organizations. AAPC and CMOR state that legiti- time. One important question for mate interviews generally range in researchers in 2002 will be how to deal According to AAPC and AAPOR, length from five minutes to over 30 with the inevitable “push polls.” research firms conducting legitimate minutes. Push poll interviews typi- research interview only a small ran- cally last 30 to 60 seconds by AAPC’s There has been little consensus in the dom sample of the population, typi- definition, and 20 to 30 seconds by political community as to what a push cally ranging from 200 to 1,000 inter- CMOR’s. poll actually is. In a 1995 press release, views. Push polls try to reach what The National Council on Public Polls AAPC calls a very high percentage of CMOR observes that push polls (NCPP) defined a push poll as voters, usually at the close of the elec- are usually conducted by campaign tion campaign. CMOR has defined workers or telemarketers, rather than ...a telemarketing technique in which research interviewers. telephone calls are used to canvass vast numbers of potential voters, feeding them AAPOR further distinguishes push false and damaging ‘information’ about polls from research by observing that a candidate under the guise of taking a 17 data from the former are rarely or never poll to see how this ‘information’ affects saved or analyzed. voter preferences. In fact, the intent is to ‘push’ the voters away from one candi- 98 eading opinion researchers of date and toward the opposing candidate. ? both parties, and all professional Lorganizations with political poll- This definition closely matches those used by the American Association for Public Opinion Research (AAPOR), % 25 the American Association of Political Consultants (AAPC), and the Council for Marketing and Opinion Research 10 3 (CMOR). However, each organization has its own variations in the ways in which it distinguishes legitimate polls from push polls:

All three specify that legitimate polling firms open each interview by iden- tifying the name of the firm or the telephone research center. However, CMOR has held, and AAPOR members have unofficially observed, that the candidate or organization sponsor-

Karl G. Feld is vice president, research services, humanvoice, Inc., a Western Wats Center Company.

Public Perspective, September/October 2001 37 based upon their sentiments toward “The use of push polls has spread those doing the polling. throughout all levels of US politics.” The blurring of push polling definitions by media and campaigns is also detrimental to survey response rates. ster memberships, have condemned the those arguments might be in Along with continued “sugging” (Sell- practice of push polling. In the words of future advertising. In many ing Under the Guise of research) and Republican pollster Ed Goeas of the cases, these push questions [con- “frugging” (Fund Raising Under the Tarrance Group, “When political re- tain] attacks on the candidate Guise of research), push polling accel- searchers put a survey into the field they sponsoring the poll, to test how erates the current rate of decline, as do so using recognized scientific tech- vulnerable that candidate may Democratic pollster Mark Melman niques to find out what the public is be against anticipating attacks observed in a 1996 interview with John thinking or feeling. Push polls, on the from the other party. These are Nielson broadcast on National Public other hand, are meant to inform the not only legitimate tools of sur- Radio’s All Things Considered. Michael electorate with no accountability.” vey research, but any political Traugott, last year’s president of pollster who did not use them AAPOR, made a similar statement in The practice of push polling violates would be doing their clients a the wake of the February 2000 contro- the code of ethics to which members real disservice. versy over alleged push polling by the of AAPC, CMOR, and AAPOR agree Bush primary campaign. He declared upon joining. The NCPP has issued Questions used in push polls often that “[Push polls] breed cynicism about a statement to the effect that it does sound similar to those used as push politics, and we believe they contrib- not recognize push polls as legiti- questions—an intentional camou- ute to declining response rates for polls.” mate research. flage on the part of push pollsters. As a result, many respondents, political Inevitably, push polling will remain a ournalists, university professors, opponents, and journalists often campaign tool. In his new book, Elec- employees of research firms, staffs lump push questions and push poll- tions Polls, the News Media, and De- Jof local and statewide candidates, ing together. Their uninformed or mocracy, Traugott observes that the and even members of Congress have misleading statements do little to use of push polls has spread through- made statements about push polling clarify the confusion in the out all levels of US politics. Research which reflect their lack of understand- electorate’s mind. Today, pollsters for Larry Sabato and Glenn Simpson’s ing of how it differs from legitimate are often accused of conducting push Dirty Little Secrets: The Persistence of research. The perplexity usually oc- polling when they are, in fact, con- Corruption in American Politics revealed curs because push polling tends to be ducting legitimate research. that 35 of 45 candidates interviewed in confused with the frequent and proper 1996 claimed they had been victim- use of survey questions designed to test ne result of this befuddlement ized by a covert negative phoning cam- negative campaign messages. F. Chris- is that the research industry paign. Sabato and Simpson also give topher Arterton, Dean of George Ohas lately been threatened by details of numerous cases in which the Washington University’s Graduate various legislators and civic groups who technique had been used by both can- School of Political Management, calls seek to restrict both political didates’ campaigns, or by advocacy this latter practice “push questioning.” telemarketing and research. The usual groups, without a candidate’s knowl- demand, varied by state, is that the edge or consent. Push questions are generally recog- person or organization paying for the nized as a valid research tool. They are poll and the company conducting the ducating active political cam- widely used throughout the research interviewing be required to identify paigners and consultants on the industry, whereas push polling is not. themselves at the beginning of any Edifferences between push poll- In a February 2000 article in The telephone conversation. Push polling ing and push questions will do little to Hotline, Charlie Cook of the National is explicitly cited as the reason for this change the behavior of those conduct- Journal observed that legislative hazing. Should any of this ing push polls. Even experienced cam- legislation pass, the mandatory disclo- paigners in national races have indi- …there are legitimate polls that sure statements will severely bias the cated they care little about adhering to can ask push questions, which collected data, as respondents will be the standards and guidelines set by test potential arguments against predisposed to respond to survey ques- professional organizations. Eighty-one a rival to ascertain how effective tions (or refuse to participate altogether) percent of leading political pollsters

38 Public Perspective, September/October 2001 polled in a 1998 Pew Research Center esearchers can also support the The internet has a bright future in study thought that association guide- activities of the various orga- politics, part of which will include lines had little effect on their peers, and R nizations that fight on their web-based surveys and, inevitably, the 54% admitted the guidelines had little behalf in the legislative arena. CMOR web’s own version of push polling. effect on their own behavior! and AAPC have led successful cam- There is already a great deal of litera- paigns to amend or defeat draft legisla- ture on the poor quality of most online So what can researchers do as an indus- tion restricting telephone contacts. surveys, including problems with the try to protect themselves and the qual- Their efforts have advanced legislation way some of these “surveys” are writ- ity of research they do from the harm- against push polling while protecting ten. Many are not surveys at all, but ful side effects of push polling? There research in many states. The financial rather attempts to fundraise or smear are a number of possible recourses. support and membership of research- opponents, much as push polls do. ers in these organizations will continue First, professional associations can be to advance the cause of research. A prime example is the weekly web used to adjudicate disputes among re- “survey” hosted until the beginning of searchers, candidates, journalists, and Finally, these same organizations, as February 2000 on the site of the Demo- respondents. During the 2000 elec- well as the NCPP and AAPOR, should cratic National Committee (DNC). As tions, the NCPP established a Polling be encouraged by their research mem- described by all industry definitions of Review Board comprised of recognized bership to educate consumers of poll- push polling, it was designed to reach authorities in public opinion research ing information actively and publicly the many thousands who would log expressly for this purpose. A review through outreach initiatives. The onto the DNC site. The survey also board or similar asked leading questions with slanted organization to wording. It is not known whether the standardize and “Even experienced campaigners in data collected were used in a manner clarify the defini- consistent with push polling; however, tion of a push poll national races have indicated they the combination of the two other fea- would provide re- tures would suggest that the survey was search firms with care little about adhering to the intended to inform and influence, rather a third-party de- standards and guidelines set by than test messages and create data. fense against the uninformed and professional organizations.” s online survey research be the slanderous. comes more prevalent, profes- A sional organizations are also Second, CMOR changing with the times. For the has established the Caller Hotline NCPP took a first step in this direc- 2000 election season, the NCPP’s Research Information Systems, or tion with its March 2000 meeting on Polling Review Board intentionally “CALL CHRIS,” where consumers “Monitoring Polls & Poll Coverage.” included Humphrey Taylor of Harris can learn more about a specific inter- AAPOR also took a stand last year Interactive. This gave the board an view or surveys in general. Frontline with a New York Times article on the authority on cyber surveys useful in data collectors can refer respondents subject by Michael Traugott. Special adjudicating web push polls, much as who do not understand the difference attention should be paid to influenc- it does telephone pushing. In addi- between push polling and push ques- ing course work at political manage- tion, educational efforts by NCPP tioning to this interactive voice infor- ment and journalism schools. It would and AAPOR are beginning to include mation system, or to CMOR’s website also be useful to provide clear talking discussion of this growing medium, at www.cmor.org. Both services will points to frontline data collectors so as the 2000 and 2001 AAPOR con- provide further clarification on push they can respond to poorly informed ference schedules indicated. polling by a third-party source. Wider research consumers in a knowledge- use of these resources will also serve to able fashion. Encouraging the efforts of AAPOR, increase respondent confidence in the AAPC, CMOR, and the NCPP will legitimacy of a particular survey, head o much for taking care of the somewhat mitigate the effects of inevi- off potential misinformed complaints, present. What about dealing with table future push polls. The more and make a small contribution to pre- Sthe problem of push polling in researchers use these resources, the venting the further erosion of coop- the years to come, as it makes its way greater their credibility with voters and eration rates. online? consumers of research data will be.

Public Perspective, September/October 2001 39