Pioneer DaysThe promise of online polling By Anna Greenberg and Douglas Rivers

t is clear that Campaign 2000 served Because Knowledge Networks’ sample Knowledge Networks surveys con- as the first great experiment of includes respondents previously with ducted during the primary season Ionline survey research with elec- and without internet access, it over- yielded higher levels of undecided toral politics. Numerous organiza- comes some of the sampling challenges response than polls that pushed re- tions attempted to forecast the out- facing other online ventures. Panel spondents to make a choice. come based on internet straw polls and weights are applied to correct for un- panels, experimented with online de- equal probability of selection due to ne significant advantage of bates and focus groups, and creatively multiple phone lines, and post-stratifi- web interviewing is the low employed multimedia technologies to cation weights are based on Current Ocost of contact, compared to test reactions to campaign ads and de- Population Study data. traditional telephone methods. Knowl- bates. Other ventures tried to sell edge Networks, like nearly every other information about their registered us- hile Knowledge Networks polling organization, tracked vote in- ers as “public opinion data” and re- did not attempt to forecast tentions over the fall campaign. Un- cruited their members to participate in W the election outcome in like most other organizations, how- focus groups for clients. 2000, the organization conducted na- ever, Knowledge Networks was able to tional sample polls throughout the elec- reinterview the same respondents im- As we face declining telephone response toral cycle. These data consistently mediately following the election. rates and the limitations of phone tech- yielded results remarkably similar to nology, we need to assess how well simultaneous phone surveys. For ex- With a sample of 17,704 adults, the online polling takes our nation’s po- ample, a survey of likely voters con- post-election reinterview conducted litical pulse and fosters innovative ways ducted for iVillage.com in late Sep- from November 8-15, 2000 was able to measure our preferences. tember 2000 showed Al Gore with a 3- to identify defections of Ralph Nader point lead over George W. Bush, a voters (mostly toward Gore) as well as nowledge Networks is a firm result comparable to a series of tele- a swing of undecided voters over the that conducts commercial and phone surveys from the same period. weekend (again in Gore’s favor). K political research for a wide Twenty-eight percent of respondents variety of corporations, media outlets, This recurring consistency suggests that who said that they intended to vote for and political and non-profit organiza- the Knowledge Networks panel pro- Nader ended up voting for Gore, com- tions. Formerly known as InterSurvey, duces accurate information. It also, pared to 13% for Bush. This is consis- Knowledge Networks maintains a na- however, masks a significant mode ef- tent with strategic voting behavior (vot- tionally representative, web-enabled fect in online surveys. In our experi- ing for the second most preferred can- panel, recruited using random digit di- ence, there is a relatively higher “don’t didate to avoid casting a wasted vote). aling telephone methodology.1 Every know” or “undecided” response with Gore also garnered 55% of the pre- participating Knowledge Networks online surveys. Without interviewers election undecided voters, compared household receives free hardware, free to probe “voluntary” don’t know re- to 33% for Bush and only 6% for internet access, free email accounts, and sponses further, respondents frequently Nader. The late shift toward Gore ongoing technical support. Participants remain undecided. helped account for the closeness of the receive a short multimedia survey about race, even as the public polls showed once a week. Surveys are delivered by This effect was demonstrated in a Bush with an approximate 2-point ad- email on the same standardized hard- late 1999 Knowledge Networks sur- vantage on Election Day. ware, through the television set. vey conducted for The Washington Post, in which Gore received 38% of n additional benefit of web in- the vote and Bush received 48%, terviewing is its suitability for Anna Greenberg is assistant professor of while 14% remained undecided. In A conducting complex and in- public policy, John F. Kennedy School of a simultaneous phone survey com- teresting experiments. For example, in Government, Harvard University. Dou- missioned by ABC News/Washing- a study designed by Simon Jackman, glas Rivers is CEO, Knowledge Networks ton Post, Gore garnered 39% of the Josh Clinton and Doug Rivers (all of and professor of political science, vote, Bush received 55%, and only Stanford University), panel members Stanford University. 1% offered no opinion. Similarly, rated the relative impact of potential

40 Public Perspective, March/April 2001 vice presidential picks on their pro- dents saw three of the eight commer- Assisted Telephone Interviewing pensity to vote for Bush or Gore. Ap- cials in random order to create eleven (CATI), such as randomization, split proximately 8,000 panelists were given treatment groups, such that every pair- sampling, and branching schemes with 40 different hypothetical match-ups, ing of ads was seen by at least one visual material. pairing Bush with Elizabeth Dole, group. When asked the purpose or Frank Keating, John McCain, Colin objective of the interest group adver- Internet panels can also target sub- Powell, Tom Ridge, Fred Thompson tising, panelists clearly believed that populations more efficiently than tele- and Christine Todd Whitman, and the interest group ads were urging them phone surveys because they maintain Gore with Evan Bayh, Bill Bradley, to vote for or against a particular can- detailed demographic and behavioral Richard Durbin, Diane Feinstein, John didate. Less than 10% considered these information on respondents. This al- Kerry and Robert Rubin. ads as primarily concerning issues, while lows researchers to target surveys to 80% thought the ads suggested they low-incidence populations, such as Jew- Leaving aside the poor prognostica- vote a particular way in the election. ish Americans, African Americans, us- tion about the eventual vice presiden- ers of particular products, or opinion tial selection, the experiment yielded A second multimedia study compared elites. The panel design creates flex- interesting results. Not surprisingly, the results of online “insta-polls” con- ibility, allowing the researcher to con- Bush/Powell and Bush/McCain eas- ducted during the presidential debates duct follow-up interviews with par- ily defeated any Democratic ticket, with post-debate telephone insta-polls. ticular respondents. while the Gore/Bradley match-up was Using the Knowledge Networks panel, the strongest Democratic pair. More CBS News pre-recruited a national As its panel size increases, Knowl- remarkably, female candidates such sample of registered voters to log in at edge Networks will have the ability as Elizabeth Dole and Christine Todd a particular time on the night of the to study specific congressional dis- Whitman greatly strengthened the debates. After answering an initial tricts or target even more specialized Bush ticket, which beat every Demo- screening question on whether or not subgroups, such as single-issue vot- cratic pair with the exception of Gore/ they watched the debate, panelists were ers. We are further refining the ca- Bradley. Moreover, selecting a fe- asked a series of questions about the pability to utilize dial-meter tech- male vice presidential running mate candidates’ performance. nology and online focus groups, in muted the gender gap, leading to which participants electronically reg- greater support for Bush among When the results of the CBS News/ ister their reactions during the course women voters. Knowledge Networks online insta- of an event. CBS News utilized this poll were compared to post-debate technique during the third presiden- nother series of experiments telephone surveys by ABC and CNN, tial debate. The resulting data pro- used Knowledge Networks’ au- the former was found to be more vided a second-by-second time series A dio-visual and experimental likely than the latter to favor Gore in of voter reactions to specific candi- design capacity. In a study commis- the debates, both in the pre-debate date response and demeanor. sioned and designed by David Magleby vote choice and in assessing the de- of Brigham Young University, partici- bate winner. Other national insta- We have no doubt that the online pants in the Knowledge Networks polls had their own biases—ABC experiment will continue as more firms panel evaluated election advertisements News and CNN/USA Today/Gallup and organizations attempt to harness produced by candidates (hard money), showed larger Bush leads in their pre- the power of the web in measuring party committees (soft money), inter- recruited panels than in their own public attitudes and preferences. The est groups, and pure issue campaigns. daily tracking polls. challenge for the survey research com- The aim of the study was to determine munity is to maintain the integrity of whether interest group advertising and he Knowledge Networks panel, the data and the spirit of innovation as soft money ads adhere to the intent of and online survey research we incorporate the increasingly sophis- Buckley v. Valeo, which permits inde- Tmore generally, holds even ticated technology and speed offered pendent expenditures by groups of in- more potential to conduct public opin- by the internet. dividuals on behalf of candidates as ion research in innovative and cre- long as an ad does not explicitly advo- ative ways. The technology permits cate the election or defeat of a clearly the use of multimedia content with identified candidate. much larger samples than central fa- cility work, such as focus groups. It Endnote In the experimental design, a pair of allows for sophisticated experiments 1See www.knowledgenetworks.com/science/ each ad type was selected. All respon- of the type permitted by Computer methodology.html for more information.

Public Perspective, March/April 2001 41