The Influence of Voting Advice Applications on Preferences, Loyalties and Turnout: an Experimental Study

Article POLITICAL STUDIES: 2015 Political Studies doi: 10.1111/1467-9248.122132016, Vol. 64(4) 1000–1015 The Influence of Voting Advice © The Author(s) 2015 Reprints and permissions: ApplicationsThe Inﬂuence on ofPreferences, Voting Advice Loyalties Applications sagepub.co.uk/journalsPermissions.nav on DOI: 10.1111/1467-9248.12213 andPreferences, Turnout: An Loyalties Experimental and Turnout: Study An psx.sagepub.com Experimental Study

Zsolt Enyedi Central European University

Voting Advice Applications (VAAs) are increasingly popular, yet little is known about their impact. This article investigates their influence on party choice, party loyalty and electoral participation, relying on a complex experiment conducted before and after the 2010 Hungarian election. Participants were directed to two VAAs, some received advice from one and some from both, while the control group visited none. According to subjective recollections, 7 per cent changed their vote intentions, but according to the panel study the VAAs were unable to direct users to specific parties. Sheer exposure to the advice did not have mobilising or demobilising effects either, but preference- confirming outputs increased party loyalty while preference-disconfirming recommendations decreased it, and double exposure amplified further the impact of the VAAs. Converging advice from two different sources increased the rate of electoral participation, but more by provoking, rather than by persuading, the users.

Keywords: voting advice applications; party choice; electoral participation; experiment; political parties

Voting Advice Applications (VAAs) – that is, websites that identify the user’s proximity to candidates and parties based on a set of political questions – are becoming increasingly popular across Europe and have also reached North America, the Arab World and Latin America (Mendez, 2012; Wall et al., 2012). They aim to increase political knowledge, self-awareness and interest in elections and foster rational decision making. Apart from these goals, they have normatively neutral functions such as entertainment. Potentially, they can also manipulate. The more citizens use these tools, the more important it becomes for the public to know what goes on within the ‘black box’ of VAAs and what influence they have on the attitudes and behaviour of citizens. Are users stimulated to participate at elections and do they listen to the offered advice? The experiment presented below indicates that simple exposure to a VAA does not affect electoral turnout, and the number of VAA users who abandon their favourite party in order to follow the recommendations is very small. But VAAs do seem to have the power to consolidate or undermine party preferences. Those who received advice that was in line with their preferences were particularly likely to stay loyal to their party and if the preference-confirming recommendation came from two VAAs, then the level of party- switchers dropped further. Recommendations that contradicted a user’s original preferences increased the number of those who abandoned their party. Finally, identical recommendations coming from the two VAAs led to a particularly high level of electoral participation – at least compared to contradictory outputs. The results indicate that under certain conditions VAAs may influence electoral behaviour but not primarily through channeling voters from one party to another.

Spread, Infrastructure and Impact of VAAs The first VAA was intended to educate secondary school pupils (De Graaf, 2010), but VAAs soon became major media and scientific enterprises. In 2006, Kieskompas and StemWijzer generated more than 5 million recommendations in the Netherlands, involving more than a third of the electorate (Walgrave et al., 2008). VAAs soon spread across Europe and became particularly popular in Belgium, Germany, Finland and Swit- zerland (Louwerse and Rosema, 2011; Marschall, 2005, Nuytemans et al., 2010). Since the appearance of the EU Profiler and VoteMatch Europe in 2009, VAAs also contribute to the development of a pan-European political space (Threchsel and Mair, 2011). In spite of, or exactly because of, the fact that users tend to be unrepresentative of the general citizenry (i.e. they are more educated, younger and more politically active, even compared to the average internet user), political elites take VAAs seriously. In the Netherlands the VAA questionnaires are answered by a comprehensive team of party officials, sometimes including both national and European Parliament leaders (De Graaf, 2010). In Switzerland, where the websites focus on individual candidates, in 2007 about 90 per cent of future MPs answered the circulated questions. The reliability of the answers was documented by a follow-up study that showed 85 per cent of the MPs voting according to the previously provided answers (Ladner et al., 2010). Parties often get the chance to comment, justify or explain their position (Nuytemans et al., 2010). The display of supporting arguments implies that these websites can turn into platforms of persuasion and can contribute to the deliberative aspects of democracy. The structure of the questionnaire and the ways in which the answers are treated differ across the enterprises. Some VAAs give more weight to the questions considered important by the users. Some even allow the parties to rank the issues according to relevance. In Austria, both the parties and the citizens can weight individual questions (Mayer and Wassermair, 2010). The raw material on which party positions are based ranges from statements by party authorities to expert judgements or party documents, just like in mainstream political science research. Kieskompas (Krouwel et al., 2012) and the EU Profiler (Threchsel and Mair, 2011), and their later offspring, Vote Match and EUandI, rely on many data sources and on an intensive dialogue between politicians, practitioners and scholars. How user–party distance is measured also varies across the VAAs. The different algorithms are often based on varying assumptions about the spatial structure of party politics. Some VAAs assume a few-dimensional, simplified structure, while others treat each question as a separate dimension (Wall et al., 2012). Since VAAs provide the user with cheap and relevant information and since they highlight the differences between electoral alternatives, they are expected to boost turnout (Marschall and Schultze, 2012). Summarising experience across a number of countries, Lorella Cedroni has concluded that in countries where VAAs are well entrenched about 10 per cent of users may have been mobilised as a result of the advice (Cedroni, 2010, p. 255). Other reported figures range from 3 per cent (Ruusuvirta and Rosema, 2009) to 20 per cent (Mykkänen and Moring, 2006).

© 2015 The Author. Political Studies © 2015 Political Studies Association POLITICAL STUDIES: 2015 1002 Political Studies 64 (4) VAAS ON LOYALTY, PREFERENCE AND TURNOUT 3 Voting preference change can be even larger, particularly where the choice is not only between parties but also among candidates. Accordingly, in Switzerland, a particularly large number of users (15 per cent) have claimed that they voted exactly for the list of candidates recommended by the VAA. Two thirds of the users reported some sort of impact on attitudes (Ladner et al., 2010). In Finland, 7 per cent of the respondents in 2003 and 19 per cent in 2007 acknowledged the inﬂuence of VAAs, while 15 per cent claimed to have based their electoral choice entirely on the received advice (Ruusuvirta, 2010). In Germany, 6 per cent of users reported a change in voting preferences (Marschall, 2005), and in the Netherlands, their number was estimated to be 10 per cent (Kleinnijenhuis and van Hoof, 2008). There are, however, studies that found only minimal effects. In Italy, only 2–3 per cent of users indicated that they were ready to vote for a different party or to go to vote due to the VAA experience (De Rosa, 2010). In Finland, Juri Mykkänen and Toni Moring (2006) found a mere 3 per cent vote change. In Belgium, only 1.1 per cent of the respondents acknowledged a shift in their preferences, but the panel study demonstrated that a third of this tiny group stayed loyal to their original party choice (Walgrave et al., 2008). There are reasons to be sceptical about ﬁgures indicating large-scale effects. VAA users can be expected to belong to the more informed segments of society. Informed voters tend to be resistant to information that is incompatible with their predispositions, especially at the late stage of an electoral campaign (Bartels, 1993; Zaller, 1992). Most of the existing studies are based on subjective assessments by users, often collected prior to the actual elections. Humans are notoriously bad at estimating changes in their own attitudes, and high levels of distortion were documented concerning VAAs too (Walgrave et al., 2008). While in general people tend to think that their attitudes are more stable than in reality (Markus, 1986), it is reasonable to expect those who volunteer to answer questions related to VAAs to be particularly enthusiastic and consequently prone to exaggerate the role of VAAs (Wall et al., 2014). Post-election surveys alone may not provide reliable information on the nature of the advice either. In order to measure actual attitude change induced by VAAs we need information about both pre-VAA preferences and subsequent actual electoral behaviour.

Variables and Hypotheses The experiment conducted in Hungary in 2010 was designed according to these ambitions. Similarly to Walgrave et al.’s (2008) panel study, it contained information both on subjective evaluations of the VAAs and on actual subsequent voting behaviour, but it also included two VAAs. Due to the latter innovation, the project could provide variance in the strength and unequivocality of the advice and measure the consequences of such ‘treatments’. Because the sample was a good representation of internet users in Hungary, the results can be extrapolated beyond the circle of VAA enthusiasts. Given the experimental design one could investigate the impact of VAAs at various levels: observing the subjective evaluations, contrasting the original preferences with reports about how the participants voted at the election, contrasting VAA users with non-users, contrasting those who visited one VAA with those who visited two of them,

© 2015 The Author. Political Studies © 2015 Political Studies Association POLITICAL STUDIES: 2015 Enyedi 1003 4 ZSOLT ENYEDI and comparing those who received converging advice from the two websites with those who were advised by them to vote for different parties. The study contained three major dependent variables: mobilisation (operationalised as the difference between the original intentions concerning participation at the election and the actual turnout), loyalty (i.e. the rate of party list vote in line with the preferences revealed in the first wave of the experiment) and the change of party preferences (i.e. the proportion of those who voted for the VAA outputs in spite of their different original intentions). Next to exposure or no exposure to the VAAs, the treatments (or independent variables) included variation in the number of recommendations (one or two recommendations), the relationship of the recommendation to the original preference (preference-confirming or preference-disconfirming recommendations) and, concerning those who visited two VAAs, the relationship between the two recommendations (converging or diverging advice). Drawing on the persuasion studies (see particularly, Cacioppo and Petty, 1979; Sawyer, 1981; Stang, 1975; Zajonc, 1968), I expected the impact of repeated advice (i.e. advice coming from two sources) to be particularly large (Table 1). As far as party choice is concerned, I hypothesised that the VAA users adjusted their party preferences to the received recommendations (FOL), particularly if they visited two VAAs and received the same recommendation from both (CONVFOL). Note that the impact on party choice implies the ability of the treatment to move the user away from his or her original preferences to the recommended party. Therefore, it can be grasped only among those who received preference-disconfirming advice.

Table 1: Proposed Hypotheses

Effect

Vote for the Vote for recommended the original Participation party preference at the election

Treatment VAA use FOL LOY PAR VAA with confirming advice CONLOY CONPAR VAA with disconfirming advice DISCLOY DISCPAR VAA with repetition of confirming advice REPCONLOY REPCONPAR VAA with repetition of disconfirming advice REPDISCLOY REPDISCPAR VAA with diverging advice DIVPAR VAA with converging advice CONVFOL CONVPAR Confirming versus disconfirming advice CODISCLOY CODISCPAR Converging versus diverging advice CODIPAR Two versus one disconfirming advice REDISCFOL REDISCLOY REDISCPAR Two versus one confirming advice RECONLOY RECONPAR

Note: Labels in italics refer to negative relationships: depression of participation and party loyalty.

© 2015 The Author. Political Studies © 2015 Political Studies Association POLITICAL STUDIES: 2015 1004 Political Studies 64 (4) VAAS ON LOYALTY, PREFERENCE AND TURNOUT 5 VAA exposure was also expected to decrease party loyalty (LOY), unless the advice confirmed one’s previous preferences (CONLOY), in which case a higher degree of party loyalty was expected among users than among non-users. The repetition of preference- confirming advice by both VAAs was expected to strengthen loyalty (REPCONLOY), while preference-disconfirming advice (DISCLOY), especially if repeated (REPDISCLOY), was expected to reduce loyalty. The rate of party loyalty was expected to be the highest when two recommendations supported the original preference and to be the lowest when both of them disconfirmed it. Concerning turnout, VAA users were expected to participate in elections in larger numbers than the non-users even after controlling for their first-wave inclination to vote (PAR). The boost in turnout was expected to be particularly pronounced if the advice was preference-confirming (CONPAR), if the advice given by one VAA was repeated by the second VAA (CONVPAR) and if these two recommendations coincided with the original preference of the user (REPCONPAR). Electoral turnout was expected to be depressed if the advice was preference-disconfirming (DISCPAR), if the disconfirming advice was repeated (REPDISCONPAR) or if the two received recommendations contradicted each other (DIVPAR). The hypotheses above imply a contrast between the control group and the VAA users. By comparing various subgroups of VAA users one can further specify how the variation in the stimulus may affect behaviour. Preference-confirming advice, compared to a preference-disconfirming recommendation, was expected to lead to higher participation rates (CODISCPAR) and higher rates of party loyalty (CODISCLOY). The difference between the impact of converging (identical) and diverging (contradictory) advice was analysed by zooming in on the group that visited both VAAs. Here the expectation was that those who received converging advice went to vote in larger numbers (CODIPAR) than those who received diverging advice. Finally, in order to grasp the influence of repetition, the group that received one recommendation was contrasted with those participants who received two recommendations, also taking into account whether the advice was of the confirming or disconfirming sort. Repeated preference-confirming advice was expected to induce higher rates of party loyalty (RECONLOY), and higher rates of participation (RECONPAR) than a single recommendation, while preference-disconfirming advice coming from two VAAs was hypothesised to have the opposite effect: to lead to a lower rate of party loyalty (REDISCLOY) and a lower rate of participation (REDISCPAR) than advice from one VAA.

The Experimental Design The panel study began on 2 April 2010. The first wave lasted till the day of the first round of the parliamentary election on 11 April. The questionnaire was sent to 6,641 members of IPSOS’s pool of potential participants in online polling; 38 per cent of those contacted (i.e. 2,502 subjects) agreed to take part in the research. After completing the original questionnaire, four stratified sub-samples were created. The four groups were representa- tive in terms of gender, age group, residence and education of those who use the internet at least once per week.

© 2015 The Author. Political Studies © 2015 Political Studies Association POLITICAL STUDIES: 2015 Enyedi 1005 6 ZSOLT ENYEDI Two VAAs have been integrated into the experiment. The first one – the Electoral Compass (Választási Iránytu˝) – was the enterprise of the international NGO the Association of European Election Officials (ACEEEO) and was an offspring of the Europe-wide enterprise VoteMatch. The second VAA – Vote Probe (Szavazatszonda) – was operated by the Politikon Institute and the National Association of Political Science Students, and the organisers followed the scheme of Wahl-o-Mat. A total of 639 participants were asked to visit the website of the Electoral Compass, 618 were guided to Vote Probe and 631 were asked to complete both questionnaires. The fourth group received no request. Eventually, 517 panel members completed Electoral Compass, 596 Vote Probe, 349 both and 1,036 none.1 Altogether, the panel contained 926 Electoral Compass users and 945 Vote Probe users. The bulk of the panel members visited the websites during 8 and 9 April, right before the first round of the election. Between 8 and 16 April the participants were asked to report about their experiences with the VAAs: 331 panel members evaluated Electoral Compass and 335 the Vote Probe, while both were evaluated by 248 respondents. In this wave, a total of 911 panel members participated, which is 34 per cent less than in the first wave. After the second round of the parliamentary election, between 5 and 13 May, the participants were approached with a new questionnaire that included questions on the first round vote. A total of 1,767 panel members replied, which means a 29 per cent attrition rate compared to the first wave. From the point of view of the two VAAs, the distribution was as follows: 455 of the Vote Probe users, 383 of the Electoral Compass users, 286 of those who used both and 603 of those who used none filled in this questionnaire. Given that the experiment took place in the midst of an actual election, the political context needs to be taken into consideration. While in the regions party systems tend to be fluid, the Hungarian party politics is relatively institutionalised (Enyedi and Casal-Bértoa, 2011). It is also a polarised system, and polarisation implies high political stakes. Indeed, after the 2010 election, revolutionary constitutional changes followed (Enyedi, 2015). On the other hand, since Fidesz had a continuous and significant lead in the polls, the outcome was a foregone conclusion (Enyedi and Benoit, 2011), and therefore the stakes as perceived from the voters’ point of view were actually minor. The high degree of polarisation and the lack of competiveness implied relatively crystallised party preferences and disenchantment with politics in general. The newly emerged, programmatic and issue-specific political actors, Jobbik and LMP, were not expected to have any influence on the government. The other two actors in the party landscape – Civic Movement and MDF – were even more marginal, and indeed both of them remained below 1 per cent of the national vote. These characteristics of the context limited the likelihood of programme-based persuasion. On the other hand, the fact that Fidesz felt no need to campaign, MSZP was in disarray and the two new parties lacked the resources to generate major campaign events resulted in an uneventful campaign (Závecz, 2010). This means that if one finds a difference between the original party preferences and the actual vote, separated by a few days, the chances are good that the difference is due to factors related to the experiment and not to some external event. Naturally, since the experiment did not take place under laboratory circumstances, it was not possible to exclude all forms of environmental influence. But this price was worth paying given the important validity-related benefits of a study that can incorporate real-life elections.

Subjective Evaluations The impact of VAAs is supposed to be influenced by whether users find them neutral, professional and credible (Lupia and Philpot, 2005). Therefore, a battery of questions asked the opinion of participants about these aspects. According to the answers, most of the users were satisfied with the VAAs: 93 per cent of them found the experience worthwhile. The large majority also considered the VAAs trustworthy, found that the questions adequately covered the most important political issues and only 7–8 per cent reported that salient topics were missing (Table 2). There was somewhat less confidence in the impartiality of the websites (Table 3): 29 per cent (Compass) and 33 per cent (Probe) of the respondents thought that the VAA favoured some parties. Furthermore, 41 per cent claimed that their party choice is driven by different considerations than the one reflected by the questions.2

Advice Table 33 shows the contrast between the partisan composition of the sample and the distribution of the advice given by the VAAs. The right-wing Fidesz, the socialist MSZP and the green LMP received fewer recommendations than their actual support in the sample, while the extreme-right Jobbik, the populist Civic Movement and the centre-right MDF received more. Compared to the election results, Fidesz was particularly under- represented. But the most remarkable result is the relative similarity of the distribution across the two VAAs.

Table 2: Distribution of Subjective Evaluations (%)

The questions covered Taking everything well the most important together, how much political issues do you trust the VAA?

Compass Probe Compass Probe

Not at all 1.5 0.9 4.2 6.0 Rather not 6.5 6.4 14.5 13.7 Rather yes 75.7 75.8 69.0 70.4 Completely 16.4 16.9 12.3 9.9 N 550 566 550 566

Table 3: ‘Were the Issues Addressed by the VAAs Equally Important for the Parties or was the Questionnaire Biased in Favour of Some Parties?’ (%)

Compass (N = 447) Probe (N = 494)

Equally relevant for the parties 71.3 67.2 Favoured certain parties 28.7 32.8

© 2015 The Author. Political Studies © 2015 Political Studies Association POLITICAL STUDIES: 2015 Enyedi 1007 8 ZSOLT ENYEDI While the aggregate differences are not large, the individual reports show that the majority of the participants received advice that differed from their party preferences (Table 4).4 Among the analysed cases, 44.6 per cent of the advice matched the original preferences. The majority of the Socialists (51.5 per cent), Fidesz (50.9 per cent) and Jobbik (60 per cent) supporters were directed to their own party, but this was not the case with the rest of the parties (Civil Movement: 34.8 per cent; MDF: 39.1 per cent; LMP: 15.1 per cent). The subjective reports show a similar picture (Table 5), although somewhat different proportions: 21–23 per cent of respondents reported receiving advice in line with their preferences, while 48–53 per cent reported that the recommendations were close. The large size of the latter category reﬂects the fact that the VAA outputs contained not only the name of the closest party, but also the distance to all the parties. In spite of the random selection of the participants, those who were directed to the two websites differed somewhat from each other in terms of partisan composition. They may have also differed in other non-measured aspects, like the intensity of their party preferences. To have a more precise comparison one needs to examine those who visited both websites. After excluding all those with missing values on some relevant dimension, we were left with 218 such cases. Their distribution can be seen in Table 6. As shown by Table 6, 71 per cent of the cases are in the diagonal containing identical advice. As this information comes not from recorded outputs but from recollections, some degree of distortion (i.e. projection of preferences onto recalled advice or projection of one advice onto another one) is possible. But even if the actual overlap was somewhat smaller, this result can be treated as evidence of the relatively high reliability of the VAAs as detectors of political orientations. The fact that most of the diverging recommendations

Table 4: Contrast between Original Party Preferences and the VAA Advice (%)

MSZP Fidesz Jobbik MDF LMP Civil

Compass Original preferences 19.9 34.2 19.9 4.2 18.4 3.3 (N = 428) Received advice according to recall 16.1 24.4 26.7 14.3 5.6 12.9 The difference, rounded −4 −10 +7 +10 −13 +10 Probe Original preferences 19.5 32.5 18.9 4.2 21.0 3.8 (N = 466) Received advice according to recall 17.6 21.1 25.3 20.3 6.0 9.7 The difference, rounded −2 −11 +6 +16 −15 +6

Table 5: ‘How Close were the Results to Your Original Party Preferences?’ (%)

Compass (N = 507) Probe (N = 562)

Very far 10.6 10.8 Quite far 15.1 17.4 Quite close 53.0 48.4 Completely identical 21.3 23.4

Table 6: Advice Given to Those Who Visited Both VAAs (%, N = 218*)

Probe

MSZP Fidesz Jobbik LMP MDF Civil M. Compass

Compass MSZP 13.3 – – 1.8 – 0.9 16.1 Fidesz – 13.8 2.3 2.8 .5 2.3 21.6 Jobbik 1.4 4.6 22.5 1.8 – 1.8 32.1 LMP 1.4 0.5 0.9 9.6 0.5 0.5 13.3 MDF – 0.5 0.5 0.9 3.21 – 5.0 Civil M. – 1.8 1.4 – 0.5 8.3 11.9 Probe 16.1 21.1 27.5 17.0 4.6 13.8 100.0

Notes: *Rows present advice given by Electoral Compass, columns present advice by Vote Probe. Bold indicates identical advice.

involved Fidesz versus Jobbik – two parties whose electorates are known to overlap – is further proof of the converging character of the two VAAs. Electoral Compass issued somewhat more recommendations for Jobbik, while Vote Probe directed more users to the LMP. This difference indicates the relevance of statement selection. Compass asked seven questions on the favourite campaign topics of Jobbik (the Roma, the welfare-recipients, international relations and law-and-order), while Vote Probe asked only three. The number of questions on LMP’s topics (environmentalism, proportionality of the electoral system, party ﬁnance) was identical, but because the questionnaire of Vote Probe was substantially shorter than the one of Compass the representation of LMP’s concerns was more pronounced in the case of the former VAA. The overall convergence of outputs is somewhat surprising given the differences in the makeup of the two VAAs. Compass started to function seven weeks earlier than Vote Probe, it allowed the users to indicate which issues they ﬁnd particularly important, gave more weight to these positions and provided equal space in its output to all party distances. Vote Probe presented fewer questions to the users, featured the logo of a centre-right party weekly on its site and, in the output, placed somewhat more emphasis on the closest ranked party than on the others. It seems, however, that the fundamental similarity in the underlying algorithms was much more consequential than the listed differences (see Louwerse and Rosema, 2011).

Impact5 Subjective Evaluations The participants were asked to estimate the VAA’s inﬂuence on their own preferences. Different questionnaires were provided for those who visited both VAAs and those who visited only one of them.6 Members of the former group (248 respondents) could choose between three options: they could indicate whether the VAAs strengthened their original preferences, changed their voting intentions or had no impact; 50.6 per cent of them

Table 7: Subjective Reports of Those Who Completed One VAA (N = 1113)

Reported effect %

Surprised me 15.7 Convinced me 3.8 Strengthened my original opinion 19.0 Helped me decide how to vote 5.8 Changed my original voting intention 0.9 Strengthened my interest in elections 7.5 Strengthened my interest in politics 4.0 Motivated me to participate at elections 7.5 Motivated me not to participate at elections 0.2

informed us that the experience conﬁrmed their original preferences, 7.3 per cent claimed that it has changed their voting intentions and 42.1 per cent reported a lack of any impact. Those who visited only one of the VAAs were provided with nine reply options. They were also allowed to skip the entire block and to choose more than one answer, therefore the replies do not add up to 100 per cent. As Table 7 shows, the option ‘it has changed my voting intentions’ was selected by very few respondents; 5.8 per cent opted for ‘helped me to decide which party to vote for’, 7.5 per cent chose the category ‘it motivated me to participate at the elections’ and 3.8 per cent ticked ‘it has convinced me’. Considerably larger groups chose the options ‘it has surprised me’ and ‘it has strengthened my original opinion’. The battery on subjective recollections suggests that only in a small portion of cases were the VAAs directly changing party preferences, but the participants experienced many other forms of impact.

VAA Users versus Non-Users Due to the fact that experiment included actual vote recall, it is possible to investigate further whether simple exposure to VAAs induced support for the recommended parties. If the answer is ‘yes’, the first thing to expect is that the recommendations coincide to a significantly higher extent with the final vote than with the original preferences. In fact, the overlap between recommendations and vote was only 1 per cent larger (increase from 45 to 46 per cent) than between recommendations and original preferences; 26.1 per cent of the participants voted for a different party than indicated in the first round, but because the vote shifts cancelled each other out, the overall distribution of party preferences remained essentially the same and no party’s support changed by more than 2 per cent. Among those who switched party,7 24.4 per cent followed the received VAA recommendations.8 It would be tempting to attribute their movement to the advice of the VAAs, but we must remember that a random reshuffle would have brought 20 per cent of them to the respective parties anyway as the switchers could choose among five alternatives. The proportion of those who originally had a party preference but then abandoned it and voted for the recommended party was only 4.6 per cent. Almost as many (2.7 per cent)

Table 8: Hypotheses Conﬁrmed

Effect

Vote for the Vote for recommended the original Participation party preference at the election

Note: Labels in bold refer to conﬁrmed relationships.

turned against the recommended party even though they originally supported it. The 2 per cent difference between these two figures (within the entire sample only 1 per cent) indicates a very weak preference-directing power. The final evidence for the lack of impact was provided by the linear regression of the electoral party choices on the recommendations, which showed, after controlling for the original party preferences, that the recommendation is inconsequential (significance: 0.114). To conclude, the hypothesis concerning the impact of a simple VAA exposure on party choice (FOL) found no support (Table 8).9 Increasing the strength of the stimulus by focusing on those who received two recommendations pointing in the same direction (CONVFOL) does not alter the picture as the VAA advice was inconsequential in that group too (significance: 0.545). The tests of the impact of VAA exposure on turnout and party loyalty (LOY and PAR) also turned out to be negative. Those who completed the VAA test went to vote in somewhat larger numbers than those who didn’t (89.5 versus 86.4 per cent), but after controlling for the first-wave attitudes towards electoral participation the regression of actual turnout on VAA exposure showed no significant impact (significance: 0.706). The rate of party loyalty was virtually the same (73 and 75 per cent) whether one completed a VAA or not (significance: 0.234).10 The next question is whether participants who received preference-confirming (CONLOY) or preference-disconfirming (DISCLOY) advice, and those who received two such recommendations (REPCONLOY and REPDISCLOY), stayed loyal to their original preferences to a different extent than the control group. In this case, all four hypotheses proved to be right: the party loyalty of these four subgroups was significantly different from the party loyalty rate of the control group. A preference-confirming

© 2015 The Author. Political Studies © 2015 Political Studies Association POLITICAL STUDIES: 2015 Enyedi 1011 12 ZSOLT ENYEDI recommendation increased loyalty from 75.3 to 84.7 per cent (significance: 0.008), while a disconfirming recommendation brought it down to 66.4 per cent (significance: 0.009). The repetition of confirming advice boosted loyalty by almost 20 per cent to 94 per cent (significance: 0.001), while the repetition of disconfirming advice depressed it by more than 10 per cent to 61.2 per cent (significance: 0.029). As hypothesised, the rate of party loyalty was highest when two recommendations supported the original preference and lowest when both of them disconfirmed it. The participation rate was expected to increase if the recommendation endorsed the party preference (CONPAR), especially if one received two confirming recommendations (REPCONPAR). Assuming that the strength of the signal itself mobilises, the participation rate was also assumed to be high among those who received two ‘nudges’ (CONVPAR). Those who received disconfirming advice (DISCPAR), especially twice (REPDISC), and those who received contradictory recommendations (DIVPAR) were expected to be demobilised. In fact, while the participation rate of all categories of users surpassed the participation rate of the non-users, after controlling for the original inclina- tions to vote none of the treatments retained a significant impact and therefore all participation-related hypotheses have to be rejected.

Subgroups of VAA Users In the next step, the subgroups of the VAA users were compared. First, those who received confirming advice were contrasted with those who received disconfirming advice. The hypothesis predicting a mobilisation effect (CODISCPAR) has to be rejected as those who received preference-confirming advice were only marginally more active at the election than those who received disconfirming advice (significance: 0.107). At the same time, clear evidence emerges for the loyalty-consolidating power of the preference-confirming outputs: only 15.3 per cent of those who were told to vote for their favourite party broke their loyalty, while more than twice as many (33.8 per cent) of those who were pointed to a different party did so (significance: 0.000). Similar results emerge if one measures the confirmatory nature of the advice by subjective evaluations: 29 per cent of those who claimed that the VAA advice was close to their original preferences (i.e. declared that the advice was ‘close’ or ‘very close’) abandoned their original party, while 45 per cent of those who claimed to have received disconfirming advice (‘far’ or ‘very far’) did so (significance: 0.002). The next question is whether the recipients of unequivocal VAA outputs participated at the election in larger numbers than those who received contradictory recommendations (CODIPAR). Originally, 84.7 per cent of the group that received contradictory recommendations planned to vote, while the respective figure was 81.9 per cent in the case of the converging-stimulus group. Eventually those who received identical recommendations went to the polls at a rate of 93.1 per cent, while the other group only at 85.5 percent – that is, both groups became more active than originally indicated, but for one group the increase was 0.8 per cent and for the other it was 11.2 per cent. The participation rate of the recipients of converging advice surpassed the participation rate of the VAA users by 4 per cent. Convergence, as opposed to divergence, proved to increase participation, even after controlling for original predispositions (binary logistic regression, significance: 0.034). Examining the rates of mobilisation,11 instead of the rates of participation, similar differ-

© 2015 The Author. Political Studies © 2015 Political Studies Association POLITICAL STUDIES: 2015 1012 Political Studies 64 (4) VAAS ON LOYALTY, PREFERENCE AND TURNOUT 13 ences appear: 2.9 per cent of those who received contradictory advice mobilised, while the respective number was 12.6 per cent in the case of converging advice (significance: 0.017). One may wonder whether the demonstrated boost in turnout was due to the fact that many of the converging recommendations were of a confirming sort. This was not the case. In fact, within this subsample those who received confirming advice mobilised less than those who received disconfirming suggestions (significance: 0.058). It seems that the unequivocal nature of the stimulus had a mobilising impact. It is imaginable that in other contexts other mechanisms could produce similar effects. For example, learning that one is supposed to support a party not only on the basis of one’s agreement with the programme, but also due to one’s social background, economic interests, or agreement with the party leader, may have the same reassuring and provocative effect. On the other hand, the final step included the investigation of the difference between exposure to one or to two VAAs. Neither the repetition of disconfirming (REDISCPAR), nor the repetition of confirming advice (RECONPAR) influenced the level of participation. The turnout of those who received two disconfirming recommendations was essentially the same as of those who received one (significance: 0.463). The recipients of two confirming recommendations turned out to be marginally more active, but the difference was not significant (0.209), especially after controlling for first-wave predispositions (significance: 0.302). On the other hand the hypothesis according to which receiving two, instead of one, preference-confirming advice strengthens loyalty (RECONLOY) was vindicated: the rate of vote-switchers among those who learned from one VAA that their original preference was the ‘right one’ was 19.2 per cent, while among those who received confirmation from two VAAs it was 6 per cent (significance: 0.021). One can reasonably conclude that double-confirmation (repetition) contributed to the consolidation of preferences. The counter-expectation, that being told twice that there is a party which is ideologically closer to the user than the preferred party will induce party-switching (REDISCLOY), failed the test: the two groups were not different (significance: 0.258). It seems that the repetition of disconfirming advice is somewhat less consequential than the repetition of confirming suggestions. Finally, those who received two identical (preference-disconfirming) recommendations were not more likely to listen to the advice than those who received one (significance: 0.154), and therefore REDISCFOL has to be rejected too.

Conclusions The principal peculiarity of this experiment is that it involved two VAAs. In spite of the fact that the two teams worked independently from each other and that intuition played a large role in the selection of the policy statements, their outputs were relatively similar.12 At the same time, the differences in recommendations indicated that statement selection indeed shapes output, particularly with regard to the programme-oriented parties (in this case LMP and Jobbik). The results presented lend support to those studies that have found minimal impact in terms of direct preference change. The hypotheses claiming that simple exposure to VAAs determined party preferences, strengthened loyalty or induced electoral participation have to be rejected. Electoral preferences are not superﬁcial opinions, but attitudes that are well

© 2015 The Author. Political Studies © 2015 Political Studies Association POLITICAL STUDIES: 2015 Enyedi 1013 14 ZSOLT ENYEDI embedded in layers of experience (Zaller, 1992). Therefore, expectations concerning VAA-induced change need to be modest. It seems that the large majority of citizens are unwilling to follow last-minute suggestions, even if they come from trustworthy sources and even if they are formulated in unequivocal form. Exposure to VAAs also failed to change participation rates, irrespective of the nature of the stimulus. These negative findings must be qualified, however. First, about 7 per cent of the respondents reported that the VAAs changed their party preferences. This relatively high ratio may be simply an exaggeration on the part of the participants, but it may also signal an actual transformation of preferences which – apparently – does not lead to a new voting pattern in the days following the experiment, but may make later changes easier. The lack of simple direct effects may be partly due to the fact that in the examined case party preferences were highly crystallised, the number of competing parties was relatively small and the analysed population exhibited a particularly high readiness to participate in the elections. It is also important to add that the political and civic relevance of VAAs can be significant even if their ability to change preferences and alter participation rates is small. In Germany, for example, half of the respondents claimed that VAAs made them motivated to seek further information on the position of the competing parties (Marschall, 2005). In Finland in 2007, the majority of the users named VAAs as their most relevant sources of information as far as elections are concerned, ranking them higher than television or newspapers (Ruusuvirta, 2010). In almost all instances those who took the VAA tests felt better informed and better prepared (Boogers and Voerman, 2003). The subjective evaluations reported above were in line with these beneficial side-effects, pointing to the ‘soft impact’ of VAAs. The analysis also revealed tangible effects. The degree of loyalty (or its opposite: party-switching) proved to be particularly sensitive to the manipulation of experimental conditions. Those who received confirming outputs remained more loyal than the recipients of disconfirming suggestions; those who were twice reassured were more loyal than those who were reassured only once. Preference-confirming advice increased party loyalty, while preference-disconfirming recommendations decreased it, not only relative to each other but also relative to the control group. The repetition of disconfirming advice depressed party loyalty, while the repetition of confirming advice boosted it by almost 20 per cent. Thus, even if the reassurance received from the VAAs does not lead to higher rates of participation, it does help to cement party loyalties. One of the principal specificities of the analysed study was that it tested the influence of ambivalence versus uniformity of the stimuli through a double exposure to the VAAs. By including cases characterised by diverging recommendations the experimental situation became more lifelike. According to the results, not even converging advice is able to move a user from one party to another, but those who received the same recommendation decided to vote in significantly larger numbers than those who received contradictory suggestions. Persuasion must be excluded as a plausible mechanism behind the difference in participation rates given the fact that the recipients of the unequivocal message typically did not follow the advice. It seems rather that the experience of being seen by two independent sources as belonging to a particular political orientation provoked an interest in making one’s voice heard: either in line with or, as a protest, against the recommendations.

© 2015 The Author. Political Studies © 2015 Political Studies Association POLITICAL STUDIES: 2015 1014VAAS ON LOYALTY, PREFERENCE AND TURNOUT Political Studies 64 (4)15 Finally, the two most important lessons of the study should be highlighted: information about programmatic similarities from credible sources is more likely to affect the degree of party loyalty than to determine the party choice; and while most people disregard even unequivocal recommendations, exposure to a particularly strong stimulus may provoke electoral participation. Future studies should systematically look for such spillover effects. (Accepted: 11 February 2015)

Acknowledgements I thank the anonymous reviewers for Political Studies for their constructive comments.

Notes 1 They reached the websites through a link available only for the participants, therefore it was possible to monitor whether they had completed the questionnaires. 2 The questionnaire contained two items referring to specific alternative motives: ‘I will vote primarily listening to my heart’ and ‘The leader of a party is more important than its programme.’ The proportion of those who endorsed these statements was not significantly higher among those who claimed to vote based on other considerations. These statements were accepted by a minority (41 and 21 per cent, respectively), while the majority (90 per cent) claimed to vote ‘listening to reason’. 3 The sample consists of those who reported, in the first wave, preference for one of the six analysed parties, took a VAA test and recalled which party they voted for at the election. 4 After filtering out the participants who received contradictory advice, 640 cases were retained. 5 Due to the fact that no significant differences were found between the two VAAs, the rest of the article will not differentiate between them. 6 One of the objectives of the larger project that included the VAA experiment was to test the influence of question formats. In this particular case, the variance in question format overlapped with the variance in the number of VAA exposures. Given that the article gauges the impact of VAAs primarily through actual preference changes and not through subjective evaluations, this unfortunate fact poses no challenge to the causal analysis. 7 Voters of minor parties and participants without party preferences were excluded. 8 If by ‘switching’ one understands not only movement from one party to another but also the shift from the lack of preferences to the support of a party, then the rate of those who followed the VAA advice goes up to 29 per cent. 9 Source credibility is not a central issue for this article, but it is relevant to mention that those few who followed the advice of the VAAs in spite of their original preference expressed more trust in the visited VAA (significance: 0.025) and agreed less with the statement that their choice is guided by other considerations (significance: 0.044). 10 The reported significance levels here and below refer to one-tailed Chi2 statistics unless otherwise noted. 11 That is, the rate of those who originally planned to abstain but eventually went to vote. 12 The subjectivity of recalls may have also played some role in the result (see Ruusuvirta and Rosema, 2009; Wall et al., 2012).

References Bartels, L. M. (1993) ‘Messages Received: The Political Impact of Media Exposure’, American Political Science Review, 87, 267–85. Boogers, M. and Voerman, G. (2003) ‘Surﬁng Citizens and Floating Voters: Results of An Online Survey of Visitors to Political Websites during the Dutch 2002 General Elections’, Information Polity, 8 (1), 17–27. Cacioppo, J. T. and Petty, R. E. (1979) ‘Effects of Message Repetition and Position on Cognitive Response, Recall and Persuasion’, Journal of Personality and Social Psychology, 37, 97–109. Cedroni, L. (2010) ‘Voting Advice Applications in Europe: A Comparison’, in L. Cedroni and D. Garzia (eds), Voting Advice Applications in Europe: The State of the Art. Naples: CIVIS/Scriptaweb, pp. 247–57. De Rosa, R. (2010) ‘Cabina-Elettorale.It (Provides Advice to Italian Voters since 2009)’, in L. Cedroni and D. Garzia (eds), Voting Advice Applications in Europe: The State of the Art. Naples: CIVIS/Scriptaweb, pp. 187–98. Enyedi, Z. (2015) ‘Plebeians, Citoyens and Aristocrats, or Where is the Bottom of Bottom-Up? The Case of Hungary’, in H. Kriesi and T. Pappas (eds), Populism in the Shadow of the Great Recession. Colchester: ECPR Press, pp. 338–62. Enyedi, Z. and Benoit, K. (2011) ‘Kritikus Szavazás 2010’, in Z. Enyedi, R. Tardos and A. Szabo (eds), Új Képlet. Budapest: DKMKA, pp. 17–42. Enyedi, Z. and Casal-Bértoa, F. (2011) ‘Patterns of Party Competition (1990–2009)’, in P. G. Lewis and R. Markowski (eds), Europeanising Party Politics? Comparative Perspectives on Central and Eastern Europe after Enlargement. Manchester: Manchester University Press, pp. 116–42. De Graaf, J. (2010) ‘The Irresistible Rise of Stemwijzer’, in L. Cedroni and D. Garzia (eds), Voting Advice Applications in Europe: The State of the Art. Naples: CIVIS/Scriptaweb, pp. 35–46.

Kleinnijenhuis, J. and van Hoof, A. M. J. (2008) ‘The Influence of Internet Consultants’. Paper presented at the Voting Advice Applications between Charlatanism and Political Science conference, Antwerp, 16 May. Krouwel, A., Vitiello, T. and Wall, M. (2012) ‘The Practicalities Issuing Vote Advice: A New Methodology for Profiling and Matching’, International Journal of Electronic Governance, 5 (3–4), 223–43. Ladner, A., Fivaz, J. and Felder, G. (2010) ‘More than Toys? A First Assessment of Voting Advice Applications in Switzerland’, in L. Cedroni and D. Garzia (eds), Voting Advice Applications in Europe: The State of the Art. Naples: CIVIS/Scriptaweb, pp. 91–123. Louwerse, T. and Rosema, M. (2011) ‘The Design Effects of Voting Advice Applications: Comparing Methods of Calculating Results’. Paper presented at the ECPR General Conference, Reykjavik, 25–27 August. Lupia, A. and Philpot, T. S. (2005) ‘Views from Inside the Net: How Websites Affect Young Adults’ Political Interest’, Journal of Politics, 26 (4), 1123–42. Markus, G. B. (1986) ‘Stability and Change in Political Attitudes: Observed, Recalled and Explained’, Political Behavior, 8, 21–44. Marschall, S. (2005) ‘The Online Making of Citizens: Wahl-o-Mat’, in V. B. Georgi (ed.), The Making of Citizens in Europe: New Perspectives on Citizenship Education. Bonn: Bundeszentrale für Politische Bildung, pp. 137–41. Marschall, S. and Schultze, M. (2012) ‘Voting Advice Applications and Their Effect on Voter Turnout: The Case of the German Wahl-o-Mat’, International Journal of Electronic Governance, 5 (3–4), 349–66. Mayer, C. and Wassermair, M. (2010) ‘Wahlkabine.at: Promoting an Enlightened Understanding of Politics’, in L. Cedroni and D. Garzia (eds) Voting Advice Applications in Europe: The State of the Art. Naples: CIVES/Scriptaweb, pp. 173–85. Mendez, F. (2012) ‘Matching Voters with Political Parties and Candidates: An Empirical Test of Four Algorithms’, International Journal of Electronic Governance, 5 (3–4), 264–78. Mykkänen, J. and Moring, T. (2006) Dealigned Politics Comes of Age? The Effects of Online Candidate Selectors on Finnish Voters. Paper presented at the Politics on the Internet: New Forms of Media for Political Action conference, Tampere, 24–25 November. Nuytemans, M., Walgrave, S. and Deschouwer, K. (2010) ‘Do the Vote Test: The Belgian Voting Aid Application’, in L. Cedroni and D. Garzia (eds), Voting Advice Applications in Europe: The State of the Art. Naples: CIVIS/Scriptaweb, pp. 125–56. Ruusuvirta, O. (2010) ‘Much Ado about Nothing? Online Voting Advice Applications in Finland’, in L. Cedroni and D. Garzia (eds), Voting Advice Applications in Europe: The State of the Art. Naples: CIVIS/Scriptaweb, pp. 47–64. Ruusuvirta, O. and Rosema, M. (2009) Do Online Vote Selectors Influence Electoral Participation and the Direction of the Vote? ’. Paper presented at the ECPR Conference, Potsdam, Germany, 10–12 September. Sawyer, A. (1981) ‘Repetition, Cognitive Responses and Persuasion’, in R. E. Petty, T. M. Ostrom and T. C. Brock (eds), Cognitive Response in Persuasion. Hillsdale, NJ: Lawrence Erlbaum Associates, pp. 237–62. Stang, D. J. (1975) ‘Effects of Mere Exposure on Learning and Affect’, Journal of Personality and Social Psychology, 31, 7–12. Threchsel, A. and Mair, P. (2011) ‘When Parties (also) Position Themselves: An Introduction to the EU Profiler’, Journal of Information Technology and Politics, 8 (1), 1–20. Walgrave, S., van Aelst, P. and Nuytemans, M. (2008) ‘ “Do the Vote Test”: The Electoral Effects of a Popular Vote Advice Application at the 2004 Belgian Elections’, Acta Politica, 43 (1), 50–70. Wall, M., Krouwel, A. and Kleinnijenhuis, J. (2012) ‘Political Matchmakers: How do the Decision Rules Employed by Vote Advice Application Sites Influence Their Advice?’ Paper presented at the 2012 ELECDEM Conference, Florence, 28–30 June. Wall, M., Krouwel, A. and Vitiello, T. (2014) ‘Do Voters Follow the Recommendations of Voter Advice Application Websites? A Study of the Effects of Kieskompas.nl on Its Users’ Vote Choices in the 2010 Dutch Legislative Elections’, Party Politics, 20 (3), 416–28. Zajonc, R. B. (1968) ‘The Attitudinal Effects of Mere Exposure’, Journal of Personality and Social Psychology, 9 (2), 1–27. Zaller, J. R. (1992) The Nature and Origins of Mass Opinion. Cambridge: Cambridge University Press. Závecz, T. (2010) ‘Pártpreferenciák dinamikája a 2010. évi országgyulési választásokat megelo˝zo˝en’. Available from: http://www.valasztaskutatas.hu/eredmenyek/kutatas-jelentesek/rkkutjel/zavecz-2010-partpreferenciak- dinamikaja/view [Accessed 28 April 2015].

About the Author Zsolt Enyedi is Professor of Political Science at the Central European University. He works on various aspects of party politics, comparative government, church and state relations, and political psychology. He has (co-)authored two books and (co-)edited eight, and published close to a hundred articles and book chapters. His articles have appeared in journals such as Political Psychology, European Journal of Political Research, West European Politics, Party Politics, Europe-Asia Studies, European Review, Journal of Legislative Studies, Democratization and Social Thought and Research. In 2003 he received the Rudolf Wildenmann Award. Zsolt Enyedi, Department of Political Science, Central European University, Nador 9, H-1051 Budapest, Hungary; email: [email protected]