Pre-Analysis Plan Democratic Backsliding, Understandings of Democracy, and Political Choice: a Survey Experiment in Poland

Pre-Analysis Plan Democratic Backsliding, Understandings of Democracy, and Political Choice: A Survey Experiment in Poland

Natasha Wunsch, ETH Zurich & Sciences Po Paris Marc S. Jacob, ETH Zurich Laurenz Derksen, ETH Zurich

First draft: January 2021 Current version: 7 July 2021

Abstract In electoral democracies, political actors engaged in democratic backsliding require at least some degree of voter consent. So why do citizens support candidates who endorse undemocratic practices? The bulk of existing research assumes that a common understanding of democracy underpins citizens’ evaluations of different candidates, leading them to actively trade off undemocratic or illiberal practices against partisan or economic considerations. We question this view by sug- gesting that different understandings of democracy may coexist in a given electorate, including ones that are at odds with some fundamental stipulations of liberal democracy, such as the separa- tion of powers and independent media. Drawing on the literatures on political culture, political behaviour, and democratisation, we develop a series of hypotheses to probe the influence of divergent understandings of democracy upon candidate choice. We outline a survey experiment articu- lated around a candidate choice conjoint that serves to evaluate these hypotheses against alternative explanatory factors.

1 - Project overview Whereas dominant executives tend to drive the gradual erosion of domestic checks and balances and civil liberties, citizens play a central role in supporting or at least tolerating undemocratic practices by elected leaders. Our study proposes to probe the sources of voter support for candidates espousing such illiberal practices. We posit that political actors in democracies do not only represent different policy preferences, but may also stand for distinct system-level preferences to which voters respond. We outline the experimental design through which we aim to probe the mechanism(s) underpinning citizens’ support for candidates advocating democratically question- able practices.

What unites the bulk of existing research on candidate choice in contexts of democratic backsliding (Carey et al. 2020; Graham and Svolik 2020; McCoy, Simonovits, and Littvay 2020) is the (gen- erally implicit) assumption that a common understanding of democracy underpins citizens’ evaluations of different candidates, leading them to actively trade off undemocratic practices against competing candidates’ personal, partisan or policy-related characteristics. We challenge this view by contending that understandings of democracy may comprise alternative views of the concept, including ones at odds with the emphasis on checks and balances that are the hallmarks of liberal democracy. We expect such divergent understandings of democracy to co-exist even in consoli- dated democracies and to inform citizens’ electoral choices and evaluations of alternative candidates. In other words, we submit that voters may endorse specific candidates not despite the undemocratic practices they sponsor, but precisely because these candidates’ professed preferences align with their own understanding of the meaning and purpose of democracy.

To explain citizen support for illiberal practices, we evaluate the relevance of democratic preferences against a series of alternative mechanisms. Our research design consists of a survey experiment in the form of a candidate choice conjoint. We place respondents into a hypothetical election situation and confront them with two candidate profiles that diverge according to their stated democratic, outcome-related, and cultural positions as well as their partisan affiliation. Respondents are asked to choose which candidate they would be most likely to vote for as well as to provide an

individual rating for each profile. We expect our results to provide an indication of which element(s) citizens consider most crucial in their evaluation of competing candidates and to reveal the relative importance of divergent understandings of democracy for their candidate choice.

The experiment will be fielded in Poland, which represents a paradigmatic case of democratic backsliding. Poland is experiencing deepening political and societal polarisation (Fomina 2019; Tworzecki 2019), making it a particularly promising context in which to study the relevance of divergent understandings of democracy and their interactions with partisan and policy-based preferences as drivers of candidate support. Finally, our study is, to the best of our knowledge, the first to analyse democratic backsliding via a candidate choice conjoint in a European, multi-party set- ting.

2 - Research design Our survey consists of a candidate choice conjoint experiment. The survey will be divided into two parts, with the conjoint experiment complemented by a series of general questions related to socio-economic as well as political information which we will use for subset analyses. A set of open-ended questions as well as further items are included for additional analyses as explained below. Throughout the survey, we plan to use attention checks and time stamps to be able to re- move response profiles that do not fulfil minimal attention and temporal requirements (Berinsky, Margolis, and Sances 2014).

2.1 Sample The survey into which our conjoint experiment is embedded will be fielded online by the Warsaw- based market research company Inquiry. Respondents will be sent to our own website containing our survey programmed via Qualtrics. Sampling will be based on national representativeness regarding age, gender, education, and vote choice at the last national election. The full sample will comprise around 2’700 respondents.

2.2 Conjoint design The central feature of our experiment consists of a paired candidate choice conjoint, which consists of twelve tasks during which participants will be asked to choose between two randomly generated

profiles of candidates running for seats in the national parliament. We will ask respondents to choose between two candidates (forced choice) and to rate each candidate on a scale from 1 (strongly disapprove) to 7 (strongly approve). The rating question allows us to estimate how well on average randomly generated candidate profiles reflected the respondents’ preferences and serves as the basis to assess individual marginal component effects (IMCEs) (Zhirkov 2021). Each candidate profile will be identified with a neutral label (‘Candidate A’ vs. ‘Candidate B’) and will display randomized information on seven attributes. The order of attributes is fully randomized anew for each choice task. The interface for the candidate choice experiment is displayed in Ap- pendix B.

Candidate profiles will consist of a set of preferences related to democratic preferences, cultural views, economic outcomes as well as a party label. The first set of attributes on democratic preferences includes a procedural element of democracy relating to nomination of judges and an attribute relating to civil liberties that addresses the role of public media. These two attributes were selected to reflect two distinct dimensions on which democratic erosion may occur and to probe the salience of such forms of erosion for respondents’ candidate choice. We seek to capture divergent understandings of democracy by formulating the levels for the two democratic attributes in line with liberal, majoritarian, and authoritarian understandings. We add an outcome-oriented attribute relating to tax reform that may influence voters’ preferences due to trade-offs between economic interests and support for democratic standards. Moreover, we add an attribute probing cultural identity that provides differing candidate positions on the highly salient issue of abortion rights. Finally, we choose to include an explicit party label to probe the impact of partisanship on respondents’ evaluation of candidates’ adhesion to democratic standards. Whereas some studies suggest removing party labels completely by using hypothetical profiles to avoid triggering partisan identification (Kirkland and Coppock 2018; McCoy, Simonovits, and Littvay 2020), we are interested precisely in assessing whether party labels drive candidate choice or interact in any other way with democratic preferences. We therefore choose to present respondents with a mix of candidates from all parties above 5% of vote share according to polls in June 2021, with choice situ- ations also including run-offs between candidates of the same party background. We include age and gender of candidates to create more realistic profiles. Table 1 provides a full list of attributes and levels along with the corresponding theoretical rationales for inclusion.

Table 1. Overview of attributes and levels used in conjoint Attribute Levels Concept

Female (weighted at 35%) Socio-de- Gender Male mographics Gender

Socio-de- Age Randomize (random integer 30-65) Age Age mographics Law and Justice (PiS) Poland 2050 Partisan affili- Civic Coalition (KO) Partisanship

ship ation The Left Partisan- Confederation Tax reform should increase taxes for medium- and high- income households. Economic trade- Tax reform Tax reform should increase taxes for all households. off Tax reform should decrease taxes for low-income house- oriented Outcome- holds. Abortion legislation should grant greater freedom of Abortion leg- choice to women. Women’s rights islation Abortion legislation should protect the unborn child’s life identity Cultural Cultural in all but exceptional circumstances. Lib: Judges should be selected based on cross-party con- sensus. Judicial ap- Judicial inde- Maj: Judges should be selected by the government. pointments pendence Auth: Judges should be selected by the leader of the ruling party. Lib: The role of public media is to report independently on political developments.

Democratic Role of public Maj: The role of public media is to justify government Media pluralism media policy towards the wider public. Auth: The role of public media is to defend government policy against criticism.

2.3 General survey questions Following the conjoint, we ask respondents a series of questions related to their socio-economic background, political preferences, and views of democracy. For the latter, we adapt an established battery of questions used in the World Values Survey on liberal and authoritarian understandings of democracy and formulate own items to capture majoritarian orientations. We tested formula- tions in two separate pre-tests to ensure our distinct items correspond to a single latent concept. We define the latent constructs and observed variables below to avoid ambiguous construct build- ing after collecting the data. For partisanship, we include questions about perceived closeness to

different parties that will allow us to address the issue of affective polarisation (Reiljan 2020). We ask these questions after the conjoint to avoid priming respondents.

2.4 Outcome measures 2.4.1 Primary outcome measures Our primary outcome measure concerns respondents’ rating of candidate profiles, which we use to calculate individual marginal component effects (IMCEs) (Zhirkov 2021). We use those scores as outcome variables in regression models, where we define individual scores on the various understandings of democracy constructs, partisanship and socio-economic controls as independent variables. In addition, we use respondents’ candidate preference resulting from forced choices among two alternative candidate profiles to estimate the average marginal component effect (AMCE) (Hainmueller, Hopkins, and Yamamoto 2014) for one of our hypotheses. We plan to conduct the analysis both at the level of individual attributes, but also regarding the aggregated arbitration between democratic and outcome-related preferences.

2.4.2 Secondary outcome measures Our survey enables a closer study of divergent understandings of democracy among respondents. We measure such divergent understandings as described in section 4.1 and plan to use this measure to further explore how distinct understandings of democracy are associated with partisanship and certain socio-demographic attributes.

3 – Hypotheses In this section, we outline the main theoretical expectations underpinning our experimental design. We distinguish between primary hypotheses, which concern the relevance of divergent understandings of democracy as our main theoretical focus, and secondary hypotheses that detail our expectations regarding alternative explanatory factors.

3.1 Primary hypotheses We designed our survey experiment primarily to explore the link between understandings of democracy and political behaviour. Our main hypotheses therefore concern the impact of divergent understandings of democracy upon a series of candidate choices in a hypothetical voting situation.

H1a: Respondents are more likely to prefer candidates whose democratic preferences are congruent with their own understanding of democracy.

We expect this relationship to be particularly salient for respondents with more liberal understandings of democracy:

H1b: Respondents with more liberal understandings of democracy lend greater weight to candidates’ democratic preferences than those with more majoritarian or authoritarian understandings of democracy.

Moreover, we expect democratic preferences as revealed by the conjoint to be associated with partisan groups, with democratic preferences varying along partisan platforms. We formulate a general hypothesis on this relationship at the respondent level:

H2a: Respondents’ party preferences predict their revealed democratic preferences.

In addition, we formulate a specific hypothesis for supporters of the Law and Justice Party (PiS) that is currently in power in Poland and for whose partisans we therefore expect to find higher levels of majoritarian understandings of democracy. We do not have any directional expectations for respondents with other party preferences.

H2b: Partisans of PiS are more likely to hold majoritarian understandings of democracy than supporters of other parties.

3.2 Secondary hypotheses 3.2.1 Partisanship Prior research has indicated a ‘partisan double standard’ (Graham and Svolik 2020) whereby voters punish the violation of democratic standards less severely when it concerns a candidate repre- senting their own party. We test this hypothesis in the Polish multi-party context:

H3: Respondents punish candidates who correspond to their party preference less severely for expressing democratic preferences that violate democratic standards.

4 - Analysis We describe our analysis strategy for the pooled sample as well as planned subgroup analyses in the following paragraphs.

4.1 Measurement constructs We use employ confirmatory factor analysis (CFA) to measure different understandings of democracy. The question wordings for the observed variables are listed in Appendix A. We estimate our measurement model with ordered confirmatory factor analysis and compute the factors scores for each latent variable based on the model. The measurement model is displayed in Figure 1. We consider simple indices of each latent variable as an alternative measure of understandings of democracy.

Figure 1. Measurement model

4.2 Neutrality checks To test whether the conjoint profiles were correctly randomized, we run a regression of respondents’ socio-economic variables (age, gender, education) on the treatment indicators, i.e., all cate-

gorical variables for each attribute displayed in Table 1. We report imbalances in the randomiza- tion in case we find a statistically significant effect of any of the socio-economic or moderator variables.

4.3 Aggregate conjoint analysis We will estimate and plot the Average Marginal Component Effect (AMCE) of each attribute level, where each attribute is a factor variable consisting of the factor levels developed in our research design. The model for the pooled sample reads as follows:

퐶ℎ표푖푐푒 = 훽 + 훽퐸푐표푛표푚푖푐푇푟푎푑푒푂푓푓 + 훽푊표푚푒푛푠푅푖푔ℎ푡푠 + 훽퐽푢푑푖푐푖푎푙퐼푛푑푒푝푒푛푑푒푛푐푒

+ 훽푀푒푑푖푎푃푙푢푟푎푙푖푠푚 + 훽푆표푐푖표퐷푒푚표푔푟푎푝ℎ푖푐푠 + 휀

Where 퐶ℎ표푖푐푒 is individual 푖′푠 choice to vote for a candidate, 훽 to 훽 the characteristics of the candidate, and 휀 an error term. This regression model allows us to test the AMCES on the entire population.

4.4 Individual conjoint analysis To test our hypothesis referring to individuals’ candidate preferences, we rely on the Individual Marginal Component Effect (IMCE) approach developed by (Zhirkov 2021). This approach allows us to compute individual marginal effects for every attribute level. Next, using the IMCE point estimates for each democracy attribute (judicial independences and media pluralism) as separate outcome variables, we implement the following linear regression model: 퐷푒푚표푐푟푎푐푦푆푐표푟푒

= 훽 + 훽푈푛푑푒푟푠푡푎푛푑푖푛푔푠푂푓퐷푒푚표푐푟푎푐푦 + 훽푃푎푟푡푖푠푎푛푠ℎ푖푝

+ 훽퐷푒푚표푔푟푎푝ℎ푖푐푠 + 휀, where the independent variables correspond to the predictors defined in the hypotheses, controlling for socio-demographic characteristics. Since we have an overall number of 6 items referring to candidates’ democratic position (3x judicial independence & 3x role of media), we obtain 6 scores relating to democracy using the ICME approach. Consequently, we implement 6 separate regression models.

4.5 Power analysis 4.5.1 Conjoint analysis We conducted a power analysis for conjoint designs as developed by Stefanelli and Lukac (2020). The respondents will perform twelve choice tasks on a conjoint in which an attribute consists of a maximum of three levels. Assuming an effect size of 0.05 and a sample size of 2,700 respondents, the statistical power is predicted to converge to 100% (see Figure 1) already for nine choice tasks (which is the maximum it is possible to indicate with our chosen power analysis tool, as displayed in the figures below). Given our intention to use IMCEs for our analysis and Zhirkov’s recommen- dation to have respondents ideally rate close to 30 profiles (2021: 12), we choose to present respondents with twelve choice tasks and thus to collect their ratings on 24 profiles.

Figure 2: Statistical power with 2,700 respond- Figure 3: Type S error with 2,700 respondents ents

Figure 4: Type M error with 2,700 respondents

As Figures 1 to 3 show, a number of 2,700 respondents and nine choice tasks will almost certainly allow us to detect a statistical effect if present in the data, as well as almost fully rule out a Type M (receiving opposite coefficient sign) error and a Type S error (estimating an exaggerated effect size).

4.5.2 Regression analysis We run another power analysis for the regression analysis. Assuming an effect size of 0.07 to also identify weaker associations in the data and ∝ error probability of 0.05, we would require a sample size of 2,641 respondents. As Figure 5 shows, statistical power passes the 0.95 threshold at a sample size of about 2’600 respondents. We will thus focus only on coefficients yielding an effect size of 0.07 or higher in our analysis.

Figure 5. Power analysis: range of values

5 - Pre-test We ran two pre-tests to assess a number of design choices before conducting our study with the full sample. All respondents were invited by Inquiry and redirected to our survey implemented via Qualtrics on a separate website. We asked each respondent to evaluate twelve pairs of candidate profiles. None of the respondents who participated in the pilots will be invited to participate in the final conjoint task.

5.1 Sample Our first pre-test was fielded in February/March 2021 among 200 respondents. The second pre- test was conducted in June 2021 among 300 respondents. In both cases, the pre-test sample con- served the representative sampling strategy (based on age, region, socio-economic background, and vote choice in last election) that we will use for the full survey.

5.2 Adjustments following the pre-test We used the pre-test to assess the measurement and distribution of different understandings of democracy across our sample as discussed above. For the first pre-test, we had included a larger

number of attributes (10 in total) into our conjoint (three for democratic preferences and two each for cultural orientations and economic outcomes). In light of this more complex design, we found party labels to act as a heuristic in particular for speedy respondents, with AMCEs (in particular of partisanship) increasing with each iteration of the conjoint, rather than being independent from each other, thus violating the stability assumption for the calculations of AMCEs (Hainmueller, Hopkins, and Yamamoto 2014). This led us to reduce the number of attributes (reducing the number of attributes per theoretically defined category) and to fully randomize them rather than pin the party label at the top along with gender and age. Our second pre-test confirmed these choices and no further adjustments were introduced for the full survey.

Appendix A - General questions wording A.1 Understandings of democracy Democracy can mean different things to different people. Please indicate to what extent you agree that each of the following items is an essential element for a democracy.

 People choose their leaders in free elections [L1].  Civil rights protect people from state oppression [L2].  Women have the same rights as men [L3].

 The government uses violence to enforce public order [A1].  Elections only serve to confirm the ruling party in office [A2].  The government limits civic freedoms to rule efficiently [A3].

 The majority can always overrule the minority [M1].  Any law can be changed if there is a majority for it [M2].  The minority must accept the will of the majority in all circumstances [M3].

Response set:  Strongly disagree  Disagree  Somewhat disagree  Neither agree or disagree  Somewhat agree  Agree  Strongly agree

B - Survey design Figure A1. Candidate choice task interface

References Berinsky, Adam J., Michele F. Margolis, and Michael W. Sances. 2014. “Separating the Shirkers from the Workers? Making Sure Respondents Pay Attention on Self-Administered Surveys.” American Journal of Political Science 58 (3): 739–53. Carey, John, Katherine Clayton, Gretchen Helmke, Brendan Nyhan, Mitchell Sanders, and Susan Stokes. 2020. “Who will defend democracy? Evaluating tradeoffs in candidate support among partisan donors and voters.” Journal of Elections, Public Opinion & Parties: 1–16. Fomina, Joanna. 2019. “Of 'Patriots' and Citizens: Asymmetric Populist Polarization in Poland.” In Democracies Divided: The Global Challenge of Political Polarization, eds. Thomas Carothers and Andrew O'Donohue. Brookings Institution Press. Graham, Matthew H., and Milan W. Svolik. 2020. “Democracy in America? Partisanship, Polari- zation, and the Robustness of Support for Democracy in the United States.” American Politi- cal Science Review 114 (2): 392–409. Hainmueller, Jens, Daniel J. Hopkins, and Teppei Yamamoto. 2014. “Causal Inference in Con- joint Analysis: Understanding Multidimensional Choices via Stated Preference Experiments.” Political Analysis 22 (1): 1–30. Kirkland, Patricia A., and Alexander Coppock. 2018. “Candidate Choice Without Party Labels.” Political Behavior 40 (3): 571–91. Leeper, Thomas J., Sara B. Hobolt, and James Tilley. 2020. “Measuring Subgroup Preferences in Conjoint Experiments.” Political Analysis 28 (2): 207–21. McCoy, Jennifer, Gabor Simonovits, and Levente Littvay. 2020. “Democratic Hypocrisy: Polar- ized citizens support democracy-eroding behavior when their own party is in power.” APSA pre-print. Stefanelli, Alberto, and Martin Lukac. 2020. Subjects, Trials, and Levels: Statistical Power in Conjoint Experiments. Tworzecki, Hubert. 2019. “Poland: A Case of Top-Down Polarization.” The ANNALS of the American Academy of Political and Social Science 681 (1): 97–119. Zhirkov, Kirill. 2021. “Estimating and Using Individual Marginal Component Effects from Con- joint Experiments.” Political Analysis: 1–14.