Everybody out of the Pool! Reconstructing the Democratic Peace
Total Page:16
File Type:pdf, Size:1020Kb
EVERYBODY OUT OF THE POOL! RECONSTRUCTING THE DEMOCRATIC PEACE CSSS WORKING PAPER 55 MICHAEL D. WARD, RANDOLPH M. SIVERSON, AND XUN CAO Abstract. Research in international politics may have produced misleading results because (1) typical data contain dependencies that have been ignored, (2) popula- tions are treated as samples, with unwarranted reliance on misleading significance tests, and (3) scant attention is devoted to how well the model can predict the events of interest. Using the democratic peace research program as an example, we show that the three elements of the Kantian model-mutual democracy, high trade and common membership in IGOs–have at best weak effects on dampening the oc- currence of militarized international disputes within dyads. Neither do they offer meaningful predictions about which dyads will be involved in these disputes. A model incorporating several types of dependencies among countries yields results with high levels of predictive accuracy and provides new substantive insight about the prominence of dependencies in international relations. In a recent paper, Frieden & Lake (2005) take stock of the state of research in interna- tional politics. Their overall argument is that “progress in the study of international politics– including. making its lessons more relevant to policy–depends on more not less, rigorous theory and, more not less, systematic empirical testing” (p. 137). While we agree with their overall sen- timents, the argument we present here is that different modes of systematic empirical research are needed to avoid problems that have had unrecognized consequences for the quality of the research results reported in the literature. In this paper we identify three major problems with prevailing practices in the empirical analysis of international politics. These are (1) the lack of independence in the cases, (2) an unwarranted reliance on tests of significance and, as a consequence (3) a neglect of how well a “tested” model predicts the outcome of interest. The combination of these problems can produce misleading results. We show in this article that because popular statistical procedures overlook the dependencies in the data, they result in an over-confidence in the Kantian hypothesis that joint democracy, mutual trade, and shared participation in the international governmental organizations each independently reduce international conflict. Authors’ Note: Version of February 3, 2006. Ward’s research was supported by a grant from the Methods, Measurement, and Statistics Program at the National Science Foundation, grant number: SES-0417559. Peter Hoff and Anton Westveld provided much guidance and helpful discussions on this topic, as well as many others. This research greatly benefitted from the scrutiny and helpful suggestions of Neal Beck, Bruce Bueno de Mesquita, James Caporaso, Jeff Gill, Jonathan Mercer, and Aseem Prakash. David Callaway and John E. Daniels of the Social Science Data Service at the Institute of Governmental Affairs at the University of California at Davis were especially helpful to us at a critical point in our quest for distributed computing resources which we found at the San Diego Supercomputer Center at the University of California at San Diego. Special thanks go to SS and SW for their forbearance and editorial guidance during their vacations, and after. And before. And, then again. 1 Everybody Out of the Pool! February 3, 2006 Three Problems The first problem is that both dyadic data and international relations are known to con- tain dependencies that are omitted by most popular procedures for analyzing data.1 Because regression-based approaches assume that the data are exchangeable and the errors independently and identically distributed, they are unable to capture the extent to which dependent data may actually reflect the ebb and flow of international politics. International Organization published a symposium in 2001 that examined the robustness of statistical findings in many dyadic studies of international conflict. Green, Kim & Loon (2001) surveyed more than four dozen panel studies in the field of international relations that appeared in prominent scholarly journals over the period from 1996-1999. They raised the question of whether the democratic peace is a powerful expla- nation, or whether dyad-based fixed effects were statistically more powerful. In summarizing this debate, King (2001) noted that the thorniest problem is to unravel the dependencies in dyadic data that result in biased estimates of coefficients and covariance structures, and further suggested that “[a] logical methodological starting point for addressing the problems at hand would be based on Bayesian hierarchical, random effects, or split population models” (page 506). Below we present one such model to examine the role of dependencies in the democratic peace literature.2 What are these dependencies? Maoz (2004) has shown for example that countries which are involved in conflicts in one decade are quite prone to them in the next. He illustrates that Sweden, Switzerland, and Venezuela, for example, generally are not involved in militarized disputes and wars, while historically, Israel, Pakistan, India, Jordan, and Syria (among others) are repeatedly involved in both disputes and wars. These historical facts result in dependencies among the ebb and flow of, as well as the data we collect on, interstate disputes and wars. These dependencies are at the core of this research. They reflect the fact that it is common to observe patterns of interstate conflict having the same initiator but different targets. For example, pursuing an active foreign policy may lead a country into conflict with many different countries over time. Similarly, disputes having different initiators but the same targets are also common, as in the case of the first Gulf War where a large number of countries were engaged in conflict with Iraq. These actions are at least partially dependent upon one another and can not be treated as completely independent occurrences. Additional, there are often pairs of countries that are in repeated, or protracted, conflicts that are typically described as rivalries (Diehl, Goertz & Jones 2005). Episodic conflicts and disputes are not independent, either statistically or historically. The geopolitics of the Middle East is replete with such dyadic dependencies. Table 3, below, displays some of the most prominent dependencies in international conflict data. A second important problem we address concerns the basic inferential model that dominates popular practice in the field of international relations, as well as many other disciplines. Since most research in international relations is based on observational studies that have a large number of cases, there is an appearance of statistical significance in almost all the findings in the literature, even considering the well-known publication bias for positive and statistically significant results. Actually, the prevailing popular statistical approach reflects the ability of the tests to detect small differences–i.e., the power of the tests–as well as the number of observations more than it reveals any underlying statistical significance of hypothesized causal linkages (Savage 1957, Gill 1999). This research tradition somehow expects more from tests of significance, an expectation that probably creates more problems than it solves. 1One early effort examined strategic dependencies in the context of endogenous choice calculus (Smith 1999). Another important initiative (Signorino 1999) extends the Quantile Response Equilibrium in the context of a solution to certain game-theoretic models of strategic choice. 2Clark & Regan (2003) have examined split population models, examining heterogeneity in different samples, but not examining dependencies. 2 Everybody Out of the Pool! February 3, 2006 While observational data rarely comprise a random sample, they nevertheless are frequently treated as an enormous one. Large samples have immense statistical power that large collections of observational data normally do not. The null hypothesis implicit in most of the statistical examinations of these models is that the quantities of primary interest are all equal to zero. Why this would be the case in observational data is not clear. In situations characterized by rare events with a preponderance of non-occurrences, we should not expect the pooled correlations to be identical for all groupings of observations. Zero correlation is a meaningful expectation if and only if one knows that the two groups of cases (for example: the disputes and the non-disputes) are either randomly assigned to the treatment (dispute) or randomly selected from the covering population in such a way that they are matched on the important covariates. Neither of these conditions can plausibly be argued to be the case for observational data on militarized interstate disputes, nor, for that matter, in almost any data currently being used in the study of international politics. As a result, statistical significance tests can be badly misleading in terms of producing information about the underlying data generating processes by allowing scholars to attribute statistical significance to summary characteristics known a priori to be different. Below we show that it is not necessary to embrace the sampling framework in order to extract useful information from observational data in the context of the democratic peace. Third, most studies have focused solely on statistical significance as a measure of “fit,” while at the same time avoiding