Mistaking the Forest for the Trees: the Mistreatment of Group-Level Treatments in the Study of American Politics
Total Page:16
File Type:pdf, Size:1020Kb
Mistaking the Forest for the Trees: The Mistreatment of Group-Level Treatments in the Study of American Politics Kelly T. Rader Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Graduate School of Arts and Sciences COLUMBIA UNIVERSITY 2012 c 2012 Kelly T. Rader All Rights Reserved ABSTRACT Mistaking the Forest for the Trees: The Mistreatment of Group-Level Treatments in the Study of American Politics Kelly T. Rader Over the past few decades, the field of political science has become increasingly sophisticated in its use of empirical tests for theoretical claims. One particularly productive strain of this devel- opment has been the identification of the limitations of and challenges in using observational data. Making causal inferences with observational data is difficult for numerous reasons. One reason is that one can never be sure that the estimate of interest is un-confounded by omitted variable bias (or, in causal terms, that a given treatment is ignorable or conditionally random). However, when the ideal hypothetical experiment is impractical, illegal, or impossible, researchers can often use quasi-experimental approaches to identify causal effects more plausibly than with simple regres- sion techniques. Another reason is that, even if all of the confounding factors are observed and properly controlled for in the model specification, one can never be sure that the unobserved (or error) component of the data generating process conforms to the assumptions one must make to use the model. If it does not, then this manifests itself in terms of bias in standard errors and incor- rect inference on statistical significance of quantities of interest. In this case, one can either turn to standard error “fixes” that are robust to generic forms of deviance from standard assumptions or to non-parametric solutions that do not require such assumptions but may be less powerful than their parametric counterparts. The following essays, I develop the use of some of these techniques for inference with obser- vational data and explore their limitations. Collectively, these essays challenge the conventional application of quasi-experimental techniques and standard error fixes. They also contribute to important substantive debates over legislative organization by producing more cleanly identified effects of the power of Congressional representatives as individuals and as members of parties to bargain over distributive goods. Table of Contents I Dissertation Chapters1 1 Overview 2 1.1 Introduction..........................................3 1.2 Chapter Summaries......................................4 1.2.1 Randomization Tests and Inference with Grouped Data.............4 1.2.2 Party Effects on the Distribution of Federal Outlays...............7 1.2.3 Malapportionment in the U.S. House of Representatives............ 11 2 Randomization Tests for Grouped Data 14 2.1 Introduction.......................................... 15 2.2 Inference with Clustered Data................................ 16 2.3 Cluster-Robust Standard Errors............................... 19 2.4 Randomization Tests..................................... 21 2.5 Evaluation Criteria...................................... 26 2.6 Monte Carlo Experiment................................... 29 2.7 Results and Discussion.................................... 30 i 2.8 Applications.......................................... 32 2.8.1 State Postregistration Laws and Voting...................... 33 2.8.2 Precedent and Voting on the Supreme Court................... 34 2.8.3 Democratic Trade................................... 36 2.9 Conclusion........................................... 37 2.10 Appendix........................................... 39 3 Party Effects on Outlays 49 3.1 Introduction.......................................... 50 3.2 The Theory of Party Effects................................. 52 3.2.1 Universalistic Theories................................ 52 3.2.2 Party-Centered Theories............................... 53 3.2.3 Democrats versus Republicans........................... 54 3.2.4 Empirical Tests of Party Effects on Spending................... 55 3.3 Regression Discontinuity Design.............................. 56 3.3.1 Implementation.................................... 57 3.3.2 Assumptions..................................... 58 3.4 Data and Results....................................... 59 3.4.1 Majority Party Power................................ 61 3.4.2 Democrats vs. Republicans............................. 63 3.5 Conclusion........................................... 65 3.6 Appendix........................................... 77 ii 4 Malapportionment in the House 82 4.1 Introduction.......................................... 83 4.2 Apportioning the House................................... 84 4.3 How Malapportioned is the House?............................ 85 4.4 The Apportionment-Funding Connection......................... 87 4.5 Relative Federal Spending in the States.......................... 89 4.6 Method and Results...................................... 90 4.7 Discussion and Conclusion................................. 93 4.8 Appendix........................................... 103 II Bibliography 106 Bibliography 107 iii List of Figures 2.1 Size results from Monte Carlo experiments. Results shown for nominal α levels .01, .05, and .1. Shaded gray area represents 95% binomial confidence intervals around the nominal α level, denoted with dotted line. Confidence intervals calculated using the Wilson method recommended by Agresti and Coull(1998). ......................................... 43 2.2 Power results from Monte Carlo experiments. Results shown for nominal α levels .01, .05, and .1, and number of groups equal to 10, 20, and 50. Dotted lines at nominal α level and one (maximum power). .............................................. 44 2.3 Randomization Test Results. Mailed polling place information and its interaction with individual education level on an individual’s propensity to vote. ............... 45 2.4 Randomization Test Results. Regime change in freedom of expression cases before and after Grayned vs. Rockford ................................... 46 2.5 Randomization Test Results. Minimum dyadic democracy score and bilateral trade .... 47 2.6 Size results from Monte Carlo experiments. Results shown for data in which Z and X are uncorrelated, correlated at 0.3, and correlated at 0.8. Shaded gray area represents 95% binomial confidence intervals around the nominal α level, denoted with dotted line. Confidence intervals calculated using the Wilson method recommended by Agresti and Coull(1998). ........... 48 3.1 Estimates correspond to table 3.1, column 1. 95% confidence intervals shown..... 68 3.2 Estimates correspond to table 3.1, column 2. 95% confidence intervals shown..... 68 iv 3.3 Estimates correspond to table 3.1, column 3. 95% confidence intervals shown..... 69 3.4 Estimates correspond to table 3.1, column 5. 95% confidence intervals shown..... 69 3.5 Estimates correspond to table 3.1, column 6. 95% confidence intervals shown..... 70 3.6 Estimates correspond to table 3.1, column 7. 95% confidence intervals shown..... 70 3.7 Effect of Majority Party Win on Spending. Locally weighted regressions model ln(spending) as a function of majority party victory margin in the district up to zero and above zero. Sample includes all districts matched to either 14 or 20 fiscal years............................................... 71 3.8 Vertical line indicates the IK optimal bandwidth. Dots are point estimates, and bands are 95% confidence intervals. Sample includes all districts matched to either 14 or 20 fiscal years....................................... 71 3.9 Estimates correspond to table 3.2, column 1. 95% confidence intervals shown..... 73 3.10 Estimates correspond to table 3.2, column 2. 95% confidence intervals shown..... 73 3.11 Estimates correspond to table 3.2, column 3. 95% confidence intervals shown..... 74 3.12 Estimates correspond to table 3.2, column 5. 95% confidence intervals shown..... 74 3.13 Estimates correspond to table 3.2, column 6. 95% confidence intervals shown..... 75 3.14 Estimates correspond to table 3.2, column 7. 95% confidence intervals shown..... 75 3.15 Effect of Democratic Party Win on Spending. Locally weighted regressions model ln(spending) as a function of Democratic victory margin in the district up to zero and above zero. Sample includes all districts matched to either 14 or 20 fiscal years. 76 3.16 Vertical line indicates the IK optimal bandwidth. Dots are point estimates, and bands are 95% confidence intervals. Sample includes all districts matched to either 14 or 20 fiscal years....................................... 76 4.1 Relative Representation in the U.S. House by State, 1972-1982............. 96 v 4.2 Relative Representation in the U.S. House by State, 1973-1992............. 96 4.3 Relative Representation in the U.S. House by State, 1983-2002............. 97 4.4 Relative Representation in the U.S. House by State, 1993-2004............. 97 4.5 Relative Federal Spending in States, 1972-1982...................... 98 4.6 Relative Federal Spending in States, 1973-1992...................... 98 4.7 Relative Federal Spending in States, 1983-2002...................... 99 4.8 Relative Federal Spending in States, 1993-2004...................... 99 4.9 Relative Federal Spending in States, 1983-2002 (FAADS Data)............. 104 vi List of Tables 2.1 Hypothesis Testing...................................... 27 3.1 Regressions