Georgia State University ScholarWorks @ Georgia State University Economics Dissertations 8-1-2012 Essays on Experimental and Quasi-Experimental Policy Design and Evaluation Juan Jose Miranda Montero Follow this and additional works at: https://scholarworks.gsu.edu/econ_diss Recommended Citation Miranda Montero, Juan Jose, "Essays on Experimental and Quasi-Experimental Policy Design and Evaluation." Dissertation, Georgia State University, 2012. https://scholarworks.gsu.edu/econ_diss/86 This Dissertation is brought to you for free and open access by ScholarWorks @ Georgia State University. It has been accepted for inclusion in Economics Dissertations by an authorized administrator of ScholarWorks @ Georgia State University. For more information, please contact [email protected]. PERMISSION TO BORROW In presenting this dissertation as a partial fulfillment of the requirements for an advanced degree from Georgia State University, I agree that the Library of the University shall make it available for inspection and circulation in accordance with its regulations governing materials of this type. I agree that permission to quote from, to copy from, or to publish this dissertation may be granted by the author or, in his or her absence, the professor under whose direction it was written or, in his or her absence, by the Dean of the Andrew Young School of Policy Studies. Such quoting, copying, or publishing must be solely for scholarly purposes and must not involve potential financial gain. It is understood that any copying from or publication of this dissertation which involves potential gain will not be allowed without written permission of the author. _________________ Signature of Author NOTICE TO BORROWERS All dissertations deposited in the Georgia State University Library must be used only in accordance with the stipulations prescribed by the author in the preceding statement. The author of this dissertation is: Juan José Miranda Montero 5480 Wisconsin Ave, Apartment 811 Chevy Chase, MD 20815 The director of this dissertation is: Dr. Paul J. Ferraro Andrew Young School of Policy Studies Georgia State University P. O. Box 3992 Atlanta, GA 30302-3992 Users of this dissertation not regularly enrolled as students at Georgia State University are required to attest acceptance of the preceding stipulations by signing below. Libraries borrowing this dissertation for the use of their patrons are required to see that each user records here the information requested. Type of use Name of User Address Date (Examination only or copying) ESSAYS ON EXPERIMENTAL AND QUASI-EXPERIMENTAL POLICY DESIGN AND EVALUATION BY JUAN JOSE MIRANDA MONTERO A Dissertation Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in the Andrew Young School of Policy Studies of Georgia State University GEORGIA STATE UNIVERSITY 2012 Copyright Juan José Miranda Montero 2012 ACCEPTANCE This dissertation was prepared under the direction of the candidate's Dissertation Committee. It has been approved and accepted by all members of that committee, and it has been accepted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Economics in the Andrew Young School of Policy Studies of Georgia State University. Dissertation Chair: Dr. Paul J. Ferraro Committee: Dr. H. Spencer Banzhaf Dr. Craig McIntosh Dr. Rusty Tchernis Electronic Version Approved: Mary Beth Walker, Dean Andrew Young School of Policy Studies Georgia State University July 2012 ACKNOWLEDGMENTS This dissertation was made possible by the support of many people. I owe a debt of gratitude to all of the friends, family members, and teachers who have facilitated my personal and academic growth over the years. All of you know who you are. First and foremost, I would like to thank to Dr. Paul Ferraro for his great mentorship, encouragement, support, and friendship. He has been instrumental in the achievement of my Ph.D. His influence on my professional life has been far greater than that of a dissertation chair. Second, I would like to thank to Dr. Roxana Barrantes, my Peruvian mentor, former boss, and friend, for her unconditional support, both personally and academically. I am also indebted to the members of my dissertation committee, Dr. Spencer H. Banzhaf, Dr. Craig McIntosh, and Dr. Rusty Tchernis, for their insightful comments and suggestions. I would like to dedicate this dissertation to Lorena, my wife, with love and gratitude for her patience and encouragement. I am grateful for her support at all times and everywhere. To Jaime and Juana, my parents, their dedication and hard work have been my examples. To Jaime, my brother, for all that he has done for me. To Andrea and Akemi, my nieces, who brought happiness and light to my family. vi CONTENTS ACKNOWLEDGMENTS ............................................................................................................. vi LIST OF FIGURES ....................................................................................................................... ix LIST OF TABLES .......................................................................................................................... x LIST OF APPENDICES ................................................................................................................ xi ABSTRACT .................................................................................................................................. xii CHAPTER I: HETEROGENEOUS TREATMENT EFFECTS AND MECHANISMS IN INFORMATION-BASED ENVIRONMENTAL POLICIES: EVIDENCE FROM A LARGE- SCALE NATURAL FIELD EXPERIMENT ................................................................................. 1 1. Introduction .......................................................................................................................... 1 2. Experiments with Information-based Environmental Programs .......................................... 3 3. Methodology ........................................................................................................................ 4 3.1. Heterogeneity in Treatment Responses ............................................................................. 5 3.2. Nonparametric Tests ......................................................................................................... 7 3.3. Quantile Treatment Effects ............................................................................................... 9 3.4. Subgroup Analysis .......................................................................................................... 10 4. Experimental Design and Data ........................................................................................... 11 5. Results ................................................................................................................................ 13 5.1. Nonparametric Tests ....................................................................................................... 13 5.2. Quantile Regressions ....................................................................................................... 15 5.3. Subgroup Analysis .......................................................................................................... 16 5.4. Targeting Information Campaigns .................................................................................. 17 5.5. Mechanisms of the Strong Social Norm Message (Treatment 3) ................................... 18 5.5.1. Recurring behavioral changes versus one-shot technological investments ................. 19 5.5.2. Outdoor versus indoor water use changes .................................................................... 20 5.5.3. Social versus private preferences ................................................................................. 21 6. Conclusions ........................................................................................................................ 23 7. References .......................................................................................................................... 26 CHAPTER II: COMPARING EXPERIMENTAL AND NON-EXPERIMENTAL EVALUATION DESIGNS USING A LARGE-SCALE RANDOMIZED EXPERIMENT ....... 53 1. Introduction ........................................................................................................................ 53 vii 2. The Performance of Non-Experimental Evaluation Designs ............................................. 56 3. Study Site and Experiment ................................................................................................. 59 4. Non-Experimental Designs ................................................................................................ 60 4.1. Single Difference ............................................................................................................. 61 4.2. Difference-in-Differences ............................................................................................... 65 4.3. Summary of Non-experimental Designs and Methods ................................................... 67 5. Data .................................................................................................................................... 67 6. Experimental Benchmark ................................................................................................... 70 7. Non-experimental Evaluation Criteria ............................................................................... 72 8. Non-Experimental Sample