The Marketplace of Ideas for Policy Change Who do developing world leaders listen to and why? Contact AidData April 2015 | Full Report [email protected] www.aiddata.org ACKNOWLEDGMENTS This publication was prepared by a team led by Brad Parks, Zachary Rice, and Samantha Custer. Rebecca Latourell and Katie Paulson-Smith also made substantial contributions. However, the data and analysis contained in this report is the result of thousands of contributions from individuals and organizations over a five-year period. As such, we have accumulated many debts. The authors are indebted to Ronald Rapoport, Matt Andrews and Sarah Rose, for their insights on the survey questionnaire, weighting methodology, and early draft report. We extend a special thanks to Michael Tierney for his generous institutional support to facilitate our research from 2010 to 2015. The authors also acknowledge the generous financial support received from the Smith Richardson Foundation, the John Templeton Foundation, the College of William & Mary, the London School of Economics and Political Science, and the World Bank. The authors wish to thank our outstanding team of student research assistants: Kelvin Abrokwa-Johnson, Melanie Aguilar-Rojas, Arvin Alaigh, Justin Anderson, India Braver, Ben Bunch, Paul Burgess, Megan Coolidge, Anca Cretu, Caroline Davis, Jerusalem Demsas, Jake Douglas, Thomas Fringer, Taylor Gaukel, Caitlin Grobarek, Lauren Hoy, Nakul Kadaba, Emily Kelly, Dan Kent, Greg Kirwin, Rachel Kling, Kristin Krejmas, Joe Lahouchuc, Alina Lam, Lisa Lee, Gabi Levi, Ziwei Li, Nic Martinez, Kate McGinnis, Nina Moreno, Robert Mosolgo, Ashley Napier, Rachel Nelson, James O'Leary, James Page, Alyse Prawde, Suzanne Reed, Emily Reeder, Kristin Ritchey, Sarah Sajewski, Salil Singhal, Austin Strange, Chelsea Stresler, Michael Testa, Jessica Tiblin, Marie Tummarello, Linh Vinh, Taylor Wilson, and Diana Winter. We also wish to thank Raneen Ak, Sofía Cevallos, Waly Faye, Gabriel Gaspar, Hagar Hisham, Sophia Kosar, Sarah Mellman, Sarah Parks, Alexander Prokhorov, Daria Prokhorova, Vincent Rampino, and Sarah Wall for their translation work and assistance. We are also deeply grateful to each of the individuals who helped us pre-test the survey questionnaire and online survey interface: Jean Antonin, Hovhannes Avetisyan, Nicky Bell, Victor Borges, Michael Butler, Raúl Chambote, Eduardo Cruz, Amacodou Diouf, Sonia Flamenco, Francis Gatare, Vanndy Hem, Justin Mosa, Decontee King-Sackie, Nicolas Lemay-Hebert, Gibrill Jalloh, Carlos Melara, Dan Nielson, Ifueko Okauru, Charles Perla, Masroor Qureshi, Rija Ranaivoarison, Henri Ranaivoson, Marius Ratolojanahary, Ivohasina Razafimahefa, Haja Razafinjatovo, Elva Richardson, Phil Roessler, Said Salime, Alpha Sesay, Anna Sevortyan, Pranay Sinha, Ian Walker, Matt Winters, Alex Wooley, and Xolani Zitha. We are also indebted to the following people for the feedback they provided on the design and content of the 2014 Reform Efforts Survey questionnaire: Victor Borges, Jeff Chwieroth, Luis Cosenza, Casey Dunning, John Evans, Christian Henning, Sheila Herrling, John Hicklin, Stephen Knack, Eva Krampe, Julia Maciel, Mishkat Moumin, Lucrecia Palacios, Garry Robins, Said Salime, Sarah Jane Staats, Christian Stein, David Wheeler, Mark Buntaine, and Jennifer Widner. For their feedback on an early draft of this report, we are also grateful to Ingo Keilitz, Takaaki Masaki, and Lodewijk Smets. We could not have successful gathered the data necessary to produce this report without an exceptional team of researchers and programmers at NORC at the University of Chicago: Renee Hendley, Kate Hobson, David Kordus, Elise Le, Aparna Ramakrishnan, Alex Rigaux, Bhanuj Soni, and Stacy Stonich. We thank them for their many hours of hard work and dedication. We are also thankful for the assistance of various staff at and friends of the College of William & Mary’s Institute for the Theory and Practice of International Relations, including Aneta Baltes, Priscilla Caldwell, Albert Decatur, Joyce Deuley, Josephine Garcia, Marisa LeForge, Sasha Maliniak, and Jason Pully. Last, but certainly not least, we wish to thank participants in the 2014 Reform Efforts Survey. Were it not for their willingness to graciously share of their time and experiences, this report would not have been possible. The authors are solely responsible for any findings and content found in this report; we realize that those who we have acknowledged may hold views that are different from our own. TABLE OF CONTENTS 1. Introduction ...................................................................................................................................................................................... 1 1.1. Making Reform Incentives Work for Developing Countries ........................................................................................................ 1 1.2. External Assessments and the Marketplace of Policy Ideas ...................................................................................................... 1 1.3. The Missing Data Problem in Understanding External Assessment Influence ........................................................................... 2 2. Methodology .................................................................................................................................................................................... 4 2.1. Defining the Population of Interest .............................................................................................................................................. 5 2.2. Creating the Sampling Frame ..................................................................................................................................................... 5 2.3. Designing the Survey Questionnaire ......................................................................................................................................... 6 2.4. Survey Implementation ............................................................................................................................................................... 6 2.5. Composition and Representativeness of the Survey Sample ..................................................................................................... 7 3. External Assessment Influence: Global-Level Findings ............................................................................................................ 17 3.1. External assessments can influence reform efforts, but some are far more influential than others. ....................................... 17 3.2. External assessment influence is strongest at the agenda-setting stage of the policymaking process.................................... 28 3.3. In a crowded marketplace of policy ideas, assessment familiarity delivers an influence dividend. .......................................... 30 3.4. Influencing reform efforts from the outside is substantially easier in some policy domains than in others. .............................. 37 3.5. Geography matters: assessments achieve widely varying levels of uptake across regions and countries. ............................. 45 4. External Assessment Influence: Assessment-Level Findings ................................................................................................. 56 4.1. Do the types of policy problems that external assessments seek to address affect their influence? ....................................... 56 4.2. External Assessment Characteristics and Uptake Among Developing Countries .................................................................... 64 4.3. Which Assessment Attributes Do Not Appear to Influence Behavior Change? ........................................................................ 68 4.4. Alignment with local priorities is a key determinant of external assessment influence. ............................................................ 70 4.5. Survey Participants Speak to Factors Contributing to External Assessment Influence ............................................................ 74 4.6. Position-Driven Perceptions of Contributing Factors to External Assessment Influence .......................................................... 76 5. External Assessment Influence: Country-Level Findings ......................................................................................................... 80 5.1. External assessments are more influential in smaller countries. .............................................................................................. 80 5.2. External assessments are more influential in countries with democratically elected leaders and a free press. ....................... 82 5.3. Countries with effective public sector institutions are more amenable to external analysis and advice. .................................. 84 5.4. Reform-minded governments are more likely to draw upon external sources of advice and analysis. .................................... 85 5.5. Aid-dependent countries are more influenced by external assessments. ................................................................................ 86 5.6. The influence of external assessments may depend upon the training and work experience of political leaders. ................... 87 6. Findings: Intended and Unintended Effects ............................................................................................................................... 89 6.1. Assessments have largely positive and rarely negative effects on the policy