Edmundo Beteta Mobile money and its impact on Giovanna Aguilar improving living conditions in Niger Oliver Elorreaga Jean Pierre Meneses A replication study Edgar Ventura César Huaroto December 2018 Replication Finance, Information and Communications Technology Paper 19 About 3ie The International Initiative for Impact Evaluation (3ie) is an international grant-making NGO promoting evidence-informed development policies and programs. We are the global leader in funding, producing and synthesizing high-quality evidence of what works, for whom, how, why and at what cost. We believe that using better and policy-relevant evidence helps to make development more effective and improve people’s lives. 3ie Replication Paper Series The 3ie Replication Paper Series is designed to be a publication and dissemination outlet for internal replication studies of development impact evaluations. Internal replication studies are those that reanalyze the data from an original paper in order to validate the results. The series seeks to publish replication studies with findings that reinforce or challenge the results of an original paper. To be eligible for submission, a replication study needs to be of a paper in 3ie’s online Impact Evaluation Repository and needs to include a pure replication. 3ie invites formal replies from the original authors. These are published on the 3ie website together with the replication study. The 3ie Replication Program also includes grant-making windows to fund replication studies of papers identified by donors and implementing organizations. Requests for proposals are issued based on support from donors or crowdsourced recommendations. The studies are chosen by a demand-driven process and include published studies that are considered influential, innovative or counterintuitive. The aim of the 3ie Replication Program is to improve the quality of evidence from development impact evaluations for use in policymaking and program design. About this report The Bill & Melinda Gates Foundation helped fund this report. All content, errors and omissions are the sole responsibility of the authors and do not represent the opinions of 3ie, its donors or its Board of Commissioners. Please direct any comments or queries to the corresponding author, Jean Pierre Meneses at [email protected]. Suggested citation: Meneses, JP, Ventura, E, Elorreaga, O, Huaroto, C, Aguilar, G, and Beteta, E, 2018. Mobile money and its impact on improving living conditions in Niger: a replication study. 3ie Replication Paper 19. Washington, DC: International Initiative for Impact Evaluation (3ie). Available at: http://doi.org/10.23846/RPS0019 3ie Replication Paper Series executive editor: Marie Gaarder Managing editors: Scott Neilitz and John Creamer Production manager: Brigid Monaghan Copy editor: Jaime L Jarvis Proof reader: Yvette Charboneau Cover design: Akarsh Gupta © International Initiative for Impact Evaluation (3ie), 2018 Mobile money and its impact on improving living conditions in Niger: a replication study Jean Pierre Meneses Centro de Investigaciones Sociológicas, Económicas, Políticas y Antropológicas (CISEPA) - Pontificia Universidad Católica del Perú. Lima, Peru School of Medicine "Alberto Hurtado", Universidad Peruana Cayetano Heredia. Lima, Peru Edgar Ventura Centro de Investigaciones Sociológicas, Económicas, Políticas y Antropológicas (CISEPA) - Pontificia Universidad Católica del Perú. Lima, Peru Oliver Elorreaga Centro de Investigaciones Sociológicas, Económicas, Políticas y Antropológicas (CISEPA) - Pontificia Universidad Católica del Perú. Lima, Peru César Huaroto Centro de Investigaciones Sociológicas, Económicas, Políticas y Antropológicas (CISEPA) - Pontificia Universidad Católica del Perú. Lima, Peru Giovanna Aguilar Centro de Investigaciones Sociológicas, Económicas, Políticas y Antropológicas (CISEPA) - Pontificia Universidad Católica del Perú. Lima, Peru Edmundo Beteta Centro de Investigaciones Sociológicas, Económicas, Políticas y Antropológicas (CISEPA) - Pontificia Universidad Católica del Perú. Lima, Peru Replication Paper 19 December 2018 Summary Around the world, there has been an exponential growth in cash-based programs, especially unconditional cash transfer programs. By 2013, these public policies had been placed in at least 119 countries, including 37 in Africa. This context presents an unprecedented opportunity to increase financial inclusion using a new payment channel that can decrease the operating costs, improve the security and increase the outreach. Replacing cash transfers with other payment mechanisms, such as mobile money, could have additional benefits and advantages for recipients. A 2016 study by Aker and colleagues, Payment mechanisms and antipoverty programs: evidence from a mobile money cash transfer experiment in Niger, suggests that the use of mobile money might change intra-household socio-economic dynamics. Our replication study evaluates the original paper’s findings. The pure replication – our independent reproduction of findings – confirms that our results are comparable to the original authors’ results. Our measurement and estimation analysis includes analysis of heterogeneity and a robustness test. The heterogeneity evaluation suggests that the Zap mobile money transfer intervention had a different impact on older beneficiaries than younger ones. The robustness analysis, which considers multiple imputation and Lee bounds analysis, confirms that the original results are robust to the evaluation of these methods. In the theory of change analysis, we describe stunting and wasting status, as well as their severity in children under 5 years using anthropometric measures available in the data set. We are able to find a protective effect, via Zap, for the reduction of wasting in children 25–60 months old. i Contents Summary ................................................................................................................... i List of figures and tables ....................................................................................... iii Abbreviations and acronyms ................................................................................. v 1. Introduction ......................................................................................................... 1 2. The pure replication ............................................................................................ 2 2.1 The data .......................................................................................................... 2 2.2 Tables ............................................................................................................. 3 2.3 Figures: mean cost by transfer mechanism ................................................... 14 2.4 Challenges with our pure replication .............................................................. 15 3. Measurement and estimation analysis ............................................................ 16 3.1 Exploring heterogeneous impact ................................................................... 16 3.2 Robustness analysis: multiple imputation and Lee bounds ............................ 23 4. Theory of change analysis ............................................................................... 28 4.1 Nutrition evaluation ........................................................................................ 28 4.2 Heterogeneity analysis by age ....................................................................... 31 5. Conclusion ......................................................................................................... 33 Appendix A: Push-button replication .................................................................. 34 Appendix B: Figures ............................................................................................. 52 Appendix C: Quartile results ................................................................................ 53 Appendix D: Multiple imputation results ............................................................. 61 Appendix E: Lee bounds results .......................................................................... 67 Appendix F: Summary of multiple imputation and Lee bounds ........................ 71 Appendix G: Demographic statistics of Niger ..................................................... 73 References ............................................................................................................. 75 ii List of figures and tables Table 1: Timeline of data collection and implementation ................................................. 4 Table 2: Baseline individual and household covariates (by treatment status) .................. 5 Table 3: Baseline individual and household outcomes (by treatment status) ................... 6 Table 4: Uses of cash transfer ......................................................................................... 7 Table 5: Impact on food security and nutritional status .................................................... 9 Table 6: Leakage .......................................................................................................... 10 Table 7: Location, knowledge and timing of cash transfer expenses ............................. 11 Table 8: Mobile phone ownership and usage ................................................................ 12 Table 9: Intrahousehold decision-making ...................................................................... 13 Table 10: Alternative explanations ................................................................................