Public Disclosure Authorized Comparative Assessment of Software Public Disclosure Authorized Programs for the Development of Computer-Assisted Personal Interview (CAPI) Applications Public Disclosure Authorized July 2011 Public Disclosure Authorized Table of Contents 1. INTRODUCTION………………………………………………………….………….……………………………………………………... 6 2. METHODOLOGY………………………………………………………….………….……………………………………………………...8 2.1 Search for software ..................................................................................................................8 2.2 Inclusion criteria .......................................................................................................................9 2.2.1 Exclusion criteria .................................................................................................................. 11 2.3 Development of evaluation criteria and a rating system .......................................................... 11 2.3.1 The ideal CAPI package ..................................................................................................... 11 2.4 Evaluation areas ..................................................................................................................... 13 2.4.1 Tools for Design ................................................................................................................... 13 2.4.2 Performance in the field ....................................................................................................... 13 2.4.3 Tools for managing survey cases and survey data ................................................................. 14 2.4.4 Needs in human, computing, and financial resources ............................................................ 15 2.4.5 Tools for expanding and extending current capabilities ......................................................... 15 2.5 Evaluation of software packages ............................................................................................. 16 3. BRIEF OVERVIEW OF EACH SOFTWARE PACKAGE .......................................................................... 16 3.1 BLAISE .................................................................................................................................... 17 3.2 CASES ..................................................................................................................................... 18 3.3 CSPROX .................................................................................................................................. 19 3.4 ENTRYWARE ........................................................................................................................... 20 3.5 MMIC ..................................................................................................................................... 20 3.6 OPEN DATA KIT ....................................................................................................................... 22 3.7 PENDRAGON FORMS .............................................................................................................. 24 3.8 SURVEYBE .............................................................................................................................. 25 4. COMPARATIVE EVALUATION OF EACH SOFTWARE PACKAGE ......................................................... 26 4.1 Programming .......................................................................................................................... 26 4.2 Development environment ..................................................................................................... 29 4.3 Interface for field users ........................................................................................................... 32 4.4 Questionnaire implementation ............................................................................................... 34 4.5 Questionnaire navigation ........................................................................................................ 34 4.6 Case management .................................................................................................................. 37 4.7 Data transfer .......................................................................................................................... 39 4.8 Data exporting ........................................................................................................................ 40 4.9 Support and documentation ................................................................................................... 41 4.10 Cost effectiveness ................................................................................................................. 43 4.11 Extensibility .......................................................................................................................... 45 4.12 Operating requirements ........................................................................................................ 46 5. EVALUATIONS OF INDIVIDUAL CAPI PROGRAMS ........................................................................... 48 5.1 BLAISE ....................................................................................................................................... 48 2 5.2 CASES ........................................................................................................................................ 55 5.3 CSPROX ..................................................................................................................................... 62 5.4 ENTRYWARE .............................................................................................................................. 69 5.5 MMIC ........................................................................................................................................ 75 5.6 OPEN DATA KIT ......................................................................................................................... 80 5.7 PENDRAGON ............................................................................................................................. 88 5.8 SURVEYBE ................................................................................................................................. 94 APPENDIX A – COMPACT CHECKLIST ............................................................................................. 1021 APPENDIX B – DETAILED CHECKLIST .............................................................................................. 1021 APPENDIX C – META-EVALUATION ............................................................................................... 1021 APPENDIX D – FEATURED USERS .................................................................................................. 1021 APPENDIX E – SOFTWARE SCREENSHOTS ...................................................................................... 1021 APPENDIX F – GLOSSARY .............................................................................................................. 1032 3 ACKNOWLEDGEMENTS This publication was prepared by Arthur Shaw, Lena Nguyen, Ulrike Nischan, and Herschel Sy of the IRIS Center at the University of Maryland, with assistance from Margaret Richards. However, a report of this length and breadth is never really undertaken by its authors alone. Therefore, we would like to thank several people for their invaluable contributions. For funding, inspiring, and guiding the software comparison project, we would like to thank the staff of the Living Standards Measurement Study team at the World Bank. In particular, we would like to thank Talip Kilic for supporting, coordinating, and managing this effort; Gero Carletto and Kinnon Scott for directing and guiding this project through their valuable feedback; Kathleen Beegle for her insightful CAPI questions before and during this project; and others on the larger team for reviewing and commenting on an early draft of this report. For their informative explanations and helpful clarifications, we would like to thank representatives from the software reviewed in this report: Jim O’Reilly from Westat, the providers of Blaise technical support for North America; Jeff Royal from the University of California Berkeley’s Social Science Computing Lab, the developers of CASES; Julio Ortuzar and Ruben Hume from Serpro, the developers of CSProX; Nicolette Chin-Shue and Rodel Flores from Techneos, the developers of Entryware; Bart Orriens and Bas Weerman from RAND Corporation, the developers of MMIC; Yaw Anokwa and Mitchell Sundt from the ODK development team at the University of Washington in Seattle; Thomas Roth and Ivan Phillips of Pendragon Software; Joachim De Weerdt and Louise Broadbent from Economic Development Initiatives, the developers of Surveybe; and, although the software was not included in this review, Carmita Signes of Nova Research Company, the makers of QDS. For their invaluable insights from the field, we would also like to thank the users of CAPI software with whom we spoke: Neil Hendrick from the Human Rights Center at UC Berkeley; Clayton Sims and Anton DeWinter at Dimagi; Marcos Vera-Hernandez and Bansi Malde at the Institute for Financial Studies; the staff at the CHARLS project at Peking University; Fred Wensing from Softscape Solutions, Australia; Dave Hockman-Wert, a user from the Pendragon user group; and Michelle McConnaughay and Elizabeth Koshy from Innovations for Poverty Action. We have benefited from the collective wisdom