Developing and testing poverty assessment tools: Results from accuracy tests in Kazakhstan Manfred Zeller Gabriela Alcaraz V. June, 2005 Contract Number: GEG-I-02-02-00029-00 Task Order No. 2: Developing Poverty Assessment Tools Contractor: The IRIS Center at the University of Maryland Manfred Zeller is a full professor for Socioeconomics of Rural Development at the Georg-August-University of Göttingen, Germany. Gabriela Alcaraz V. is a doctoral candidate of the Institute of Rural Development, Georg-August-University of Göttingen, Germany. The IRIS Center is a research and technical assistance provider housed at the University of Maryland, College Park. Contents Chapter One: Introduction................................................ 1 1.1 Field survey for accuracy tests in Kazakhstan .....................................................................2 1.2 Sampling frame ....................................................................................................................2 1.2.1 Selection of the 7 oblasts ..................................................................................................3 1.2.2 Allocation of the 40 clusters within the 7 oblasts.............................................................4 1.2.3 Selection of 40 rayons and cities within the oblasts .........................................................5 1.2.4 Selection of clusters within the cities ...............................................................................5 1.3 Poverty line...........................................................................................................................6 Chapter Two: Overview of Regression Analysis ........... 15 2.1 Introduction ........................................................................................................................15 2.2 Composite Questionnaire...................................................................................................15 2.3 Selection of indicators........................................................................................................16 2.4 Specification of regression models.....................................................................................18 2.5 Differences between the models.........................................................................................19 Chapter Three: Results from Regression Models........... 24 3.1 Model 1...............................................................................................................................25 3.2 Model 2...............................................................................................................................29 3.3 Model 3...............................................................................................................................31 3.4 Model 4...............................................................................................................................33 3.5 Model 5...............................................................................................................................35 3.6 Model 6...............................................................................................................................37 3.7 Model 7...............................................................................................................................39 3.8 Model 8...............................................................................................................................41 3.9 Model 9...............................................................................................................................43 Chapter Four: Two-step models ..................................... 45 4.1 Introduction ........................................................................................................................45 4.1.1 First step: Model 1 - Best 15 set on full sample........................................................46 4.1.2 Second step: Model 1 - Best 15 set on subsample ....................................................46 4.1.3 Combined accuracy of the two-step models...............................................................49 4.1.4 Two-step model at the national poor cutoff ...............................................................50 4.2 Two-step model 7.....................................................................................................................52 4.2.1 Accuracy results for the very-poor .................................................................................52 4.2.2 Accuracy results at the national poor cutoff ...................................................................52 4.3 Two-step model 9.....................................................................................................................55 4.3.1 Accuracy results at the bottom 50% cutoff.....................................................................55 4.3.2 Accuracy results at the national poor cutoff ...................................................................55 Chapter Five: Summary.................................................. 57 5.1 Synthesis of results.............................................................................................................57 5.2 The issue of unbalanced accuracies revisited.......................................................................57 Bibliography .................................................................... 59 Annexes ........................................................................... 60 Annex A: Characteristics of sample..............................................................................................60 Table A.1 List of selected clusters, rayons and cities, villages and districts...........................60 Annex B: Descriptives of all regressors (n= 260), by type of model (N = 817) ............................62 Annex C: Gender-specific variables used in regression analysis...................................................71 Annex D: Verifiability scores provided by SANGE.....................................................................72 Annex E: Accuracy performance of regression models.................................................................83 Annex E.1: Single-step OLS models with per-capita daily expenditures as continuous dependent variable (selection of regressors by MAXR)..........................................................83 Annex E.2: Two-step models with a continuous dependent variable (OLS estimation) (with selection of variables by MAXR) ............................................................................................87 Annex F: Variables included in the BEST15 models....................................................................89 Annex F.1: Variables included in the single-step OLS models (BEST 15 sets) .....................89 Annex F.2: Variables included in the two-step OLS models 1, 7 and 9..................................92 Endnotes .......................................................................... 95 2 Developing Poverty Assessment Tools Chapter One: Introduction USAID has commissioned the IRIS Center to develop, test and disseminate poverty assessment tools which will meet Congressional requirements for accuracy and cost of implementation. Accuracy tests of poverty indicators have been implemented by IRIS in Bangladesh, Peru, Uganda, and Kazakhstan. Comprehensive information on the project is available at www.povertytools.org, and will not be summarized in this report. The purpose of this report is to present the results of the accuracy tests in Kazakhstan1. Chapter 1 provides an overview of the design of the field research for the accuracy test and the computation of the applicable poverty line. Chapter 2 provides an overview of the analysis presented in this report. In chapter 3, we present the results on selected poverty indicators from 9 regression models (known as “ordinary least squares,” or OLS, regression models). Each of these models potentially represents a newly designed poverty assessment tool, calibrated for Kazakhstan using a nationally representative sample. The regression models are run in SAS, using the function MAXR that seeks to maximize the explained variance of the dependent variable (per capita daily expenditure) by a set of best 5, best 10, and best 15 regressors. Any set of five, ten, or fifteen poverty indicators can be considered a poverty assessment tool for purposes of identifying the poverty status of a household. The first 6 regression models differ with respect to the set of poverty indicators allowed in the model, starting from a model with a full set of potential regressors and gradually restricting the set of regressors on the basis of practicality in implementation. A seventh model is run as an example of a tool that considers only those poverty indicators that were rated as “highly verifiable” by Sange Research Center, the survey firm in Kazakhstan. A subsequent model compiles these indicators with powerful subjective as well as monetary indicators. Finally, the last model makes use of poverty indicators usually available in Living Standards Measurement (LSMS) surveys. Thus, the first eight models can be considered alternative best combinations of poverty indicators which were mainly derived from existing practitioner tools for poverty assessment, while model 9 is a tool derived from poverty indicators usually available in LSMS surveys. Chapter 4 presents results from an alternative