NSAF Methodology R 19991999 NSAFNSAF CollectionCollection ofof PapersPapers Report No. 7 Prepared by: Tamara Black Kenneth Finegold A. Bowen Garrett Adam Safir Fritz Scheuren Kevin Wang Douglas Wissoker Urban Institute eports J. Michael Brick David Cantor David Ferraro Deborah Ingram Jill Montaquila Assessing John O'Hare the New Teresa Strickler Federalism Westat An Urban Institute Program to Assess Changing Social Policies Preface Introduction 1999 NSAF Technical Papers is the seventh report in a series describing the methodology of the 1999 National Survey of America’s Families (NSAF). It is a companion to the recently re-issued 1997 NSAF report on the same subject (which is No. 16 in that series). See also the related 1997 Methodology Report No. 7 on nonresponse in the first round of the NSAF. About the National Survey of America’s Families (NSAF) As discussed elsewhere (especially Report No. 1 in the 1997 NSAF methodology series), the NSAF is part of the Urban Institute’s Assessing the New Federalism project, which is being done in partnership with Child Trends. Data collection for the NSAF was conducted by Westat. In each round of the NSAF carried out so far, over 40,000 households have been interviewed, yielding information on over 100,000 people. The NSAF has focused on the economic, health, and social characteristics of children, adults under the age of 65, and their families. The sample is representative of the nation as a whole and of 13 states in particular. Due to its large state sample sizes, the NSAF has an unprecedented ability to measure differences between the 13 states it targeted. About the 1997 and 1999 NSAF Methodology Series The 1997 and 1999 series of methodology reports have been developed to provide readers with a detailed description of the methods employed to conduct the NSAF. The two series are nearly parallel, except for the documentation of the public use files, where an on-line system is being used for the 1999 survey, and we are planning to reissue the 1997 files on a similar basis. Report No. 1 in the 1997 series introduces the survey. Reports 2 through 4 in both series—plus Report No. 14 in the 1997 series—describe the sample design, how survey results were estimated and how variances were calculated. Reports 5 and 9 in each series describe the interviewing done in for the telephone (random-digit dial, or RDD) and in-person samples. Reports 6 and 15 in the 1997 series and Report No. 6 in the 1999 series display and discuss the comparisons we made to surveys that partially overlapped NSAF in content—including the Current Population Survey and the National Health Interview Survey. Reports 7 and 8 in both series cover what we know about nonresponse rates and nonresponse biases. Report No. 10 in both series covers the details of the survey processing, after the fieldwork was completed, including the imputation done for items that were missing. Report No. 11 in both series introduces the public use files made available. In the 1997 series, there were additional reports on the public use files available in a PDF: Reports No. 13, and 17 through 22. These will all eventually be superceded by the on-line data file codebook system that we are going to employ for the 1999 survey. The 1997 and 1999 NSAF questionnaires are available as Report No. 12 in the 1997 series and Report No. 1 in the 1999 series. Report No. 16 for the 1997 series contains occasional papers of methodological interest given at professional meetings though 1999, regarding the NSAF work as it has progressed since 1996, when the project began. About This Report This seventh report in the 1999 Methodology Series collects occasional papers given at professional meetings and workshops at the Urban Institute on technical issues in the design, implementation, and operation of the 1999 round of the NSAF. For More Information For more information about the National Survey of America’s Families, contact Assessing the New Federalism, Urban Institute, 2100 M Street, NW, Washington, D.C. 20037, telephone: (202) 261-5886 fax: (202) 293-1918, Web site: http://newfederalism.urban.org. For information about this report, contact [email protected]. Kevin Wang and Fritz Scheuren January 2001 Table of Contents Page Chapter 1 1 National Survey of America’s Families: Survey Methods and Data Reliability ........ 1-1 Adam Safir, Fritz Scheuren, and Kevin Wang Chapter 2 2A Assessing Nonresponse Bias in the National Survey of America’s Families............. 2A-1 Tamara Black and Adam Safir 2B Panel Conditioning in a Random-Digit Dial Survey .................................................. 2B-1 Kevin Wang, David Cantor, and Adam Safir 2C Macro and Micro Paradata for Survey Assessment .................................................... 2C-1 Fritz Scheuren 2D Estimating Residency Rates for Undetermined Telephone Numbers......................... 2D-1 J. Michael Brick, Jill Montaquila, and Fritz Scheuren 2E Correlates of Measurement Error When Screening on Poverty Status for a Random-Digit Dial Survey ......................................................................................... 2E-1 David Cantor and Kevin Wang 2F Weighting Issues in an RDD Panel Survey ................................................................ 2F-1 David Ferraro, J. Michael Brick, and Teresa Strickler Chapter 3 3A Use of STATA with NSAF......................................................................................... 3A-1 A. Bowen Garrett 3B Notes on Weighting in Regression ............................................................................. 3B-1 Douglas Wissoker 3C Milwaukee Study........................................................................................................ 3C-1 Pat Cunningham, David Cantor, Carl Fowlkes, and Judy Meader 3D Report on the NSAF Cycle 2 Capture/Recapture Study............................................. 3D-1 Mike Brick, Nancy Vaden-Kiernan, Carl Fowlkes, and Pat Cunningham i List of Tables Table Page 1-1 Maximum Margins of Error, 1997 and 1999 NSAF..................................................1-4 1-2 Earnings from Employment: Distributions for Adults 18 to 64 Years Old, 1997 and 1999 NSAF and CPS Compared (in percent).............................................1-8 1-3 Household Size: Distributions for 1997 and 1999, NSAF and CPS Compared (in percent) ...............................................................................................1-8 2A-1 Analysis Groups.........................................................................................................2A-5 2A-2 Results of Controlling for Same Telephone Number ................................................2A-5 2A-3 Distribution of Matchers and Joiners.........................................................................2A-6 2A-4 Comparison of Completes vs. Refusals and Noncontacts .........................................2A-8 2B-1 Variables Compared in the Analysis of Panel Conditioning .....................................2B-8 2B-2 Sample Size for Child Sample, by Sample Group.....................................................2B-9 2B-3 Demographic Characteristics, by Sample Group.......................................................2B-9 2B-4 Statistically Significant Differences between New and Matched Sample.................2B-10 2D-1 Example of Survival Method Computations .............................................................2D-16 2D-2 Estimated Residency Rates and Distribution of Undetermined Numbers in the NHES:1999 Sample, by Listed Status and Primary Coding of Answering- Machine Call Results.................................................................................................2D-16 2D-3 Estimated Residency Rate for Undetermined Telephone Numbers and Overall in the NSAF:1999, by Site.........................................................................................2D-17 2E-1 Percent of Respondents Switching Poverty Status between Screener and Extended Interview, by Type of Family.....................................................................2E-10 2E-2 Switching Rates, by Respondent, Design Characteristics, and Type of Family ........2E-11 2E-3 Switching Rates, by Detailed Poverty Status and Type of Family ............................2E-12 ii List of Tables (Continued) Table Page 2E-4 Switching Rates, by Measures of Assets and Income and by Type of Family...........2E-13 2F-1 Sample Sizes by Round and Subsampling Retention Rates, by Stratum...................2F-4 2F-2 Expected and Observed Residency and Screener Response Rates, by Stratum.........2F-4 3B-1 Estimates of Wage Equation for Welfare Leavers, Unweighted and Weighted Regressions, with Varying Controls for Sample Selection........................................3B-9 3C-1 Source of Address for Milwaukee Sample ................................................................3C-7 3C-2 Match Between Address and Telephone Number by Source of Address ..................3C-7 3C-3 Match Between Address and telephone Number by Final NSAF Result ..................3C-8 3D-1 Number of Cases, by Stratum and Disposition, Capture-Recapture Experiment.....3D-3 3D-2 Percentage of Cases, by Stratum and Disposition, Capture-Recapture Experiment.................................................................................................................3D-3 3D-3 1998 Poverty Threshold Amounts .............................................................................3D-15 3D-4 Final Distribution of Total Sample............................................................................3D-18