National Survey of Crime Victimization Companion Study - Pilot Summary of Pilot Results Authors J. Michael Brick Sharon Lohr W. Sherman Edwards Pamela Giambo Pam Broene Douglas Williams Sarah Dipko March 12, 2013 Prepared for: Prepared by: Bureau of Justice Statistics Westat Washington, D.C. 1600 Research Boulevard Rockville, Maryland 20850-3129 (301) 251-1500 Table of Contents Chapter Page 1 Design and Objectives ....................................................................................... 1-1 1.1 Pilot Test Sample Design ..................................................................... 1-5 1.2 Pilot Telephone Instrumentation ....................................................... 1-6 1.3 Additional Experiment – Test of Survey Name ............................... 1-8 2 Data Collection Results ..................................................................................... 2-1 2.1 Counts of Yield ..................................................................................... 2-1 2.2 Level of Effort ....................................................................................... 2-5 2.3 Response Rates ...................................................................................... 2-6 2.4 Summary of Data Collection Results ................................................. 2-10 3 Results of Other Experiments .......................................................................... 3-1 4 Cost Comparisons .............................................................................................. 4-1 4.1 Mail Costs ............................................................................................... 4-2 4.2 Telephone Costs .................................................................................... 4-4 4.3 Relationship of Mail and Telephone Costs ....................................... 4-8 4.4 Effect of Changing Parameters ........................................................... 4-9 4.5 Comparison with RDD Costs ............................................................. 4-10 5 Questionnaire Assessment ................................................................................ 5-1 5.1 Overview and CATI Instrument Timing .......................................... 5-1 5.2 Item Nonresponse and Response Frequencies ................................ 5-3 5.3 Interviewer Debriefing ......................................................................... 5-6 5.4 Behavior Coding and Monitoring ....................................................... 5-9 6 Data Processing .................................................................................................. 6-1 6.1 Crime Classification Process ............................................................... 6-1 6.2 Sensitivity Analysis ................................................................................ 6-4 6.3 Recommendations ................................................................................ 6-6 NCVS-CS Pilot Report iii Contents (continued) Chapter Page 7 Weighting ............................................................................................................. 7-1 7.1 Household Weights .............................................................................. 7-2 7.1.1 Household Base Weights ..................................................... 7-2 7.1.2 Household Nonresponse AAdjustments ............................. 7-3 7.1.3 Household Poststratification .............................................. 7-4 7.2 Person Weights ...................................................................................... 7-5 7.2.1 Person Base Weights ............................................................ 7-6 7.2.2 Person Weight Raking .......................................................... 7-6 7.2.3 Person Weight Trimming .................................................... 7-6 7.3 Victimization and Incident Weights ................................................... 7-7 7.4 Variance Estimation ............................................................................. 7-7 8 Estimation Results .............................................................................................. 8-1 8.1 Victimization Rates: Comparison with National NCVS Estimates and Estimates from the Chicago Poliice Department ............................................................................................ 8-1 8.1.1 Why Might Estimates from the NCVS, the CS, and the City of Chicago be Expected to Differ? .............. 8-1 8.1.2 Property Victimization ......................................................... 8-4 8.1.3 Violent Victimization ........................................................... 8-8 8.2 Effectiveness of Mail Screener for Predicting NCVS Victimization .......................................................................................... 8-12 8.3 Nonresponse Bias Assessment ........................................................... 8-15 8.4 Analysis of Interviewer Effects ........................................................... 8-19 8.5 Recall Effects ......................................................................................... 8-20 8.6 Summary and Recommendations ....................................................... 8-21 9 Summary and Next Steps .................................................................................. 9-1 9.1 Lessons from the Pilot Study .............................................................. 9-1 9.2 Objectives for Further Evaluation ..................................................... 9-4 9.3 Suggested Approach ............................................................................. 9-6 NCVS-CS Pilot Report iv Contents (continued) Appendices Page A Mail Screeners ..................................................................................................... A-1 B Variation Between Core NCVS Instruments and the Westat Pilot ............ B-1 C Results from Behavior Coding ......................................................................... C-1 D Behavior Coding Questions Reviewed With Recommendations ................ D-1 E Census Flow Chart of TOC Codes .................................................................. E-1 F Type of Crime Specifications ............................................................................ F-1 G Description of the Type of Crime Codes ....................................................... G-1 H Census Instructions for Using the Automated NCVS Editing and Coding Blaise Instrument .................................................................................. H-1 I How To Use the Weights .................................................................................. I-1 Tables 1-1 Comparison of Approach 1 and Approach 2 methodologies used in the Pilot Data Collection ............................................................................... 1-2 1-2 Expected pilot sample performance, Approach 1 ......................................... 1-7 1-3 Expected pilot sample performance, Approach 2 ......................................... 1-7 2-1 Number of sampled and completed sampled addresses, by approach ............................................................................................................... 2-2 2-2 Percentage completing household screener of those sent for telephone data collection, by approach and telephone match status ..................................................................................................................... 2-3 2-3 Number of telephone numbers sent for telephone data collection, by approach and phone type ............................................................................. 2-4 2-4 Cases to complete ratio, by approach .............................................................. 2-5 2-5 Estimated response rates, by approach ........................................................... 2-7 NCVS-CS Pilot Report v Contents (continued) Tables (continued) Page 2-6 Estimated NCVS-1 household response rates, by approach and characteristic ........................................................................................................ 2-8 2-7 Estimated Approach 2 mail response rates, by characteristic ...................... 2-9 4-1 Approach 1 mail screener costs ........................................................................ 4-2 4-2 Approach 2 mail screener costs ........................................................................ 4-4 4-3 Approach 1 telephone and total costs ............................................................. 4-5 4-4 Approach 2 telephone and total costs ............................................................. 4-7 5-1 Interviewer and respondent behavior codes used in NCVS-CS pilot ....................................................................................................................... 5-10 7-1 General guide to using the household, person, victimization and incident weights .................................................................................................. 7-7 8-1 Estimated property victimization rates, per thousand households; 95 percent confidence intervals are given beelow the estimates. The last two columns give the ratios of the CS victimization rates to the NCVS victimization rates from the Chicago CBSA .......................... 8-6 8-2 Estimated property victimization rates, per thousand households, for geographic poststrata; 95 percent confidence intervals are given below the estimates .................................................................................