Evaluating and Improving Surveys of American Jews
Total Page:16
File Type:pdf, Size:1020Kb
Numbering the Jews: Evaluating and Improving Surveys of American Jews Volume I of II A Dissertation Presented to The Faculty of the Graduate School of Arts and Sciences Brandeis University Near Eastern and Judaic Studies and Sociology Sylvia Barack Fishman, Advisor Shulamit Reinharz, Advisor In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy by Benjamin Phillips February 2007 This dissertation, directed and approved by Benjamin Phillips’s Committee, has been accepted and approved by the Faculty of Brandeis University in partial fulfillment of the requirements for the degree of: DOCTOR OF PHILOSOPHY ___________________________________________________ Adam B. Jaffe, Dean of Arts and Sciences Dissertation Committee: ___________________________________________________ Sylvia Barack Fishman, Near Eastern and Judaic Studies ___________________________________________________ Shulamit Reinharz, Sociology ___________________________________________________ Jonathan Sarna, Near Eastern and Judaic Studies ___________________________________________________ David Cunningham, Sociology ___________________________________________________ Leonard Saxe, Hornstein Jewish Professional Leadership Program Copyright by Benjamin Phillips 2007 ACKNOWLEDGMENTS I have looked forward to writing this section since the inception of the dissertation, not only because it meant I was almost finished, but because so many people participated in so many ways to make this possible. A major debt of gratitude is owed to Len Saxe and Charles Kadushin, who enabled me to play a major role in the Boston Jewish Community Study and whose way of thinking undergirds this dissertation. The Boston study would not have come together, however, without the assistance of many others at the Cohen Center for Modern Jewish Studies and the Steinhardt Social Research Institute. Dara Brzoska, MeLena Hessel, and Beverley Batyah Levine all played major roles in various aspects of the project. Graham Wright in particular enabled me to take the time to write my dissertation by taking over the reins of the Boston Jewish Community Study in spring and summer 2006 and made major contributions, particularly with respect to weighting and analysis; much of my analysis is grounded on his SPSS and Stata code. The Boston survey also benefitted from the experience and effort of staff at Schulman, Ronca & Bu- cuvalas, Inc.: Mark Schulman, Chintan Turakhia, and Shermarkeh Hussein. Chintan’s comments on the potentially self-defeating nature of a large list frame informed my anal- ysis of survey frames. Sherm not only managed the study’s field operations but made possible the study’s metamorphosis from a design to a fielded survey. Christopher Win- iv ship played a far greater role in the study’s design than his position as co-chair of the community study committee of Combined Jewish Philanthropies of Greater Boston would suggest, and is acknowledged at numerous points throughout the text. I am also grateful for the foresight and willingness to support sophisticated social scientific re- search of Combined Jewish Philanthropies and Barry Shrage and Martin Abramowitz in particular. Mark Niedergang, a consultant to Combined Jewish Philanthropies, made the expanded list frame possible by obtaining organizational lists. I am also deeply grateful for the financial support provided by Mrs. May Gruber and Mr. Mandell (Bill) Berman, which enabled me to write this dissertation. v ABSTRACT Numbering the Jews: Evaluating and Improving Surveys of American Jews A dissertation presented to the Faculty of the Graduate School of Arts and Sciences of Brandeis University, Waltham, Massachusetts by Benjamin Phillips American Jews have long understood the vitality of the Jewish community by its num- bers: initially its population size and later its characteristics. Acquiring such data, though, has always been complicated by the exclusion of questions on religion from the U.S. Census of Population and Households. In the absence of official counts of the Jewish population, American Jews have sought other ways of counting themselves. The methods used to this end have, however, varied considerably across the decades, as have the quali- ty of the methods used. Initial attempts to count the Jewish population relied on institu- tional censuses of Jewish congregations supplemented, and ultimately supplanted, by guesstimates of dubious quality from local reporters. From the beginning of the twentieth century, these efforts were joined by proxy methods that relied on readily ascertainable statistics with some type of relationship with Jewish identity, which also failed to gener- ate accurate estimates. More accurate counts were finally arrived initially in censuses based primarily on organizational lists and later via various forms of geographically based samples. List-based methods were no longer sufficient as the number of Jews not vi found on lists grew due to assimilation. Costs for these methods, though, were too great and the 1980s saw a move to random digit dialing methods. Costs, however, continued to increase, driving studies to become more economically efficient. The late 1990s saw a decline in response rates to surveys of all types, further raising costs and increasing me- thodological concerns. The long-awaited National Jewish Population Survey of 2000-01 was the culmination of these trends, with a vast cost of $6 million and multiple methodo- logical flaws. I examine a variety of responses to the challenges of the present survey research environment. Indirect screening questions appear to be less effective than a direct approach. Use of expanded list frames drawn from the 10 or so largest Jewish or- ganizations in a community in combination with random digit dialing appears to be the most efficient sampling scheme. I conclude that cost-variance optimization for estimating Jewish incidence leads to dramatic increases in accuracy compared to nonoptimized dis- tribution of the sample across frames and strata. vii TABLE OF CONTENTS List of Tables ix List of Figures x Chapter 1. The Population Wars 1 Chapter 2. History of Jewish Population Studies in the United States 18 Chapter 3. Contemporary Challenges 136 Chapter 4. Sampling Frames 224 Chapter 5. Nonresponse Analysis and Weighting 300 Chapter 6. Evaluating and Improving Surveys of American Jews 340 Appendix A. Census Estimates of Number of Jewish Congregations by County 362 Appendix B. Compendium of Studies of United States Jewish Communities 367 Appendix C. Lists of Distinctive Jewish Names 379 Appendix D. Most Commonly Occurring Surnames in the Boston Study 383 Appendix E. NJPS 2000-01 Weighting Adjustment for Household Type 385 Appendix F. Letters Requesting Permission to Use Lists 396 Appendix G. Development of the List Frame 400 Appendix H. Sampling Strata 411 Appendix I. Estimation of Stratum Population Size 414 Appendix J. Multiplicity Screener Questions 417 Appendix K. Prenotification Letters 423 Appendix L. Refusal Conversion Letter 430 Appendix M. Final Dispositions and Outcome Rates 432 Appendix N. Estimation of Survey Costs 437 Appendix O. Cost-Variance Optimization 450 References 453 viii LIST OF TABLES Table 2:1. Number of Synagogues 36 Table 2:2. Number of Synagogue Seats 39 Table 2:3. Number of Synagogue Members 42 Table 3:1. Screening Questions in Jewish Population Studies 185 Table 3:2. NJPS 2000-01 Jewishness Classification 192 Table 3:3. NJPS 1990 and 2000-01 Screening Questions 196 Table 3:4. Current Denominational Identification 197 Table 4:1. Number of Households on List Frame at Various Stages of Development 234 Table 4:2. List Frame Coverage 238 Table 4:3. Screener Outcome Rates by Sampling Frame 257 Table 4:4. Unweighted Percentage of Jewish Households by Frame and Stratum 258 Table 4:5. Estimates of Jewish Population Size 260 Table 4:6. Estimates of Intermarriage Rate 260 Table 4:7. Characteristics of Alternate Non-Universal Sampling Frames 273 Table 4:8. Group Quarters Population of Boston and Brockton MSAs 279 Table 4:9. Estimated Jewish Undergraduate Population at Major Universities 280 Table 4:10. List and Survey Based Estimates of Jewish Population Incidence 286 Table 4:11. Estimated Percentage Orthodox by Age Group 288 Table 5:1. List Sample and Population Sizes by Stratum 303 Table 5:2. Multinomial Logit Regression of List Household Response 307 Table 5:3. RDD Domain 314 Table 5:4. Overlap Domain 315 Table 5:5. Multinomial Logit Regression of RDD Household Response 318 Table 5:6. Variables Used in the Calculation of Composite Weights 323 Table 5:7. Odds Ratios of Logistic Regression on Selected Variables 329 ix LIST OF FIGURES Figure 2:1. Coverage Error 27 Figure 2:2. Sampling Distribution of S Realizations of Normal Samples of n=1,000 29 Figure 2:3. Nonresponse Error 31 Figure 2:4. Estimates of State Jewish Population, 1905-07 54 Figure 2:5. American Jewish Year Book Population Estimates 55 Figure 2:6. Number of Jewish Community Studies by Year 107 Figure 3:1. Ownership of Answering Machines 150 Figure 3:2. Subscription to Cellular and Landline Telephones 165 Figure 3:3. Telephone Coverage, 2000-05 166 Figure 3:4. Prevalence of Listed Numbers in 100 Blocks 170 Figure 3:5. Screening Cost for n=1,000 Elements of a Rare Population by Incidence 178 Figure 3:6. Comparing Complete and Incomplete Household Types 209 Figure 4:1. Dual-frame Sample Coverage 230 Figure 4:2. Conceptual Diagram of Multiple List Frame Approach 232 Figure 4:3. Estimated Proportion of Jewish Households by Town 236 Figure 4:4. Location of List Households 237 Figure 4:5. Pilot Test Random Digit Dialing Strata 242 Figure 4:6. Final Random Digit Dialing Strata 243 Figure 4:7. Distribution of List Sizes 264 Figure 4:8. Percentage of Jewish Population in List Frame by Number of Lists Used 265 Figure 4:9. Cost of Developing List Frame by Number of Lists Sorted by Size 267 Figure 4:10. Cost of Surveying by Number of Lists in List Frame 268 Figure 4:11. 95% Confidence Intervals for Jewish Incidence by Number of Lists 269 Figure 4:12. 95% Confidence Intervals for Intermarriage Rate by Number of Lists 272 Figure 4:13.