The Basics of Social Research, 4Th Edition

THE BASICS OF SOCIAL RESEARCH A Note from the Author

Writing is my joy, sociology my pas- I ventured into the outer world sion. I delight in putting words to- by way of Harvard, the USMC, U.C. gether in a way that makes people Berkeley, and 12 years teaching at learn or laugh or both. Sociology is the University of Hawaii. Along the one way I can do just that. It repre- way, I married Sheila two months sents our last, best hope for planet- after our first date, and we created training our race and finding ways Aaron three years after that: two of for us to live together. I feel a special my wisest acts. I resigned from teach- excitement at being present when ing in 1980 and wrote full-time for sociology, at last, comes into focus seven years, until the call of the class- as an idea whose time has come. room became too loud to ignore. I grew up in small-town Vermont For me, teaching is like playing jazz. and New Hampshire. When I announced I wanted Even if you perform the same number over and to be an auto-body mechanic, like my dad, my over, it never comes out the same twice, and you teacher told me I should go to college instead. don’t know exactly what it’ll sound like until you When Malcolm X announced he wanted to be a hear it. Teaching is like writing with your voice. lawyer, his teacher told him a colored boy should At last, I have matured enough to rediscover be something more like a carpenter. The difference and appreciate my roots in Vermont each summer. in our experiences says something powerful about Rather than a return to the past, it feels more like the idea of a level playing field. The inequalities the next turn in a widening spiral. I can’t wait to among ethnic groups run deep. see what’s around the next bend. THE BASICS OF SOCIAL RESEARCH

Fourth Edition

Earl Babbie Chapman University

Australia • Brazil • Canada • Mexico • Singapore • Spain United Kingdom • United States The Basics of Social Research, Fourth Edition Earl Babbie

Acquisitions Editor: Chris Caldeira Art Director: John Walker Development Editor: Sherry Symington Print Buyer: Nora Massuda Assistant Editor: Christina Ho Permissions Editor: Roberta Broyer Editorial Assistant: Tali Beesley Production Service: Greg Hubit Bookworks Technology Project Manager: Dave Lionetti Copy Editor: Molly D. Roth Marketing Manager: Michelle Williams Illustrator: Lotus Art Marketing Assistant: Jaren Boland Cover Designer: Yvo Riezebos Marketing Communications Manager: Linda Yip Cover Image: © Chad Ehlers/Alamo Project Manager, Editorial Production: Matt Ballantyne Compositor: Newgen Creative Director: Rob Hugel Text and Cover Printer: Courier—Westford

© 2008, 2005 Thomson Wadsworth, a part of The Thomson Thomson Higher Education Corporation. Thomson, the Star logo, and Wadsworth are 10 Davis Drive trademarks used herein under license. Belmont, CA 94002-3098 USA ALL RIGHTS RESERVED. No part of this work covered by the copyright hereon may be reproduced or used in any form or by any means—graphic, electronic, or mechanical, including photocopying, recording, taping, web distribution, information storage and retrieval systems, or in any other manner—without the written permission of the publisher.

Printed in the United States of America 1 2 3 4 5 6 7 11 10 09 08 07

For more information about our products, contact us at: Thomson Learning Academic Resource Center 1-800-423-0563 For permission to use material from this text or product, submit a request online at http://www.thomsonrights.com. Any additional questions about permissions can be submitted by e-mail to [email protected].

Library of Congress Control Number: 2006938498 Student Edition: ISBN-13: 978-0-495-09468-5 ISBN-10: 0-495-09468-4 International Student Edition: ISBN: 0-495-10233-4 (Not for sale in the United States) Dedication

Evelyn Fay Babbie Henry Robert Babbie This page intentionally left blank CONTENTS IN BRIEF

Part One Part Four AN INTRODUCTION TO INQUIRY 1 ANALYSIS OF DATA: 1 Human Inquiry and Science 3 QUANTITATIVE 2 Paradigms, Theory, and Research 32 AND QUALITATIVE 412 3 The Ethics and Politics of Social 13 Qualitative Data Analysis 414 Research 64 14 Quantitative Data Analysis 442 15 Reading and Writing Social Research 470 Part Two THE STRUCTURING OF INQUIRY: Appendixes QUANTITATIVE A Using the Library 498 AND QUALITATIVE 92 B Random Numbers 506 4 Research Design 94 C Distribution of Chi Square 508 5 Conceptualization, Operationalization, D Normal Curve Areas 510 and Measurement 130 E Estimated Sampling Error 511 6 Indexes, Scales, and Typologies 168 7 The Logic of Sampling 198

Part Three MODES OF OBSERVATION: QUANTITATIVE AND QUALITATIVE 242 8 Experiments 244 9 Survey Research 268 10 Qualitative Field Research 312 11 Unobtrusive Research 348 12 Evaluation Research 382

vii This page intentionally left blank CONTENTS

Preface xvii Review Questions 30 Online Study Resources 30 Additional Readings 30 Part One Chapter 2 AN INTRODUCTION TO INQUIRY 1 Paradigms, Theory, Chapter 1 and Research 32 What Do You Think? 33 Human Inquiry and Science 3 Introduction 33 What Do You Think? 4 Some Social Science Paradigms 34 Introduction 4 Macrotheory and Microtheory 36 Looking for Reality 6 Early Positivism 36 Ordinary Human Inquiry 6 Conﬂ ict Paradigm 37 Tradition 7 Symbolic Interactionism 37 Authority 8 Ethnomethodology 38 Errors in Inquiry and Some Solutions 8 Structural Functionalism 39 What’s Really Real? 10 Feminist Paradigms 40 The Foundations of Social Science 12 Critical Race Theory 42 Theory, Not Philosophy or Belief 13 Rational Objectivity Reconsidered 42 Social Regularities 13 Two Logical Systems Revisited 45 Aggregates, Not Individuals 15 The Traditional Model of Science 45 A Variable Language 16 Deduction and Induction Compared 48 The Purposes of Social Research 21 Deductive Theory Construction 54 The Ethics of Human Inquiry 22 Getting Started 54 Some Dialectics of Social Research 22 Constructing Your Theory 54 Idiographic and Nomothetic Explanation 22 An Example of Deductive Theory: Inductive and Deductive Theory 23 Distributive Justice 54 Quantitative and Qualitative Data 25 Inductive Theory Construction 56 Pure and Applied Research 27 An Example of Inductive Theory: What Do You Think? Revisited 28 Why Do People Smoke Marijuana? 57 Main Points 29 The Links between Theory and Research 58 Key Terms 30 The Importance of Theory in the “Real World” 59

ix x CONTENTS

Research Ethics and Theory 60 Three Purposes of Research 97 What Do You Think? Revisited 60 Exploration 97 Main Points 61 Description 99 Key Terms 62 Explanation 99 Review Questions 62 The Logic of Nomothetic Explanation 99 Online Study Resources 62 Criteria for Nomothetic Causality 100 Additional Readings 62 Nomothetic Causal Analysis and Hypothesis Testing 101 Chapter 3 False Criteria for Nomothetic Causality 102 The Ethics and Politics Necessary and Sufﬁ cient Causes 102 of Social Research 64 Units of Analysis 104 What Do You Think? 65 Individuals 105 Introduction 65 Groups 106 Ethical Issues in Social Research 66 Organizations 106 Voluntary Participation 67 Social Interactions 108 No Harm to the Participants 68 Social Artifacts 108 Anonymity and Conﬁ dentiality 69 Units of Analysis in Review 109 Deception 72 Faulty Reasoning about Units of Analysis: Analysis and Reporting 73 The Ecological Fallacy and Institutional Review Boards 74 Reductionism 109 Professional Codes of Ethics 77 The Time Dimension 111 Two Ethical Controversies 79 Cross-Sectional Studies 111 Trouble in the Tearoom 79 Longitudinal Studies 112 Observing Human Obedience 80 Approximating Longitudinal The Politics of Social Research 81 Studies 115 Objectivity and Ideology 82 Examples of Research Strategies 117 Politics with a Little “p” 86 How to Design a Research Project 117 Politics in Perspective 87 Getting Started 119 What Do You Think? Revisited 88 Conceptualization 120 Main Points 88 Choice of Research Method 120 Key Terms 89 Operationalization 120 Review Questions 89 Population and Sampling 121 Online Study Resources 90 Observations 121 Additional Readings 91 Data Processing 121 Analysis 122 Part Two Application 122 Research Design in Review 122 THE STRUCTURING OF INQUIRY: The Research Proposal 123 QUANTITATIVE Elements of a Research Proposal 124 AND QUALITATIVE 92 The Ethics of Research Design 125 What Do You Think? Revisited 125 Chapter 4 Main Points 126 Key Terms 127 Research Design 94 Review Questions 127 What Do You Think? 95 Online Study Resources 128 Introduction 96 Additional Readings 128 CONTENTS xi

Chapter 5 Chapter 6 Conceptualization, Indexes, Scales, and Operationalization, Typologies 168 and Measurement 130 What Do You Think? 169 What Do You Think? 131 Introduction 169 Introduction 131 Indexes versus Scales 170 Measuring Anything That Exists 132 Index Construction 173 Conceptions, Concepts, Item Selection 173 and Reality 133 Examination of Empirical Conceptions as Constructs 134 Relationships 174 Conceptualization 136 Index Scoring 179 Indicators and Dimensions 136 Handling Missing Data 180 The Interchangeability Index Validation 182 of Indicators 139 The Status of Women: An Illustration Real, Nominal, and Operational of Index Construction 185 Defi nitions 139 Scale Construction 186 Creating Conceptual Order 140 Bogardus Social Distance Scale 186 An Example of Conceptualization: Thurstone Scales 187 The Concept of Anomie 142 Likert Scaling 188 Defi nitions in Descriptive Semantic Differential 189 and Explanatory Studies 145 Guttman Scaling 190 Operationalization Choices 147 Typologies 193 Range of Variation 147 What Do You Think? Revisited 194 Variations between the Extremes 148 Main Points 195 A Note on Dimensions 148 Key Terms 196 Defi ning Variables and Attributes 149 Review Questions 196 Levels of Measurement 149 Online Study Resources 196 Single or Multiple Indicators 154 Additional Readings 196 Some Illustrations of Operationalization Choices 154 Chapter 7 Operationalization Goes On and On 155 The Logic of Sampling 198 Criteria of Measurement Quality 156 What Do You Think? 199 Precision and Accuracy 156 Introduction 199 Reliability 157 A Brief History of Sampling 200 Validity 160 President Alf Landon 201 Who Decides What’s Valid? 162 President Thomas E. Dewey 202 Tension between Reliability Two Types of Sampling Methods 203 and Validity 163 Nonprobability Sampling 203 What Do You Think? Revisited 163 Reliance on Available Subjects 203 The Ethics of Measurement 164 Purposive or Judgmental Sampling 204 Main Points 164 Snowball Sampling 205 Key Terms 165 Quota Sampling 205 Review Questions 165 Selecting Informants 206 Online Study Resources 166 The Theory and Logic of Probability Additional Readings 166 Sampling 207 xii CONTENTS

Conscious and Unconscious The Classical Experiment 246 Sampling Bias 208 Independent and Dependent Variables 247 Representativeness and Probability Pretesting and Posttesting 247 of Selection 210 Experimental and Control Groups 248 Random Selection 211 The Double-Blind Experiment 249 Probability Theory, Sampling Distributions, and Selecting Subjects 250 Estimates of Sample Error 212 Probability Sampling 250 Populations and Sampling Frames 221 Randomization 251 Types of Sampling Designs 223 Matching 251 Simple Random Sampling 224 Matching or Randomization? 252 Systematic Sampling 224 Variations on Experimental Design 253 Stratifi ed Sampling 227 Preexperimental Research Designs 253 Implicit Stratifi cation in Systematic Validity Issues in Experimental Research 254 Sampling 230 An Illustration of Experimentation 259 Illustration: Sampling University Students 230 Web-Based Experiments 262 Sample Modifi cation 230 “Natural” Experiments 263 Multistage Cluster Sampling 231 Strengths and Weaknesses of the Multistage Designs and Sampling Error 232 Experimental Method 264 Stratifi cation in Multistage Ethics and Experiments 265 Cluster Sampling 234 What Do You Think? Revisited 265 Probability Proportionate to Size Main Points 266 (PPS) Sampling 235 Key Terms 267 Disproportionate Sampling Review Questions 267 and Weighting 236 Online Study Resources 267 Probability Sampling in Review 238 Additional Readings 267 The Ethics of Sampling 238 What Do You Think? Revisited 238 Main Points 239 Chapter 9 Key Terms 240 Survey Research 268 Review Questions 240 What Do You Think? 269 Online Study Resources 240 Introduction 269 Additional Readings 241 Topics Appropriate for Survey Research 270 Guidelines for Asking Questions 271 Part Three Choose Appropriate Question Forms 272 MODES OF OBSERVATION: Make Items Clear 273 QUANTITATIVE Avoid Double-Barreled Questions 273 Respondents Must Be Competent AND QUALITATIVE 242 to Answer 274 Respondents Must Be Willing to Answer 274 Chapter 8 Questions Should Be Relevant 274 Short Items Are Best 276 Experiments 244 Avoid Negative Items 276 What Do You Think? 244 Avoid Biased Items and Terms 277 Introduction 244 Questionnaire Construction 278 Topics Appropriate to Experiments 246 General Questionnaire Format 278 CONTENTS xiii

Formats for Respondents 278 The Various Roles of the Observer 317 Contingency Questions 279 Relations to Subjects 319 Matrix Questions 280 Some Qualitative Field Research Ordering Items in a Questionnaire 281 Paradigms 321 Questionnaire Instructions 282 Naturalism 321 Pretesting the Questionnaire 283 Ethnomethodology 322 A Sample Questionnaire 283 Grounded Theory 324 Self-Administered Questionnaires 286 Case Studies and the Extended Mail Distribution and Return 286 Case Method 326 Monitoring Returns 287 Institutional Ethnography 328 Follow-up Mailings 288 Participatory Action Research 329 Acceptable Response Rates 288 Conducting Qualitative Field Research 333 A Case Study 289 Preparing for the Field 333 Interview Surveys 291 Qualitative Interviewing 335 The Role of the Survey Interviewer 291 Focus Groups 338 General Guidelines for Recording Observations 340 Survey Interviewing 292 Strengths and Weaknesses of Qualitative Coordination and Control 294 Field Research 342 Telephone Surveys 295 Validity 343 Positive and Negative Factors 295 Reliability 344 Computer-Assisted Telephone What Do You Think? Revisited 344 Interviewing (CATI) 297 Ethics in Qualitative Field Research 345 Response Rates in Interview Surveys 297 Main Points 345 New Technologies and Survey Research 299 Key Terms 346 Comparison of the Different Review Questions 346 Survey Methods 302 Online Study Resources 346 Strengths and Weaknesses Additional Readings 347 of Survey Research 303 Secondary Analysis 304 Ethics and Survey Research 307 Chapter 11 What Do You Think? Revisited 307 Unobtrusive Research 348 Main Points 308 What Do You Think? 349 Key Terms 309 Introduction 349 Review Questions 309 Content Analysis 350 Online Study Resources 310 Topics Appropriate to Content Analysis 350 Additional Readings 310 Sampling in Content Analysis 352 Coding in Content Analysis 355 Chapter 10 Illustrations of Content Analysis 359 Strengths and Weaknesses Qualitative Field Research 312 of Content Analysis 361 What Do You Think? 313 Analyzing Existing Statistics 362 Introduction 313 Durkheim’s Study of Suicide 362 Topics Appropriate to Field Research 314 The Consequences of Globalization 364 Special Considerations in Qualitative Units of Analysis 365 Field Research 317 Problems of Validity 365 xiv CONTENTS

Problems of Reliability 366 Part Four Sources of Existing Statistics 366 ANALYSIS OF DATA: Comparative and Historical Research 369 Examples of Comparative and Historical QUANTITATIVE Research 369 AND QUALITATIVE 412 Sources of Comparative and Historical Data 374 Chapter 13 Analytical Techniques 376 Ethics and Unobtrusive Measures 378 Qualitative Data Analysis 414 What Do You Think? Revisited 378 What Do You Think? 415 Main Points 379 Introduction 415 Key Terms 379 Linking Theory and Analysis 416 Review Questions 380 Discovering Patterns 416 Online Study Resources 380 Grounded Theory Method 417 Additional Readings 380 Semiotics 419 Conversation Analysis 421 Chapter 12 Qualitative Data Processing 421 Coding 422 Evaluation Research 382 Memoing 426 What Do You Think? 383 Concept Mapping 427 Introduction 383 Computer Programs for Qualitative Data 428 Topics Appropriate to Evaluation Research 385 QDA Programs 428 Formulating the Problem: Leviticus as Seen through NUD*IST 429 Issues of Measurement 386 Using NVivo to Understand Women Specifying Outcomes 387 Film Directors, by Sandrine Zerbib 433 Measuring Experimental Contexts 388 The Qualitative Analysis of Specifying Interventions 388 Quantitative Data 438 Specifying the Population 389 Ethics and Qualitative Data Analysis 438 New versus Existing Measures 389 What Do You Think? Revisited 439 Operationalizing Success/Failure 389 Main Points 439 Types of Evaluation Research Designs 390 Key Terms 440 Experimental Designs 390 Review Questions 440 Quasi-Experimental Designs 391 Online Study Resources 441 Qualitative Evaluations 395 Additional Readings 441 Logistical Problems 397 Use of Research Results 400 Chapter 14 Social Indicators Research 406 The Death Penalty and Deterrence 406 Quantitative Data Analysis 442 Computer Simulation 407 What Do You Think? 443 Ethics and Evaluation Research 408 Introduction 443 What Do You Think? Revisited 409 Quantiﬁ cation of Data 444 Main Points 410 Developing Code Categories 445 Key Terms 410 Codebook Construction 447 Review Questions 410 Data Entry 448 Online Study Resources 411 Univariate Analysis 448 Additional Readings 411 Distributions 449 CONTENTS xv

Central Tendency 450 Evaluation of Research Reports 474 Dispersion 453 Using the Internet Wisely 478 Continuous and Discrete Variables 454 Writing Social Research 486 Detail versus Manageability 454 Some Basic Considerations 486 Subgroup Comparisons 455 Organization of the Report 488 “Collapsing” Response Categories 456 Guidelines for Reporting Analyses 491 Handling Don’t Knows 457 Going Public 492 Numerical Descriptions What Do You Think? Revisited 492 in Qualitative Research 458 The Ethics of Reading and Writing Bivariate Analysis 459 Social Research 493 Percentaging a Table 460 Main Points 493 Constructing and Reading Key Terms 494 Bivariate Tables 462 Review Questions and Exercises 494 Introduction to Multivariate Analysis 463 Online Study Resources 495 Sociological Diagnostics 464 Additional Readings 495 Ethics and Quantitative Data Analysis 466 What Do You Think? Revisited 466 Appendixes Main Points 467 Key Terms 468 A Using the Library 498 Review Questions 468 B Random Numbers 506 Online Study Resources 469 C Distribution of Chi Square 508 Additional Readings 469 D Normal Curve Areas 510 E Estimated Sampling Error 511 Chapter 15

Reading and Writing Glossary 513 Social Research 470 References 525 What Do You Think? 471 Introduction 471 Index 535 Reading Social Research 471 Organizing a Review of the Literature 471 Journals versus Books 472 This page intentionally left blank PREFACE

The book in your hands has been about four de- plied research, some alternative teaching formats cades in the making. It began in the classroom, have called for a shorter book, and student eco- when I was asked to teach a seminar in survey re- nomics have argued for a paperback. While stan- search. Frustrated with the lack of good textbooks dard methods courses have continued using The on the subject, I began to dream up something I Practice of Social Research, I’ve been delighted to called “A Survey Research Cookbook and Other see that the fi rst three editions of Basics seem to Fables,” which was published in 1973 with a more have satisfi ed a substantial group of instructors as sober title: Survey Research Methods. well. The fi ne-tuning in this fourth edition is in- The book was an immediate success. However, tended to help Basics serve this group even better there were few courses limited to survey research. than before. Several instructors around the country asked if “the same guy” could write a more general methods book, and The Practice of Social Research appeared CHANGES IN THE FOURTH EDITION two years later. The latter book has become a fi x- ture in social research instruction, with the 11th A revision like this depends heavily on the input edition published in 2006. The offi cial Chinese edi- from students and faculty, who have been using tion was published in Beijing in 2000. earlier editions. Some of those suggestions re- Over the life of this fi rst book, successive revi- sulted in two new features that have been added sions have been based in large part on sugges- across all chapters: tions, comments, requests, and corrections from my colleagues around the country and, increas- • Increased ethics coverage: Although Chapter 3, ingly, around the world. Many also requested a The Ethics and Politics of Social Research, deals shorter book with a more applied orientation. with research ethics in depth, each of the other Whereas the third quarter of the twentieth cen- chapters has a section discussing some of the tury saw a greater emphasis on quantitative, pure ethical issues involved in specifi c aspects of research, the century ended with a renaissance of research. concern for applied sociological research (some- • Increased coverage of qualitative research: times called sociological practice) and also a re- Besides the new material on qualitative newed interest in qualitative research. The Basics research featured in Chapter 10, Qualitative of Social Research was fi rst published in 1999 in Field Research, and Chapter 13, Qualitative Data support of these trends. The fourth edition aims at Analysis, additional qualitative discussions increasing and improving that support. have been highlighted where appropriate, such The book can also be seen as a response to as the discussion of the uses of qualitative and changes in teaching methods and in student de- quantitative methods in the study of terror- mographics. In addition to the emphasis on ap- ism in Chapter 4, the new section on response xvii xviii PREFACE

rates in interview surveys in Chapter 9, and tion, explanation—plus pure versus applied the example of qualitative content analysis of research. gangsta rap and nihilism in Chapter 11. • Box on Ross Koppel’s research on medi- • A series of “In the Real World” boxes suggests cation errors, near the beginning of the ways in which the topics of a chapter apply chapter, to show the relevance of social within real research settings and also how they research. may be useful to students outside the realm of research—in their real lives. 2. Paradigms, Theory, and Research • Each of the chapters begins with a “What Do You • New section on critical race theory. Think?” box intended to present students with • Box on the power of political and religious a puzzle that they will be able to resolve as a paradigms. result of reading the chapter. In this edition, I’ve added some photos to accompany each of 3. The Ethics and Politics of Social Research these boxes. I hope you will ﬁ nd some of them • Discussion of the debate over “politicization amusing. of science.” • Many of the ﬁ gures and diagrams in the book • Discussion of how sloppiness in research is have been redrawn for both increased peda- an ethical violation. gogical value and visual appeal. • Introduction of the concept of participatory • “Issues and Insights” boxes throughout now action research. showcase fascinating insights by researchers • New section: “Politics and the Census.” and more in-depth discussion of key issues discussed in the chapters. Part Two: The Structuring of Inquiry

4. Research Design Chapter Changes • New section: “Nomothetic Causal Analysis In addition to those bookwide changes, here are and Hypothesis Testing.” some of the additional changes you’ll fi nd in spe- • Expanded discussion of the literature re- cifi c chapters of the book. Many of these changes view in the design of a study. were made in response to comments and requests • Discussion of the uses of qualitative and from students and faculty. quantitative methods in the study of terrorism. Part One: An Introduction to Inquiry • Example of a cohort analysis overturn- ing the conclusions of a more simplistic 1. Human Inquiry and Science analysis. • Data in the birthrate illustration have been • A new fi gure gives a graphic portrayal of the updated to strengthen this illustration of so- cohort study design. cial phenomena as distinct from individual phenomena. 5. Conceptualization, Operationalization, and • Examples have been generally increased Measurement and updated. • The chapter now begins with an example • New table on anti-gay prejudice and of measuring college satisfaction, which I education. expand on later in a box. • More applied/activist examples to expand • Discussion of Clifford Geertz’s “thick that theme of this book. description.” • New subsection introducing the purposes • Added British example of “fear of crime” of social research: exploration, descrip- index. PREFACE xix

• Example of Inuit words for snow to illus- • Expanded discussion of online surveys. trate ambiguity in concepts and words. • Comment on secondary analysis of qualita- • Box on the importance of conceptualization tive data. in political debates. • Box applying conceptualization and opera- 10. Qualitative Field Research tionalization to college satisfaction. • Introduction of the terms emic perspective and etic perspective. 6. Indexes, Scales, and Typologies • Introduction of the terms “virtual ethnogra- • Discussion of how minority group members phy” and “autoethnography.” view items in a Bogardus Social Distance • Updated Strauss and Corbin guidelines for Scale. grounded theory. • Example of indicators being independent of • Comment/quotation of challenge of control each other. in PAR • Example of typology involving racial hege- • New glossary term: emancipatory research. mony and colonialism. • Example of focus groups constituting in- depth interviews. 7. The Logic of Sampling • Example using the Internet to identify sub- • Updated presidential election polling data jects for in-depth interviews. now includes 2004 election. • New box: “Pencils and Photos in the Hands • Discussion of “cell phone only” problem. of Research Subjects.” • New example of snowball sampling in Australia. 11. Unobtrusive Research • Updated, more sophisticated example of • Example of study on the consequences of sampling error in the mass media. globalization. • A note that sampling error can be cal- • Historical research on the rise of Christianity. culated for several measures, not just • Qualitative content analysis of gangsta rap percentages. and nihilism. • Example of the multistage sampling of cities in Iran. 12. Evaluation Research • Discussion of controversy over weighting in • New glossary term: program evaluation/ political polls. outcome assessment. • New section on the Sabido methodology. Part Three: Modes of Observation • Example of monitoring studies in environmental research. 8. Experiments • Chapter recast in terms of its pedagogical Part Four: Analysis of Data value. • Introduction of the term ﬁ eld experiment. 13. Qualitative Data Analysis • Expanded discussion of forms of coding. 9. Survey Research • New section on computer programs for • Discussion of politicians’ reactions to politi- qualitative data analysis. cal polls. • New section on response rates in interview 14. Quantitative Data Analysis surveys. • Updated data on religious attendance. • Discussion of ways to improve response • Updated data on marijuana legalization, rates. age, and political orientations. xx PREFACE

• Updated data on religious attendance, age, have made sense to me in graphical form. and sex. Whereas my task here has been to translate • Mention of GapMinder software online. those mental pictures into words, I’ve also • Updated data on education, gender, and included some illustrations. Advances in com- income. puter graphics have helped me communicate to the Wadsworth artists what I see in my head 15. Reading and Writing Social Research and would like to share with students. I’m de- • New section on organizing a review of the lighted with the new graphics in this edition. literature. • Boxed examples and discussions: Stu- • New section on evaluating content analysis. dents tell me they like the boxed materials that • New section on evaluating comparative and highlight particular ideas and studies as well as historical research. vary the format of the book. In this edition, I’ve • Updated and improved illustrations of web added “Issues and Insights” boxes to explore searches. key topics, and “In the Real World” boxes to • Introduction of SourceWatch to help help students see how the ideas they’re read- students judge trustworthiness of web ing about apply to real research projects, as sources. well as to their lives. • New section, “Going Public,” about student • Running Glossary: There is a running glos- presentations/publications. sary throughout the text. Key terms are highlighted in the text, and the defi nition for each term is listed at the bottom of the page where Pedagogical Features it appears. This will help students learn the Although students and instructors both have told defi nitions of these terms and locate them in me that the past editions of this book were effec- each chapter to review them in context. tive tools for learning research methods, I have • Main Points: At the end of each chapter, a seen this edition as an opportunity to review the concise list of main points provides both a brief book from a pedagogical standpoint—fi ne-tuning chapter summary and a useful review. The some elements, adding others. Here’s the resulting main points let students know exactly what package for the fourth edition. ideas they should focus on in each chapter. • Key Terms: A list of key terms follows the • Chapter Overview: Each chapter is preceded main points. These lists reinforce the students’ by a pithy focus paragraph that highlights the acquisition of necessary vocabulary. The new principal content of the chapter. vocabulary in these lists is defi ned in context • Chapter Introduction: Each chapter opens in the chapters. The terms are boldfaced in the with an introduction that lays out the main text, defi ned in the running glossary that ap- ideas in that chapter and, importantly, relates pears at the bottom of the page throughout the them to the content of other chapters in the text, and included in the glossary at the back of book. the book. • Clear and provocative examples: Students • Review Questions and Exercises: This often tell me that the examples—real and review aid allows students to test their under- hypothetical—have helped them grasp diffi cult standing of the chapter concepts and apply and/or abstract ideas, and this edition has what they have learned. many new examples as well as some that have • Resources on the Internet: As I mentioned proven particularly valuable in earlier editions. earlier, each chapter ends with this new sec- • Graphics: From the fi rst time I took a course tion. This edition continues previous editions’ in research methods, most of the key concepts movement into cyberspace. PREFACE xxi

• Additional Readings: In this section, I teaching. Students tell me they use it heavily as a include an annotated list of references that review of the text, and I count the exercises as half students can turn to if they would like to learn their grade in the course. more on the topics discussed in the chapter. In this edition, Ted and I have once again sorted • Appendixes: As in previous editions, a set through the exercises and added new ones we’ve of appendixes provides students with some created in our own teaching or heard about from research tools, such as a guide to the library, a colleagues. These include matching, multiple- table of random numbers, and so forth. There choice, and open-ended discussion questions for is an SPSS primer on the book’s website along each chapter, along with four to six exercises de- with primers for NVivo and Qualrus. signed to reinforce the material learned in the text • Clear and accessible writing: This is per- with examples from everyday life. Also included haps the most important “pedagogical aid” of are the answers to the matching and multiple- all. I know that all authors strive to write texts choice review questions, as well as a General that are clear and accessible, and I take some Social Survey appendix, plus chapter objectives, pride in the fact that this “feature” of the book chapter summaries, and key terms. has been one of its most highly praised attributes through nine previous editions. It is the SPSS Student Version CD-ROM 14.0 (Windows one thing students write most often about. For only) Based on the professional version of one the fourth edition, the editors and I have taken of the world’s leading desktop statistical software special care to reexamine literally every line packages, SPSS Student Version for Windows pro- in the book—pruning, polishing, embellishing, vides real-world software for students to do socio- and occasionally restructuring for a maximally logical data analysis, such as interpreting the GSS “reader-friendly” text. Whether you’re new to data sets found on the companion website. this book or intimately familiar with previous editions, I invite you to open to any chapter Learning How to Use SPSS: with Exercises This and evaluate the writing for yourself. handy guide is coordinated with the text and SPSS CD-ROM 14.0 to help students learn basic navi- SUPPLEMENTS gation in SPSS, including how to enter their own data; create, save, and retrieve fi les; produce and interpret data summaries; and much more. Also The Basics of Social Research, Fourth Edition, is ac- included are SPSS practice exercises correlated companied by a wide array of supplements pre- with each chapter. The guides comes free when pared for both the instructor and student to create bundled with the text. the best learning environment inside as well as outside the classroom. All the continuing supple- GSS Data Disk Over the years, we have sought to ments have been thoroughly revised and updated, provide up-to-date personal computer support for and several are new to this edition. I invite you to students and instructors. Because there are now examine and take full advantage of the teaching many excellent programs for analyzing data, we and learning tools available to you. have provided data to be used with them. With this edition we have updated the data disk to include For the Student the GSS data. Guided Activities for Babbie’s The Basics of So- Experiencing Social Research: An Introduction cial Research, Fourth Edition. The student study Using MicroCase, Second Edition This supple- guide and workbook Ted Wagenaar and I have mentary workbook and statistical package, written prepared continues to be a mainstay of my own by David J. Ayers of Grove City College, includes xxii PREFACE short discussions, quizzes, and computerized ex- exercises, Internet exercises, and possible study ercises in which students will learn and apply key guide answers. In addition, for each chapter of the methodological concepts and skills by analyzing, text, the Test Bank has multiple-choice, true-false, and in some cases collecting and building, simple short-answer, and essay questions, with answers data fi les for real sociological data. Designed to ac- and page references. All questions are labeled as company The Basics of Social Research, the work- new, modifi ed, or pickup so instructors know if book and statistical package take a step-by-step the question is new to this edition of the test bank, approach to show students how to do real socio- modifi ed but picked up from the previous edition of logical research, using the same data and tech- the test bank, or picked up straight from the previ- niques used by professional researchers, to rein- ous edition of the test bank. force, build on, and complement course materials. ExamView Computerized Testing for Macintosh Readings in Social Research, Third Edition The and Windows. This allows instructors to create, concepts and methodologies of social research deliver, and customize printed and online tests come to life in this interesting collection of articles and study guides. ExamView includes a Quick Test specifi cally designed to accompany The Basics of Wizard and an Online Test Wizard to guide instruc- Social Research. Diane Kholos Wysocki includes an tors step by step through the process of creating interdisciplinary range of readings from the fi elds tests. The test appears on screen exactly as it will of psychology, sociology, social work, criminal jus- print or display online. Using ExamView’s com- tice, and political science. The articles focus on the plete word-processing capabilities, instructors can important methods and concepts typically covered enter an unlimited number of new questions or in the social research course and provide an illus- edit questions included with ExamView. trative advantage. Multimedia Manager with Instructor’s Re- Researching Sociology on the Internet, Third sources: A Microsoft ® PowerPoint ® Tool This Edition This guide is designed to help sociology one-stop lecture and class preparation tool makes students do research on the Internet. Part 1 con- it easy to assemble, edit, publish, and present tains general information necessary to get started custom lectures for your course, using Microsoft and answers questions about security, the type of PowerPoint. The Multimedia Manager brings to- sociology material available on the Internet, the in- gether art (fi gures, tables, maps) from this text, formation that is reliable and the sites that are not, preassembled Microsoft PowerPoint lecture slides, the best ways to fi nd research, and the best links to and sociology-related videos, along with video take students where they want to go. Part 2 looks and animations from the web or your own ma- at each main topic in sociology and refers students terials—culminating in a powerful, personalized, to sites where they can obtain the most enlighten- media-enhanced presentation. The CD-ROM also ing research and information. contains a full Instructor’s Manual, Test Bank, and other instructor resources.

For the Instructor JoinIn™ on TurningPoint® JoinIn on Turning- Instructor’s Manual with Test Bank. Written by Point turns your lecture into an interactive expe- Margaret Platt Jendrek, this supplement offers the rience for your students. Using Microsoft Power- instructor brief chapter outlines, detailed chap- Point, you can poll students on key issues, check ter outlines, behavioral objectives, teaching sug- their comprehension of diffi cult concepts, and be- gestions and resources, InfoTrac College Edition come “critical consumers” by watching our exclu- PREFACE xxiii sive ABC video clips and answering related meth- sources for instructors and students. For instruc- odological questions about them. tors, the site offers a password-protected instructor’s manual, Microsoft PowerPoint presentation Internet-Based Supplements slides, and more. For students, there is a multitude of text-specifi c study aids including the following: ThomsonNOW™ Empowers students with the • Tutorial practice quizzing that can be scored fi rst assessment-centered student tutorial system and emailed to the instructor for Social Research/Research Methods. Seamlessly • Web links tied to the new edition, this interactive web-based • InfoTrac College Edition exercises learning tool helps students gauge their unique • Flashcards study needs with a “pretest” for each chapter to as- • GSS data sets sess their understanding of the material. They are • Data Analysis Primers then given a personalized study plan that offers in- • MicroCase Online data exercises teractive, visual and audio resources to help them • Crossword puzzles master the material. They can check their progress with an interactive posttest, as well. Thomson InSite for Writing and Research™ with WebTutor™ Toolbox on Blackboard and WebCT. Turnitin® originality checker. InSite features a This web-based software for students and instruc- full suite of writing, peer review, online grading, tors takes a course beyond the classroom to an and e-portfolio applications. It is an all-in-one tool anywhere, anytime environment. Students gain that helps instructors manage the fl ow of papers access to to the rich content from our book com- electronically and allows students to submit pa- panion websites. Available for WebCT and Black- pers and peer reviews online. Also included in the board only. suite is Turnitin, an originality checker that offers a simple solution for instructors who want a strong ® InfoTrac College Edition with InfoMarks™ deterrent against plagiarism, as well as encour- Available as a free option with newly purchased agement for students to employ proper research texts, InfoTrac College Edition gives instructors and techniques. Access is available for packaging with students four months of free access to an exten- each copy of this book. For more information, visit sive online database of reliable, full-length articles http://insite.thomson.com. (not just abstracts) from thousands of scholarly and popular publications going back as far as 22 years. Among the journals available are American ACKNOWLEDGMENTS Journal of Sociology, Social Forces, Social Research, and Sociology. InfoTrac College Edition now also comes with InfoMarks, a tool that allows you to It would be impossible to acknowledge adequately save your search parameters, as well as save links all the people who have infl uenced this book. to specifi c articles. (Available to North American My earlier methods text, Survey Research Methods, college and university students only; journals are was dedicated to Samuel Stouffer, Paul Lazarsfeld, subject to change.) and Charles Glock. I again acknowledge my debt to them. Companion Website for The Basics of Social Many colleagues helped me through the eleven Research, Fourth Edition. (http://sociology editions of The Practice of Social Research and the .wadsworth.com/babbie_basics4e). The book’s fi rst three editions of The Basics of Social Research. companion website includes chapter-specifi c re- At this point, I particularly want to thank the in- xxiv PREFACE structors who reviewed the manuscript of this edi- in books like this. Since 1973, I’ve worked with tion of Basics and made helpful suggestions: many sociology editors at Wadsworth, which has involved the kinds of adjustments you might need Melanie Arthur, Portland State University to make in as many successive marriages. Happily, James W. Cassell, Henderson State University this edition of the book has greatly profi ted from Leslie Hossfeld, University of North my partnership with Chris Caldeira and Sherry Carolina–Wilmington Symington. Perhaps you have to be a textbook Rebecca Utz, University of Utah author to appreciate how much difference editors Gary Wyatt, Emporia State University make in the writing and publishing experience, but I would also like to thank those who reviewed I want to report that I have been blessed with great earlier editions: partners. In my experience, copy editors are the invisible C. Neil Bull, University of Missouri–Kansas City heroes of publishing, and it has been my good for- Jeffrey A. Burr, University of Massachusetts– tune and pleasure to have worked with one of the Boston very best, Molly Roth, for several years and books. Karen Campbell, Vanderbilt University Among her many gifts, Molly has the uncanny abil- Douglas Forbes, University of ity to hear what I am trying to say and fi nd ways to Wisconsin–Marshfi eld help others hear it. Molly’s partnership with Greg Susan Haire, University of Georgia Hubit at Bookworks is something special in the Albert Hunter, Northwestern University publishing world, and I would not want to do a Robert Kleidman, Cleveland State University major text without them. Ross Koppel, University of Pennsylvania I have dedicated this book to my granddaugh- Susan E. Marshall, University of Texas–Austin ter, Evelyn Fay Babbie, born during the revision of Enrique Pumar, William Patterson University the second edition of the book, and my grandson, William G. Staples, University of Kansas Henry Robert Babbie, born during the revision of Stephen F. Steele, Anne Arundel Community the third edition. They continued to add joy to my College life during the revision of the fourth edition, and Thankam Sunil, University of Texas I am committed to their growing up in a more at San Antonio humane and just world than the one they were Yvonne Vissing, Salem State College born into. Over the years, I have become more and more impressed by the important role played by editors PartPart OneOne

AN INTRODUCTION TO INQUIRY

1 Human Inquiry and Science

2 Paradigms, Theory, and Research

3 The Ethics and Politics of Social Research Bonnie Kamin/PhotoEdit cience is a familiar word used by everyone. determining what scientists look for and how they S Yet images of science differ greatly. For some, interpret what they see. science is mathematics; for others, it’s white coats Whereas most of this book concerns the art and laboratories. It’s often confused with technol- and science of doing social research, Chapter 3 in- ogy or equated with tough high school or college troduces some of the political and ethical consid- courses. erations that affect social research. We’ll see the Science is, of course, none of these things per se. ethical norms that social researchers follow when Specifying exactly what science is, however, poses they design and implement research. We’ll also problems. Scientists, in fact, disagree on the proper see how social contexts affect social research. defi nition. For the purposes of this book, we’ll look Overall, Part 1 constructs a backdrop against at science as a method of inquiry—a way of learn- which to view the more specifi c aspects of research ing and knowing things about the world around design and execution. By the time you complete us. Contrasted with other ways of doing this, sci- Part 1, you should be ready to look at some of the ence has some special characteristics, which we’ll more concrete aspects of social research. examine in these opening chapters. Benjamin Spock, the renowned author and pediatrician, began his books on child care by assuring new parents that they already knew more about child care than they thought they did. I want to begin this book on a similar note. Before you’ve read very far, you’ll see that you already know a great deal about the practice of social scientifi c research. In fact, you’ve been conducting scientifi c research all your life. From that perspective, this book aims at helping you sharpen skills you already have and perhaps showing you some tricks that may not have occurred to you. By examining the fundamental characteristics and issues that make science different from other ways of knowing things, Part 1 lays the groundwork for the rest of the book. In Chapter 1, we’ll begin with a look at native human inquiry, the sort of thing you’ve been doing all your life. In the course of that examination, we’ll see some of the ways people go astray in trying to understand the world around them, and I’ll summarize the primary characteristics of scientifi c inquiry that guard against those errors. Chapter 2 deals with social scientifi c paradigms and theories, as well as the links between theory and research. We’ll look at some of the theoretical paradigms that shape the nature of inquiry, largely

2 1 HUMAN INQUIRY AND SCIENCE Bonnie Kamin/PhotoEdit

What You’ll Learn in This Chapter

We’ll examine how people learn about their world and look at the mistakes they make along the way. We’ll also begin to see what makes science different from other ways of knowing. In this chapter . . . WHAT DO YOU THINK?

Introduction The decision to have Looking for Reality a baby is deeply per- Ordinary Human Inquiry sonal. No one is in Tradition Image not available due to copyright restrictions charge of who will Authority have babies in the Errors in Inquiry and Some Solutions United States in any What’s Really Real? given year, or of The Foundations of Social Science how many will be Theory, Not Philosophy or Belief born. Whereas you must get a license to Social Regularities marry or go ﬁ shing, you do not need a li- Aggregates, Not Individuals cense to have a baby. Many couples delay A Variable Language pregnancy, some pregnancies happen by ac- The Purposes of Social Research cident, and some pregnancies are planned. The Ethics of Human Inquiry Given all these uncertainties and idiosyncra- sies, how can baby food and diaper manu- Some Dialectics of Social Research facturers know how much to produce from Idiographic and Nomothetic Explanation year to year? By the end of this chapter, you Inductive and Deductive Theory should be able to answer this question. Quantitative and Qualitative Data Pure and Applied Research See the “What Do You Think? Revisited” box toward the end of the chapter.

INTRODUCTION speak Japanese in Japan, and that made sense to you, so you didn’t question it. Perhaps your physics This book is about knowing things—not so much or astronomy instructor told you it was cold on the what we know as how we know it. Let’s start by ex- dark side of the moon, or maybe you read it on the amining a few things you probably know already. NASA website. You know the world is round. You probably also Some of the things you know seem obvious know it’s cold on the dark side of the moon, and to you. If I asked you how you know the world you know people speak Japanese in Japan. You is round, you’d probably say, “Everybody knows know that vitamin C can prevent colds and that that.” There are a lot of things everybody knows. unprotected sex can result in AIDS. Of course, at one time, everyone “knew” the world How do you know? If you think for a minute, was ﬂ at. you’ll see you know these things because some- Most of what you know is a matter of agree- body told them to you, and you believed them. You ment and belief. Little of it is based on personal may have read in National Geographic that people experience and discovery. A big part of growing

4 INTRODUCTION 5 up in any society, in fact, is the process of learning to accept what everybody around you “knows” is so. If you don’t know those same things, you can’t really be a part of the group. If you were to question seriously that the world is really round, you’d quickly ﬁ nd yourself set apart from other people. You might be sent to live in a hospital with others who ask questions like that. So, most of what you know is a matter of believing what you’ve been told. Understand that there’s nothing wrong with you in that respect. That’s simply the way human societies are structured. The basis of knowledge is agreement. Be- cause you can’t learn through personal experience and discovery alone all you need to know, things are set up so you can simply believe what others tell you. You know some things through tradition, others from “experts.” I’m not saying you shouldn’t question this received knowledge; I’m just drawing your attention to the way you and society normally get along regarding what’s so. There are other ways of knowing things, however. In contrast to knowing things through agreement, you can know them through direct expe-

rience—through observation. If you dive into a Earl Babbie glacial stream fl owing through the Canadian Rock- We learn some things by experience, others by ies, you don’t need anyone to tell you it’s cold. agreement. When your experience confl icts with what everyone else knows, though, there’s a good chance you’ll surrender your experience in favor of the The point of the story is that both of your feel- agreement. For example, imagine you’ve come to ings about the appetizer were quite real. Your ini- a party at my house. It’s a high-class affair, and the tial liking for them was certainly real, but so was drinks and food are excellent. In particular, you’re the feeling you had when you found out what you’d taken by one of the appetizers I bring around on a been eating. It should be evident, however, that the tray: a breaded, deep-fried tidbit that’s especially disgust you felt was strictly a product of the agree- zesty. You have a couple—they’re so delicious! You ments you have with those around you that worms have more. Soon you’re subtly moving around the aren’t fi t to eat. That’s an agreement you began the room to be wherever I am when I arrive with a tray fi rst time your parents found you sitting in a pile of of these nibblies. dirt with half of a wriggling worm dangling from Finally, you can contain yourself no longer. your lips. When they pried your mouth open and “What are they?” you ask. I let you in on the secret: reached down your throat for the other half of the “You’ve been eating breaded, deep-fried worms!” worm, you learned that worms are not acceptable Your response is dramatic: Your stomach rebels, food in our society. and you promptly throw up all over the living room Aside from these agreements, what’s wrong rug. What a terrible thing to serve guests! with worms? They’re probably high in protein and 6 CHAPTER 1 HUMAN INQUIRY AND SCIENCE low in calories. Bite-sized and easily packaged, out about human social life. You’ll see that some they’re a distributor’s dream. They are also a deli- of the methods coincide with the traditional image cacy for some people who live in societies that lack of science but others have been specially geared to our agreement that worms are disgusting. Some sociological concerns. people might love the worms but be turned off by In the rest of this chapter, we’ll look at inquiry the deep-fried breading. as an activity. We’ll begin by examining inquiry as Here’s a question to consider: “Are worms re- a natural human activity, something you and I have ally good or really bad to eat?” And here’s a more engaged in every day of our lives. Next, we’ll look interesting question: “How could you know which at some kinds of errors we make in normal inquiry, was really so?” This book is about answering the and we’ll conclude by examining what makes sci- second question. ence different. We’ll see some of the ways science guards against common human errors in inquiry. LOOKING FOR REALITY The box “Social Research Making a Difference” gives an example of controlled social research challenging what “everybody knows.” Reality is a tricky business. You’ve probably long suspected that some of the things you “know” may Ordinary Human Inquiry not be true, but how can you really know what’s real? People have grappled with this question for Practically all people exhibit a desire to predict thousands of years. their future circumstances. We seem quite will- One answer that has arisen out of that grap- ing, moreover, to undertake this task using causal pling is science, which offers an approach to both and probabilistic reasoning. First, we generally rec- agreement reality and experiential reality. Scien- ognize that future circumstances are somehow tists have certain criteria that must be met before caused or conditioned by present ones. We learn they’ll accept the reality of something they haven’t that swimming beyond the reef may bring an un- personally experienced. In general, an assertion happy encounter with a shark. As students we learn must have both logical and empirical support: It that studying hard will result in better grades. Sec- must make sense, and it must not contradict ac- ond, we also learn that such patterns of cause and tual observation. Why do earthbound scientists ac- effect are probabilistic in nature: The effects occur cept the assertion that it’s cold on the dark side of more often when the causes occur than when the the moon? First, it makes sense, because the sur- causes are absent—but not always. Thus, students face heat of the moon comes from the sun’s rays. learn that studying hard produces good grades in Second, the scientifi c measurements made on the most instances, but not every time. We recognize moon’s dark side confi rm the expectation. So, sci- the danger of swimming beyond the reef, with- entists accept the reality of things they don’t per- out believing that every such swim will be fatal. sonally experience—they accept an agreement re- As we’ll see throughout the book, science ality—but they have special standards for doing so. makes these concepts of causality and probability More to the point of this book, however, sci- more explicit and provides techniques for dealing ence offers a special approach to the discovery of with them more rigorously than does casual hu- reality through personal experience, that is, to the man inquiry. It sharpens the skills we already have business of inquiry. Epistemology is the science of by making us more conscious, rigorous, and ex- knowing; methodology (a subfi eld of epistemology) plicit in our inquiries. might be called the science of fi nding out. This In looking at ordinary human inquiry, we need book is an examination and presentation of social to distinguish between prediction and understand- science methodology, or how social scientists fi nd ing. Often, we can make predictions without un- LOOKING FOR REALITY 7

ISSUES AND INSIGHTS SOCIAL RESEARCH MAKING A DIFFERENCE

Medication errors in hospitals kill or injure technology. Their conclusion: CPOE was not about 770,000 patients each year, and the newly nearly as effective as claimed; it did not prevent developed Computerized Physician Order Entry errors in medication. (CPOE) systems have been widely acclaimed as As you can imagine, those manufacturing the solution to this enormous problem experi- and selling the equipment were not thrilled by enced in the traditional system of handwritten the research, and it has generated an ongoing prescriptions. discussion within the health care community. At Medical science research has generally sup- last count, the study had been cited over 20,000 ported the new technology, but an article in the times in other articles, and Koppel has become March 9, 2005, issue of the Journal of the Ameri- a sought-after expert in this regard. can Medical Association (JAMA) sent a shock wave through the medical community. The so- Source: Kathryn Goldman Schuyler, “ Medical Errors: ciologist Ross Koppel and colleagues used sev- Sociological Research Makes News,” Sociological Practice Newsletter (American Sociological Associa- eral of the research techniques you’ll be learning tion, Section on Sociological Practice), Winter 2006, in this book to test the effectiveness of the new p. 1.

derstanding—perhaps you can predict rain when part comes from the agreed-on knowledge that your trick knee aches. And often, even if we don’t others give us. This agreement reality both assists understand why, we’re willing to act on the basis and hinders our attempts to fi nd out for ourselves. of a demonstrated predictive ability. The racetrack To see how, consider two important sources of our buff who fi nds that the third-ranked horse in the secondhand knowledge—tradition and authority. third race of the day always wins will probably keep betting without knowing, or caring, why it Tradition works out that way. Whatever the primitive drives or instincts that Each of us inherits a culture made up, in part, of motivate human beings, satisfying them depends fi rmly accepted knowledge about the workings heavily on the ability to predict future circum- of the world. We may learn from others that eat- stances. However, the attempt to predict is often ing too much candy will decay our teeth, that the placed in a context of knowledge and understand- circumference of a circle is approximately twenty- ing. If we can understand why things are related to two–sevenths of its diameter, or that masturbation one another, why certain regular patterns occur, will blind us. We may test a few of these “truths” on we can predict even better than if we simply ob- our own, but we simply accept the great majority serve and remember those patterns. Thus, human of them, the things that “everybody knows.” inquiry aims at answering both “what” and “why” Tradition, in this sense of the term, offers some questions, and we pursue these goals by observing clear advantages to human inquiry. By accepting and fi guring out. what everybody knows, we avoid the overwhelm- As I suggested earlier, our attempts to learn ing task of starting from scratch in our search for about the world are only partly linked to direct, per- regularities and understanding. Knowledge is cu- sonal inquiry or experience. Another, much larger, mulative, and an inherited body of knowledge is 8 CHAPTER 1 HUMAN INQUIRY AND SCIENCE the jumping-off point for developing more of it. to start at the wrong point and push us off in the We often speak of “standing on the shoulders of wrong direction. giants,” that is, of previous generations. At the same time, tradition may be detrimental Errors in Inquiry and Some Solutions to human inquiry. If we seek a fresh understanding of something everybody already understands and Quite aside from the potential dangers of tradition has always understood, we may be marked as fools and authority, we often stumble and fall when we for our efforts. More to the point, however, most of set out to learn for ourselves. Let’s look at some us rarely even think of seeking a different under- of the common errors we make in our casual in- standing of something we all “know” to be true. quiries and the ways science guards against those errors.

Authority Inaccurate Observations Quite frequently, we make mistakes in our observations. For example, Despite the power of tradition, new knowledge ap- what was your methodology instructor wearing on pears every day. Aside from our personal inquiries, the fi rst day of class? If you have to guess, that’s we benefi t throughout life from new discoveries because most of our daily observations are casual and understandings produced by others. Often, and semiconscious. That’s why we often disagree acceptance of these new acquisitions depends on about what really happened. the status of the discoverer. You’re more likely to In contrast to casual human inquiry, scientifi c believe the epidemiologist who declares that the observation is a conscious activity. Simply making common cold can be transmitted through kissing, observation more deliberate can reduce error. If for example, than to believe your uncle Pete. you had to guess what your instructor was wear- Like tradition, authority can both assist and ing the fi rst day of class, you’d probably make a hinder human inquiry. We do well to trust in the mistake. If you had gone to the fi rst class meeting judgment of the person who has special training, with a conscious plan to observe and record what expertise, and credentials in a given matter, espe- your instructor was wearing, however, you’d likely cially in the face of controversy. At the same time, be more accurate. (You might also need a hobby.) inquiry can be greatly hindered by the legitimate In many cases, both simple and complex mea- authority who errs within his or her own special surement devices help guard against inaccurate province. Biologists, after all, do make mistakes in observations. Moreover, they add a degree of pre- the fi eld of biology. cision well beyond the capacity of the unassisted Inquiry is also hindered when we depend on the human senses. Suppose, for example, that you had authority of experts speaking outside their realm taken color photographs of your instructor that day. of expertise. For example, consider the political (See earlier comment about needing a hobby.) or religious leader with no biochemical expertise who declares that marijuana is a dangerous drug. Overgeneralization When we look for patterns The advertising industry plays heavily on this mis- among the specifi c things we observe around us, use of authority by, for example, having popular we often assume that a few similar events are athletes discuss the nutritional value of breakfast evidence of a general pattern. That is, we tend to cereals or movie actors evaluate the performance overgeneralize on the basis of limited observa- of automobiles. tions. This can misdirect or impede inquiry. Both tradition and authority, then, are double- Imagine that you’re a reporter covering an edged swords in the search for knowledge about animal-rights demonstration. You have to turn in the world. Simply put, they provide us with a start- your story in just two hours. Rushing to the scene, ing point for our own inquiry, but they can lead us you start interviewing people, asking them why LOOKING FOR REALITY 9 they’re demonstrating. If the fi rst two demonstra- number of observations on that question in a re- tors you interview give you essentially the same search project. We might select a thousand people reason, you may simply assume that the other to be interviewed on the issue. Alternately, when 3,000 would agree. Unfortunately, when your story making direct observations of an event, such as appears, your editor gets scores of letters from an animal-rights demonstration, social scientists protesters who were there for an entirely different make a special effort to fi nd “deviant cases”—those reason. who do not fi t into the general pattern. Scientists guard against overgeneralization by seeking a suffi ciently large sample of observations. Illogical Reasoning There are other ways of han- The replication of inquiry provides another safe- dling observations that contradict our conclusions guard. Basically, this means repeating a study and about the way things are in daily life. Surely one of checking to see if the same results occur each time. the most remarkable creations of the human mind Then, as a further test, the study may be repeated is “the exception that proves the rule.” This idea under slightly varied conditions. makes no sense at all. An exception can draw attention to a rule or to a supposed rule, but in no Selective Observation One danger of overgener- system of logic can it prove the rule it contradicts. alization is that it may lead to selective observation. Yet we often use this pithy saying to brush away Once you have concluded that a particular pattern contradictions with a simple stroke of illogic. exists and have developed a general understand- What statisticians have called the gambler’s fal- ing of why it does, you’ll tend to focus on future lacy is another illustration of illogic in day-to-day events and situations that fi t the pattern, and you’ll reasoning. A consistent run of either good or bad ignore those that don’t. Racial and ethnic preju- luck is presumed to foreshadow its opposite. An dices depend heavily on selective observation for evening of bad luck at poker may kindle the be- their persistence. lief that a winning hand is just around the corner; In another example, here’s how Lewis Hill re- many a poker player has stayed in a game much calls growing up in rural Vermont: too long because of that mistaken belief. Con- versely, an extended period of good weather may Haying began right after the Fourth of July. lead you to worry that it is certain to rain on the The farmers in our neighborhood believed that weekend picnic. anyone who started earlier was sure to suffer Although all of us sometimes fall into embar- all the storms of late June in addition to those rassingly illogical reasoning in daily life, scientists following the holiday which the oldtimers said avoid this pitfall by using systems of logic con- were caused by all the noise and smoke of sciously and explicitly. Chapter 2 will examine the gunpowder burning. My mother told me that logic of science in more depth. For now, it’s enough my grandfather and other Civil War veterans to note that logical reasoning is a conscious activ- claimed it always rained hard after a big battle. ity for scientists, who have colleagues around to Things didn’t always work out the way the older keep them honest. residents promised, of course, but everyone re- These, then, are a few of the ways we go astray membered only the times they did. — (2000:35) in our attempts to know and understand the world, Sometimes a research design will specify in ad- and some of the ways that science protects inquiry vance the number and kind of observations to be from these pitfalls. Accurately observing and un- made, as a basis for reaching a conclusion. If you and I wanted to learn whether women were more likely than men to support the legality of abor- replication The duplication of an experiment to expose tion, we’d commit ourselves to making a specifi ed or reduce error. 10 CHAPTER 1 HUMAN INQUIRY AND SCIENCE derstanding reality is not an obvious or trivial matter. Indeed, it’s more complicated than I’ve sug- IN THE REAL WORLD gested. (See the box “Applying Scientifi c Inquiry to APPLYING SCIENTIFIC Daily Life” for more on this topic.) INQUIRY TO DAILY LIFE

What’s Really Real? As we proceed in this examination of social science research methods, I’ll tend to talk as Philosophers sometimes use the phrase naive real- though I were training you for a career as a ism to describe the way most of us operate in our researcher. However, I realize you may not daily lives. When you sit at a table to write, you be planning on that. probably don’t spend a lot of time thinking about As such, I want to point out ways in whether the table is really made up of atoms, which which the topics of this book might relate in turn are mostly empty space. When you step into to the world at large. What you learn here the street and see a city bus hurtling down on you, may apply to your life and career in ways it’s not the best time to refl ect on methods for test- you might not realize. Most of the “In the ing whether the bus really exists. We all live with Real World” boxes will point to the everyday a view that what’s real is pretty obvious—and that uses of something you’ve learned in a par- view usually gets us through the day. ticular chapter. In this fi rst box, however, I Even so, I hope you can see that the nature of want to make the general point that even if “reality” is perhaps more complex than we tend you do not end up doing social science re- to assume. As a philosophical backdrop for the search, you’ll be a consumer of social sci- discussions to follow, let’s look at what are some- ence research throughout your life. You’ll be times called premodern, modern, and postmodern hearing about which political candidate is views of reality (W. Anderson 1990). leading another, what public opinion is on some hot issue, or which laundry detergent The Premodern View This view of reality has gets clothes cleaner. If you choose a career guided most of human history. Our early ancestors in law, you might have to deal with studies of assumed that they saw things as they really were. “community standards.” As a social worker, In fact, this assumption was so fundamental that you might need to assess research compar- they didn’t even see it as an assumption. No cave- ing different treatment modalities. mom said to her cavekid, “Our tribe makes an as- So, even if you decide not to produce so- sumption that evil spirits reside in the Old Twisted cial research yourself, you can profi t from Tree.” No, she said, “STAY OUT OF THAT TREE OR becoming an informed consumer of it. YOU’LL TURN INTO A TOAD!” As humans evolved and became aware of their diversity, they came to recognize that others did not always share their views of things. Thus, they The Modern View What philosophers call the may have discovered that another tribe didn’t be- modern view accepts such diversity as legitimate, a lieve the tree was wicked; in fact, the second tribe philosophical “different strokes for different folks.” believed that the tree spirits were holy and benefi - As a modern thinker you would say, “I regard the cial. The discovery of this diversity led members of spirits in the tree as evil, but I know others regard the fi rst tribe to conclude that “some tribes I could them as good. Neither of us is right or wrong. There name are pretty stupid.” For them, the tree was still are simply spirits in the tree. They are neither good wicked, and they expected some misguided people nor evil, but different people have different ideas to be moving to Toad City. about them.” LOOKING FOR REALITY 11

looks like. As you’re reading these words, it probably looks like Figure 1-1a. But does Figure 1-1a represent the way your book “really” looks? Or does it merely represent what the book looks like from your current point of a. b. view? Surely, Figures 1-1b, c, and d are equally valid representations. But these views of the book differ so much from one another. Which is the “reality”? As this example illustrates, there is no answer to the question, “What does the book really look like?” All we can offer is the different ways it looks from different points of view. Thus, according to c. d. the postmodern view, there is no “book,” only various images of it from different points of view. And FIGURE 1-1 A Book. All of these are the same all the different images are equally “true.” book, but it looks different when viewed from dif- Now let’s apply this logic to a social situation. ferent locations, perspectives, or “points of view.” Imagine a husband and wife arguing. When she looks over at her quarreling husband, Figure 1-2 is what the wife sees. Take a minute to imagine what you would think and feel if you were the woman in It’s probably easy for you to adopt the modern this drawing. How would you explain to your best view. Some might regard a dandelion as a beau- friend what had happened? What solutions to the tiful fl ower whereas others see only an annoying confl ict would seem appropriate if you were this weed. To the premoderns, a dandelion has to be woman? either one or the other. If you think it is a weed, it What the woman’s husband sees is another is really a weed, even though some people have matter altogether, as shown in Figure 1-3. Take a a warped sense of beauty. In the modern view, minute to imagine the situation from his point of a dandelion is simply a dandelion. The concepts view. What thoughts and feelings would you have? “beautiful fl ower” and “annoying weed” are sub- How would you tell your best friend about it? What jective points of view imposed on the plant. Nei- solutions would seem appropriate? ther is a quality of the plant itself, just as “good” Now suppose you’re an outside observer watch- and “evil” were concepts imposed on the spirits in ing this interaction. What would it look like? Unfor- the tree. tunately, we can’t easily portray the third point of view without knowing something about the per- The Postmodern View Philosophers also speak sonal feelings, beliefs, past experiences, and so of a postmodern view of reality. In this view, neither forth that you would bring to your task as “outside” the spirits nor the dandelion exists. All that’s “real” observer. (Though I call you an outside observer, are the images we get through our points of view. you are, of course, observing from inside your own Put differently, there’s nothing out there—it’s all in mental system.) here. As Gertrude Stein said of Oakland, “There’s To take an extreme example, if you were a con- no there, there.” fi rmed male chauvinist, you’d probably see the No matter how bizarre the postmodern view fi ght pretty much the same way the husband saw may seem to you on fi rst refl ection, it has a cer- it. On the other hand, if you were committed to the tain ironic inevitability. Take a moment to notice view that men are unreasonable bums, you’d see the book you’re reading; notice specifi cally what it things the way the wife saw them. 12 CHAPTER 1 HUMAN INQUIRY AND SCIENCE

FIGURE 1-2 Wife’s Point of View. There is no FIGURE 1-3 Husband’s Point of View. There question in the wife’s mind as to who is right and is no question in the husband’s mind as to who is rational and who is out of control. right and rational and who is out of control.

But imagine instead that you see two unrea- Ultimately, what you’ll see is that (1) established sonable people quarreling irrationally with each scientifi c procedures sometimes allow you to deal other. Would you see them both as irresponsible effectively with this dilemma—that is, we can study jerks, equally responsible for the confl ict? Or would people and help them through their diffi culties you see them as two people facing a diffi cult hu- without being able to view “reality” directly—and man situation, each doing the best he or she can (2) the philosophical stances I’ve presented sug- to resolve it? Imagine feeling compassion for them gest a powerful range of possibilities for structur- and noticing how each of them attempts to end the ing research. hostility, even though the gravity of the problem Let’s turn now to the foundations of the social keeps them fi ghting. scientifi c approaches to understanding. From there Notice how different these several views are. we can examine the specifi c research techniques Which is a “true” picture of what is happening be- social scientists use. tween the wife and the husband? You win the prize if you notice that your own point of view would again THE FOUNDATIONS OF SOCIAL SCIENCE color your perception of what is happening here. The postmodern view represents a critical dilemma for scientists. While their task is to observe The two pillars of science are logic and observa- and understand what is “really” happening, they tion. A scientifi c understanding of the world must are all human and, as such, have personal orienta- (1) make sense and (2) correspond with what we tions that color what they observe and how they observe. Both elements are essential to science explain it. There is ultimately no way people can and relate to three major aspects of the overall sci- totally step outside their humanness to see and un- entifi c enterprise: theory, data collection, and data derstand the world as it “really” is. There are only analysis. our several subjective views. In the most general terms, scientifi c theory We’ll return to this discussion in Chapter 2 deals with logic, data collection with observation, when we focus in on specifi c scientifi c paradigms. and data analysis with patterns in what is observed THE FOUNDATIONS OF SOCIAL SCIENCE 13 and, where appropriate, the comparison of what is these are so much a matter of opinion and belief logically expected with what is actually observed. that scientifi c inquiry is often viewed as a threat to Though most of this textbook deals with data col- what is “already known.” lection and data analysis—demonstrating how to We’ll consider this issue in more detail in Chap- conduct empirical research—recognize that social ter 12, when we look at evaluation research. As science involves all three elements. As such, Chap- you’ll see, social scientists have become increas- ter 2 of this book concerns the theoretical context of ingly involved in studying programs that refl ect research; Parts 2 and 3 focus on data collection; and ideological points of view, such as affi rmative ac- Part 4 offers an introduction to the analysis of data. tion or welfare reform. One of the biggest prob- Figure 1-4 offers a schematic view of how the book lems researchers face is getting people to agree addresses these three aspects of social science. on criteria of success and failure. Yet such criteria Let’s turn now to some of the fundamental is- are essential if social scientifi c research is to tell us sues that distinguish social science from other anything useful about matters of value. By anal- ways of looking at social phenomena. ogy, a stopwatch can’t tell us if one sprinter is better than another unless we fi rst agree that speed is Theory, Not Philosophy or Belief the critical criterion. Social science, then, can help us know only Social scientifi c theory has to do with what is, not what is and why. We can use it to determine what with what should be. For many centuries, however, ought to be, but only when people agree on the social theory has combined these two orienta- criteria for deciding what’s better than something tions. Social philosophers liberally mixed their ob- else—an agreement that seldom occurs. With that servations of what happened around them, their understood, let’s turn now to some of the funda- speculations about why, and their ideas about how mental bases on which social science allows us to things ought to be. Although modern social scien- develop theories about what is and why. tists may do the same from time to time, realize that social science has to do with how things are Social Regularities and why. This means that scientifi c theory—and science In large part, social scientifi c theory aims to fi nd itself—cannot settle debates on value. Science patterns in social life. That aim, of course, applies cannot determine whether capitalism is better or to all science, but it sometimes presents a barrier worse than socialism except in terms of agreed-on to people when they fi rst approach social science. criteria. To determine scientifi cally whether capi- Actually, the vast number of formal norms in talism or socialism most supports human dignity society create a considerable degree of regularity. and freedom we would fi rst have to agree on some For example, only people who have reached a cer- measurable defi nitions of dignity and freedom. Our tain age can vote in elections. In the U.S. military, conclusions would depend totally on this agreement until recently only men could participate in com- and would have no general meaning beyond it. bat. Such formal prescriptions, then, regulate, or By the same token, if we could agree that suicide regularize, social behavior. rates, say, or giving to charity were good measures Aside from formal prescriptions, we can ob- of a religion’s quality, then we could determine sci- serve other social norms that create more regulari- entifi cally whether Buddhism or Christianity is the ties. Republicans are more likely than Democrats better religion. Again, our conclusion would be inextricably tied to the given criterion. As a practical theory A systematic explanation for the observations that matter, people seldom agree on criteria for deter- relate to a particular aspect of life: juvenile delinquency, mining issues of value, so science is seldom use- for example, or perhaps social stratifi cation or political ful in settling such debates. In fact, questions like revolution. 14 CHAPTER 1 HUMAN INQUIRY AND SCIENCE

Theory

Religious affiliation

Prejudice Education Voting behavior

Social class

Chapter 2

Data Collection

Planning to do Sampling Observation Data processing research

Chapters 4–6 Chapter 7 Chapters 8–12 Chapters 13–14

Data Analysis

x x Y = a + x1 + x2 + x3 + x4 + e

a y 34% 78% Application c d g

b y 66% 22%

Part 4

Theory ؉ Data Collection ؉ Data Analysis ؍ FIGURE 1-4 Social Science THE FOUNDATIONS OF SOCIAL SCIENCE 15 to vote for Republican candidates. University pro- TABLE 1-1 Birthrates, United States: fessors tend to earn more money than do unskilled 1980–2003 laborers. Men earn more than do women. The list 1980 15.9 of regularities could go on and on. 1981 15.8 The objection that there are always excep- 1982 15.9 tions to any social regularity is also inappropriate. 1983 15.6 It doesn’t matter that a particular woman earns 1984 15.6 more money than a particular man if men earn 1985 15.8 more than women overall. The pattern still exists. 1986 15.6 Social regularities represent probabilistic patterns; 1987 15.7 a general pattern need not be refl ected in 100 per- 1988 16.0 cent of the observable cases. 1989 16.4 This rule applies in physical science as well as 1990 16.7 social science. In genetics, for example, the mat- 1991 16.2 ing of a blue-eyed person with a brown-eyed per- 1992 15.8 son will probably result in a brown-eyed child. The 1993 15.4 birth of a blue-eyed child does not challenge the 1994 15.0 observed regularity, however, because the geneti- 1995 14.6 cist states only that the brown-eyed offspring is 1996 14.4 more likely and, further, that brown-eyed offspring 1997 14.2 will be born in a certain percentage of the cases. 1998 14.3 The social scientist makes a similar, probabilistic 1999 14.2 prediction—that women overall are likely to earn 2000 14.4 less than are men. And the social scientist asks 2001 14.1 why this is the case. 2002 13.9 2003 14.1 Aggregates, Not Individuals *Live births per 1,000 population Social regularities do exist, then, and are worthy Source: U.S. Bureau of the Census, Statistical Abstract of the United States (Washington, DC: U.S. Government Printing Offi ce, of theoretical and empirical study. As such, social 2006), Table 72, p. 64. scientists study primarily social patterns rather than individual ones. These patterns refl ect the If you have had a baby, you could probably tell aggregate or collective actions and situations of a much more detailed, idiosyncratic story. Why did many individuals. Although social scientists often you have the baby when you did, rather than a year study motivations and actions that affect individu- earlier or later? Maybe your house burned down als, they seldom study the individual per se. That and you had to delay a year before you could af- is, they create theories about the nature of group, ford to have the baby. Maybe you felt that being a rather than individual, life. family person would demonstrate maturity, which Sometimes the collective regularities are amaz- would support a promotion at work. ing. Consider the birthrate, for example. People Everyone who had a baby last year had a dif- have babies for an incredibly wide range of per- ferent set of reasons for doing so. Yet, despite this sonal reasons. Some do it because their parents vast diversity, despite the idiosyncrasy of each in- want them to. Some think of it as a way of com- dividual’s reasons, the overall birthrate in a society pleting their womanhood or manhood. Others (the number of live births per 1,000 population) is want to hold their marriages together. Still others remarkably consistent from year to year. See Table have babies by accident. 1-1 for recent birthrates for the United States. 16 CHAPTER 1 HUMAN INQUIRY AND SCIENCE

If the U.S. birthrate were 15.9, 35.6, 7.8, 28.9, and 16.2 in fi ve successive years, demographers IN THE REAL WORLD would begin dropping like fl ies. As you can see, BIRTHRATE IMPLICATIONS however, social life is far more orderly than that. Moreover, this regularity occurs without society- Take a minute to refl ect on the practical im- wide regulation. As mentioned earlier, no one plications of the data you’ve just seen. The plans how many babies will be born or determines “What Do You Think?” box for this chapter who will have them. (See the box “Birthrate Impli- asked how baby food and diaper manufac- cations” for a look at how the analysis of birthrates turers could plan production from year to can serve many purposes.) year. The consistency of U.S. birthrates sug- Social scientifi c theories try to explain why ag- gests this is not the problem it might have gregated patterns of behavior are so regular even seemed. when the individuals participating in them may Who else might benefi t from this kind of change over time. We could say that social scien- analysis? What about health care workers tists don’t seek to explain people per se. They try and educators? Can you think of anyone else? instead to understand the systems in which people What if we organized birthrates by region operate, which in turn explain why people do what of the country, by ethnicity, by income level, they do. The elements in such a system are not and so forth? Clearly, these additional analy- people but variables. ses could make the data even more useful. As you learn about the options available A Variable Language to social researchers, I think you’ll gain an appreciation for the practical value that re- Our most natural attempts at understanding are search can have for the whole society. usually concrete and idiosyncratic. That’s just the way we think. Imagine that someone says to you, “Women ought to get back into the kitchen where they be- scribed, they might use terms such as old-fashioned long.” You’re likely to hear that comment in terms or bigot to describe the kind of person who made of what you know about the speaker. If it’s your old the comment. In other words, they try to place the uncle Harry who is also strongly opposed to day- individual in a set of similar individuals, according light saving time, zip codes, and personal comput- to a particular, defi ned concept. ers, you’re likely to think his latest pronouncement By examining an individual in this way, social simply fi ts into his rather dated point of view about scientists can make sense out of more than one things in general. person. In understanding what makes the big- If, on the other hand, the statement issues forth oted politician think the way he does, they’ll also from a politician who is trailing a female chal- learn about other people who are “like him.” In lenger and who has also begun making statements other words, they have not been studying bigots as about women being emotionally unfi t for public of- much as bigotry. fi ce and not understanding politics, you may hear Bigotry here is spoken of as a variable because it his latest comment in the context of this political varies. Some people are more bigoted than others. challenge. Social scientists are interested in understanding In both examples, you’re trying to understand the system of variables that causes bigotry to be the thoughts of a particular individual. In social sci- high in one instance and low in another. ence, researchers go beyond that level of under- The idea of a system composed of variables standing to seek insights into classes or types of may seem rather strange, so let’s look at an anal- individuals. Regarding the two examples just de- ogy. The subject of a physician’s attention is the THE FOUNDATIONS OF SOCIAL SCIENCE 17 patient. If the patient is ill, the physician’s purpose is to help that patient get well. By contrast, Some Common Social Concepts a medical researcher’s subject matter is different: Female Age the variables that cause a disease, for example. Upper class African American The medical researcher may study the physician’s Young Occupation patient, but only as a carrier of the disease. Social class Sex Of course, medical researchers care about real people, but in the actual research, patients are di- Race/ethnicity Plumber rectly relevant only for what they reveal about the disease under study. In fact, when researchers can study a disease meaningfully without involving ac- Variable Attributes tual patients, they do so. Age Young, middle-aged, old Social research involves the study of variables Occupation Plumber, lawyer, and the attributes that compose them. Social scien- data-entry clerk . . . tifi c theories are written in a language of variables, Race/ethnicity African American, Asian, and people get involved only as “carriers” of those Caucasian, Latino . . . variables. Here’s a closer look at what social scien- Sex Female, male tists mean by variables and attributes. Attributes or values are characteristics or qual- Social class Upper, middle, lower . . . ities that describe an object—in this case, a person. FIGURE 1-5 Variables and Attributes. Examples include female, Asian, alienated, conser- Variables like education and prejudice and their vative, dishonest, intelligent, and farmer. Anything attributes (educated/uneducated, prejudiced/ you might say to describe yourself or someone else unprejudiced) provide the foundation for involves an attribute. examining causal relationships in social Variables, on the other hand, are logical research. groupings of attributes. Thus, for example, male and female are attributes, and sex is the variable composed of these two attributes. The variable oc- of family income for a city is a summary of attri- cupation is composed of attributes such as farmer, butes composing that variable: $3,124; $10,980; professor, and truck driver. Social class is a vari- $35,000; and so forth. Sometimes the meanings able composed of a set of attributes such as upper of the concepts that lie behind social science con- class, middle class, and lower class. Sometimes it cepts are fairly clear. Other times they aren’t. helps to think of attributes as the categories that The relationship between attributes and vari- make up a variable. See Figure 1-5 for a schematic ables is more complicated in the case of explana- review of what social scientists mean by variables tion and gets to the heart of the variable language and attributes. of scientifi c theory. Here’s a simple example, in- The relationship between attributes and vari- volving two variables, education and prejudice. For ables lies at the heart of both description and ex- the sake of simplicity, let’s assume that the variable planation in science. For example, we might de- education has only two attributes: educated and scribe a college class in terms of the variable sex by uneducated. (Chapter 5 will address the issue of reporting the observed frequencies of the attributes how such things are defi ned and measured.) Simi- male and female: “The class is 60 percent men and 40 percent women.” An unemployment rate can be thought of as a description of the variable em- attribute A characteristic of a person or a thing. ployment status of a labor force in terms of the attri- variable A logical grouping of attributes. The variable sex butes employed and unemployed. Even the report is made of up of the attributes male and female. 18 CHAPTER 1 HUMAN INQUIRY AND SCIENCE a. The uneducated are more prejudiced than the educated.

Educated Uneducated Prejudiced Unprejudiced

b. There is no apparent relationship between education and prejudice.

Educated Uneducated Prejudiced Unprejudiced

FIGURE 1-6 Illustration of Relationship between Two Variables (Two Possibilities). Variables such as education and prejudice and their attributes (educated/uneducated, prejudiced/unprejudiced) are the foundation for the examination of causal relationships in social research.

larly, let’s give the variable prejudice two attributes: are two predominant pairings: (1) those who are prejudiced and unprejudiced. educated and unprejudiced and (2) those who are Now let’s suppose that 90 percent of the unedu- uneducated and prejudiced. Here are two other cated are prejudiced, and the other 10 percent are useful ways of viewing that relationship. unprejudiced. And let’s suppose that 30 percent of First, let’s suppose that we play a game in the educated people are prejudiced, and the other which we bet on your ability to guess whether a 70 percent are unprejudiced. This is illustrated person is prejudiced or unprejudiced. I’ll pick the graphically in Figure 1-6a. people one at a time (not telling you which ones Figure 1-6a illustrates a relationship or asso- I’ve picked), and you have to guess whether each ciation between the variables education and preju- person is prejudiced. We’ll do it for all 20 people in dice. This relationship can be seen in terms of the Figure 1-6a. Your best strategy in this case would pairings of attributes on the two variables. There be to guess prejudiced each time, because 12 out of THE FOUNDATIONS OF SOCIAL SCIENCE 19 the 20 are categorized that way. Thus, you’ll get 12 Notice, at the same time, that educational varia- right and 8 wrong, for a net success of 4. tions can be found to depend on something else— Now let’s suppose that when I pick a person such as the educational level of our subjects’ par- from the fi gure, I have to tell you whether the per- ents. People whose parents have a lot of education son is educated or uneducated. Your best strategy are more likely to get a lot of education than are now would be to guess prejudiced for each unedu- those whose parents have little education. In this cated person and unprejudiced for each educated relationship, the subject’s education is the depen- person. If you follow that strategy, you’ll get 16 dent variable, the parents’ education the indepen- right and 4 wrong. Your improvement in guessing dent variable. We can say the independent variable prejudice by knowing education illustrates what it is the cause, the dependent variable the effect. means to say that variables are related. Returning to our fi rst example, we can see that Second, by contrast, let’s consider how the 20 the discussion of Figure 1-6 involved the interpre- people would be distributed if education and prej- tation of data. We looked at the distribution of the udice were unrelated to each other. This is illus- 20 people in terms of the two variables. In con- trated in Figure 1-6b. Notice that half the people structing a social scientifi c theory, we would derive are educated, and half are uneducated. Also notice an expectation regarding the relationship between that 12 of the 20 (60 percent) are prejudiced. If 6 of the two variables, based on what we know about the 10 people in each group were prejudiced, we each. We know, for example, that education ex- would conclude that the two variables were un- poses people to a wide range of cultural variation related to each other. Knowing a person’s educa- and to diverse points of view—in short, it broadens tion would not be of any value to you in guessing their perspectives. Prejudice, on the other hand, whether that person was prejudiced. represents a narrower perspective. Logically, then, We’ll be looking at the nature of relationships we might expect education and prejudice to be among variables in some depth in Part 4 of this somewhat incompatible. We might therefore arrive book. In particular, we’ll see some of the ways re- at an expectation that increasing education would lationships can be discovered and interpreted in reduce the occurrence of prejudice, an expectation research analysis. A general understanding of re- that our observations would support. lationships now, however, will help you appreciate Because Figure 1-6 has illustrated two possi- the logic of social scientifi c theories. bilities—that education reduces the likelihood of Theories describe the relationships we might prejudice or that it has no effect—you might be logically expect among variables. Often, the expectation involves the idea of causation. A person’s attributes on one variable are expected to cause, independent variable A variable with values that are predispose, or encourage a particular attribute on not problematical in an analysis but are taken as simply another variable. In our example, something about given. An independent variable is presumed to cause being educated apparently leads people to be less or determine a dependent variable. If we discover that religiosity is partly a function of sex—women are more re- prejudiced than if they were uneducated. ligious than are men—sex is the independent variable and As I’ll further discuss later in the book, education religiosity is the dependent variable. Note that any given and prejudice in this example would be regarded as variable might be treated as independent in one part of independent and dependent variables, respec- an analysis and dependent in another part of it. Religiosity tively. Because prejudice depends on something, might become an independent variable in an explanation of crime. we call it the dependent variable, which depends on an independent variable, in this case education. dependent variable A variable assumed to depend on or be caused by another (called the independent variable). Although the educational levels of the people be- If you fi nd that income is partly a function of amount of ing studied vary, that variation is independent of formal education, income is being treated as a dependent prejudice. variable. 20 CHAPTER 1 HUMAN INQUIRY AND SCIENCE interested in knowing what is actually the case. TABLE 1-2 Education and Anti-Gay Prejudice There are, of course, many types of prejudice. For Percent Saying this illustration, let’s consider prejudice against Homosexuality gays and lesbians. Over the years, the General So- Level of Education Is Always Wrong cial Survey (GSS) has asked respondents whether Less than high school graduate 72 homosexual relations between two adults is “al- High school graduate 62 ways wrong, almost always wrong, sometimes Junior college 56 wrong, or not wrong at all.” In 2004, 56 percent Bachelor’s degree 44 of those interviewed said that homosexuality was Graduate degree 30 always wrong. However, this response is strongly related to the respondents’ education, as Table 1-2 indicates.

ISSUES AND INSIGHTS ANALYZING DATA ONLINE

You can test the relationship between prejudice variable (Column). YEAR(2004) indicates that and education for yourself if you have a connec- you only want to analyze data from the 2004 tion to the Internet. The data just presented in survey (Filter). Leaving the fi lter blank would the text are taken from the General Social Sur- cause the program to analyze the data from all vey (GSS), which you can locate at http://www. the surveys done since 1972 that included the icpsr.umich.edu/GSS/. two variables under study. Once you reach that location, you’ll discover Click “Run the Table” at the bottom, and you several buttons along the top of the page. Click should be presented with a table containing the the box labeled “Analyze.” This will open up a data shown in the text. Once you’ve done that, you data-analysis program created by the University might want to do some exploration on your own. of California–Berkeley. The “Subject” hotlink will take you to a cata- Click the second button, for “Frequencies or log of topics studied by the GSS over the years. crosstabulation,” and then click “Start” near the You might want to examine other indicators of bottom of the page. This will take you to a page other forms of prejudice: against racial groups, where you can specify the analysis you would women, and the like. like to see. To replicate the analysis presented in With the increased interest in online data the text, you will use three variables in the GSS analysis, you may fi nd this GSS site overused data set. HOMOSEX asks people whether they and slow to respond. In that case, you might look feel homosexuality is wrong or not. DEGREE around for other analysis engines, such as the gives us the respondent’s educational level, and Cultural Policy and the Arts’ CPANDA-FACTOID YEAR let’s you specify which GSS survey you are at http://www.cpanda.org/codebookDB/sdalite interested in: 2004 in this case. .jsp?ida00079. Or you may want to search the Enter this information in the online form as web for something like “Analyze ‘General Social shown here. Survey.’” Because the web is an evolving re- This specifi es HOMOSEX as the dependent source, new tools will likely appear by the time variable (Row) and DEGREE as the independent this textbook reaches you. THE FOUNDATIONS OF SOCIAL SCIENCE 21

Notice that the theory has to do with the two The Purposes of Social Research variables education and prejudice, not with peo- Although Chapter 4 will examine the various pur- ple as such. People are the carriers of those two poses of social research in some detail, preview- variables, so we can see the relationship between ing them here will be useful. To begin, sometimes the variables only when we observe people. Ul- social research is a vehicle for mapping out a timately, however, the theory uses a language of topic that may warrant further study later: look- variables. It describes the associations that we ing into a new political or religious group, learn- might logically expect to exist between particular ing something about use of a new street drug, and attributes of different variables. (You can do this so forth. The methods vary greatly and the con- data analysis for yourself with nothing more than a clusions are usually suggestive rather than defi ni- connection to the Internet. See the box “Analyzing tive. Still, careful exploratory social research can Data Online.”) 22 CHAPTER 1 HUMAN INQUIRY AND SCIENCE dispel some misconceptions and help focus future The ethical concerns will make more sense to research. you as you learn more about the actual techniques Some social research is done for the purpose of of doing research. Be sure to consider this impor- describing the state of social affairs: What is the un- tant issue as you read each chapter. employment rate? What is the racial composition of a particular city? What percentage of the popu- SOME DIALECTICS lation holds a particular opinion or plans to vote OF SOCIAL RESEARCH for a particular political candidate? Careful empirical description takes the place of speculation and impressions. There is no one way to do social research. (If there Often, social research has an explanatory pur- were, this would be a much shorter book.) In fact, pose—providing reasons for phenomena, in terms much of the power and potential of social research of causal relationships. Why do some cities have lies in the many valid approaches it comprises. higher unemployment rates than others? Why are Four broad and interrelated distinctions under- some people more prejudiced than others? Why lie these approaches. Though these distinctions are women likely to earn less than men for doing can be seen as competing choices, a good social the same job? Ordinary, everyday discourse offers researcher thoroughly learns each. This is what I an abundance of answers to such questions, but mean by the “dialectics” of social research: a fruitful some of those answers are simply wrong. Explan- tension between these complementary concepts. atory social research provides more trustworthy explanations. Idiographic and Later in this chapter, we’ll compare pure and Nomothetic Explanation applied research. It’s worth noting here, though, that the purpose of some research is limited to All of us go through life explaining things. We do it understanding, whereas other research efforts are every day. You explain why you did poorly or well deliberately intended to bring about social change, on an exam, why your favorite team is winning creating a more workable or just society. or losing, why you may be having trouble getting dates. In our everyday explanations, we engage in The Ethics of Human Inquiry two distinct forms of causal reasoning, though we do not ordinarily distinguish them. Most of this book is devoted to the logic and tech- Sometimes we attempt to explain a single situ- niques of doing social research, but you will soon ation in idiosyncratic detail. Thus, for example, you discover an ethical dimension running throughout may have done poorly on an exam because (1) you the discussion. You’ll learn that medical, social, had forgotten there was an exam that day, (2) it and other studies of human beings have often used was in your worst subject, (3) a traffi c jam made methods later condemned as unethical. In Chap- you late for class, (4) your roommate had kept you ter 3 and throughout the book, we examine the up the night before the exam with loud music, (5) various concerns that distinguish ethical from un- the police kept you until dawn demanding to know ethical research. what you had done with your roommate’s stereo— and with your roommate, for that matter—and (6) a wild band of coyotes ate your textbook. Given idiographic An approach to explanation in which we all these circumstances, it is no wonder that you seek to exhaust the idiosyncratic causes of a particular did poorly. condition or event. Imagine trying to list all the reasons why you chose to attend your particular college. Given This type of causal reasoning is called an idio- all those reasons, it’s diffi cult to imagine your making any graphic explanation. Idio- in this context means other choice. unique, separate, peculiar, or distinct, as in the SOME DIALECTICS OF SOCIAL RESEARCH 23 word idiosyncrasy. When we have completed an id- understanding of the inner workings of a particu- iographic explanation, we feel that we fully under- lar juvenile gang or the corporate leadership of a stand the causes of what happened in this particular particular multinational conglomerate engage in instance. At the same time, the scope of our expla- idiographic research: they try to understand that nation is limited to the case at hand. Although parts particular group as fully as possible. of the idiographic explanation might apply to other A. Libin and J. Cohen-Mansfi eld (2000) have situations, our intention is to explain one case fully. contrasted the way these two approaches are used Now consider a different kind of explanation. in studies of the elderly (gerontology). Some stud- Every time you study with a group, you do better ies focus on the experiences of individuals in the on an exam than if you study alone. Your favorite totality of their life situations, whereas other stud- team does better at home than on the road. Ath- ies look for statistical patterns describing the el- letes get more dates than do members of the bi- derly in general. The authors suggest ways to com- ology club. Notice that this type of explanation is bine idiographic and nomothetic approaches in more general, covering a wider range of experience gerontology. The box “Idiographic and Nomothetic or observation. It speaks implicitly of the relation- Reasoning in Everyday Life” shows how these ap- ship between variables: for example, (1) whether proaches appear in less formal settings. or not you study in a group and (2) how well you As you can see, social scientists can access do on the exam. This type of explanation—labeled two distinct kinds of explanation. Just as physicists nomothethic—seeks to explain a class of situa- sometimes treat light as a particle and other times tions or events rather than a single one. Moreover, as a wave, social scientists can search for relatively it seeks to explain “economically,” using only one superfi cial universals today and probe the narrowly or just a few explanatory factors. Finally, it settles particular tomorrow. Both are good science, both for a partial rather than a full explanation. are rewarding, and both can be fun. In each of these examples, you might qualify your causal statements with on the whole, usually, Inductive and Deductive Theory all else being equal, and the like. Thus, you usually do better on exams when you’ve studied in Like idiographic and nomothetic forms of explana- a group, but not always. Similarly, your team has tion, inductive and deductive thinking both play a won some games on the road and lost some at role in our daily lives. They, too, represent an im- home. And the gorgeous head of the biology club portant variation in social research. may get lots of dates, while the defensive lineman There are two routes to the conclusion that you Pigpen-the-Terminator may spend a lot of Satur- do better on exams if you study with others. On day nights alone punching heavy farm equipment. the one hand, you might fi nd yourself puzzling, Such exceptions are acceptable within a broader halfway through your college career, about why range of overall explanation. As we noted earlier, you do so well on exams sometimes but poorly patterns are real and important even when they at other times. You might list all the exams you’ve are not perfect. taken, noting how well you did on each. Then you Both the idiographic and the nomothetic ap- might try to recall any circumstances shared by all proaches to understanding can serve you in your the good exams and all the poor ones. Did you do daily life. The nomothethic patterns you discover better on multiple-choice exams or essay exams? might offer a good guide for planning your study habits, but the idiographic explanation is more nomothetic convincing to your parole offi cer. An approach to explanation in which we seek to identify a few causal factors that generally impact By the same token, both idiographic and no- a class of conditions or events. Imagine the two or three mothetic reasoning are powerful tools for social key factors that determine which colleges students research. Researchers who seek an exhaustive choose, such as proximity, reputation, and so forth. 24 CHAPTER 1 HUMAN INQUIRY AND SCIENCE

ISSUES AND INSIGHTS IDIOGRAPHIC AND NOMOTHETIC REASONING IN EVERYDAY LIFE

The difference between idiographic and nomo- ly complete explanation for the behavior of the thetic explanations can be found in everyday one boy in question. In the nomothetic mode, life. Consider the following: we have a simpler, more general, explanation, Idiographic: “He’s like that because his fa- which wouldn’t necessarily be true of all teen- ther and mother kept giving him mixed signals. age boys but portrays a general pattern. The fact that his family moved seven times by Be warned, of course, that neither explana- the time he was 12 years old didn’t help. More- tion is necessarily true. When these models are over, his older brother is exactly the same and used in social research, other elements of the probably served as a role model.” inquiry—how subjects were chosen, how mea- Nomothetic: “Teenage boys are like that.” surements were made, and so forth—strengthen In the idiographic mode, we have a seeming- the validity of conclusions drawn.

Morning exams or afternoon exams? Exams in the Then you consider whether to study at a measured natural sciences, the humanities, or the social sci- pace or pull an all-nighter just before the exam. ences? Times when you studied alone or . . . BAM! Among these musings, you might ask whether you It occurs to you that you have almost always done should get together with other students in the class best on exams when you studied with others. This or just study on your own. You could evaluate the mode of inquiry is known as induction. pros and cons of both options. Inductive reasoning moves from the particular Studying with others might not be as effi cient, to the general, from a set of specifi c observations because a lot of time might be spent on things to the discovery of a pattern that represents some you already understand. On the other hand, you degree of order among all the given events. No- can understand something better when you’ve tice, incidentally, that your discovery doesn’t nec- explained it to someone else. And other students essarily tell you why the pattern exists—just that might understand parts of the course you haven’t it does. gotten yet. Several minds can reveal perspectives Here’s a very different way you might have ar- that might have escaped you. Also, your commit- rived at the same conclusion about studying for ment to study with others makes it more likely that exams. Imagine approaching your fi rst set of ex- you’ll study rather than watch the special Brady ams in college. You wonder about the best ways Bunch retrospective. to study—how much to review, how much to focus In this fashion, you might add up the pros and on class notes. You learn that some students pre- cons and conclude, logically, that you’d benefi t pare by rewriting their notes in an orderly fashion. from studying with others. It seems reasonable to you, the way it seems reasonable that you’ll do better if you study rather than not. Sometimes we induction The logical model in which general principles say things like this are true “in theory.” To complete are developed from specifi c observations. Having noted the process, we test whether they’re true in prac- that Jews and Catholics are more likely to vote Demo- tice. For a complete test, you might study alone cratic than are Protestants, you might conclude that religious minorities in the United States are more affi li- for half your exams and study with others for the ated with the Democratic party, and then your task is other exams. This procedure would test your logi- to explain why. cal reasoning. SOME DIALECTICS OF SOCIAL RESEARCH 25

This second mode of inquiry, deduction, moves tinction between numerical and nonnumerical from the general to the specifi c. It moves from (1) a data. When we say someone is intelligent, we’ve pattern that might be logically or theoretically ex- made a qualitative assertion. When psychologists pected to (2) observations that test whether the ex- and others measure intelligence by IQ scores, they pected pattern actually occurs. Notice that deduc- are attempting to quantify such a qualitative as- tion begins with “why” and moves to “whether,” sessment. For example, the psychologist might say whereas induction moves in the opposite direction. that a person has an IQ of 120. As you’ll see later in this book, these two very Every observation is qualitative at the outset, different approaches present equally valid av- whether it be your experience of someone’s intel- enues for science. Each of these approaches can ligence, the location of a pointer on a measuring stimulate the research process, prompting the re- scale, or a check mark entered in a questionnaire. searcher to take on specifi c questions and framing None of these things is inherently numerical or the manner in which they are addressed. More- quantitative, but converting them to a numerical over, you’ll see how induction and deduction work form is useful at times. (Chapter 14 deals specifi - together to provide ever more powerful and com- cally with the quantifi cation of data.) plete understandings. Quantifi cation often makes our observations Notice, by the way, that the deductive/inductive more explicit. It can also make aggregating and distinction is not necessarily linked to the nomo- summarizing data easier. Further, it opens up the thetic and idiographic modes. For example, idio- possibility of statistical analyses, ranging from sim- graphically and deductively, you might prepare for ple averages to complex formulas and mathemati- a particular date by taking into account everything cal models. Thus, a social researcher might ask you know about the person you’re dating, trying whether you tend to date people older or younger to anticipate logically how you can prepare—what than yourself. A quantitative answer to this seems kinds of clothing, behavior, hairstyle, oral hygiene, easily attained. The researcher asks how old each and so forth will likely produce a successful date. of your dates has been and calculates an average. Or, idiographically and inductively, you might try to Case closed. fi gure out what it was exactly that caused your date Or is it? Although “age” here represents the to call 911. A nomothetic, deductive approach number of years people have been alive, sometimes arises when you coach others on your “rules of people use the term differently; perhaps for some dating,” wisely explaining why their dates will be people “age” really means “maturity.” Though your impressed to hear them expound on the dangers of dates may tend to be a little older than you, they satanic messages concealed in rock and roll lyrics. may act more immaturely and thus represent the When you later review your life and wonder why same “age.” Or someone might see “age” as how you didn’t date more musicians, you might engage young or old your dates look or maybe the degree in nomothetic induction. Thus, there are four pos- of variation in their life experiences, their worldli- sible approaches, which are used as much in life ness. These latter meanings would be lost in the as in research. quantitative calculation of average age. Qualita- We’ll return to induction and deduction later in tive data are richer in meaning and detail than are the book. At this point, let’s turn to a third broad quantifi ed data. This is implicit in the cliché, “He is distinction that generates rich variations in social research. deduction The logical model in which specifi c expectations of hypotheses are developed on the basis of general Quantitative and Qualitative Data principles. Starting from the general principle that all deans are meanies, you might anticipate that this one The distinction between quantitative and qualita- won’t let you change courses. This anticipation would tive data in social research is essentially the dis- be the result of deduction. 26 CHAPTER 1 HUMAN INQUIRY AND SCIENCE older than his years.” The poetic meaning of this such experiences, the more worldly we’d say they expression would be lost in attempts to specify were. If we thought of some experiences as more how much older. powerful than others, we could give those expe- This richness of meaning stems in part from am- riences more points. Once we had made our list biguity. If the expression means something to you and point system, scoring people and comparing when you read it, that particular meaning arises their worldliness would be pretty straightforward. from your own experiences, from people you’ve We would have no diffi culty agreeing on who had known who might fi t the description of being “older more points than whom. than their years” or perhaps the times you’ve heard To quantify a concept like worldliness, we need others use that expression. Two things are certain: to be explicit about what we mean. By focusing (1) You and I probably don’t mean exactly the same specifi cally on what we’ll include in our measure- thing, and (2) you don’t know exactly what I mean, ment of the concept, however, we also exclude any and vice versa. other meanings. Inevitably, then, we face a trade- I have a young friend, Ray Zhang, who was re- off: Any explicated, quantitative measure will be sponsible for communications at the 1989 freedom more superfi cial than the corresponding qualita- demonstrations in Tiananmen Square, Beijing. Fol- tive description. lowing the Army clampdown, Ray fl ed south and What a dilemma! Which approach should was arrested and then released with orders to re- we choose? Which is more appropriate to social turn to Beijing. Instead, he escaped from China and research? made his way to Paris. Eventually, he came to the The good news is that we don’t need to choose. United States, where he resumed the graduate stud- In fact, we shouldn’t. Both qualitative and quanti- ies he had been forced to abandon. He had to enroll tative methods are useful and legitimate in social in school without any transcripts from China, study research. Some research situations and topics are in a foreign language, meet his fi nancial needs—all most amenable to qualitative examination, others on his own, thousands of miles from his family. to quantifi cation. We need both. Ray strikes me as someone “older than his However, because these two approaches call years.” You’d probably agree. This qualitative de- for different skills and procedures, you may feel scription, while it fl eshes out the meaning of the more comfortable with and become more adept in phrase, still does not equip us to say how much one mode than the other. You’ll be a stronger re- older or even to compare two people in these terms searcher, however, to the extent that you can learn without the risk of disagreeing as to which one is both approaches. At the very least, you should rec- more “worldly.” ognize the legitimacy of both. It might be possible to quantify this concept, Finally, you may have noticed that the qualita- however. For example, we might establish a list of tive approach seems more aligned with idiographic life experiences that would contribute to what we explanations, whereas nomothetic explanations mean by worldliness: are more easily achieved through quantifi cation. Though this is true, these relationships are not Getting married absolute. Moreover, both approaches present con- Getting divorced siderable “gray area.” Recognizing the distinction Having a parent die between qualitative and quantitative research Seeing a murder committed doesn’t mean that you must identify your research Being arrested activities with one to the exclusion of the other. A Being exiled complete understanding of a topic often requires Being fi red from a job both techniques. Running away with the circus The contributions of these two approaches are We might quantify people’s worldliness as the widely recognized today. For example, when Stuart number of such experiences they’ve had: the more Biddle and his colleagues (2001) at the University SOME DIALECTICS OF SOCIAL RESEARCH 27 of Wales set out to review the status of research Ibrahim provides a good example of how social in the fi eld of sport and exercise psychology, they scientists deal with something like imprisonment, were careful to examine the uses of both quantita- which is, after all, an all-too-common part of mod- tive and qualitative techniques, drawing attention ern social life. to those they felt were underused. In reviewing the In those initial 45 days, my human contacts frequent disputes over the superiority of qualitative in prison were limited to prison wardens and or quantitative methods, Anthony Onwuegbuzie guards. I had little opportunity to do as much and Nancy Leech (2005) suggest that the two ap- sociological research on the prison community proaches have more similarities than differences. as I would have liked. That would have to wait They further argue that using both approaches for the second and third rounds of my impris- strengthens social research. onment which followed in 2001 and 2002. Pure and Applied Research — (2003:69) One of the charges brought against Ibrahim Social researchers have two distinct motivations: was a law that prohibits “spreading false rumors understanding and application. On the one hand, and tarnishing Egypt’s image abroad.” A more se- they are fascinated by the nature of human social rious charge was that he had accepted fi nancial life and are driven to explain it, to make sense out contributions from abroad without government of apparent chaos. Pure research in all scientifi c permission. As Ibrahim was to learn, his research fi elds is sometimes justifi ed in terms of gaining institute’s acceptance of research grants—usually “knowledge for its own sake.” a valued achievement—was regarded as a federal At the same time, perhaps inspired by their sub- crime in his case. As Ibrahim observes, ject matter, many social researchers are committed to having what they learn make a difference, Being an activist sociologist in a Third World to see their knowledge of society put into action. country is tremendously challenging. While Sometimes they focus on making things better. some elements of the work are gratifying, it When I study prejudice, for example, I’d like what is more often permeated with agony. One I discover to result in a more tolerant society. This honestly never knows when one is breaking is no different from the AIDS researcher trying to a law, violating a military order or simply step- defeat that disease. ping over an invisible red line. — (2003:70) The professional activities of some social scien- Eventually, because of his own efforts and the tists are intimately interwoven with the intention international uproar produced by his arrest and of creating a more humane society. Today, there imprisonment, Ibrahim was given a new trial and is no better role model than the Egyptian sociolo- was fi nally released from prison on his 64th birth- gist Saad Eddin Ibrahim. While addressing a great day: De cember 3, 2002. many social issues, Ibrahim’s most recent research has focused on the problems of modern Arab societies in general, and Egypt in particular. After years You can learn more about Ibrahim’s of researching and writing on the edge of political experience at his website: http://shr.aaas tolerance, Ibrahim crossed the line in 2000. .org/aaashran/alert.php?a_id=223.*

Following the publication of one of my articles on Arab presidents grooming their sons to suc- Social research is put into practice in many more ceed them in the North Korean tradition of the ordinary ways as well. Experiments and surveys, late Kim Il Sung, the old guard seemed to have for example, can be used in product marketing. gotten a green light to come after me. The day

after the article appeared on Cairo newsstands— *Each time the Internet icon appears, you’ll be given help- June 30, 2000—I was arrested. — (2003:71) ful leads for searching the World Wide Web. 28 CHAPTER 1 HUMAN INQUIRY AND SCIENCE

In-depth interviewing techniques can be especially Now that you’ve learned about the foundations useful in social work. Chapter 12 deals with evalu- of social research, I hope you can see how vibrant ation research, by which social researchers deter- and exciting such research is. All we need is an mine the effectiveness of social interventions. open mind and a sense of adventure—and a good Sometimes, seemingly mundane research ef- grounding in the basics of social research. forts can powerfully affect people’s lives. Imagine working alongside Crystal Eastman, an applied sociologist and settlement worker active in the Pitts- WHAT DO YOU THINK? REVISITED burgh area in the early twentieth century:

We got permission to use these [coroner’s This chapter opened with a question re- records] and made a record of every industrial garding uncontrolled variations in society— fatality reported to the coroner during the specifi cally, birthrates. We noted that there twelve months from July 1906 to July 1907, is no apparent control over who will or will taking down on a separate card for each case, not have a baby during a given year. Indeed, the name and address of the man killed, his many babies occur by accident. For the age, occupation and conjugal condition, the most part, different people have the babies name of his employer, the circumstances of from one year to the next, and each baby the accident, the names of important witnesses, results from idiosyncratic, deeply personal and the verdict. The plan was to learn from reasons. the evidence in the coroner’s record, how each As the data introduced in this chapter in- accident happened, and to learn from visiting dicate, however, aggregate social life oper- family what happened after the accident, e.g., ates differently from individual experiences how great a fi nancial loss was suffered by the of living in society. Although predicting family of the workman killed, how much of whether a specifi c person or couple will de- this was made up by compensation received cide to have a child at a given time is dif- from the employer, and how the family was fi cult, a greater regularity exists at the level affected in its economic life by the accident. of groups, organizations, and societies. This When we had done this with the fatalities, regularity is produced by social structure, we followed the same course with the rec- culture, and other forces that individuals ords of three months’ industrial injuries which may or may not be aware of. Refl ect, for we secured from the hospitals. — (EASTMAN example, on the impact of a housing in- 1910:789; QUOTED IN LENGERMANN AND NIEBRUGGE- dustry that provides too few residences to BRANTLEY, 2002:13) accommodate large families, in contrast to As a result of this and similar studies, American one where accommodation is the norm. workers now enjoy the protections of worker’s Whereas that single factor would not abso- compensation insurance. lutely determine the childbearing choices of As with the other dialectics just discussed, a particular person or couple, it would have some social scientists tend more toward pure re- a predictable, overall effect across the whole search, others toward application. Ultimately, both society. And social researchers are chiefl y orientations are valid and vital elements in social interested in describing and understanding research as a whole. In dealing with the basics social patterns, not individual behaviors. of social research, whether pure or applied, one This book will share with you some of the of the intentions of this book is to draw atten- logic and tools social researchers use in that tion to the ways in which such research makes a quest. difference. MAIN POINTS 29

Main Points The Foundations of Social Science ❏ Social theory attempts to discuss and ex- Introduction plain what is, not what should be. Theory ❏ The subject of this book is how we fi nd out should not be confused with philosophy or about social reality. belief. ❏ Social science looks for regularities in social Looking for Reality life. ❏ Inquiry is a natural human activity. Much of or- ❏ dinary human inquiry seeks to explain events Social scientists are interested in explaining and predict future events. human aggregates, not individuals. ❏ ❏ When we understand through direct experi- Theories are written in the language of ence, we make observations and seek patterns variables. or regularities in what we observe. ❏ A variable is a logical set of attributes. An ❏ Much of what we know, we know by agree- attribute is a characteristic, such as male ment rather than by experience. In particular, or female. Sex, for example, is a variable two important sources of agreed-on knowl- made up of these attributes. edge are tradition and authority. However, ❏ In causal explanation, the presumed cause these useful sources of knowledge can also is the independent variable, and the affected lead us astray. variable is the dependent variable. ❏ Whereas we often observe inaccurately in ❏ Social research has two main purposes: day-to-day inquiry, researchers seek to avoid describing or explaining social phenomena. such errors by making observation a careful ❏ Ethics plays a key role in the practice of and deliberate activity. social research. ❏ We sometimes jump to general conclusions on the basis of only a few observations, so Some Dialectics of Social Science scientists seek to avoid overgeneralization ❏ Whereas idiographic explanations seek by committing themselves to a suffi cient to present a full understanding of specifi c number of observations and by replicating cases, nomothetic explanations seek to studies. present a generalized account of many ❏ In everyday life we sometimes reason illogi- cases. cally. Researchers seek to avoid illogical rea- ❏ Inductive theories reason from specifi c ob- soning by being as careful and deliberate servations to general patterns. Deductive in their reasoning as in their observations. theories start from general statements and Moreover, the public nature of science means predict specifi c observations. that others can always challenge faulty ❏ Quantitative data are numerical; qualitative reasoning. data are not. Both types of data are useful ❏ Three views of “reality” are the premodern, for different research purposes. modern, and postmodern views. In the post- ❏ Both pure and applied research are valid modern view, there is no “objective” reality and vital parts of the social research independent of our subjective experiences. enterprise. Different philosophical views suggest a range of possibilities for scientifi c research. 30 CHAPTER 1 HUMAN INQUIRY AND SCIENCE

Key Terms Website for The Basics of Social Research attribute induction 4th edition deduction nomothetic dependent variable replication At the book companion website (http://sociology idiographic theory .wadsworth.com/babbie_basics4e) you will fi nd independent variable variable many resources in addition to ThomsonNow to aid you in studying for your exams. For ex- Review Questions ample, you will fi nd Tutorial Quizzes with feedback, Internet Exercises, Flashcards, and Chapter 1. How would the discussion of variables and Tutorials, as well as Extended Projects, InfoTrac attributes apply to physics, chemistry, and College Edition search terms, Social Research in biology? Cyberspace, GSS Data, Web Links, and primers 2. Identify a social problem that you feel ought for using various data analysis software such to be addressed and solved. What are the as SPSS and NVivo. variables represented in your description of the problem? Which of those variables would you monitor in determining whether the Additional Readings problem was solved? Babbie, Earl. 1994. The Sociological Spirit. Belmont, CA: 3. Suppose you were interested in studying the Wadsworth. This book is a primer in sociological quality of life among elderly people. What points of view and introduces you to many of the quantitative and qualitative indicators might concepts commonly used in social research, you examine? Babbie, Earl. 1998. Observing Ourselves: Essays in Social Research. Prospect Heights, IL: Waveland Press. This 4. How might social research be useful to such collection of essays expands some of the philosophi- professionals as physicians, attorneys, busi- cal issues you’ll see in the following chapters, includ- ness executives, police offi cers, and news- ing objectivity, paradigms, determinism, concepts, paper reporters? reality, causation, and values. Becker, Howard S. 1997. Tricks of the Trade: How to Think about Your Research while You’re Doing It. Online Study Resources Chicago: University of Chicago Press. This very approachable book offers an excellent “feel” for the enterprise of social scientifi c research, whether qualitative or quantitative. It’s fi lled with research anecdotes that show social inquiry to be a lively Go to and challenging endeavor. Cole, Stephen. 1992. Making Science: Between Nature http://sociology.wadsworth.com/babbie_basics4e and Society. Cambridge, MA: Harvard University and click on ThomsonNow for access to this Press. If you’re interested in a deeper examination of powerful online study tool. You will get a per- science as a social enterprise, you may fi nd this sonalized study plan based on your responses to a fascinating analysis. a diagnostic pretest. Once you have mastered Hoover, Kenneth R. 1992. The Elements of Social Scien- tifi c Thinking. New York: St. Martin’s Press. Hoover the material with the help of interactive learning presents an excellent overview of the key elements tools, you can take a posttest to confi rm that you in social scientifi c analysis. are ready to move on to the next chapter. ADDITIONAL READINGS 31

Steele, Stephen F., and Joyce Miller Iutcovich, eds. 1997. for Applied Sociology and provides an excellent Directions in Applied Sociology. Arnold, MD: Society overview of the issues involved in the application of for Applied Sociology. This book contains the presi- social science knowledge. dential addresses of eleven presidents of the Society 21 HUMANPARADIGMS, INQUIRY THEORY, AND ANDSCIENCE RESEARCH

Image not available due to copyright restrictions Photo credit

What You’ll Learn in thisThis ChapterChapter

HereWe’ll we’ll examine examine the some way ofpeople the theoretical learn about points their ofworld view and that the structure mistakes social they makescientiﬁ along c inquiry. the way. This We’ll lays thealso groundwork begin to see for what understanding makes sience the different speciﬁ c fromresearch

othertechniques ways discussedof knowing throughout things. the rest of the book. 2S 1S N L 32 In this chapter . . . WHAT DO YOU THINK?

Introduction Scholars such as Some Social Science Paradigms George Herbert Macrotheory and Microtheory Mead make a pow- Early Positivism Image not available due to copyright restrictions erful argument that Confl ict Paradigm social life is really Symbolic Interactionism a matter of inter- Ethnomethodology actions and their Structural Functionalism residue. You and I Feminist Paradigms meet each other for the fi rst time, feel each Critical Race Theory other out, and mutually create rules for deal- Rational Objectivity Reconsidered ing with each other. The next time we meet, Two Logical Systems Revisited we’ll probably fall back on these rules, which The Traditional Model of Science tend to stay with us. Think about your fi rst Deduction and Induction Compared encounters with a new professor or making a new friend. Mead suggests that all the so- Deductive Theory Construction cial patterns and structures that we experi- Getting Started ence are created in this fashion. Constructing Your Theory Other scholars, such as Karl Marx, argue An Example of Deductive Theory: that social life is fundamentally a struggle Distributive Justice among individuals and among groups. Ac- Inductive Theory Construction cording to Marx, society is a class struggle in An Example of Inductive Theory: Why Do People which the “haves” and the “have-nots” are Smoke Marijuana? pitted against each other in an attempt to dominate others and to avoid being domi- The Links between Theory and Research nated. He claims that, rather than being mu- The Importance of Theory in the “Real tually created by individuals, rules for behav- World” ior grow out of the economic structure of a society. Research Ethics and Theory Which of these very different views of society is true? Or does the truth lie some- INTRODUCTION where else? See the “What Do You Think? Revisited” box Some restaurants in the United States are fond of toward the end of the chapter. conducting political polls among their customers before an upcoming election. Some people take these polls very seriously because of their uncanny history of predicting winners. By the same token,

33 34 CHAPTER 2 PARADIGMS, THEORY, AND RESEARCH some movie theaters have achieved similar suc- your memory of where you had been to limit your cess by offering popcorn in bags picturing either search to more likely areas. Theory, by analogy, di- donkeys or elephants. Years ago, granaries in the rects researchers’ fl ashlights where they will most Midwest offered farmers a chance to indicate their likely observe interesting patterns of social life. political preferences through the bags of grain they This is not to say that all social science research selected. is tightly intertwined with social theory. Sometimes Such oddities offer some interest. They all pres- social scientists undertake investigations simply ent the same pattern over time, however: They to discover the state of affairs, such as an evalu- work for a while, but then they fail. Moreover, we ation of whether an innovative social program is can’t predict when or why they will fail. working or a poll to determine which candidate These unusual polling techniques point to the is winning a political race. Similarly, descriptive shortcomings of “research fi ndings” based only on ethnographies, such as anthropological accounts the observation of patterns. Unless we can offer of preliterate societies, produce valuable informa- logical explanations for such patterns, the regulari- tion and insights in and of themselves. However, ties we’ve observed may be mere fl ukes, chance even studies such as these often go beyond pure occurrences. If you fl ip coins long enough, you’ll description to ask why? Theory is directly relevant get ten heads in a row. Scientists could adapt a to “why” questions. street expression to describe this situation: “Pat- This chapter explores some specifi c ways theory terns happen.” and research work hand in hand during the adven- Logical explanations are what theories seek ture of inquiry into social life. We’ll begin by look- to provide. Further, theory functions three ways ing at several fundamental frames of reference, in research. First, it prevents our being taken in called paradigms, that underlie social theories and by fl ukes. If we can’t explain why Ma’s Diner has inquiry. Whereas theories seek to explain, para- predicted elections so successfully, we run the risk digms provide ways of looking. In and of them- of supporting a fl uke. If we know why it has hap- selves, paradigms don’t explain anything, but they pened, however, we can anticipate whether it will provide logical frameworks within which theories work in the future. are created. As you’ll see in this chapter, theories Second, theories make sense of observed pat- and paradigms intertwine throughout the search terns in ways that can suggest other possibilities. for meaning in social life. If we understand the reasons why broken homes produce more juvenile delinquency than do intact SOME SOCIAL SCIENCE PARADIGMS homes—lack of supervision, for example—we can take effective action, such as after-school youth programs. There is usually more than one way to make Finally, theories can shape and direct research sense of things. In daily life, for example, liberals efforts, pointing toward likely discoveries through and conservatives often explain the same phe- empirical observation. If you were looking for your nomenon—teenagers using guns at school, for lost keys on a dark street, you could whip your example—quite differently. So might the parents fl ashlight around randomly—or you could use and teenagers themselves. But underlying each of these different explanations, or theories, is a paradigm—one of the fundamental models or frames of reference we use to organize our observations A model or framework for observation and paradigm and reasoning. understanding, which shapes both what we see and how we understand it. The confl ict paradigm causes us to Paradigms are often diffi cult to recognize as see social behavior one way, the interactionist paradigm such because they are so implicit, assumed, taken causes us to see it differently. for granted. They seem more like “the way things SOME SOCIAL SCIENCE PARADIGMS 35

from your experience of it. As we saw in Chapter 1, however, the postmodern paradigm, which some people support, suggests that only the experience is real: The book in your hands right now is not real; only your experience of it is. Whether you think the book really exists or not refl ects the paradigm you operate within. When we recognize that we are operating within a paradigm, two benefi ts accrue. First, we are better able to understand the seemingly bizarre views and actions of others who are operating from a different paradigm. Second, at times we can profi t from stepping outside our paradigm. We Earl Babbie can see new ways of seeing and explaining things. There are many routes to understanding social life. We can’t do that as long as we mistake our paradigm for reality. Paradigms play a fundamental role in science, are” than like one possible point of view among just as they do in daily life. Thomas Kuhn (1970) many. Here’s an illustration of what I mean. drew attention to the role of paradigms in the his- Where do you stand on the issue of human tory of the natural sciences. Major scientifi c para- rights? Do you feel that individual human beings digms have included such fundamental viewpoints are sacred? Are they “endowed by their creator as Copernicus’s conception of the earth moving with certain inalienable rights,” as asserted by the around the sun (instead of the reverse), Darwin’s U.S. Declaration of Independence? Are there some theory of evolution, Newton’s mechanics, and Ein- things that no government should do to or ask of stein’s relativity. Which scientifi c theories “make its citizens? sense” depends on which paradigm scientists Consider that many other cultures today regard maintain. the Western (and particularly U.S.) commitment to Although we sometimes think of science as de- the sanctity of the individual as bizarre. Historically, veloping gradually over time, marked by important it is decidedly a minority viewpoint. For example, discoveries and inventions, Kuhn says that, histori- although many Asian countries now subscribe to cally, one paradigm would become entrenched, re- some “rights” that belong to individuals, those are sisting substantial change. Eventually, however, as balanced against the “rights” of families, organi- the shortcomings of that paradigm became obvi- zations, and society at large. When criticized for ous, a new paradigm would emerge and supplant violating human rights, Asian leaders often point the old one. Thus, the view that the sun revolves to high crime rates and social disorganization in around the earth was supplanted by the view that Western societies as the cost of what they see as the earth revolves around the sun. Kuhn’s classic our radical “cult of the individual.” book on this subject is titled, appropriately enough, No matter what our beliefs, it’s useful to recog- The Structure of Scientifi c Revolutions. nize that our views and feelings in this matter are Social scientists have developed several para- the result of the paradigm we have been socialized digms for understanding social behavior. The fate into. The sanctity of the individual is not an objec- of supplanted paradigms in the social sciences, tive fact of nature; it is a point of view, a paradigm. however, differs from what Kuhn observed in the All of us operate within many such paradigms. natural sciences. Natural scientists generally be- A traditional Western view holds that the world lieve that the succession of paradigms represents you experience has an objective reality separate progress from false views to a true ones. No mod- 36 CHAPTER 2 PARADIGMS, THEORY, AND RESEARCH ern astronomer believes that the sun revolves for a microtheoretical perspective. Such studies around the earth, for example. often come close to the realm of psychology, but In the social sciences, on the other hand, theo- whereas psychologists typically focus on what retical paradigms may gain or lose popularity, but goes on inside humans, social scientists study they’re seldom discarded. Social science para- what goes on among them. digms represent a variety of views, each of which The distinction between macro- and micro- offers insights the others lack while ignoring as- theory crosscuts the paradigms we’ll examine pects of social life that the others reveal. next. Whereas some of them, such as symbolic Each of the paradigms we’re about to examine interactionism and ethnomethodology, often work offers a different way of looking at human social best at the microlevel, others, such as the confl ict life. Each makes certain assumptions about the paradigm, can be pursued at either the micro- or nature of social reality. Ultimately, paradigms can- the macrolevel. not be true or false; as ways of looking, they can only be more or less useful. Rather than deciding Early Positivism which paradigms are true or false, try to fi nd ways they might be useful to you. As we’ll see, each When the French philosopher Auguste Comte can open up new understandings, suggest differ- (1798–1857) coined the term sociologie in 1822, ent kinds of theories, and inspire different kinds of he launched an intellectual adventure that is still research. unfolding today. Comte was arguably the fi rst to identify society as a phenomenon that can be Macrotheory and Microtheory studied scientifi cally. (Initially he wanted to label his enterprise “social physics,” but another scholar Let’s begin with a discussion that encompasses preempted that term.) many of the paradigms to be discussed. Some Prior to Comte’s time, society simply was. To theorists focus their attention on society at large the extent that people recognized different kinds of or at least on large portions of it. Topics of study societies or changes in society over time, religious for such macrotheory include the struggle among paradigms predominantly explained these differ- economic classes in a society, international rela- ences. The state of social affairs was often seen tions, and the interrelations among major institu- as a refl ection of God’s will. Alternatively, people tions in society, such as government, religion, and were challenged to create a “City of God” on earth family. Macrotheory deals with large, aggregate to replace sin and godlessness. entities of society or even whole societies. Comte separated his inquiry from religion, re- Some scholars have taken a more intimate placing religious belief with scientifi c objectivity. view of social life. Microtheory deals with issues His “positive philosophy” postulated three stages of social life at the level of individuals and small of history. A “theological stage” predominated groups. Dating behavior, jury deliberations, and throughout the world until about 1300 C.E. During student-faculty interactions provide apt subjects the next fi ve hundred years, a “metaphysical stage” replaced God with ideas such as “nature” and “natural law.” Finally, Comte felt he was launching the macrotheory A theory aimed at understanding the third stage of history, in which science would re- “big picture” of institutions, whole societies, and the place religion and metaphysics; knowledge would interactions among societies. Karl Marx’s examination of be based on observations through the fi ve senses the class struggle is an example of macrotheory. rather than on belief. Again, Comte felt that soci- microtheory A theory aimed at understanding social life at the level of individuals and their interactions. Ex- ety could be studied and understood logically and plaining how the play behavior of girls differs from that of rationally, that sociology could be as scientifi c as boys is an example of microtheory. biology or physics. SOME SOCIAL SCIENCE PARADIGMS 37

Comte’s view came to form the foundation for In theory, the chief interest being served should be subsequent development of the social sciences. In the poor people of the world or perhaps the im- his optimism for the future, he coined the term pos- poverished, Third World nations. The researcher’s itivism to describe this scientifi c approach, in con- inquiry, however, identifi ed many other interested trast to what he regarded as negative elements in parties who benefi ted: the commercial lending in- the Enlightenment. Only in recent decades has the stitutions who made loans in conjunction with the idea of positivism come under serious challenge, IMF and World Bank, and multinational corpora- as we’ll see later in this discussion. tions seeking cheap labor and markets for their goods, to name two. Chossudovsky’s analysis concluded that the interests of the banks and corpora- To explore this topic in greater depth on tions tended to take precedence over those of the the web, search for “Au guste Comte,” “positivism,” or “positivist paradigm.”* poor people, who were the intended benefi ciaries. Moreover, he found many policies were weaken- ing national economies in the Third World, as well as undermining democratic governments. Confl ict Paradigm Although applications of the confl ict paradigm Karl Marx (1818–1883) suggested that social be- often focus on class, gender, and ethnic struggles, havior could best be seen as the process of confl ict: it would be appropriate to apply it whenever differ- the attempt to dominate others and to avoid being ent groups have competing interests. For example, dominated. Marx focused primarily on the struggle we could fruitfully apply it to understanding rela- among economic classes. Specifi cally, he exam- tions among different departments in an organi- ined the way capitalism produced the oppression zation, fraternity and sorority rush weeks, or stu- of workers by the owners of industry. Marx’s inter- dent-faculty-administrative relations, to name just est in this topic did not end with analytical study: a few. He was also ideologically committed to restructur- These examples should illustrate some of the ing economic relations to end the oppression he ways you might view social life if you were taking observed. your lead from the confl ict paradigm. To explore The confl ict paradigm is not limited to economic the applicability of this paradigm, you might take a analyses. Georg Simmel (1858–1918) was particu- minute to skim through a daily newspaper or news larly interested in small-scale confl ict, in contrast magazine and identify events you could interpret to the class struggle that interested Marx. Simmel in terms of individuals and groups attempting to noted, for example, that confl icts among members dominate each other and avoid being dominated. of a tightly knit group tended to be more intense The theoretical concepts and premises of the con- than those among people who did not share feel- fl ict paradigm might help you make sense out of ings of belonging and intimacy. these events. In a more recent application of the confl ict paradigm, when Michel Chossudovsky’s (1997) analy- To explore this topic in greater depth on sis of the International Monetary Fund (IMF) and the web, search for “confl ict theory,” World Bank suggested that these two international “confl ict paradigm,” or “Karl Marx.” organizations were increasing global poverty rather than eradicating it, he directed his attention to the competing interests involved in the process. Symbolic Interactionism

As we have seen, whereas Marx chieﬂ y addressed

*Each time the Internet icon appears, you’ll be given help- macrotheoretical issues—large institutions and ful leads for searching the World Wide Web. whole societies in their evolution through the 38 CHAPTER 2 PARADIGMS, THEORY, AND RESEARCH course of history—Georg Simmel was more inter- victims, they came to identify different motivations ested in the ways individuals interacted with one among stalkers, stages in the development of a another, or the “micro” aspects of society. He be- stalking scenario, how people can recognize if they gan by examining dyads (groups of two people) are being stalked, and what they can do about it. and triads (groups of three), for example. Similarly, he wrote about “the web of group affi liations.” To explore this topic in greater depth Simmel was one of the fi rst European sociolo- on the web, search for “interactionist gists to infl uence the development of U.S. sociol- paradigm,” “interactionism,” “symbolic ogy. His focus on the nature of interactions particu- interactionism,” “George Herbert Mead,” larly infl uenced George Herbert Mead (1863–1931), “Herbert Blumer,” or “Georg Simmel.” Charles Horton Cooley (1864–1929), and others who took up the cause and developed it into a powerful paradigm for research. Ethnomethodology Cooley, for example, introduced the idea of the “primary group,” those intimate associates with Whereas some social scientifi c paradigms empha- whom we share a sense of belonging, such as our size the impact of social structure (such as norms, family, friends, and so forth. Cooley also wrote of values, and control agents) on human behavior, the “looking-glass self” we form by looking into the other paradigms do not. Harold Garfi nkel, a con- reactions of people around us. If everyone treats us temporary sociologist, takes the point of view that as beautiful, for example, we conclude that we are. people are continually creating social structure See how fundamentally this paradigm differs from through their actions and interactions—that they the society-level concerns of Marx. are, in fact, creating their realities. Thus, when Similarly, Mead emphasized the importance of you and your instructor meet to discuss your term our human ability to “take the role of the other,” paper, even though there are myriad expectations imagining how others feel and how they might be- about how you should act, the conversation will have in certain circumstances. As we gain an idea somewhat differ from any of those that have oc- of how people in general see things, we develop a curred before, and how you both act will some- sense of what Mead called the “generalized other.” what modify your future expectations. That is, Mead also felt that most interactions revolved discussing your term paper will impact your future around individuals’ reaching a common under- interactions with other professors and students. standing through language and other symbolic Given the tentativeness of reality in this view, systems, hence the term symbolic interactionism. Garfi nkel suggests that people are continuously Here’s one way you might apply this paradigm trying to make sense of the life they experience. to an examination of your own life. The next time In a way, he suggests that everyone is acting like a you meet someone new, watch how your knowl- social scientist: hence the term ethnomethodology, edge of each other unfolds through the process of or “methodology of the people.” interaction. Notice also any attempts you make to How would you go about learning about peo- manage the image you are creating in the other ple’s expectations and how they make sense out of person’s mind. their world? One technique ethnomethodologists Clearly this paradigm can lend insights into the use is to break the rules, to violate people’s ex- nature of interactions in ordinary social life, but pectations. If you try to talk to me about your term it can also help us understand unusual forms of paper, but I keep talking about football, any expec- interaction, as in the following case. Robert Em- tations you had for my behavior might come out. erson, Kerry Ferris, and Carol Brooks Gardner We might also see how you make sense out of my (1998) set out to understand the nature of “stalk- behavior. (“Maybe he’s using football as an anal- ing.” Through interviews with numerous stalking ogy for understanding social systems theory.”) SOME SOCIAL SCIENCE PARADIGMS 39

paradigm.* See the box “The Power of Paradigms” IN THE REAL WORLD for more on this topic. THE POWER OF PARADIGMS We’ll return to ethnomethodology in Chapter 10, when we discuss fi eld research. For now, let’s In this chapter, we are looking at some of the turn to a very different paradigm. social science paradigms used to organize and make sense out of social life, and we are To explore this topic in greater depth on seeing the impact those paradigms have on the web, search for “ethnomethodology” what is observed and how it is interpreted. or “Harold Garfi nkel.” The power of paradigms, however, extends well beyond the scientifi c realm. You can look almost anywhere in the world and see Structural Functionalism confl icts among religious, ethnic, political, and other cultural paradigms. Structural functionalism, sometimes also known Consider the September 11, 2001, at- as “social systems theory,” grows out of a notion tacks on the World Trade Center and Penta- introduced by Comte and others: A social entity, gon. Widely varied interpretations, refl ecting such as an organization or a whole society, can be radically different paradigms, blamed the viewed as an organism. Like organisms, a social attacks on Osama bin Laden, Saddam Hus- system is made up of parts, each of which contrib- sein, Israel, the Bush administration, God, utes to the functioning of the whole. homosexuals, and feminists. Some of these By analogy, consider the human body. Each explanations may strike you as bizarre, but component—such as the heart, lungs, kidneys, they made perfectly good sense within the skin, and brain—has a particular job to do. The worldviews of those espousing them. That’s body as a whole cannot survive unless each of the power that paradigms have in all areas these parts does its job, and none of the parts can of life. survive except as a part of the whole body. Or consider an automobile, composed of tires, steering wheel, gas tank, spark plugs, and so forth. Each of the parts serves a function for the whole; taken together, that system can get us across town. None In another example of ethnomethodology, John of the individual parts would be of much use to us Heritage and David Greatbatch (1992) examined by itself, however. the role of applause in British political speeches: The view of society as a social system, then, How did the speakers evoke applause, and what looks for the “functions” served by its various com- function did it serve (for example, to complete a ponents. We might consider a football team as a topic)? Research within the ethnomethodological social system—one in which the quarterback, run- paradigm often focuses on communication. ning backs, offensive linemen, and others have You can fi nd many interesting opportunities their own jobs to do for the team as a whole. Or, to try the ethnomethodological paradigm. For in- we could look at a symphony orchestra and exam- stance, the next time you get on an elevator, don’t ine the functions served by the conductor, the fi rst face the front (that’s the norm, or expected behav- violinist, and the other musicians. ior). Instead, just stand quietly facing the rear of the elevator. See how others react to this behavior. Just as important, notice how you feel about it. If *I am grateful to my colleague, Bernard McGrane, for this experiment. Barney also has his students eat dinner with you do this experiment a few times, you should be- their hands, watch TV without turning it on, and engage in gin to develop a feel for the ethnomethodological other strangely enlightening behavior (McGrane 1994). 40 CHAPTER 2 PARADIGMS, THEORY, AND RESEARCH

Social scientists using the structural functional Feminist Paradigms paradigm might note that the function of the po- When Ralph Linton concluded his anthropological lice, for example, is to exercise social control—en- classic, The Study of Man (1937:490), speaking of couraging people to abide by the norms of society “a store of knowledge that promises to give man and bringing to justice those who do not. We could a better life than any he has known,” no one com- just as reasonably ask what functions criminals plained that he had left women out. Linton was serve in society. Within the functionalist paradigm, using the linguistic conventions of his time; he we’d see that criminals serve as job security for the implicitly included women in all his references to police. In a related observation, Emile Durkheim men. Or did he? (1858–1917) suggested that crimes and their pun- When feminists (of both genders) fi rst began ishment provided an opportunity for the reaffi rma- questioning the use of masculine nouns and pro- tion of a society’s values. By catching and punish- nouns whenever gender was ambiguous, their ing a thief, we reaffi rm our collective respect for concerns were often viewed as petty. Many felt the private property. issue was one of women having their feelings hurt, To get a sense of the structural-functional par- their egos bruised. But be honest: When you read adigm, thumb through your college or university Linton’s words, what did you picture? An amor- catalog and assemble a list of the administrators phous, genderless human being, a hermaphrodite (such as president, deans, registrar, campus secu- at once male and female, or a male persona? rity, maintenance personnel). Figure out what each In a similar way, researchers looking at the so- of them does. To what extent do these roles relate cial world from a feminist paradigm have called to the chief functions of your college or university, attention to aspects of social life that other para- such as teaching or research? Suppose you were digms do not reveal. In fact, feminism has estab- studying some other kind of organization. How lished important theoretical paradigms for social many of the school administrators’ functions would research. In part it focuses on gender differences also be needed in, say, an insurance company? and how they relate to the rest of social organiza- In applying the functionalist paradigm to every- tion. These paradigms draw attention to the op- day life, people sometimes make the mistake of pression of women in many societies, which in thinking that functionality, stability, and integra- turn sheds light on oppression in general. tion are necessarily good, or that the functionalist Feminist paradigms not only reveal the treat- paradigm makes that assumption. However, when ment of women or the experience of oppression social researchers look for the “functions” served but often point to limitations in how other aspects by poverty, racial discrimination, or the oppres- of social life are examined and understood. Thus, sion of women, they are not justifying such things. feminist perspectives are often related to a con- Rather, they seek to understand the roles such cern for the environment, for example. As Greta things play in the larger society as a way of under- Gard suggests, standing why they persist and how they could be eliminated. The way in which women and nature have been conceptualized historically in Western intellectual tradition has resulted in devaluing whatever is associated with women, emotion, To explore this topic in greater depth animals, nature, and the body, while simultane- on the web, search for “social systems ously elevating in value those things associated theory,” “functionalism,” or “Talcott Parsons.” Parsons was the chief architect with men, reason, humans, culture, and the of the “social systems” paradigm and a mind. One task of ecofeminism has been to leading U.S. sociologist. expose these dualisms and the ways in which feminizing nature and naturalizing or animal- SOME SOCIAL SCIENCE PARADIGMS 41

izing women has served as justifi cation for the • Received knowledge: From this perspective, domination of women, animals and the earth. women feel themselves capable of taking in — (1993:5) and holding knowledge originating with external authorities. Feminist paradigms have also challenged the • Subjective knowledge: This perspective opens prevailing notions concerning consensus in so- up the possibility of personal, subjective ciety. Most descriptions of the predominant be- knowledge, including intuition. liefs, values, and norms of a society are written • Procedural knowledge: Some women feel they by people representing only portions of society. have learned the ways of gaining knowledge In the United States, for example, such analyses through objective procedures. have typically been written by middle-class white • Constructed knowledge: The authors describe men—not surprisingly, they have written about this perspective as “a position in which women the beliefs, values, and norms they themselves view all knowledge as contextual, experience share. Though George Herbert Mead spoke of the themselves as creators of knowledge, and “generalized other” that each of us becomes aware value both subjective and objective strategies of and can “take the role of,” feminist paradigms for knowing.” — (BELENKY ET AL. 1986:15) question whether such a generalized other even exists. “Constructed knowledge” is particularly interest- Further, whereas Mead used the example of ing in the context of our previous discussions. The learning to play baseball to illustrate how we learn positivistic paradigm of Comte would have a place about the generalized other, Janet Lever’s research neither for “subjective knowledge” nor for the idea suggests that understanding the experience of that truth might vary according to its context. The boys may tell us little about girls. ethnomethodological paradigm, on the other hand, would accommodate these ideas. Girls’ play and games are very different. They To try out feminist paradigms, you might want are mostly spontaneous, imaginative, and free to look into the possibility of discrimination against of structure or rules. Turn-taking activities women at your college or university. Are the top like jumprope may be played without setting administrative positions held equally by men and explicit goals. Girls have far less experience with women? How about secretarial and clerical posi- interpersonal competition. The style of their tions? Are men’s and women’s sports supported competition is indirect, rather than face to face, equally? Read through the offi cial history of your individual rather than team affi liated. Leader- school; does it include men and women equally? ship roles are either missing or randomly fi lled. (If you attend an all-male or all-female school, of — (LEVER 1986:86) course, some of these questions won’t apply.) Social researchers’ growing recognition of the intellectual differences between men and women To explore this topic in greater depth led the psychologist Mary Field Belenky and her on the web, search for “feminist para- colleagues to speak of Women’s Ways of Knowing digm,” “feminist sociology,” or “feminist (1986). In-depth interviews with 45 women led theory,” and don’t miss http://www.cddc.vt the researchers to distinguish fi ve perspectives on .edu/feminism/. knowing that challenge the view of inquiry as obvious and straightforward: As we just saw, feminist paradigms refl ect a • Silence: Some women, especially early in life, concern for the unequal treatment of women but feel themselves isolated from the world of also an epistemological recognition that men and knowledge, their lives largely determined by women overall perceive and understand society external authorities. differently. Social theories created solely by men, 42 CHAPTER 2 PARADIGMS, THEORY, AND RESEARCH which has been the norm, run the risk of an unrec- litical interests of the white majority, not by edu- ognized bias. A similar case can be made for theo- cational equality for African American students. In ries created almost exclusively by white people. his analysis, he introduced the concept of interest convergence, suggesting that laws will only be Critical Race Theory changed to benefi t African Americans if and when those changes are seen to further the interests of The roots of critical race theory are generally as- whites. Richard Delgado (2002) provides an excel- sociated with the civil rights movement of the mid- lent overview of how subsequent critical race the- 1950s and race-related legislation of the 1960s. By ory scholars have pursued Bell’s reasoning. the mid-1970s, with fears that the strides toward As a general rule, whenever you fi nd the word equality were beginning to bog down, civil rights critical in the name of a paradigm or theory, it will activists and social scientists began the codifi ca- likely refer to a nontraditional view, one that may tion of a paradigm based on race awareness and a be at odds with the prevailing paradigms of an ac- commitment to racial justice. ademic discipline and also at odds with the main- This was not the fi rst time sociologists paid at- stream structure of society. tention to the status of nonwhites in American society. Perhaps the best known African Ameri- Rational Objectivity Reconsidered can sociologist in the history of the discipline was W. E. B. DuBois, who published The Souls of Black We began with Comte’s assertion that we can Folk in 1903. Among other things, DuBois pointed study society rationally and objectively. Since his out that African Americans lived their lives through time, the growth of science, the decline of supersti- a “dual consciousness”: as Americans and as black tion, and the rise of bureaucratic structures have people. By contrast white Americans seldom re- put rationality more and more at the center of so- fl ect on being white. If you are American, white is cial life. As fundamental as rationality is to most simply assumed. If you are not white, you are seen of us, however, some contemporary scholars have as and feel like the exception. So imagine the dif- raised questions about it. ference between an African American sociologist For example, positivistic social scientists have and a white sociologist creating a theory of social sometimes erred in assuming that humans will identity. Their theories of identity would likely dif- always act rationally. I’m sure your own experi- fer in some fundamental ways, even if they were ence offers ample evidence to the contrary. Many not limiting their analyses to their own race. modern economic models also assume that people Much of the contemporary scholarship in criti- will make rational choices in the economic sector: cal race theory has to do with the role of race in They will choose the highest-paying job, pay the politics and government, often undertaken by legal lowest price, and so forth. This assumption, how- scholars as well as social scientists. Thus, for ex- ever, ignores the power of such matters as tradi- ample, Derrick Bell (1980) critiqued the Supreme tion, loyalty, and image that compete with reason Court’s landmark Brown vs. Board of Education de- in determining human behavior. cision, which struck down the “separate but equal” A more sophisticated positivism would as- system of school segregation. He suggested that sert that we can rationally understand even non- the Court was motivated by the economic and po- rational human behavior. Here’s an example. In the famous “Asch Experiment” (Asch 1958), a group of subjects is presented with a set of lines on a screen and asked to identify the two lines of equal length. interest convergence The thesis that majority group Imagine yourself a subject in such an experi- members will only support the interests of minorities when those actions also support the interests of the majority ment. You’re sitting in the front row of a classroom group. in a group of six subjects. A set of lines is pro- SOME SOCIAL SCIENCE PARADIGMS 43

More radically, we can question whether social life abides by rational principles at all. In the physical sciences, developments such as chaos theory, fuzzy logic, and complexity have suggested that we may need to rethink fundamentally the orderli- ness of physical events. The contemporary challenge to positivism, however, goes beyond the question of whether people behave rationally. In part, the criticism of X ABC positivism challenges the idea that scientists can be as objective as the scientifi c ideal assumes. Most FIGURE 2-1 The Asch Experiment. Subjects in scientists would agree that personal feelings can the Asch Experiment have a seemingly easy task: to and do infl uence the problems scientists choose to determine whether A, B, or C is the same length as study, their choice of what to observe, and the con- X. But there’s more here than meets the eye. clusions they draw from their observations. As with rationality, there is a more radical critique of objectivity. Whereas scientifi c objectivity jected on the wall in front of you (see Figure 2-1). has long stood as an unquestionable ideal, some The experimenter asks the subjects, one at a time, contemporary researchers suggest that subjectiv- to identify the line to the right (A, B, or C) that ity might actually be preferred in some situations, matches the length of line X. The correct answer as we glimpsed in the discussions of feminism and (B) is obvious to you. To your surprise, you fi nd that ethnomethodology. Let’s take a moment to return all the other subjects agree on a different answer! to the dialectic of subjectivity and objectivity. The experimenter announces that all but one of To begin with, all our experiences are inescap- the group has gotten the correct answer; that is, ably subjective. There is no way out. We can see you’ve gotten it wrong. Then a new set of lines is only through our own eyes, and anything peculiar presented, and you have the same experience. The to our eyes will shape what we see. We can hear obviously correct answer is wrong, and everyone things only the way our particular ears and brain but you seems to understand that. transmit and interpret sound waves. You and I, to As it turns out, of course, you’re the only real some extent, hear and see different realities. And subject in the experiment—all the others are work- both of us experience quite different physical “re- ing with the experimenter. The purpose is to see alities” than do bats, for example. In what to us is whether you would be swayed by public pressure total darkness, a bat “sees” things such as fl ying and go along with the incorrect answer. In one- insects by emitting a sound we humans can’t hear. third of the initial experiments, Asch found that his The refl ection of the bat’s sound creates a “sound subjects did just that. picture” precise enough for the bat to home in on Choosing an obviously wrong answer in a the moving insect and snatch it up. In a similar simple experiment is an example of nonrational vein, scientists on the planet Xandu might develop behavior. But as Asch went on to show, experi- theories of the physical world based on a sensory menters can examine the circumstances that lead apparatus that we humans can’t even imagine. more or fewer subjects to go along with the incor- Maybe they see X rays or hear colors. rect answer. For example, in subsequent studies, Despite the inescapable subjectivity of our ex- Asch varied the size of one group and the number perience, we humans seem to be wired to seek an of “dissenters” who chose the “wrong” (that is, the agreement on what is “really real,” what is objec- correct) answer. Thus, it is possible to study nonra- tively so. Objectivity is a conceptual attempt to get tional behavior rationally and scientifi cally. beyond our individual views. It is ultimately a mat- 44 CHAPTER 2 PARADIGMS, THEORY, AND RESEARCH ter of communication, as you and I attempt to fi nd crease. Because each of us can discover this inde- a common ground in our subjective experiences. pendently, we say it is objectively true. Whenever we succeed in our search, we say we From the seventeenth century through the are dealing with objective reality. This is the agree- middle of the twentieth, the belief in an objective ment reality discussed in Chapter 1. reality that people could ever more clearly see pre- Perhaps the most signifi cant studies in the dominated in science. For the most part, it was history of social science were conducted in the held not simply as a useful paradigm but as The 1930s by a Turkish American social psychologist, Truth. The term positivism generally represents the Muzafer Sherif (1935), who slyly said he wanted belief in a logically ordered, objective reality that to study “auto-kinetic effects.” To do this, he put we can come to know. This is the view challenged small groups in totally darkened rooms, save for a today by postmodernists and others. single point of light in the center of the wall in front Some say that the ideal of objectivity conceals of the participants. Sherif explained that the light as much as it reveals. As we saw earlier, much of would soon begin to move about, and the subjects what was regarded as scientifi c objectivity in years were to determine how far it was moving—a dif- past was actually an agreement primarily among fi cult task with nothing else visible as a gauge of white, middle-class, European men. Experiences length or distance. common to women, to ethnic minorities, or to Amazingly, each of the groups agreed on the the poor, for example, were not necessarily repre- distance the point of light moved about. Oddly, sented in that reality. however, the different groups of subjects arrived at The early anthropologists are now criticized for very different conclusions. Strangest of all—as you often making modern, Westernized “sense” out may have guessed—the point of light had remained of the beliefs and practices of nonliterate tribes stationary. If you stare at a fi xed point of light long around the world—sometimes portraying their enough it will seem to move (Sherif’s “auto-kinetic subjects as superstitious savages. We often call effect”). Notice, however, that each of the groups orally transmitted beliefs about the distant past agreed on a specifi c delusion. The movement of “creation myth,” whereas we speak of our own the light was real to them, but it was a reality cre- beliefs as “history.” Increasingly today, there is a ated out of nothing: a socially constructed reality. demand to fi nd the native logic by which various Whereas our subjectivity is individual, our peoples make sense out of life. search for objectivity is social. This is true in all Ultimately, we’ll never know whether there is aspects of life, not just in science. While you and I an objective reality that we experience subjectively prefer different foods, we must agree to some ex- or whether our concepts of an objective reality tent on what is fi t to eat and what is not, or else are illusory. So desperate is our need to know just there could be no restaurants, no grocery stores, what is going on, however, that both the positivists no food industry. The same argument could be and the postmodernists are sometimes drawn into made regarding every other form of consumption. the belief that their view is real and true. There is a There could be no movies or television, no sports. dual irony in this. On the one hand, the positivist’s Social scientists as well have found benefi ts in belief in the reality of the objective world must ul- the concept of objective reality. As people seek to timately be based on faith; it cannot be proved by impose order on their experience of life, they fi nd “objective” science, because that’s precisely what’s it useful to pursue this goal as a collective ven- at issue. And the postmodernists, who say nothing ture. What are the causes and cures of prejudice? is objectively so, do at least feel the absence of ob- Working together, social researchers have uncov- jective reality is really the way things are. ered some answers that hold up to intersubjective Postmodernism is often portrayed as a denial scrutiny. Whatever your subjective experience of of the possibility of social science. Because this things, for example, you can discover for yourself book has already expressed sympathy for some that as education increases, prejudice tends to de- postmodern views and concerns, a word of expla- TWO LOGICAL SYSTEMS REVISITED 45 nation may be in order. This textbook makes no next that the issues involved in theory construction assumption about the existence or absence of an are of interest and use to all social researchers, objective reality. At the same time, human beings from the positivists to the postmodernists—and all demonstrate an extensive and robust ability to those in between. establish agreements as to what’s “real.” This appears in regard to rocks and trees, as well as ghosts TWO LOGICAL SYSTEMS REVISITED and gods, and even more elusive ideas such as loyalty and treason. Whether something like “prejudice” really exists, research into its nature can take In Chapter 1, I introduced deductive and induc- place, because enough people agree that prejudice tive theory with a promise that we would return to does exist, and researchers can use techniques of them later. It’s later. inquiry to study it. This book will not require you to choose be- The Traditional Model of Science tween positivism and postmodernism. In fact, I invite you to look for value in both as you seek to Years of learning about “the scientifi c method,” understand the world that may or may not exist especially in the physical sciences, tends to create around you. For social researchers, each approach in students’ minds a particular picture of how sci- brings special strengths, and each compensates ence operates. Although this traditional model of for the weaknesses of the other. It’s often most science tells only a part of the story, it’s helpful to useful to “work both sides of the street,” tapping understand its logic. into the rich variety of theoretical perspectives There are three main elements in the traditional that can be brought to bear on the study of human model of science, typically presented in the order in social life. which they are implemented: theory, operational- The renowned British physicist, Stephen Hawk- ization, and observation. Let’s look at each in turn. ing, has elegantly described the appealing sim- Theory At this point we’re already well ac- plicity of the positivistic model and tempers his quainted with the idea of theory. According to the remarks with a recognition of the way science is traditional model of science, scientists begin with practiced. a theory, from which they derive hypotheses that According to this way of thinking, a scientifi c they can test. So, for example, as social scientists theory is a mathematical model that describes we might have a theory about the causes of juve- and codifi es the observations we make. A good nile delinquency. Let’s assume that we have arrived theory will describe a large range of phenomena at the hypothesis that delinquency is inversely re- on the basis of a few simple postulates and will lated to social class. That is, as social class goes make defi nite predictions that can be tested. If up, delinquency goes down. the predictions agree with the observations, the theory survives that test, though it can never be Operationalization To test any hypothesis, we proved to be correct. On the other hand, if the must specify the meanings of all the variables observations disagree with the predictions, one involved in it: social class and delinquency in the has to discard or modify the theory. (At least, present case. For example, delinquency might be that is what is supposed to happen. In practice, people often question the accuracy of the observations and the reliability and moral character of hypothesis A specifi ed testable expectation about em- those making the observations.) — (2001:31) pirical reality that follows from a more general proposition; more generally, an expectation about the nature of things The attempt to establish formal theories of so- derived from a theory. It is a statement of something that ciety has been closely associated with the belief in ought to be observed in the real world if the theory is a discoverable, objective reality. Even so, we’ll see correct. 46 CHAPTER 2 PARADIGMS, THEORY, AND RESEARCH specifi ed as “being arrested for a crime,” or “being In this respect, scientists are very much like convicted of a crime,” and so forth. For this par- Humpty Dumpty in Lewis Carroll’s Through the ticular study, social class might be specifi ed as fam- Looking Glass. “When I use a word,” Humpty ily income. Dumpty tells Alice, “it means just what I choose it Next, we need to specify how we’ll measure the to mean—neither more nor less.” variables we have defi ned. Operationalization “The question is,” Alice replies, “whether you literally means the operations involved in measur- can make words mean so many different things.” ing a variable. There are many ways we can pur- To which Humpty Dumpty responds, “The question sue this topic, each of which allows for different is, which is to be master—that’s all.” ways of measuring our variables. Scientists have to be “masters” of their opera- For simplicity, let’s assume we’re planning tional defi nitions for the sake of precision in ob- to conduct a survey of high school students. We servation, measurement, and communication. might operationalize delinquency in the form of the Otherwise, we would never know whether a study question “Have you ever stolen anything?” Those that contradicted ours did so only because it used who answer “yes” will be classifi ed as delinquents a different set of procedures to measure one of the in our study; those who say “no” will be classifi ed variables and thus changed the meaning of the hy- as nondelinquents. Similarly, we might operation- pothesis being tested. Of course, this also means alize family income by asking respondents, “What that to evaluate a study’s conclusions about juve- was your family’s income last year?” and providing nile delinquency and social class, or any other vari- them with a set of family income categories: under ables, we need to know how those variables were $10,000; $10,000–$24,999; $25,000–$49,999; and operationalized. $50,000 and above. The way we have operationalized the variables At this point someone might object that “de- in our imaginary study could be open to other prob- linquency” can mean something more or different lems, however. Perhaps some respondents will from having stolen something at one time or an- lie about having stolen anything, in which case other, or that social class isn’t necessarily the same we’ll misclassify them as nondelinquent. Some re- as family income. Some parents might think body spondents will not know their family incomes and piercing is a sign of delinquency even if their chil- will give mistaken answers; others may be embar- dren don’t steal, and to some “social class” might rassed and lie. We’ll consider such issues in detail include an element of prestige or community in Part 2. standing as well as how much money a family has. Our operationalized hypothesis now is that For the researcher testing a hypothesis, however, the highest incidence of delinquents will be found the meaning of variables is exactly and only what among respondents who select the lowest family the operational defi nition specifi es. income category (under $10,000), a lower percentage of delinquents will be found in the $10,000– $24,999 category, still fewer delinquents will be found in the $25,000–$49,999 category, and the operationalization One step beyond conceptual- lowest percentage of delinquents will be found in ization. Operationalization is the process of developing the $50,000 and above category. operational defi nitions, or specifying the exact operations involved in measuring a variable. operational defi nition The concrete and specifi c Observation The fi nal step in the traditional defi nition of something in terms of the operations by model of science involves actual observation, which observations are to be categorized. The operational defi nition of “earning an A in this course” might be looking at the world and making measurements “correctly answering at least 90 percent of the fi nal exam of what is seen. Having developed theoretical clar- questions.” ity and expectations and having created a strategy TWO LOGICAL SYSTEMS REVISITED 47 for looking, all that remains is to look at the way things actually appear. Let’s suppose our survey produced the follow- Idea/interest ing data: “What causes X?”

Percent Delinquent

Under $10,000 20 THEORETICAL UNDERSTANDING $10,000–$24,999 15 Y causes X $25,000–$49,999 10 $50,000 and above 5 HYPOTHESIS X = f(Y ) Theoretical expectation Observations producing such data would conﬁ rm our hypothesis. But suppose our ﬁ ndings were as Operationalization follows: x = f(y ) Testable hypothesis

? Percent x = f(y ) Observation Delinquent (hypothesis testing)

Under $10,000 15 FIGURE 2-2 The Traditional Image of Science. The deductive model of scientiﬁ c inquiry begins $10,000–$24,999 15 with a sometimes vague or general question, which $25,000–$49,999 15 is subjected to a process of speciﬁ cation, resulting $50,000 and above 15 in hypotheses that can be tested through empirical observations.

These fi ndings would disconfi rm our hypothesis regarding family income and delinquency. Discon- fi rmability is an essential quality in any hypothesis. development of a theoretical understanding. The In other words, if there is no chance that our hy- theoretical considerations result in a hypothesis, pothesis will be disconfi rmed, it hasn’t said any- or an expectation about the way things ought to thing meaningful. be in the world if the theoretical expectations are For example, the hypothesis that “juvenile de- correct. The notation X f(Y) is a conventional linquents” commit more crimes than do “non- way of saying that X (for example, delinquency) delinquents” cannot possibly be disconfi rmed, is a function of (is in some way caused by) Y (for because criminal behavior is intrinsic to the no- example, poverty). At that level, however, X and Y tion of delinquency. Even if we recognize that have general rather than specifi c meanings. some young people commit crimes without being In the operationalization process, general con- caught and labeled as delinquents, they couldn’t cepts are translated into specifi c indicators and threaten our hypothesis, because our observations procedures. The lowercase x, for example, is a would lead us to conclude they were law-abiding concrete indicator of capital X. Thus, whereas X nondelinquents. is theoretical, x is something we could actually Figure 2-2 provides a schematic diagram of observe. If X is the theoretical variable “juvenile the traditional model of scientifi c inquiry. In it we delinquency,” x could be measured as “self- see the researcher beginning with an interest in reported crimes” on a survey. If Y stands for “pov- something or an idea about it. Next comes the erty” in general, y might stand for “family income.” 48 CHAPTER 2 PARADIGMS, THEORY, AND RESEARCH

ISSUES AND INSIGHTS HINTS FOR STATING HYPOTHESES

by Riley E. Dunlap to word a hypothesis that states a relationship Department of Sociology, Oklahoma State University between the two variables (the one that fi lls in the “blank” and “attitudes toward women’s lib- A hypothesis is the basic statement that is tested eration”). You need to do so in a precise manner in research. Typically a hypothesis states a reso that you can determine clearly whether the lationship between two variables. (Although it hypothesis is supported or not when you exam- is possible to use more than two variables, you ine the results (in this case, most likely the re- should stick to two for now.) Because a hypoth- sults of a survey). esis makes a prediction about the relationship The key is to word the hypothesis carefully so between the two variables, it must be testable that the prediction it makes is quite clear to you so you can determine if the prediction is right or as well as others. If you use age, note that say- wrong when you examine the results obtained ing “Age is related to attitudes toward women’s in your study. A hypothesis must be stated in liberation” does not say precisely how you think an unambiguous manner to be clearly testable. the two are related (in fact, the only way this hy- What follows are suggestions for developing pothesis could be falsifi ed is if you fail to fi nd a testable hypotheses. statistically signifi cant relationship of any type Assume you have an interest in trying to between age and attitudes toward women’s lib- predict some phenomenon such as “attitudes eration). In this case a couple of steps are neces- toward women’s liberation” and that you can sary. You have two options: measure such attitudes on a continuum ranging from “opposed to women’s liberation” to 1. “Age is related to attitudes toward women’s “neutral” to “supportive of women’s liberation.” liberation, with younger adults being more Also assume that, lacking a theory, you’ll rely supportive than older adults.” (Or, you could on “hunches” to come up with variables that state the opposite, if you believed older might be related to attitudes toward women’s people are likely to be more supportive.) liberation. 2. “Age is negatively related to support for In a sense, you can think of hypothesis con- women’s liberation.” Note here that I specify struction as a case of fi lling in the blank: “______“support” for women’s liberation (SWL) and is related to attitudes toward women’s libera- then predict a negative relationship—that tion.” Your job is to think of a variable that might is, as age goes up, I predict that SWL will plausibly be related to such attitudes, and then go down.

This operationalization process results in the Deduction and Induction Compared formation of a testable hypothesis: for example, increasing family income reduces self-reported theft. The traditional model of science uses deductive Observations aimed at fi nding out whether this is logic (see Chapter 1). In this section, we’re going true are part of what is typically called hypothesis to see how deductive logic fi ts into social scien- testing. (See the box “Hints for Stating Hypotheses” tifi c research and contrast it with inductive logic. for more on this.) W. I. B. Beveridge, a philosopher of science, describes these two systems of logic as follows: TWO LOGICAL SYSTEMS REVISITED 49

In this hypothesis, note that both of the vari- which makes the identical prediction. (Of ables (age, the independent variable or likely course, you could also make the opposite “cause,” and SWL, the dependent variable or prediction, that men are more supportive likely “effect”) range from low to high. This fea- than women are, if you wished.) ture of the two variables is what allows you to 4. Equally legitimate would be “Women are use “negatively” (or “positively”) to describe the more likely to support women’s libera- relationship. tion than are men.” (Note the need for the Notice what happens if you hypothesize a second “are,” or you could be construed as relationship between gender and SWL. Since hypothesizing that women support women’s gender is a nominal variable (as you’ll learn in liberation more than they support men—not Chapter 5) it does not range from low to high— quite the same idea.) people are either male or female (the two attributes of the variable gender). Consequently, The above examples hypothesized relation- you must be careful in stating the hypothesis ships between a “characteristic” (age or gender) unambiguously: and an “orientation” (attitudes toward women’s liberation). Because the causal order is pretty 1. “Gender is positively (or negatively) related clear (obviously age and gender come before to SWL” is not an adequate hypothesis, attitudes, and are less alterable), we could because it doesn’t specify how you ex- state the hypotheses as I’ve done, and every- pect gender to be related to SWL—that is, one would assume that we were stating causal whether you think men or women will be hypotheses. more supportive of women’s liberation. Finally, you may run across references to the 2. It is tempting to say something like “Women null hypothesis, especially in statistics. Such a are positively related to SWL,” but this re- hypothesis predicts no relationship (technically, ally doesn’t work because female is only an no statistically signiﬁ cant relationship) between attribute, not a full variable (gender is the the two variables, and it is always implicit in variable). testing hypotheses. Basically, if you have hy- 3. ”Gender is related to SWL, with women be- pothesized a positive (or negative) relationship, ing more supportive than men” would be my you are hoping that the results will allow you recommendation. Or, you could say, “with to reject the null hypothesis and verify your hy- men being less supportive than women,” pothesized relationship.

Logicians distinguish between inductive rea- starts from some general law and applies it to a soning (from particular instances to general particular instance. — (1950:113) principles, from facts to theories) and deductive reasoning (from the general to the particular, applying a theory to a particular case). In null hypothesis In connection with hypothesis testing induction one starts from observed data and and tests of statistical signiﬁ cance, the hypothesis that develops a generalization which explains the suggests there is no relationship among the variables un- relationships between the objects observed. der study. You may conclude that the variables are related On the other hand, in deductive reasoning one after having statistically rejected the null hypothesis. 50 CHAPTER 2 PARADIGMS, THEORY, AND RESEARCH

The classical illustration of deductive logic is the esis were correct, women should appear more re- familiar syllogism “All men are mortal; Socrates is ligious than men. Once the survey data had been a man; therefore Socrates is mortal.” This syllo- collected and analyzed, our expectation about gen- gism presents a theory and its operationalization. der and religion was clearly confi rmed. On three To prove it, you might then perform an empirical separate measures of religious involvement—ritual test of Socrates’ mortality. That is essentially the (for example, church attendance), organizational approach discussed as the traditional model. (for example, belonging to church organizations), Using inductive logic, you might begin by noting and intellectual (for example, reading church pub- that Socrates is mortal and by observing several lications)—women were more religious than men. other men as well. You might then note that all the On our overall measure, women scored 50 percent observed men were mortals, thereby arriving at higher than men. the tentative conclusion that all men are mortal. In another test of the Comfort Hypothesis, we Let’s consider an actual research project as a reasoned that in a youth-oriented society, old peo- vehicle for comparing the roles of deductive and ple would be more deprived of secular gratifi cation inductive logic in theory and research. than the young would be. Once again, the data confi rmed our expectation. The oldest parishioners A Case Illustration Years ago, Charles Glock, were more religious than were the middle-aged, Benjamin Ringer, and I (1967) set out to discover who were more religious than were the young what caused differing levels of church involvement adults. among U.S. Episcopalians. Several theoretical or Social class—measured by education and in- quasi-theoretical positions suggested possible an- come—afforded another test, which was suc- swers. I’ll focus on only one here—what we came cessful. Those with low social status were more to call the “Comfort Hypothesis.” involved in the church than were those with high In part, we took our lead from the Christian social status. injunction to care for “the halt, the lame, and the The hypothesis was even confi rmed in a test that blind” and those who are “weary and heavy laden.” went against everyone’s commonsense expecta- At the same time, ironically, we noted the Marxist tions. Despite church posters showing worshipful assertion that religion is “the opium for the peo- young families and bearing the slogan “The Family ple.” Given both, it made sense to expect the fol- That Prays Together Stays Together,” the Comfort lowing, which was our hypothesis: “Parishioners Hypothesis suggested that parishioners who were whose life situations most deprive them of satis- married and had children—the clear U.S. ideal at faction and fulfi llment in the secular society turn that time—would enjoy secular gratifi cation in that to the church for comfort and substitute rewards” regard. As a consequence, they should be less re- (Glock, Ringer, and Babbie 1967:107–8). ligious than those who lacked one or both family Having framed this general hypothesis, we set components. Thus, we hypothesized that parish- about testing it. Were those deprived of satisfac- ioners who were both single and childless should tion in the secular society in fact more religious be the most religious; those with either spouse or than those who received more satisfaction from child should be somewhat less religious; and those the secular society? To answer this, we needed to married with children—representing the ideal pic- distinguish who was deprived. Our questionnaire tured on all those posters—should be least reli- included items that intended to indicate whether gious of all. That’s exactly what we found. parishioners were relatively deprived or gratifi ed in Finally, the Comfort Hypothesis suggested that secular society. the various kinds of secular deprivation should be To start, we reasoned that men enjoy more cumulative: Those with all the characteristics as- status than do women in our generally male- sociated with deprivation should be the most reli- dominated society. It followed that, if our hypoth- gious; those with none should be the least. When TWO LOGICAL SYSTEMS REVISITED 51 we combined the four individual measures of de- That question launched a process of reasoning privation into a composite measure (see Chapter 6 about what the several variables had in common, for methods of doing this), the theoretical expecta- aside from their impact on religiosity. (The com- tion was exactly confi rmed. Comparing the two ex- posite index was originally labeled “Predisposition tremes, we found that single, childless, old, lower- to Church Involvement.”) Eventually we saw that class female parishioners scored more than three each of the four variables also refl ected differen- times as high on the measure of church involve- tial status in the secular society, and then we had ment than did young, married, upper-class fathers. the thought that perhaps the issue of comfort was This research example clearly illustrates the involved. Thus, the inductive process had moved logic of the deductive model. Beginning with gen- from concrete observations to a general theoreti- eral, theoretical expectations about the impact of cal explanation. See the box “Church Involvement” social deprivation on church involvement, we de- for more on the Glock study. rived concrete hypotheses linking specifi c measurable variables, such as age and church attendance. A Graphic Contrast As the preceding case il- We then analyzed the actual empirical data to de- lustration shows, theory and research can be ac- termine whether empirical reality supported the complished both inductively and deductively. Fig- deductive expectations. Sounds good, right? ure 2-3 shows a graphic comparison of the deduc- Alas, I’ve been fi bbing a little bit just now. To tive and inductive methods. In both cases, we are tell the truth, although we began with an interest in discovering what caused variations in church involvement among Episcopalians, we didn’t actually begin with a Comfort Hypothesis, or any IN THE REAL WORLD other hypothesis for that matter. (In the interest CHURCH INVOLVEMENT of further honesty, Glock and Ringer initiated the study, and I joined it years after the data had been collected.) Although many church leaders believe that A questionnaire was designed to collect infor- the function of the churches is to shape mation from parishioners that might shed some members’ behavior in the community, the light on why some participated in the church more Glock study suggests that church involve- than others, but questionnaire construction was ment primarily refl ects a need for comfort not guided by any precise, deductive theory. Once by those who are denied gratifi cation in the the data were collected, the task of explaining secular society. How might churches apply differences in religiosity began with an analysis these research results? of variables that have a wide impact on people’s On the one hand, churches might adjust lives, including gender, age, social class, and family their programs to the needs that were draw- status. Each of these four variables was found to ing their members to participation. They relate strongly to church involvement in the ways might study members’ needs for gratifi cation already described. Rather than being good news, and develop more programs to satisfy them. this presented a dilemma. On the other hand, churches could seek Glock recalls discussing his fi ndings with col- to remind members that the purpose of par- leagues over lunch at the Columbia faculty club. ticipation is to learn and practice proper be- Once he had displayed the tables illustrating the havior. Following that strategy would proba- impact of the variables and their cumulative effect, bly change participation patterns, attracting a colleague asked, “What does it all mean, Char- new participants to the church while driving lie?” Glock was at a loss. Why were those variables away others. so strongly related to church involvement? 52 CHAPTER 2 PARADIGMS, THEORY, AND RESEARCH

1. Deductive Method 2. Inductive Method

a. Hypothesis a. Observations 100 100

50 50 Grades Grades

0 0 0 10 20 30 40 0 10 20 30 40 Hours studying Hours studying

b. Observations b. Finding a pattern 100 100

50 50 Grades Grades

0 0 0 10 20 30 40 0 10 20 30 40 Hours studying Hours studying

c. Accept or reject hypothesis? c. Tentative conclusion 100 100

50 50 Grades Grades

0 0 0 10 20 30 40 0 10 20 30 40 Hours studying Hours studying FIGURE 2-3 Deductive and Inductive Methods. Both deduction and induction are legitimate and valuable approaches to understanding. Deduction begins with an expected pattern that is tested against observations, whereas induction begins with observations and seeks to ﬁ nd a pattern within them.

interested in the relationship between the number of these abilities should be increased by exposure of hours spent studying for an exam and the grade to the information before the exam. In this fash- earned on that exam. Using the deductive method, ion, we would arrive at a hypothesis suggesting a we would begin by examining the matter logi- positive relationship between the number of hours cally. Doing well on an exam refl ects a student’s spent studying and the grade earned on the exam. ability to recall and manipulate information. Both That is, we expect grades to increase as the hours TWO LOGICAL SYSTEMS REVISITED 53 of studying increase. If increased hours produced decreased grades, we would call it a negative relationship. The hypothesis is represented by the line in part 1a of Figure 2-3. Our next step would be to make observations relevant to testing our hypothesis. The shaded area in part 1b of the fi gure represents perhaps hundreds of observations of different students, noting how many hours they studied and what grades they got. Finally, in part 1c, we compare the hypothesis and Text not available due to copyright restrictions the observations. Because observations in the real world seldom if ever match our expectations perfectly, we must decide whether the match is close enough to confi rm the hypothesis. In other words, can we conclude that the hypothesis describes the general pattern that exists, granting some variations in real life? Now let’s address the same research question by using the inductive method. We would begin—as in part 2a of the fi gure—with a set of observations. Curious about the relationship be- this process as a circle, which is presented in a tween hours spent studying and grades earned, modifi ed form in Figure 2-4. we might simply arrange to collect some relevant When Emile Durkheim ([1897] 1951) pored over data. Then we’d look for a pattern that best repre- table after table of offi cial statistics on suicide rates sented or summarized our observations. In part 2b in different areas, he was struck by the fact that of the fi gure, the pattern is shown as a curved line Protestant countries consistently had higher sui- running through the center of the curving mass of cide rates than did Catholic ones. Why should that points. be the case? His initial observations led him to cre- The pattern found among the points in this ate a theory of religion, social integration, anomie, case suggests that with 1 to 15 hours of study- and suicide. His theoretical explanations led to fur- ing, each additional hour generally produces a ther hypotheses and further observations. higher grade on the exam. With 15 to about 25 In summary, the scientifi c norm of logical rea- hours, however, more study seems to lower the soning provides a two-way bridge between theory grade slightly. Studying more than 25 hours, on and research. Scientifi c inquiry in practice typically the other hand, results in a return to the initial pat- involves an alternation between deduction and tern: More hours produce higher grades. Using the induction. During the deductive phase, we reason inductive method, then, we end up with a tenta- toward observations; during the inductive phase, tive conclusion about the pattern of the relation- we reason from observations. Both deduction and ship between the two variables. The conclusion is induction offer routes to the construction of so- tentative because the observations we have made cial theories, and both logic and observation are cannot be taken as a test of the pattern—those essential. observations are the source of the pattern we’ve Although both inductive and deductive meth- created. ods are valid in scientifi c inquiry, individuals may In actual practice, theory and research interact feel more comfortable with one approach than through a never-ending alternation of deduction the other. Consider this exchange in Sir Arthur and induction. Walter Wallace (1971) represents Conan Doyle’s “A Scandal in Bohemia,” as Sherlock 54 CHAPTER 2 PARADIGMS, THEORY, AND RESEARCH

Holmes answers Dr. Watson’s inquiry (Doyle [1891] stand out as important determinants of attitudes 1892:13): about abortion. Findings such as these will be quite useful to you in creating your own theory. “What do you imagine that it means?” Throughout this process, introspection is help- “I have no data yet. It is a capital mistake ful. If you can look at your own personal pro- to theorise before one has data. Insensibly one cesses—including reactions, fears, and prejudices begins to twist facts to suit theories, instead of you aren’t especially proud of—you may be able theories to suit facts.” to gain important insights into human behavior in Some social scientists would rally behind this general. inductive position (see especially the discussion of grounded theory in Chapter 10), whereas oth- Constructing Your Theory ers would take a deductive stance. Most, however, concede the legitimacy of both. With this under- Although theory construction is not a lockstep af- standing of the deductive and inductive links be- fair, the following list of elements in theory con- tween theory and research, let’s delve a little more struction should organize the activity for you. deeply into how theories are constructed using 1. Specify the topic. these two different approaches. 2. Specify the range of phenomena your theory addresses. Will your theory apply to all of DEDUCTIVE THEORY CONSTRUCTION human social life, will it apply only to U.S. citizens, only to young people, or what? 3. Identify and specify your major concepts and To see what is involved in deductive theory con- variables. struction and hypothesis testing, let’s imagine that 4. Find out what is known (or what propositions you are going to construct a deductive theory. How have been demonstrated) about the relation- would you go about it? ships among those variables. 5. Reason logically from those propositions to the Getting Started speciﬁ c topic you are examining.

The fi rst step in deductive theory construction is We’ve already discussed items (1) through (3), to pick a topic that interests you. It can be broad, so let’s focus now on (4) and (5). As you identify such as “What’s the structure of society?” or nar- the relevant concepts and discover what has al- rower, as in “Why do people support or oppose a ready been learned about them, you can begin to woman’s right to an abortion?” Whatever the topic, create a propositional structure that explains the it should be something you’re interested in under- topic under study. For the most part, social scien- standing and explaining. tists have not created formal, propositional theo- Once you’ve picked your topic, you then un- ries. Still, looking at a well-reasoned example is dertake an inventory of what is known or thought useful. Let’s look now at an example of how these about it. In part, this means writing down your building blocks fi t together in actual deductive the- own observations and ideas about it. Beyond that, ory construction and empirical research. you need to learn what other scholars have said about it. You can do this by talking to other people An Example of Deductive and by reading what others have written about it. Theory: Distributive Justice Appendix A provides guidelines for using the library—you’ll probably spend a lot of time there. A topic of central interest to scholars is the con- Your preliminary research will probably uncover cept of distributive justice, people’s perception of consistent patterns discovered by prior scholars. whether they’re being treated fairly by life, whether For example, religious and political variables will they’re getting “their share.” Guillermina Jasso de- DEDUCTIVE THEORY CONSTRUCTION 55 scribes the theory of distributive justice more for- cause some of the goods to be examined are con- mally, as follows: crete and commonly measured (such as money), whereas others are less tangible (such as respect). The theory provides a mathematical description The former kind, she says, will be measured con- of the process whereby individuals, refl ecting on ventionally, whereas the latter will be measured their holdings of the goods they value (such as “by the individual’s relative rank . . . within a spe- beauty, intelligence, or wealth), compare them- cially selected comparison group.” The theory will selves to others, experiencing a fundamental provide a formula for making that measurement instantaneous magnitude of the justice evalu- (Jasso 1988:13). ation ( J), which captures their sense of being Jasso continues in this fashion to introduce ad- fairly or unfairly treated in the distributions of ditional elements, weaving them into mathemati- natural and social goods. — (1988:11) cal formulas for deriving predictions about the Notice that Jasso has assigned a letter to her key workings of distributive justice in a variety of social variable: J will stand for distributive justice. She settings. Here is a sampling of where her theoriz- does this to support her intention of stating her ing takes her (1988:14–15). theory in mathematical formulas. Though theo- • Other things [being] the same, a person will ries are often expressed mathematically, we’ll not prefer to steal from a fellow group member delve too deeply into that practice here. rather than from an outsider. Jasso indicates that there are three kinds of pos- • The preference to steal from a fellow group tulates in her theory. “The fi rst makes explicit the member is more pronounced in poor groups fundamental axiom which represents the substan- than in rich groups. tive point of departure for the theory.” She elabo- • In the case of theft, informants arise only in rates as follows: cross-group theft, in which case they are mem- The theory begins with the received Axiom of bers of the thief’s group. Comparison, which formalizes the long-held • Persons who arrive a week late at summer view that a wide class of phenomena, includ- camp or for freshman year of college are more ing happiness, self-esteem, and the sense of likely to become friends of persons who play distributive justice, may be understood as the games of chance than of persons who play product of a comparison process. — (1988:11) games of skill. • A society becomes more vulnerable to defi cit Thus, our sense of whether we are receiving a spending as its wealth increases. “fair” share of the good things of life comes from • Societies in which population growth is wel- comparing ourselves with others. If this seems ob- comed must be societies in which the set of vious to you, that’s good. Remember, axioms are valued goods includes at least one quantity- the taken-for-granted beginnings of theory. good, such as wealth. Jasso continues to do the groundwork for her theory. First, she indicates that our sense of dis- Jasso’s theory leads to many other propositions, tributive justice is a function of “Actual Holdings but this sampling should provide a good sense of (A)” and “Comparison Holdings (C )” of some good. where deductive theorizing can take you. To get Let’s consider money. My sense of justice in this a feeling for how she reasons her way to these regard is a function of how much I actually have, propositions, let’s look briefl y at the logic involved compared with how much others have. By specify- in two of the propositions that relate to theft within ing the two components of the comparison, Jasso and outside one’s group. can use them as variables in her theory. Jasso then offers a “measurement rule” that • Other things [being] the same, a person will further specifi es how the two variables, A and C, prefer to steal from a fellow group member will be conceptualized. This step is needed be- rather than from an outsider. 56 CHAPTER 2 PARADIGMS, THEORY, AND RESEARCH

Beginning with the assumption that thieves thief’s comparison group. We’ve just seen how my want to maximize their relative wealth, ask your- relative standing was decreased by your theft. How self whether that goal would be best served by about other members of the other group? Each of stealing from those you compare yourself with or them would actually profi t from the theft, because from outsiders. In each case, stealing will increase you would have reduced the total with which they your Actual Holdings, but what about your Com- compare themselves. Hence, the theory of distribu- parison Holdings? tive justice predicts that informants arise from the A moment’s thought should suggest that steal- thief’s comparison group. ing from people in your comparison group will This brief and selective peek into Jasso’s deriva- lower their holdings, further increasing your relations should give you some sense of the enterprise tive wealth. To simplify, imagine there are only of deductive theory. Realize, of course, that the two people in your comparison group: you and theory guarantees none of the given predictions. I. Suppose we each have $100. If you steal $50 The role of research is to test each of them em- from someone outside our group, you will have pirically to determine whether what makes sense increased your relative wealth by 50 percent com- (logic) occurs in practice (observation). pared with me: $150 versus $100. But if you steal There are two important elements in science, $50 from me, you will have increased your relative then: logical integrity and empirical verifi cation. wealth 200 percent: $150 to my $50. Your goal is Both are essential to scientifi c inquiry and discov- best served by stealing from within the compari- ery. Logic alone is not enough, but on the other son group. hand, the mere observation and collection of empirical facts does not provide understanding—the • In the case of theft, informants arise only in telephone directory, for example, is not a scien- cross-group theft, in which case they are mem- tifi c conclusion. Observation, however, can be the bers of the thief’s group. springboard for the construction of a social scien- Can you see why it would make sense for infor- tifi c theory, as we shall now see in the case of in- mants (1) to arise only in the case of cross-group ductive theory. theft and (2) to come from the thief’s comparison group? This proposition again depends on the fun- INDUCTIVE THEORY CONSTRUCTION damental assumption that everyone wants to increase his or her relative standing. Suppose you and I are in the same comparison group, but this time Quite often, social scientists begin constructing a the group contains additional people. If you steal theory through the inductive method by fi rst ob- from someone else within our comparison group, serving aspects of social life and then seeking my relative standing in the group does not change. to discover patterns that may point to relatively Although your wealth has increased, the average universal principles. Barney Glaser and Anselm wealth in the group remains the same (because Strauss (1967) coined the term grounded theory in someone else’s wealth has decreased by the same reference to this method. amount). I have no incentive to inform on you. Field research—the direct observation of events If you steal from someone outside our compari- in progress—is frequently used to develop theories son group, your nefarious income increases the through observation (see Chapter 10). A long and total wealth in our group, so my own wealth rela- rich anthropological tradition has seen this method tive to that total is diminished. Because my rela- used to good advantage. tive wealth has suffered, I’m more likely to bring Among social scientists of the twentieth cen- an end to your stealing. tury, no one was more adept at seeing the pat- This last deduction also begins to explain why terns of human behavior through observation than informants are more likely to arrive from within the Erving Goffman: INDUCTIVE THEORY CONSTRUCTION 57

A game such as chess generates a habitable favor of marijuana, on the other hand, often spoke universe for those who can follow it, a plane of the search for new values: Marijuana smokers, of being, a cast of characters with a seemingly they said, were people who had seen through the unlimited number of different situations and hypocrisy of middle-class values. acts through which to realize their natures and David Takeuchi’s (1974) analysis of the data destinies. Yet much of this is reducible to a small gathered from University of Hawaii students, how- set of interdependent rules and practices. If the ever, did not support any of the explanations being meaningfulness of everyday activity is similarly offered. Those who reported smoking marijuana dependent on a closed, fi nite set of rules, then had essentially the same academic records as explication of them would give one a powerful those who didn’t smoke it, and both groups were means of analyzing social life. — (1974:5) equally involved in traditional “school spirit” activities. Both groups seemed to feel equally well inte- In a variety of research efforts, Goffman uncov- grated into campus life. ered the rules of such diverse behaviors as living There were differences, however: in a mental institution (1961) and managing the “spoiled identity” of disfi guration (1963). In each 1. Women were less likely than men to smoke case, Goffman observed the phenomenon in depth marijuana. and teased out the rules governing behavior. Goff- 2. Asian students (a large proportion of the stu- man’s research provides an excellent example of dent body) were less likely than non-Asians to qualitative fi eld research as a source of grounded smoke marijuana. theory. 3. Students living at home were less likely than Our earlier discussion of the Comfort Hypothe- those living in apartments to smoke marijuana. sis and church involvement shows that qualitative As in the case of religiosity, the three variables fi eld research is not the only method of observa- independently affected the likelihood of a student’s tion appropriate to the development of inductive smoking marijuana. About 10 percent of the Asian theory. Here’s another detailed example to illus- women living at home had smoked marijuana, trate further the construction of inductive theory compared with about 80 percent of the non-Asian using quantitative methods. men living in apartments. And, as in the religiosity study, the researchers discovered a powerful pat- An Example of Inductive Theory: tern of drug use before they had an explanation for Why Do People Smoke Marijuana? that pattern. In this instance, the explanation took a pecu- During the 1960s and 1970s, marijuana use on liar turn. Instead of explaining why some students U.S. college campuses was a subject of consider- smoked marijuana, the researchers explained why able discussion in the popular press. Some people some didn’t. Assuming that all students had some were troubled by marijuana’s popularity; others motivation for trying drugs, the researchers sug- welcomed it. What interests us here is why some gested that students differed in the degree of “so- students smoked marijuana and others didn’t. cial constraints” preventing them from following A survey of students at the University of Hawaii through on that motivation. (Takeuchi 1974) provided data to answer that U.S. society is, on the whole, more permissive question. with men than with women when it comes to de- At the time of the study, people were offering viant behavior. Consider, for example, a group of numerous explanations for drug use. Those who men getting drunk and boisterous. We tend to dis- opposed drug use, for example, often suggested miss such behavior with references to “camarade- that marijuana smokers were academic failures rie” and “having a good time,” whereas a group of trying to avoid the rigors of college life. Those in women behaving similarly would probably be re- 58 CHAPTER 2 PARADIGMS, THEORY, AND RESEARCH garded with at least some disapproval. We have an neither case do theory and research really interact idiom, “Boys will be boys,” but no comparable id- for the purpose of developing new explanations. iom for girls. The researchers reasoned, therefore, Some studies make no use of theory at all, aim- that women would have more to lose by smoking ing specifi cally, for example, at an ethnographic marijuana than would men. Being female, then, description of a particular social situation, such as provided a constraint against smoking marijuana. an anthropological account of food and dress in a Students living at home had obvious constraints particular society. against smoking marijuana, compared with stu- As you read social research reports, however, dents living on their own. Quite aside from differ- you’ll often fi nd that the authors are conscious of ences in opportunity, those living at home were the implications of their research for social theo- seen as being more dependent on their parents— ries and vice versa. Here are a few examples to il- hence more vulnerable to additional punishment lustrate this point. for breaking the law. When W. Lawrence Neuman (1998) set out to Finally, the Asian subculture in Hawaii has examine the problem of monopolies (the “trust traditionally placed a higher premium on obedi- problem”) in U.S. history, he saw the relevance of ence to the law than have other subcultures. As theories about how social movements transform such, Asian students would have more to lose if society (“state transformation”). He became con- they were caught violating the law by smoking vinced, however, that existing theories were inad- marijuana. equate for the task before him: Overall, then, a “social constraints” theory was offered as the explanation for observed differences State transformation theory links social move- in the likelihood of smoking marijuana. The more ments to state policy formation processes by fo- constraints a student had, the less likely he or she cussing on the role of cultural meaning in orga- would be to smoke marijuana. It bears repeating nized political struggles. Despite a resemblance that the researchers had no thoughts about such among concepts and concerns, constructionist a theory when their research began. The theory ideas found in the social problems, social move- came from an examination of the data. ments, and symbolic politics literatures have not been incorporated into the theory. In this paper, I draw on these three literatures to enhance THE LINKS BETWEEN state transformation theory. — (1998:315) THEORY AND RESEARCH Having thus modifi ed state transformation theory, Neuman had a theoretical tool that could Throughout this chapter, we have seen various guide his inquiry and analysis into the political aspects of the links between theory and research maneuverings related to monopolies beginning in in social scientifi c inquiry. In the deductive model, the 1880s and continuing until World War I. Thus, research is used to test theories. In the inductive theory served as a resource for research and at the model, theories are developed from the analysis of same time was modifi ed by it. research data. This section looks more closely into In a somewhat similar study, Alemseghed the ways theory and research are related in actual Kebede and J. David Knottnerus (1998) set out to social scientifi c inquiry. investigate the rise of Rastafarianism in the Ca- Although we have discussed two idealized logi- ribbean. However, they felt that recent theories cal models for linking theory and research, social on social movements had become too positivistic scientifi c inquiries have developed a great many in focusing on the mobilization of resources. Re- variations on these themes. Sometimes theoreti- source mobilization theory, they felt, downplays cal issues are introduced merely as a background for empirical analyses. Other studies cite selected the motivation, perceptions, and behavior of empirical data to bolster theoretical arguments. In movement participants . . . and concentrates THE IMPORTANCE OF THEORY IN THE “REAL WORLD” 59

instead on the whys and hows of mobilization. search, from psychology to social reform. Think, Typically theoretical and research problems for instance, how someone could make use of Da- include: How do emerging movement organiza- vid Takeuchi’s research on marijuana use. tions seek to mobilize and routinize the fl ow of But how does theory work in such applications? resources and how does the existing political In some minds, theoretical and practical matters apparatus affect the organization of resources? are virtual opposites. Social scientists committed — (1998:500) to the use of science know differently, however. Lester Ward, the fi rst president of the American To study Rastafarianism more appropriately, Sociological Association, was committed to the ap- the researchers felt the need to include several plication of social research in practice, or the use concepts from contemporary social psychology. of that research toward specifi c ends. Ward distin- In particular, they sought models to use in dealing guished pure and applied sociology as follows: with problems of meaning and collective thought. Frederika E. Schmitt and Patricia Yancey Martin Just as pure sociology aims to answer the ques- (1999) were particularly interested in discovering tions What, Why, and How, so applied sociol- what produced successful rape crisis centers and ogy aims to answer the question What for. The how such centers dealt with the organizational former deals with facts, causes, and principles, and political environments within which they oper- the latter with the object, end, or purpose. ated. The researchers found theoretical constructs — (1906:5) appropriate to their inquiry: No matter how practical and/or idealistic your This case study of unobtrusive mobilizing by aims, a theoretical understanding of the terrain [the] Southern California Rape Crisis Center uses may spell the difference between success and fail- archival, observational, and interview data to ure. As Ward saw it, “Reform may be defi ned as explore how a feminist organization worked to the desirable alteration of social structures. Any change police, schools, prosecutor[s], and some attempt to do this must be based on a full knowl- state and national organizations from 1974 to edge of the nature of such structures, otherwise its 1994. Mansbridge’s concept of street theory and failure is certain” (1906:4). Katzenstein’s concepts of unobtrusive mobiliza- Suppose you were concerned about poverty in tion and discursive politics guide the analysis. the United States. The sociologist Herbert Gans — (1999:364) (1971) suggests that understanding the functions that poverty serves for people who are not poor In summary, there is no simple recipe for con- is vital. For example, the persistence of poverty ducting social science research. It is far more means there will always be people willing to do open-ended than the traditional view of science the jobs no one else wants to do—and they’ll work suggests. Ultimately, science depends on two cat- for very little money. The availability of cheap la- egories of activity: logic and observation. As you’ll bor provides many affordable comforts for the see throughout this book, they can be fi t together nonpoor. in many patterns. By the same token, poverty provides many job opportunities for social workers, unemployment THE IMPORTANCE OF offi ce workers, police, and so forth. If poverty were THEORY IN THE “REAL WORLD” to disappear, what would happen to social work colleges, for example? I don’t mean to suggest that people conspire to At this point you may be saying, “Sure, theory and keep the poor in their place or that social work- research are OK, but what do they have to do with ers secretly hope for poverty to persist. Nor do I the real world?” As we’ll see later in this book, want to suggest that the dark cloud of poverty has there are many practical applications of social re- a silver lining. I merely want you to understand the 60 CHAPTER 2 PARADIGMS, THEORY, AND RESEARCH point made by Ward, Gans, and many other soci- WHAT DO YOU THINK? REVISITED ologists: If you want to change society, you need to understand how it operates. As William White (1997) argues, “Theory helps create questions, As we’ve seen, many different paradigms shapes our research designs, helps us anticipate have been suggested for the study of so- outcomes, helps us design interventions.” ciety. The opening “What Do You Think?” box asked which one was true. You should RESEARCH ETHICS AND THEORY see by now that the answer is “None of the above.” However, none of the paradigms is false, either. In this chapter, we have seen how the paradigms By their nature, paradigms are neither and theories that guide research inevitably im- true or false. They are merely different ways pact what is observed and how it is interpreted. of looking and of seeking explanations. Choosing a particular paradigm or theory does not Thus, they may be judged as useful or not guarantee a particular research conclusion, but it useful in a particular situation, but not true will affect what you look for and what you ignore. or false. Whether you choose a functionalist or a confl ict Imagine that you and some friends are paradigm to organize your research on police- in a totally darkened room. Each of you community relations will make a big difference. has a fl ashlight. When you yourself turn on This choice can produce certain ethical issues. your fl ashlight, you create a partial picture Choosing a theoretical orientation for the purpose of what’s in the room, whereby some things of encouraging a particular conclusion, for ex- are revealed but others remain concealed. ample, would be regarded as unethical as a gen- Now imagine your friends taking turns turn- eral matter. However, when researchers intend ing on their fl ashlights. Every person’s fl ash- to bring about social change through their work, light presents a different picture of what’s in they will likely choose a theoretical orientation the room, revealing part but not all of it. appropriate to that intention. Let’s say you’re con- Paradigms are like the fl ashlights in this cerned about the treatment of homeless people by gripping tale. Each offers a particular point the police in your community. You might very well of view that may or may not be useful in a organize your research in terms of interactionist given circumstance. None reveals the full or confl ict paradigms and theories that would re- picture, or the “truth.” veal any instances of mistreatment that may occur. The danger lies in the bias this might cause in your research. Two factors counter this potential bias. First, as we’ll see in the remainder of the book, social Second, the collective nature of social research science research techniques—the various meth- offers further protection. As indicated in Chapter ods of observation and analysis—place a damper 1, peer review in which researchers evaluate each on our simply seeing what we expect. Even if you other’s efforts will point to instances of shoddy expect to fi nd the police mistreating the homeless or biased research. Moreover, with several re- and use theories and methods that will reveal such searchers studying the same phenomenon, per- mistreatment, you will not observe that which isn’t haps using different paradigms, theories, and meth- there—if you apply those theories and methods ods, the risk of biased research fi ndings is further appropriately. reduced. RESEARCH ETHICS AND THEORY 61

Main Points ❏ Like feminist paradigms, critical race theory both examines the disadvantaged position of a Introduction social group (African Americans) and offers a ❏ Theories seek to provide logical explanations. different vantage point from which to view and understand society. Some Social Science Paradigms ❏ Some contemporary theorists and researchers ❏ A paradigm is a fundamental model or scheme have challenged the long-standing belief in an that organizes our view of something. objective reality that abides by rational rules. ❏ Social scientists use a variety of paradigms to They point out that it is possible to agree on an organize how they understand and inquire into “intersubjective” reality. social life. Two Logical Systems Revisited ❏ A distinction between types of theories that ❏ In the traditional image of science, scientists cuts across various paradigms is macrotheory proceed from theory to operationalization to (theories about large-scale features of society) observation. But this image is not an accurate versus microtheory (theories about smaller picture of how scientifi c research is actually units or features of society). done. ❏ The positivistic paradigm assumes we can sci- ❏ Social scientifi c theory and research are linked entifi cally discover the rules governing social through two logical methods: Deduction life. involves the derivation of expectations or hy- ❏ The confl ict paradigm focuses on the attempt potheses from theories. Induction involves the of one person or group to dominate others and development of generalizations from specifi c to avoid being dominated. observations. ❏ The symbolic interactionist paradigm exam- ❏ Science is a process involving an alternation of ines how shared meanings and social pat- deduction and induction. terns are developed in the course of social interactions. Deductive Theory Construction ❏ Ethnomethodology focuses on the ways people ❏ Guillermina Jasso’s theory of distributive justice make sense out of life in the process of living illustrates how formal reasoning can lead to a it, as though each were a researcher engaged variety of theoretical expectations that can be in an inquiry. tested by observation. ❏ The structural functionalist (or social systems) Inductive Theory Construction paradigm seeks to discover what functions ❏ David Takeuchi’s study of factors infl uenc- the many elements of society perform for the ing marijuana smoking among University of whole system—for example, the functions of Hawaii students illustrates how collecting mothers, labor unions, and radio talk observations can lead to generalizations and shows. an explanatory theory. ❏ Feminist paradigms, in addition to drawing attention to the oppression of women in most The Links between Theory and Research societies, highlight how previous images of ❏ In practice, there are many possible links be- social reality have often come from and rein- tween theory and research and many ways of forced the experiences of men. going about social inquiry. 62 CHAPTER 2 PARADIGMS, THEORY, AND RESEARCH

❏ Using theories to understand how society Online Study Resources works is key to offering practical solutions to society’s problems.

Research Ethics and Theory Go to ❏ Researchers must guard against letting their choice of theory or paradigms bias their re- http://sociology.wadsworth.com/babbie_basics4e search results. and click on ThomsonNow for access to this ❏ The collective nature of social research powerful online study tool. You will get a per- offers protection against biased research sonalized study plan based on your responses to fi ndings. a diagnostic pretest. Once you have mastered the material with the help of interactive learning tools, you can take a posttest to confi rm that you Key Terms are ready to move on to the next chapter. hypothesis null hypothesis Website for interest convergence operational defi nition The Basics of Social Research macrotheory operationalization 4th edition microtheory paradigm

At the book companion website (http://sociology Review Questions .wadsworth.com/babbie_basics4e) you will fi nd many resources in addition to ThomsonNow to 1. Consider the possible relationship between aid you in studying for your exams. For example, education and prejudice (mentioned in Chap- you will fi nd Tutorial Quizzes with feedback, Inter- ter 1). How might that relationship be exam- net Exercises, Flashcards, and Chapter Tutorials, as ined through (a) deductive and (b) inductive well as Extended Projects, InfoTrac College Edition methods? search terms, Social Research in Cyberspace, GSS 2. Select a social problem that concerns you, Data, Web Links, and primers for using various such as war, pollution, overpopulation, preju- data analysis software such as SPSS and NVivo. dice, or poverty, Then use one of the paradigms in this chapter to address that problem. Additional Readings What would be the main variables involved in the study of that problem, including variables Berger, Joseph, Morris Zelditch, Jr., and Bo Anderson, that may cause it or hold the key to its eds. 1989. Sociological Theories in Progress. Thousand solution? Oaks, CA: Sage. Several authors develop parts of a theory of social interaction, many of which focus 3. What, in your own words, is the difference on how we create expectations for each other’s between a paradigm and a theory? behavior. 4. You have been hired to evaluate how well a Delgado, Richard, and Jean Stefancic. 2001. Critical particular health maintenance organization Race Theory: An Introduction. New York: New York (HMO) serves the needs of its clients. How University Press. This is a good introduction to this might you implement this study using each of alternative paradigm for viewing racial and ethnic the following: (1) the interactionist paradigm, issues, presenting key concepts and fi ndings. Denzin, Norman K., and Yvonna S. Lincoln, eds. 1994. (2) the social systems or functionalist para- Handbook of Qualitative Research. Thousand Oaks, digm, (3) the confl ict paradigm? CA: Sage. Various authors discuss the process of ADDITIONAL READINGS 63

qualitative research from the perspective of various approach to society and to the craft of sociology. The paradigms, showing how they infl uence the nature book is useful for both students and faculty. of inquiry. The editors also critique positivism from a Reinharz, Shulamit. 1992. Feminist Methods in Social Re- postmodern perspective. search. New York: Oxford University Press. This book DeVault, Marjorie L. 1999. Liberating Method: Feminism explores several social research techniques (such and Social Research. Philadelphia: Temple University as interviewing, experiments, and content analysis) Press. This book elaborates on some of the methods from a feminist perspective. associated with the feminist paradigm and is com- Ritzer, George. 1988. Sociological Theory. New York: mitted to both rigorous inquiry and the use of social Knopf. This is an excellent overview of the major research to combat oppression. theoretical traditions in sociology. Harvey, David. 1990. The Condition of Postmodernity: An Rosenau, Pauline Marie. 1992. Post-Modernism and the Enquiry into the Origins of Cultural Change. Cam- Social Sciences: Insights, Inroads, and Intrusions. Princ- bridge, MA: Blackwell. Here’s a wide-ranging analysis eton, NJ: Princeton University Press. Regarded as a of the history and meaning of postmodernism, linking modern classic, this book examines some of the main political and historical factors to experiences of time variations on postmodernism and shows how they and space. have impacted different realms of society. Kuhn, Thomas. 1970. The Structure of Scientifi c Revolu- Sprague, Joey. 1997. “Holy Men and Big Guns: The tions. Chicago: University of Chicago Press. In this Can[n]on in Social Theory.” Gender and Society 11 exciting and innovative recasting of the nature of (1): 88–107. This is an excellent analysis of the ways scientifi c development, Kuhn disputes the notion of in which conventional social theory misses aspects gradual change and modifi cation in science, arguing of society that might be revealed in a feminist instead that established “paradigms” tend to persist examination. until the weight of contradictory evidence brings their Turner, Jonathan H., ed. 1989. Theory Building in Sociol- rejection and replacement by new paradigms. This ogy: Assessing Theoretical Cumulation. Thousand short book is at once stimulating and informative. Oaks, CA: Sage. This collection of essays on socio- Lofl and, John, and Lyn H. Lofl and. 1995. Analyzing logical theory construction focuses specifi cally on Social Settings: A Guide to Qualitative Observation the question posed by Turner’s introductory chapter, and Analysis. 3rd ed. Belmont, CA: Wadsworth. An “Can Sociology Be a Cumulative Science?” excellent text on how to conduct qualitative inquiry Turner, Stephen Park, and Jonathan H. Turner. 1990. with an eye toward discovering the rules of social life. The Impossible Science: An Institutional Analysis of Includes a critique of postmodernism. American Sociology. Thousand Oaks, CA: Sage. Two McGrane, Bernard. 1994. The Un-TV and 10 mph Car: authors bring two very different points of view to the Experiments in Personal Freedom and Everyday Life. history of U.S. sociologists’ attempt to establish a Fort Bragg, CA: The Small Press. Some excellent and science of society. imaginative examples of an ethnomethodological THE ETHICS AND POLITICS 31 HUMANOF SOCIAL INQUIRY RESEARCH AND SCIENCE

Image not available due to copyright restrictions Photo credit Hazel Hanken/Stock Boston

What You’ll Learn in thisThis ChapterChapter

SocialWe’ll research examine takes the wayplace people in a social learn context. about their Researchers world and must the mistakestherefore theytake makeinto account along the many way. ethical We’ll andalso political begin to considerations see what makes alongside sience different scientiﬁ cfrom ones

otherin designing ways of and knowing executing things. their research. Often, however, clear-cut answers to 2S thorny ethical and political issues are hard to come by. 1S N L 64 In this chapter . . . WHAT DO YOU THINK?

Introduction Whenever research Ethical Issues in Social Research ethics are discussed, Voluntary Participation the Nazi medical ex- No Harm to the Participants Image not available due to copyright restrictions periments of World Anonymity and Conﬁ dentiality War II often surface Deception as the most hid- Analysis and Reporting eous breach of eth- Institutional Review Boards ical standards in his- Professional Codes of Ethics tory. Civilian and military prisoners in Nazi concentration camps were subjected to Two Ethical Controversies freezing; malaria; mustard gas; sulfanil- Trouble in the Tearoom amide; bone, muscle, and nerve transplants; Observing Human Obedience jaundice; sterilization; spotted fever; vari- The Politics of Social Research ous poisons; and phosphorus burns, among Objectivity and Ideology other tortures. Many died, and others were Politics with a Little “p” permanently maimed. All suffered tremen- Politics in Perspective dous physical and psychological pain. Some have argued, however, that the real breach of ethics did not lie in the suffering or the INTRODUCTION deaths per se. What could possibly be a worse ethical breach than that?

To present a realistic and useful introduction to do- See the “What Do You Think? Revisited” box ing social research, this book must consider four toward the end of the chapter. main constraints on research projects: scientifi c, administrative, ethical, and political. Most of the book focuses on scientifi c and administrative constraints. We’ll see that the logic of science suggests certain research procedures, but we’ll also see too impractical to use, others are either ethically that some scientifi cally “perfect” study designs are prohibitive or politically diffi cult or impossible. not administratively feasible, because they would Here’s a story to illustrate what I mean. be too expensive or take too long to execute. Several years ago, I was invited to sit in on a Throughout the book, therefore, we’ll deal with planning session to design a study of legal educa- workable compromises. tion in California. The joint project was to be con- Before we get to the scientifi c and administra- ducted by a university research center and the state tive constraints on research, it’s useful to explore bar association. The purpose of the project was to the other important considerations in doing re- improve legal education by learning which aspects search in the real world: ethics and politics, which of the law school experience were related to suc- this chapter covers. Just as certain procedures are cess on the bar exam. Essentially, the plan was to 65 66 CHAPTER 3 THE ETHICS AND POLITICS OF SOCIAL RESEARCH prepare a questionnaire that would get detailed that we interview people about their sex lives and information about the law school experiences of then publish what they said in the local newspa- individuals. People would be required to answer per. But, as ethical as you are, you’ll totally miss the questionnaire when they took the bar exam. the ethical issues in some other situations—we By analyzing how people with different kinds of all do. law school experiences did on the bar exam, we The fi rst half of this chapter deals with the eth- could fi nd out what sorts of things worked and ics of social research. In part, it presents some of what didn’t. The fi ndings of the research could be the broadly agreed-on norms describing what’s made available to law schools, and ultimately legal ethical in research and what’s not. More important education could be improved. than simply knowing the guidelines, however, is The exciting thing about collaborating with the becoming sensitized to the ethical component in bar association was that all the normally irritating research so that you’ll look for it whenever you logistical hassles would be handled. There would plan a study. Even when the ethical aspects of a be no problem getting permission to administer situation are debatable, you should know that questionnaires in conjunction with the exam, for there’s something to argue about. It’s worth noting example, and the problem of nonresponse could that many professions operate under ethical con- be eliminated altogether. straints and that these constraints differ from one I left the meeting excited about the prospects profession to another. Thus, the ethics of priests, for the study. When I told a colleague about it, I physicians, lawyers, reporters, and television pro- glowed about the absolute handling of the nonre- ducers differ. In this chapter, we’ll look only at the sponse problem. Her immediate comment turned ethical principles that govern social research. everything around completely. “That’s unethical. Political considerations in research are also There’s no law requiring the questionnaire, and subtle, ambiguous, and arguable. Notice that the participation in research has to be voluntary.” The law school example involves politics as well as study wasn’t done. ethics. Although social researchers have an ethical In retelling this story, it’s obvious to me that re- norm that participation in research should be vol- quiring participation would have been inappropri- untary, this norm clearly grows out of U.S. political ate. You may have seen this even before I told you norms protecting civil liberties. In some nations, about my colleague’s comment. I still feel a little the proposed study would not have been consid- embarrassed over the matter, but I have a specifi c ered unethical at all. purpose in telling this story about myself. In the second half of this chapter, we’ll look All of us consider ourselves to be ethical—not at social research projects that were crushed or perfect perhaps, but as ethical as anyone else and nearly crushed by political considerations. As with perhaps more so than most. The problem in social ethical concerns, there is often no “correct” take research, as probably in life, is that ethical consid- on a given situation. People of goodwill disagree. erations are not always apparent to us. As a result, I won’t try to give you a party line about what is we often plunge into things without seeing ethi- and what is not politically acceptable. As with eth- cal issues that may be apparent to others and may ics, the point is to become sensitive to the political even be obvious to us when pointed out. When I dimension of social research. reported back to the others in the planning group, for example, no one disagreed with the inappropri- ETHICAL ISSUES IN SOCIAL RESEARCH ateness of requiring participation. Everyone was a bit embarrassed about not having seen it. Any of us can immediately see that a study that In most dictionaries and in common usage, eth- requires small children to be tortured is unethical. ics is typically associated with morality, and both I know you’d speak out immediately if I suggested deal with matters of right and wrong. But what is ETHICAL ISSUES IN SOCIAL RESEARCH 67

Anyone involved in social scientiﬁ c research, then, needs to be aware of the general agreements shared by researchers about what is proper and improper in the conduct of scientiﬁ c inquiry. This section summarizes some of the most important ethical agreements that prevail in social research.

Voluntary Participation

Often, though not always, social research represents an intrusion into people’s lives. The interviewer’s knock on the door or the arrival of a questionnaire in the mail signals the beginning of an activity that the respondent has not requested and that may require signifi cant time and energy. Participation in a social experiment disrupts the subject’s regular activities. Social research, moreover, often requires that people reveal personal information about themselves—information that may be unknown to their friends and associates. Further, social research often requires that such information be revealed to strangers. Other professionals, such as physicians and lawyers, also ask for such information. Earl Babbie Their requests may be justifi ed, however, by their As in other aspects of life, there are limits to what aims: They need the information in order to serve is acceptable in social research. the personal interests of the respondent. Social researchers can seldom make this claim. Like medical scientists, they can only argue that the research right and what is wrong? What is the source of the effort may ultimately help all humanity. distinction? For individuals the sources vary. They A major tenet of medical research ethics is that may include religions, political ideologies, or the experimental participation must be voluntary. The pragmatic observation of what seems to work and same norm applies to social research. No one what doesn’t. should be forced to participate. This norm is far Webster’s New World Dictionary is typical among easier to accept in theory than to apply in practice, dictionaries in defi ning ethical as “conforming to however. the standards of conduct of a given profession or Again, medical research provides a useful par- group.” Although this defi nition may frustrate those allel. Many experimental drugs used to be tested in search of moral absolutes, what we regard as on prisoners. In the most rigorously ethical cases, morality and ethics in day-to-day life is a matter of the prisoners were told the nature and the possi- agreement among members of a group. And, not ble dangers of the experiment, they were told that surprisingly, different groups have agreed on dif- participation was completely voluntary, and they ferent codes of conduct. Part of living successfully were further instructed that they could expect no in a particular society is knowing what that society special rewards—such as early parole—for partici- considers ethical and unethical. The same holds pation. Even under these conditions, it was often true for the social research community. clear that volunteers were motivated by the be- 68 CHAPTER 3 THE ETHICS AND POLITICS OF SOCIAL RESEARCH lief that they would personally benefi t from their No Harm to the Participants cooperation. Social research should never injure the people be- When the instructor in an introductory sociology ing studied, regardless of whether they volunteer class asks students to fi ll out a questionnaire that for the study. Perhaps the clearest instance of this he or she hopes to analyze and publish, students norm in practice concerns the revealing of infor- should always be told that their participation in the mation that would embarrass subjects or endan- survey is completely voluntary. Even so, most stu- ger their home life, friendships, jobs, and so forth. dents will fear that nonparticipation will somehow We’ll discuss this aspect of the norm more fully in affect their grade. The instructor should therefore a moment. be especially sensitive to such implications and Because subjects can be harmed psychologi- make special provisions to eliminate them. For cally in the course of a social research study, the example, the instructor could insure anonymity researcher must look for the subtlest dangers and by leaving the room while the questionnaires are guard against them. Quite often, research subjects being completed. Or, students could be asked to are asked to reveal deviant behavior, attitudes they return the questionnaires by mail or to drop them feel are unpopular, or personal characteristics that in a box near the door just before the next course may seem demeaning, such as low income, the meeting. receipt of welfare payments, and the like. Reveal- As essential as it is, this norm of voluntary par- ing such information usually makes subjects feel at ticipation goes directly against several scientifi c least uncomfortable. concerns. In the most general terms, the scientifi c Social research projects may also force par- goal of generalizability is threatened if experi- ticipants to face aspects of themselves that they mental subjects or survey respondents are all the don’t normally consider. This can happen even kinds of people who willingly participate in such when the information is not revealed directly to the things. Because this orientation probably refl ects researcher. In retrospect, a certain past behavior more general personality traits, the results of the may appear unjust or immoral. The project, then, research might not be generalizable to all kinds of can cause continuing personal agony for the sub- people. Most clearly, in the case of a descriptive ject. If the study concerns codes of ethical conduct, survey, a researcher cannot generalize the sample for example, the subject may begin questioning his survey fi ndings to an entire population unless a or her own morality, and that personal concern substantial majority of the scientifi cally selected may last long after the research has been reported. sample actually participates—the willing respon- For instance, probing questions can injure a fragile dents and the somewhat unwilling. self-esteem. As you’ll see in Chapter 10, fi eld research has Clearly, just about any research you might its own ethical dilemmas in this regard. Very of- conduct runs the risk of injuring other people in ten, the researcher cannot even reveal that a some way. Further, some study designs make study is being done, for fear that that revelation such injuries more likely than do others. If a par- might signifi cantly affect the social processes be- ticular research procedure seems likely to produce ing studied. Clearly, the subjects of study in such unpleasant effects for subjects—asking survey cases do not receive the opportunity to volunteer respondents to report deviant behavior, for ex- or refuse to participate. In cases where you feel ul- ample—the researcher should have the fi rmest of timately justifi ed in violating the norm of voluntary scientifi c grounds for doing it. If the research de- participation, observing the other ethical norms sign is essential and also likely to be unpleasant of scientifi c research, such as bringing no harm for subjects, you’ll fi nd yourself in an ethical neth- to the people under study, becomes all the more erworld and may face some personal agonizing. important. ETHICAL ISSUES IN SOCIAL RESEARCH 69

Although it has little value in itself, agonizing may to his congregation, saying that many of them were be a healthy sign that you’ve become sensitive to atheists and were going to hell. Even though he the problem. could not identify the people who gave particular Increasingly, the ethical norms of voluntary responses, it seems certain that the survey ended participation and no harm to participants have up harming many respondents. become formalized in the concept of informed Like voluntary participation, avoiding harm to consent. This norm means that subjects must people is easy in theory but often diffi cult in prac- base their voluntary participation in research proj- tice. Sensitivity to the issue and experience with ects on a full understanding of the possible risks its applications, however, should improve the re- involved. In a medical experiment, for example, searcher’s tact in delicate areas of research. prospective subjects will be presented with a dis- In recent years, social researchers have gained cussion of the experiment and all the possible support for abiding by this norm. Federal and other risks to themselves. They will be required to sign funding agencies typically require an independent a statement indicating that they are aware of the evaluation of the treatment of human subjects for risks and that they choose to participate anyway. research proposals, and most universities now Although the value of such a procedure is obvious have human-subject committees to serve this when subjects will be injected with drugs designed evaluative function. Although sometimes trouble- to produce physical effects, for example, it’s hardly some and inappropriately applied, such require- appropriate when a participant-observer rushes ments not only guard against unethical research to the scene of urban rioting to study deviant be- but can also reveal ethical issues overlooked by havior. The researcher in this latter case is not ex- even the most scrupulous researchers. cused from the norm of not bringing harm to those observed, but gaining informed consent is not the Anonymity and Confi dentiality means to achieving that end. Another, often unrecognized source of poten- The clearest concern in guarding subjects’ inter- tial harm to subjects lies in the analysis and report- ests and well-being is the protection of their iden- ing of data. Every now and then, research subjects tity, especially in survey research. If revealing their read the books published about the studies they survey responses would injure them in any way, participated in. Reasonably sophisticated subjects adherence to this norm becomes all the more can locate themselves in the various indexes and important. Two techniques—anonymity and con- tables. Having done so, they may fi nd themselves fi dentiality—assist researchers in this regard, al- characterized—though not identifi ed by name—as though people often confuse the two. bigoted, unpatriotic, irreligious, and so forth. At the very least, such characterizations will likely trou- Anonymity A research project guarantees ano- ble them and threaten their self-images. Yet the nymity when the researcher—not just the people whole purpose of the research project may be to who read about the research—cannot identify a explain why some people are prejudiced and others are not. In one survey of churchwomen (Babbie 1967), ministers in a sample of churches were asked to informed consent A norm in which subjects base their voluntary participation in research projects on a full distribute questionnaires to a specifi ed sample of understanding of the possible risks involved. members, collect them, and return them to the re- anonymity Anonymity is guaranteed in a research search offi ce. One of these ministers read through project when neither the researchers nor the readers of the questionnaires from his sample before return- the fi ndings can identify a given response with a given ing them, and then he delivered a scathing sermon respondent. 70 CHAPTER 3 THE ETHICS AND POLITICS OF SOCIAL RESEARCH given response with a given respondent. This im- easy norm to follow, because, for example, the plies that a typical interview survey respondent courts have not recognized social research data as can never be considered anonymous, because the kind of “privileged communication” accepted an interviewer collects the information from an in the case of priests and attorneys. identifi able respondent. An example of anonymity This unprotected guarantee of confi dentiality is a mail survey in which nothing can identify the produced a near disaster in 1991. Two years ear- respondent when the questionnaire arrives at the lier, the Exxon Valdez supertanker had run aground research offi ce. near the port of Valdez in Alaska, spilling ten mil- As we’ll see in Chapter 9, assuring anonymity lion gallons of oil into the bay. The economic and makes keeping track of who has or hasn’t returned environmental damage was widely reported. the questionnaires diffi cult. Despite this problem, Less attention was given to the psychological you may be advised to pay the necessary price in and sociological damage suffered by residents of some situations. In one study of drug use among the area. There were anecdotal reports of increased university students, I decided that I specifi cally did alcoholism, family violence, and other secondary not want to know the identity of respondents. I felt consequences of the disruptions caused by the oil that honestly assuring anonymity would increase spill. Eventually, 22 communities in Prince William the likelihood and accuracy of responses. Also, I Sound and the Gulf of Alaska sued Exxon for the did not want to be in the position of being asked by economic, social, and psychological damages suf- authorities for the names of drug offenders. In the fered by their residents. few instances in which respondents volunteered To determine the amount of damage done, the their names, such information was immediately communities commissioned a San Diego research obliterated on the questionnaires. fi rm to undertake a household survey asking residents very personal questions about increased problems in their families. The sample of residents Confi dentiality A research project guarantees were asked to reveal painful and embarrassing in- confi dentiality when the researcher can identify formation, under the guarantee of absolute con- a given person’s responses but essentially prom- fi dentiality. Ultimately, the results of the survey ises not to do so publicly. In an interview survey, confi rmed that a variety of personal and family for example, the researchers could make public the problems had increased substantially following income reported by a given respondent, but they the oil spill. assure the respondent that this will not be done. When Exxon learned that survey data would be Whenever a research project is confi dential presented to document the suffering, they took an rather than anonymous, it is the researcher’s re- unusual step: They asked the court to subpoena sponsibility to make that fact clear to the respon- the survey questionnaires. The court granted the dent. Moreover, researchers should never use the defendant’s request and ordered the researchers to term anonymous to mean confi dential. turn them over—with all identifying information. It With few exceptions (such as surveys of pub- appeared that Exxon’s intention was to call survey lic fi gures who agree to have their responses pub- respondents to the stand and cross-examine them lished), the information respondents give must be regarding answers they had given interviewers kept at least confi dential. This is not always an under the guarantee of confi dentiality. Moreover, many of the respondents were Native Americans, whose cultural norms made such public revelations all the more painful. confi dentiality A research project guarantees confi dentiality when the researcher can identify a given Happily, the Exxon Valdez case was settled be- person’s responses but promises not to do so publicly. fore the court decided whether it would force sur- ETHICAL ISSUES IN SOCIAL RESEARCH 71 vey respondents to testify in open court. Unhap- tion as soon as it’s no longer necessary. In a sur- pily, the potential for disaster remains. vey, for example, all names and addresses should The seriousness of this issue is not limited be removed from questionnaires and replaced by to established research fi rms. Rik Scarce was a identifi cation numbers. An identifi cation fi le should graduate student at Washington State University be created that links numbers to names to permit when he undertook participant observation among the later correction of missing or contradictory in- animal-rights activists. In 1990 he published a formation, but this fi le should not be available ex- book based on his research, entitled Ecowarriors: cept for legitimate purposes. Understanding the Radical Environmental Move- Similarly, in an interview survey you may need ment. In 1993 Scarce was called before a grand to identify respondents initially so that you can jury and asked to identify the activists he had stud- recontact them to verify that the interview was ied. In keeping with the norm of confi dentiality, conducted and perhaps to get information that the young researcher refused to answer the grand was missing in the original interview. As soon as jury’s questions and spent 159 days in the Spokane you’ve verifi ed an interview and assured yourself County jail. He reports, that you don’t need any further information from the respondent, however, you can safely remove Although I answered many of the prosecutor’s all identifying information from the interview questions, on 32 occasions I refused to answer, booklet. Often, interview booklets are printed so saying, “Your question calls for information that that the fi rst page contains all the identifi ers—it can I have only by virtue of a confi dential disclosure be torn off once the respondent’s identifi cation is given to me in the course of my research activi- no longer needed. J. Steven Picou (1996a, 1996b) ties. I cannot answer the question without actu- points out that even removing identifi ers from data ally breaching a confi dential communication. fi les does not always suffi ciently protect respon- Consequently, I decline to answer the question dent confi dentiality, a lesson he learned during under my ethical obligations as a member of the nearly a year in federal court. A careful examina- American Sociological Association and pursuant tion of all the responses of a particular respondent to any privilege that may extend to journalists, sometimes allows others to deduce that person’s researchers, and writers under the First Amend- identity. Imagine, for example, that someone said ment.” — (SCARCE 1999:982) they were a former employee of a particular com- At the time of his grand jury appearance and pany. Knowing the person’s gender, age, ethnicity, his incarceration, Scarce felt his ethical stand was and other characteristics could make it possible for strongly supported by the ASA Code of Ethics, and the company to identify that person. the association fi led a friend of the court brief on Even if you intend to remove all identifying his behalf. In 1997 the ASA revised its Code of Eth- information, suppose you have not yet done so. ics and, while still upholding the norm of confi den- What do you do when the police or a judge or- tiality, warned researchers to inform themselves ders you to provide the responses given by your regarding laws and rules that may limit their ability research subjects? See the box “Confi dentiality” for to promise confi dentiality to research subjects. more on this issue. You can use several techniques to guard against In this, as all other aspects of research ethics, such dangers and ensure better performance professional researchers avoid settling for mere on the guarantee of confi dentiality. To begin, in- rote compliance with established ethical rules. terviewers and others with access to respondent Rather, they continually ask what would be most identifi cations should be trained in their ethical appropriate in protecting the interests of those be- responsibilities. Beyond training, the most funda- ing studied. Here’s the way Penny Edgell Becker mental technique is to remove identifying informa- addressed the issue of confi dentiality in connec- 72 CHAPTER 3 THE ETHICS AND POLITICS OF SOCIAL RESEARCH

researchers knows the community’s real name IN THE REAL WORLD and the rest of the world is kept in the dark. . . . CONFIDENTIALITY In all cases individual identities are disguised, except for Jack Finney, the Lutheran pastor, who Upholding confi dentiality is a real issue for gave permission to be identifi ed. “City Baptist” is practicing social researchers, even though a pseudonym used at the request of the church’s they sometimes disagree about how to pro- leadership. The leaders of Good Shepherd tect subjects. Harry O’Neill, the vice chair of Lutheran Church (GSLC) gave permission to use the Roper Organization, for example, sug- the church’s real name. — (1998:452) gested that the best solution is to avoid altogether the ability to identify respondents Deception with their responses: We’ve seen that the handling of subjects’ identi- So how is this accomplished? Quite ties is an important ethical consideration. Han- simply by not having any respondent- dling your own identity as a researcher can also be identifi able information available for the tricky. Sometimes it’s useful and even necessary to court to request. In my initial contact identify yourself as a researcher to those you want with a lawyer-client, I make it unmis- to study. You’d have to be an experienced con artist takably clear that, once the survey is to get people to participate in a laboratory experi- completed and validated, all respondent- ment or complete a lengthy questionnaire without identifi able information will be removed letting on that you were conducting research. and destroyed immediately. Everything Even when you must conceal your research else connected with the survey— identity, you need to consider the following. Be- completed questionnaires, data tapes, cause deceiving people is unethical, deception methodology, names of interviewers within social research needs to be justifi ed by com- and supervisors—of course will be made pelling scientifi c or administrative concerns. Even available. — (O’NEILL 1992:4) then, the justifi cation will be arguable. Board Chairman Burns Roper (1992:5) dis- Sometimes researchers admit that they’re do- agreed, saying that such procedures might ing research but fudge about why they’re doing it raise questions about the validity of the re- or for whom. Suppose a public welfare agency has search methods. Instead, Roper said that he asked you to conduct a study of living standards felt he must be prepared to go to jail if neces- among aid recipients. Even if the agency is looking sary. (He noted that Vice Chair O’Neill prom- for ways of improving conditions, the recipient- ised to visit him in that event.) subjects will likely fear a witch-hunt for “cheaters.” They might be tempted, therefore, to give answers that make them seem more destitute than they really are. Unless they provide truthful answers, however, the study will not produce accurate data tion with a qualitative research project studying that will contribute to an effective improvement of religious life in a community: living conditions. What do you do? Following the lead of several recent studies, I One solution would be to tell subjects that identify the real name of the community, Oak you’re conducting the study as part of a university Park, rather than reducing the complexity of research program—concealing your affi liation with the community’s history to a few underlying the welfare agency. Doing that improves the sci- dimensions or [creating] an “insider/outsider” entifi c quality of the study, but it raises a serious dynamic where some small group of fellow ethical issue. ETHICAL ISSUES IN SOCIAL RESEARCH 73

Lying about research purposes is common in them at the outset. This must be done with care, laboratory experiments. Although it’s diffi cult to however, making sure the subjects aren’t left with conceal that you’re conducting research, it’s usu- bad feelings or doubts about themselves based on ally simple—and sometimes appropriate—to con- their performance in the experiment. If this seems ceal your purpose. Many experiments in social complicated, it’s simply the price we pay for using psychology, for example, test the extent to which other people’s lives as the subject matter for our subjects will abandon the evidence of their own research. observations in favor of the views expressed by As a social researcher, then, you have many others. Figure 2-1 (p. 43) shows the stimulus from ethical obligations to the subjects in your studies. the classic Asch experiment—frequently replicated The box “Ethical Issues in Research on Human by psychology classes—in which subjects are Sexuality” illustrates some of the ethical questions shown three lines of differing lengths (A, B, and C) involved in a specifi c research area. and asked to compare them with a fourth line (X). Subjects are then asked, “Which of the fi rst three Analysis and Reporting lines is the same length as the fourth?” In addition to their ethical obligations to subjects, researchers have ethical obligations to their col- You can learn more about the Asch ex- leagues in the scientifi c community. These obliga- periments by searching the web for “Asch experiment” or “Solomon Asch.”* tions concern the analysis of data and the way the results are reported. In any rigorous study, the researcher should be Recall from Chapter 2 that the purpose of the more familiar than anyone else with the study’s experiment is to see whether a person would technical limitations and failures. Researchers give up his or her own judgment in favor of group have an obligation to make such shortcomings agreement. I think you can see that conformity is known to their readers—even if admitting qualifi - a useful phenomenon to study and understand, cations and mistakes makes them feel foolish. and it couldn’t be studied experimentally without Negative fi ndings, for example, should be re- deceiving the subjects. We’ll examine a similar ported if they are at all related to the analysis. situation in the discussion of a famous experi- There is an unfortunate myth in scientifi c report- ment by Stanley Milgram later in this chapter. The ing that only discoveries of strong, causal relation- question is, how do we get around the ethical is- ships among variables are worth reporting (journal sue that deception is necessary for an experiment editors are sometimes guilty of believing this as to work? well). In science, however, it’s often as important One appropriate solution researchers have to know that two variables are not related as to found is to debrief subjects following an experi- know that they are. ment. Debriefi ng entails interviews to discover any Similarly, researchers must avoid the tempta- problems generated by the research experience so tion to save face by describing their fi ndings as that those problems can be corrected. Even though the product of a carefully preplanned analytical subjects can’t be told the true purpose of the study strategy when that is not the case. Many fi ndings prior to their participation in it, there’s usually no reason they can’t know afterward. Telling them the truth afterward may make up for having to lie to debriefi ng Interviewing subjects to learn about their experience of participation in the project and to inform them of any unrevealed purpose. This is especially important if *Each time the Internet icon appears, you’ll be given help- there’s a possibility that they have been damaged by that ful leads for searching the World Wide Web. participation. 74 CHAPTER 3 THE ETHICS AND POLITICS OF SOCIAL RESEARCH

ISSUES AND INSIGHTS ETHICAL ISSUES IN RESEARCH ON HUMAN SEXUALITY

by Kathleen McKinney tion may be used when describing an embar- Department of Sociology, Illinois State University rassing or personal topic of study, because the researchers fear high rates of refusal or false When studying any form of human behavior, data. Covert research, such as some observa- ethical concerns are paramount. This statement tional studies, also violate the informed consent may be even truer for studies of human sexu- standard since subjects are unaware that they ality because of the topic’s highly personal, sa- are being studied. Informed consent may cre- lient, and perhaps threatening nature. Concern ate special problems with certain populations. has been expressed by the public and by legis- For example, studies of the sexuality of children lators about human sexuality research. Three are limited by the concern that children may be commonly discussed ethical criteria have been cognitively and emotionally unable to give in- related speciﬁcally to research in the area of hu- formed consent. Although there can be problems man sexuality. such as those discussed, most research is clearly voluntary, with informed consent from those Informed Consent This criterion emphasizes participating. the importance of both accurately informing your subject or respondent as to the nature of Right to Privacy Given the highly personal na- the research and obtaining his or her verbal or ture of sexuality and society’s tremendous con- written consent to participate. Coercion is not cern with social control of sexuality, the right to to be used to force participation, and subjects privacy is a very important ethical concern for may terminate their involvement in the research research in this area. Individuals may risk los- at any time. There are many possible violations ing their jobs, having family difﬁculties, or be- of this standard. Misrepresentation or decep- ing ostracized by peers if certain facets of their

arrive unexpectedly—even though they may seem Finally, simple carelessness or sloppiness can obvious in retrospect. So an interesting relation- be considered an ethical problem. If the research ship was uncovered by accident—so what? Em- project uses up limited resources or imposes on broidering such situations with descriptions of subjects but produces no benefi ts, many in the refi ctitious hypotheses is dishonest. It also does a search community would consider that an ethical disservice to less experienced researchers by lead- violation. This is not to say that all research must ing them into thinking that all scientifi c inquiry is produce positive results, but it should be conducted rigorously preplanned and organized. in a manner that promotes that possibility. In general, science progresses through honesty and openness; ego defenses and deception retard Institutional Review Boards it. Researchers can best serve their peers—and scientifi c discovery as a whole—by telling the truth The issue of research ethics in studies involving about all the pitfalls and problems they’ve experi- humans is now also governed by federal law. Any enced in a particular line of inquiry. Perhaps they’ll agency (such as a university or a hospital) wishing save others from the same problems. to receive federal research support must establish ETHICAL ISSUES IN SOCIAL RESEARCH 75

sexual lives are revealed. This is especially true tional or survey research. Emotional distress, for individuals involved in sexual behavior cate- however, is a possibility in all studies of human gorized as deviant (such as transvestism). Viola- sexuality. Respondents may be asked questions tions of right to privacy occur when researchers that elicit anxiety, dredge up unpleasant memo- identify members of certain groups they have ries, or cause them to evaluate themselves criti- studied, release or share an individual’s data or cally. Researchers can reduce the potential for responses, or covertly observe sexual behavior. such distress during a study by using anony- In most cases, right to privacy is easily main- mous, self-administered questionnaires or well- tained by the researchers. In survey research, trained interviewers and by wording sensitive self-administered questionnaires can be anony- questions carefully. mous and interviews can be kept confidential. All three of these ethical criteria are quite In case and observational studies, the identity subjective. Violations are sometimes justified by of the person or group studied can be disguised arguing that risks to subjects are outweighed by in any publications. In most research methods, benefits to society. The issue here, of course, is analysis and reporting of data should be at the who makes that critical decision. Usually, such group or aggregate level. decisions are made by the researcher and often a screening committee that deals with ethical Protection from Harm Harm may include concerns. Most creative researchers have been emotional or psychological distress, as well as able to follow all three ethical guidelines and physical harm. Potential for harm varies by re- still do important research. search method; it is more likely in experimental studies where the researcher manipulates or does something to the subject than in observa-

sources and even research done at no cost, such You can fi nd the “Protection of Human as student projects. Subjects” law online at http://www The chief responsibility of an IRB is to ensure .hhs.gov/ohrp/humansubjects/guidance/ that the risks faced by human participants in re- 45cfr46.htm#46.101. search are minimal. In some cases, the IRB may refuse to approve a study or may ask the researcher to revise the study design. Where some minimal an Institutional Review Board (IRB), a panel of fac- risks are deemed unavoidable, researchers must ulty (and possibly others) who review all research prepare an “informed consent” form that describes proposals involving human subjects so that they those risks clearly. Subjects may participate in the can guarantee that the subjects’ rights and inter- study only after they have read the statement and ests will be protected. The law applies specifi cally signed it. to federally funded research, but many universi- Much of the original impetus for establishing ties apply the same standards and procedures to IRBs had to do with medical experimentation on all research, including that funded by nonfederal humans, and many social research study designs 76 CHAPTER 3 THE ETHICS AND POLITICS OF SOCIAL RESEARCH are generally regarded as exempt from IRB review. such a manner that subjects cannot be identi- An example is an anonymous survey sent to a fi ed, directly or through identifi ers linked to the large sample of respondents. The guideline to be subjects. followed by IRBs, as contained in the Federal Ex- (5) Research and demonstration projects emption Categories (45 CFR 46.101 [b]) exempts a which are conducted by or subject to the ap- variety of research situations: proval of Department or Agency heads, and which are designed to study, evaluate, or other- (1) Research conducted in established or wise examine: commonly accepted educational settings, in- (i) Public benefi t or service programs; (ii) volving normal educational practices, such as procedures for obtaining benefi ts or services (i) research on regular and special education under those programs; (iii) possible changes in instructional strategies, or (ii) research on the or alternatives to those programs or procedures; effectiveness of or the comparison among or (iv) possible changes in methods or levels instructional techniques, curricula, or classroom of payment for benefi ts or services under those management methods. programs. (2) Research involving the use of educa- (6) Taste and food quality evaluation and tional tests (cognitive, diagnostic, aptitude, consumer acceptance studies, (i) if wholesome achievement), survey procedures, interview foods without additives are consumed or (ii) if a procedures or observation of public behavior, food is consumed that contains a food ingredi- unless: ent at or below the level and for a use found to (i) information obtained is recorded in such be safe, or agricultural chemical or environmen- a manner that human subjects can be identi- tal contaminant at or below the level found to fi ed, directly or through identifi ers linked to be safe, by the Food and Drug Administration the subjects; and (ii) any disclosure of the hu- or approved by the Environmental Protection man subjects’ responses outside the research Agency or the Food Safety and Inspection Ser- could reasonably place the subjects at risk of vice of the U.S. Department of Agriculture. criminal or civil liability or be damaging to the subjects’ fi nancial standing, employability, or Paragraph (2) of the excerpt exempts much of reputation. the social research described in this book. None- (3) Research involving the use of educa- theless, universities sometimes apply the law’s tional tests (cognitive, diagnostic, aptitude, provisions inappropriately. As chair of a university achievement), survey procedures, interview IRB, for example, I was once asked to review the procedures, or observation of public behavior letter of informed consent that was to be sent to that is not exempt under paragraph (b)(2) of medical insurance companies, requesting their this section, if: agreement to participate in a survey that would (i) the human subjects are elected or ap- ask which medical treatments were covered under pointed public offi cials or candidates for public their programs. Clearly the humans involved were offi ce; or (ii) Federal statute(s) require(s) without not at risk in the sense anticipated by the law. In a exception that the confi dentiality of the person- case like that, the appropriate technique for gain- ally identifi able information will be maintained ing informed consent is to mail the questionnaire. throughout the research and thereafter. If a company returns it, they’ve consented. If they (4) Research involving the collection or study don’t, they haven’t. of existing data, documents, records, patho- Other IRBs have suggested that researchers logical specimens, or diagnostic specimens, need to obtain permission before observing par- if these sources are publicly available or if the ticipants in public gatherings and events, before information is recorded by the investigator in conducting surveys on the most mundane mat- ETHICAL ISSUES IN SOCIAL RESEARCH 77 ters, and so forth. Christopher Shea (2000) has of informed consent, in which the risks and chronicled several such questionable applications benefi ts of the research are disclosed to the sub- of the law while supporting the ethical logic that ject. The second principle, benefi cence, involves originally prompted the law. maximizing possible benefi ts and good for the subject, while minimizing the amount of possible harm and risks resulting from the research. Shea’s critical article regarding IRBs can be Since the fruits of knowledge can come at a found in print as Christopher Shea, “Don’t cost to those participating in research, the last Talk to the Humans: The Crackdown on Social Science Research,” Lingua Franca principle, justice, seeks a fair distribution of the 10, no. 6 (Sep tember 2000): 27–34. Or burdens and benefi ts associated with research, you can fi nd the article online by searching so that certain individuals or groups do not bear the web for “shea lingua franca irb.” disproportionate risks while others reap the benefi ts. — (FRANKEL AND SIANG 1999:2–3)

Don’t think that these critiques of IRBs mini- Professional Codes of Ethics mize the importance of protecting human subjects. Indeed, some universities exceed the federal Ethical issues in social research are both impor- requirements in reasonable and responsible ways: tant and ambiguous. For this reason, most of the requiring IRB review of nonfederally funded proj- professional associations of social researchers ects, for example. have created and published formal codes of con- Research ethics is an ever-evolving subject, be- duct describing what is considered acceptable and cause new research techniques often require revis- unacceptable professional behavior. As one exam- iting old concerns. For example, the increased use ple, Figure 3-1 presents the code of conduct of the of public databases for secondary research recently American Association for Public Opinion Research caused some IRBs to worry whether they would (AAOPR), an interdisciplinary research association need to reexamine such projects as the General in the social sciences. Most professional associa- Social Survey every time a researcher proposed to tions have such codes of ethics. See, for example, use those data. Most IRBs have decided this is un- the American Sociological Association (ASA), the necessary. (See Skedsvold 2002 for more on this.) American Psychological Association (APA), the Similarly, the prospects for research of and through the Internet has raised ethical concerns. For more on codes of ethics, see the For example, the American Association for the following: Advancement of Science held a workshop on this AAOPR: The code of conduct is currently topic in November 1999. The overall conclusion of under revision; you can download a copy the report produced by the workshop summarizes of the proposed changes at http://www some of the primary concerns already examined in .aapor.org/?pagesurvey_methods/ this chapter: standards_and_best_practices/code_for _professional_ethics_and_practices The current ethical and legal framework for ASA: http://www.asanet.org/page.ww protecting human subjects rests on the princi- ?section Ethics&name Code+of +Ethics+Table+of+Contents ples of autonomy, benefi cence, and justice. The APA: http://www.apa.org/ethics/ fi rst principle, autonomy, requires that subjects APSA: http://www.apsanet.org/content be treated with respect as autonomous agents _9347.cfm and affi rms that those persons with diminished AoIR: http://www.aoir.org/reports/ autonomy are entitled to special protection. In ethics.pdf practice, this principle is refl ected in the process CODE OF PROFESSIONAL ETHICS AND PRACTICES We, the members of the American Association for Public Opinion Research, subscribe to the principles expressed in the following code. Our goal is to support sound practice in the profession of public opinion research. (By public opinion research we mean studies in which the principal source of information about individual beliefs, preferences, and behavior is a report given by the individual himself or herself.) We pledge ourselves to maintain high standards of scientific competence and integrity in our work, and in our relations both with our clients and with the general public. We further pledge ourselves to reject all tasks or assignments which would be inconsistent with the principles of this code. THE CODE I. Principles of Professional Practice in the Conduct of Our Work A. We shall exercise due care in gathering and processing data, taking all reasonable steps to assume the accuracy of results. B. We shall exercise due care in the development of research designs and in the analysis of data. 1. We shall employ only research tools and methods of analysis which, in our professional judgment, are well suited to the research problem at hand. 2. We shall not select research tools and methods of analysis because of their special capacity to yield a desired conclusion. 3. We shall not knowingly make interpretations of research results, nor shall we tacitly permit interpretations, which are inconsistent with the data available. 4. We shall not knowingly imply that interpretations should be accorded greater confidence than the data actually warrant. C. We shall describe our findings and methods accurately and in appropriate detail in all research reports. II. Principles of Professional Responsibility in Our Dealings with People A. The Public: 1. We shall cooperate with legally authorized representatives of the public by describing the methods used in our studies. 2. We shall maintain the right to approve the release of our findings whether or not ascribed to us. When misinterpretation appears, we shall publicly disclose what is required to correct it, notwithstanding our obligation for client confidentiality in all other respects. B. Clients or Sponsors: 1. We shall hold confidential all information obtained about the client's general business affairs and about the findings of research conducted for the client, except when the dissemination of such information is expressly authorized. 2. We shall be mindful of the limitations of our techniques and facilities and shall accept only those research assignments which can be accomplished within these limitations. C. The Profession: 1. We shall not cite our membership in the Association as evidence of professional competence, since the association does not so certify any persons or organizations. 2. We recognize our responsibility to contribute to the science of public opinion research and to disseminate as freely as possible the ideas and findings which emerge from our research. D. The Respondent: 1. We shall not lie to survey respondents or use practices and methods which abuse, coerce, or humiliate them. 2. We shall protect the anonymity of every respondent, unless the respondent waives such anonymity for specified uses. In addition, we shall hold as privileged and confidental all information which tends to identify the respondent.

FIGURE 3-1 Code of Conduct of the American Association for Public Opinion Research Source: American Association for Public Opinion Research, By-Laws (May 1977). Used by permission. TWO ETHICAL CONTROVERSIES 79

American Political Science Association (APSA), lookout, called the “watchqueen.” Humphreys be- and so forth. You can fi nd many of these on the gan showing up at public restrooms, offering to associations’ websites. In addition, the Association serve as watchqueen whenever it seemed ap- of Internet Researchers (AoIR) has a code of ethics propriate. Because the watchqueen’s payoff was accessible online. the chance to watch the action, Humphreys was able to conduct fi eld observations as he would in a study of political rallies or jaywalking behavior at TWO ETHICAL CONTROVERSIES intersections. To round out his understanding of the tearoom trade, Humphreys needed to know some- As you may already have guessed, the adoption thing more about the people who participated. and publication of professional codes of conduct Because the men probably would not have been have not totally resolved the issue of research eth- thrilled about being interviewed, Humphreys de- ics. Social researchers still disagree on some gen- veloped a different solution. Whenever possible, eral principles, and those who agree in principle he noted the license numbers of participants’ often debate specifi cs. cars and tracked down their names and addresses This section briefl y describes two research proj- through the police. Humphreys then visited the ects that have provoked ethical controversy and men at their homes, disguising himself enough discussion. The fi rst project studied homosexual to avoid recognition, and announced that he was behavior in public restrooms; the second exam- conducting a survey. In that fashion, he collected ined obedience in a laboratory setting. the personal information he couldn’t get in the restrooms. Trouble in the Tearoom As you can imagine, Humphreys’ research provoked considerable controversy both inside and As a graduate student, Laud Humphreys became outside the social scientifi c community. Some crit- interested in the study of homosexual behavior. ics charged Humphreys with a gross invasion of He developed a special interest in the casual and privacy in the name of science. What men did in fl eeting same-sex acts engaged in by some male public restrooms was their own business. Others nonhomosexuals. In particular, his research inter- were mostly concerned about the deceit involved— est focused on homosexual acts between strang- Humphreys had lied to the participants by leading ers meeting in the public restrooms in parks, called them to believe he was only a voyeur-participant. “tearooms” among homosexuals. The result was Even people who felt that the tearoom participants the publication in 1970 of Tearoom Trade. were fair game for observation because they used What particularly interested Humphreys about a public facility protested the follow-up survey. the tearoom activity was that the participants They felt it was unethical for Humphreys to trace seemed otherwise to live conventional lives as the participants to their homes and to interview “family men” and accepted members of the com- them under false pretenses. munity. They did nothing else that might qualify Still others justifi ed Humphreys’ research. The them as homosexuals. Thus, it was important to topic, they said, was worth study. It couldn’t be them that they remain anonymous in their tearoom studied any other way, and they regarded the de- visits. How would you study something like that? ceit as essentially harmless, noting that Humphreys Humphreys decided to take advantage of the was careful not to harm his subjects in disclosing social structure of the situation. Typically, the tea- their tearoom activities. room encounter involved three people: the two The tearoom trade controversy is still debated, men actually engaging in the sexual act and a and it probably always will be, because it stirs 80 CHAPTER 3 THE ETHICS AND POLITICS OF SOCIAL RESEARCH emotions and involves ethical issues people dis- in day-to-day civilian life. Few would disagree agree about. What do you think? Was Humphreys that this reliance on authority exists, yet Stanley ethical in doing what he did? Are there parts of Milgram’s study (1963, 1965) of the topic provoked the research that you believe were acceptable and considerable controversy. other parts that were not? To observe people’s willingness to harm others when following orders, Milgram brought 40 adult men from many different walks of life into a labo- See the discussion by Joan Sieber online at ratory setting designed to create the phenomenon http://www.missouri.edu/ϳphilwb/Laud .html for more on the political and ethical under study. If you had been a subject in the ex- context of the “tearoom” research. periment, you would have had something like the following experience. You’ve been informed that you and another Observing Human Obedience subject are about to participate in a learning experiment. Through a draw of lots, you’re as- The second illustration differs from the fi rst in signed the job of “teacher” and your fellow subject many ways. Whereas Humphreys’ study involved the job of “pupil.” The “pupil” is led into another participant observation, this study took place in room and strapped into a chair; an electrode is at- the laboratory. Humphreys’ study was sociological, tached to his wrist. As the teacher, you’re seated this one psychological. And whereas Humphreys in front of an impressive electrical control panel examined behavior considered by many to be a covered with dials, gauges, and switches. You form of deviance, the researcher in this study ex- notice that each switch has a label giving a dif- amined obedience and conformity. ferent number of volts, ranging from 15 to 315. One of the most unsettling clichés to come out The switches have other labels, too, some with of World War II was the German soldier’s common the ominous phrases “Extreme-Intensity Shock,” excuse for atrocities: “I was only following orders.” “Danger—Severe Shock,” and “XXX.” From the point of view that gave rise to this com- The experiment runs like this. You read a list of ment, any behavior—no matter how reprehen- word pairs to the learner and then test his ability sible—could be justifi ed if someone else could be to match them up. Because you can’t see him, a assigned responsibility for it. If a superior offi cer light on your control panel indicates his answer. ordered a soldier to kill a baby, the fact of the order Whenever the learner makes a mistake, you’re in- supposedly exempted the soldier from personal re- structed by the experimenter to throw one of the sponsibility for the action. switches—beginning with the mildest—and ad- Although the military tribunals that tried the minister a shock to your pupil. Through an open war crime cases did not accept this excuse, social door between the two rooms, you hear your pupil’s researchers and others have recognized the extent response to the shock. Then you read another list to which this point of view pervades social life. of word pairs and test him again. People often seem willing to do things they know As the experiment progresses, you administer would be considered wrong by others, if they can ever more intense shocks, until your pupil screams claim that some higher authority ordered them to for mercy and begs for the experiment to end. do it. Such was the pattern of justifi cation in the My You’re instructed to administer the next shock Lai tragedy of Vietnam, when U.S. soldiers killed anyway. After a while, your pupil begins kicking more than 300 unarmed civilians—some of them the wall between the two rooms and continues young children—simply because their village, My to scream. The implacable experimenter tells you Lai, was believed to be a Viet Cong stronghold. to give the next shock. Finally, you read a list and This sort of justifi cation appears less dramatically ask for the pupil’s answer—but there is no reply THE POLITICS OF SOCIAL RESEARCH 81 whatever, only silence from the other room. The The web has a wealth of discussion regard- experimenter informs you that no answer is con- ing the Milgram experiments. Search for sidered an error and instructs you to administer the “Milgram experiments,” “human obedi- next higher shock. This continues up to the “XXX” ence experiments,” or “Stanley Milgram.” shock at the end of the series. What do you suppose you really would have done when the pupil fi rst began screaming? When he began kicking on the wall? Or when he became THE POLITICS OF SOCIAL RESEARCH totally silent and gave no indication of life? You’d refuse to continue giving shocks, right? And surely the same would be true of most people. As I indicated earlier, both ethics and politics hinge So we might think—but Milgram found out oth- on ideological points of view. What is unaccept- erwise. Of the fi rst 40 adult men Milgram tested, able from one point of view will be acceptable nobody refused to continue administering the from another. Although political and ethical issues shocks until they heard the pupil begin kicking the are often closely intertwined, I want to distinguish wall between the two rooms. Of the 40, 5 did so between them in two ways. then. Two-thirds of the subjects, 26 of the 40, con- First, the ethics of social research deals mostly tinued doing as they were told through the entire with the methods employed; political issues tend series—up to and including the administration of to center on the substance and use of research. the highest shock. Thus, for example, some critics raise ethical objec- As you’ve probably guessed, the shocks were tions to the Milgram experiments, saying that the phony, and the “pupil” was a confederate of the methods harmed the subjects. A political objection experimenter. Only the “teacher” was a real sub- would be that obedience is not a suitable topic for ject in the experiment. As a subject, you wouldn’t study, either because (1) we should not tinker with actually have been hurting another person, but people’s willingness to follow orders from higher you would have been led to think you were. The authority or (2) because the results of the research experiment was designed to test your willingness could be misused to make people more obedient. to follow orders to the point of presumably killing The second distinction between ethical and po- someone. litical aspects of social research is that there are Milgram’s experiments have been criticized no formal codes of accepted political conduct. Al- both methodologically and ethically. On the ethi- though some ethical norms have political aspects— cal side, critics have particularly cited the effects for example, specifi c guidelines for not harming of the experiment on the subjects. Many seem to subjects clearly relate to the U.S. protection of civil have experienced personally about as much pain liberties—no one has developed a set of political as they thought they were administering to some- norms that all social researchers accept. one else. They pleaded with the experimenter to The only partial exception to the lack of politi- let them stop giving the shocks. They became ex- cal norms is the generally accepted view that a re- tremely upset and nervous. Some had uncontrol- searcher’s personal political orientation should not lable seizures. interfere with or unduly infl uence his or her scien- How do you feel about this research? Do you tifi c research. It would be considered improper for think the topic was important enough to justify a researcher to use shoddy techniques or to dis- such measures? Would debriefi ng the subjects be tort or lie about his or her research as a way of suffi cient to ameliorate any possible harm? Can furthering the researcher’s political views. As you you think of other ways the researcher might have can imagine, however, studies are often enough examined obedience? attacked for allegedly violating this norm. 82 CHAPTER 3 THE ETHICS AND POLITICS OF SOCIAL RESEARCH Earl Babbie Politics can intrude in research as in other aspects of life.

Objectivity and Ideology Most social researchers have agreed with this abstract ideal, but not all. Marxist and neo- In Chapter 1, I suggested that social research can Marxist scholars, for example, have argued that never be totally objective, because researchers social science and social action cannot and should are human and therefore necessarily subjective. not be separated. Explanations of the status quo in Science, as a collective enterprise, achieves the society, they contend, shade subtly into defenses of equivalent of objectivity through intersubjectivity. that same status quo. Simple explanations of the That is, different scientists, having different sub- social functions of, say, discrimination can easily jective views, can and should arrive at the same become justiﬁ cations for its continuance. By the results when they employ accepted research tech- same token, merely studying society and its ills niques. Essentially, this will happen to the extent without a commitment to making society more that each can set personal values and views aside humane has been called irresponsible. for the duration of the research. In Chapter 10, we’ll examine participatory ac- The classic statement on objectivity and neu- tion research, which is explicitly committed to us- trality in social science is Max Weber’s “Science as ing social research for the purposes designed and a Vocation” ([1925] 1946). In this lecture, Weber valued by the subjects of the research. Thus, for coined the phrase value-free sociology and urged example, researchers committed to improving the that sociology, like other sciences, needed to re- conditions for workers at a factory would ask the main unencumbered by personal values if it was workers to deﬁ ne the outcomes they would like to to make a special contribution to society. Liberals see and to have a hand in conducting social re- and conservatives alike could recognize the “facts” search relevant to achieving them. The task of the of social science, regardless of how those facts ac- researchers is to ensure that the workers have accorded with their personal politics. cess to professional research methods. THE POLITICS OF SOCIAL RESEARCH 83

Quite aside from abstract disagreements about For the most part, social researchers in this cen- whether social science should be value-free, many tury have supported the cause of African American have argued about whether particular research equality in the United States, and their convictions undertakings are value-free or whether they repre- often have provided the impetus for their research. sent an intrusion of the researcher’s own political Moreover, they’ve hoped that their research will values. Typically, researchers deny such intrusion, lead to social change. There is no doubt, for ex- and others then challenge their denials. Let’s look ample, that Gunnar Myrdal’s classic two-volume at some examples of the controversies concerning study (1944) of race relations in the United States this issue. signifi cantly infl uenced race relations themselves. Myrdal amassed a great deal of data to show that Social Research and Race Nowhere have social the position of African Americans directly contra- research and politics been more controversially in- dicted U.S. values of social and political equality. tertwined than in the area of racial relations. Social Further, Myrdal did not attempt to hide his own researchers studied the topic for a long time. Of- point of view in the matter. ten, the products of the social research found their way into practical politics. A few brief references You can pursue Myrdal’s landmark research should illustrate the point. online by searching for “Gunnar Myrdal” In 1896, when the U.S. Supreme Court estab- or “An American Dilemma.” lished the principle of “separate but equal” as a means of reconciling the Fourteenth Amendment’s guarantee of equality to African Americans with Many social researchers have become directly the norms of segregation, it neither asked for nor involved in the civil rights movement, some more cited social research. Nonetheless, it is widely be- radically than others. Given the broad support for lieved that the Court was infl uenced by the writings ideals of equality, research conclusions supporting of William Graham Sumner, a leading social scien- the cause of equality draw little or no criticism. To tist of his era. Sumner was noted for his view that recognize how solid the general social science po- the mores and folkways of a society were relatively sition is in this matter, we need only examine a impervious to legislation and social planning. His few research projects that have produced conclu- view has often been paraphrased as “stateways do sions disagreeing with the predominant ideologi- not make folkways.” Thus the Court ruled that it cal position. could not accept the assumption that “social preju- Most social researchers have—overtly, at least— dices may be overcome by legislation” and denied supported the end of school segregation. Thus, an the wisdom of “laws which confl ict with the gen- immediate and heated controversy was provoked eral sentiment of the community” (Blaunstein and in 1966 when James Coleman, a respected sociolo- Zangrando 1970:308). As many a politician has gist, published the results of a major national study said, “You can’t legislate morality.” of race and education. Contrary to general agree- When the doctrine of “separate but equal” was ment, Coleman found little difference in academic overturned in 1954 (Brown v. Board of Education performance between African American students of Topeka), the new Supreme Court decision was attending integrated schools and those attending based in part on the conclusion that segregation segregated ones. Indeed, such obvious things as li- had a detrimental effect on African American chil- braries, laboratory facilities, and high expenditures dren. In drawing that conclusion, the Court cited per student made little difference. Instead, Coleman several sociological and psychological research reported that family and neighborhood factors had reports (Blaunstein and Zangrando 1970). the most infl uence on academic achievement. 84 CHAPTER 3 THE ETHICS AND POLITICS OF SOCIAL RESEARCH

Coleman’s fi ndings were not well received point out how ideology often shows up in matters by many of the social researchers who had been of social research. Although the abstract model of active in the civil rights movement. Some schol- science is divorced from ideology, the practice of ars criticized Coleman’s work on methodological science is not. grounds, but many others objected hotly on the ground that the fi ndings would have segregationist To examine a more recent version of the political consequences. The controversy that raged controversy surrounding race and achieve- around the Coleman report hearkened back to the ment, search the web for differing points uproar provoked a year earlier by Daniel Moynihan of view concerning “The Bell Curve”— (1965) in his critical analysis of the African Ameri- sparked by a book with that title by can family in the United States. Richard J. Herrnstein and Charles Murray. Another example of political controversy surrounding social research in connection with race concerns IQ scores. In 1969 Arthur Jensen, a The Politics of Sexual Research As I indicated Harvard psychologist, was asked to prepare an earlier, the Laud Humphreys study of “tearoom article for the Harvard Educational Review examin- trade” raised ethical issues that researchers still ing the data on racial differences in IQ test results discuss and debate. At the same time, much of the (Jensen 1969). In the article, Jensen concluded furor was related to the subject matter itself. As I that genetic differences between African Ameri- have written elsewhere, cans and whites accounted for the lower average Laud Humphreys didn’t just study S-E-X but IQ scores of African Americans. Jensen became observed and discussed homosexuality. And it so identifi ed with that position that he appeared wasn’t even the caring-and-committed-relation- on college campuses across the country discuss- ships-between-two-people-who-just-happen-to- ing it. be-of-the-same-sex homosexuality but tawdry Jensen’s research has been attacked on numer- encounters between strangers in public toilets. ous methodological bases. Critics charged that Only adding the sacrifi ce of Christian babies much of the data on which Jensen’s conclusion could have made this more infl ammatory for was based were inadequate and sloppy—there are the great majority of Americans in 1970. many IQ tests, some worse than others. Similarly, — (BABBIE 2004:12) it was argued that Jensen had not taken social- environmental factors suffi ciently into account. Although Humphreys’ research topic was un- Other social researchers raised still other method- usually provocative for many, much tamer sexual- ological objections. ity research has also engendered outcries of public Beyond the scientifi c critique, however, many horror. During the 1940s and 1950s, Alfred Kinsey, condemned Jensen as a racist. Hostile crowds a biologist, published landmark studies of sexual drowned out his public presentations by booing. practices of American men (1948) and women Ironically, Jensen’s reception by several university (1953). Kinsey’s extensive interviewing allowed audiences did not differ signifi cantly from the re- him to report on frequency of sexual activity, pre- ception received by abolitionists a century before, marital and extramarital sex, homosexual behav- when the prevailing opinion favored leaving the ior, and so forth. His studies produced public out- institution of slavery intact. rage and efforts to close his research institute at Many social researchers limited their objections Indiana University. to the Moynihan, Coleman, and Jensen research Although today most people no longer get to scientifi c, methodological grounds. The politi- worked up about the Kinsey reports, Americans cal fi restorms ignited by these studies, however, tend to remain touchy about research on sex. THE POLITICS OF SOCIAL RESEARCH 85

In 1987 the National Institutes of Health (NIH), of the various states to determine their proper charged with fi nding ways to combat the AIDS representation in the House of Representatives. epidemic, found they needed hard data on con- Whereas each state gets two senators, large states temporary sexual practices if they were to design get more representatives than do small ones. So effective anti-AIDS programs. Their request for re- what could be simpler? Just count the number of search proposals resulted in a sophisticated study people in each state. design by Edward O. Laumann and colleagues. The From the beginning, there was nothing simple proposed study focused on the different patterns of about counting heads in a dispersed, national pop- sexual activity characterizing different periods of ulation like the United States. Even the defi nition life, and it received rave reviews from the NIH and of a “person” was anything but straightforward. A their consultants. slave, for example, counted as only three-fourths Enter Senator Jesse Helms (R-North Carolina) of a person for purposes of the census. This de- and Congressman William Dannemeyer (R-Cali- creased the representation of the slave-holding fornia). In 1989, having learned of the Laumann Southern states, though counting slaves as whole study, Helms and Dannemeyer began a campaign people might have raised the dangerously radical to block the study and shift the same amount of idea that they should be allowed to vote. money to a teen celibacy program. Anne Fausto- Further, the logistical problems of counting Sterling, a biologist, sought to understand the op- people who reside in suburban tract houses, ur- position to the Laumann study. ban apartments, college dorms, military barracks, farms, cabins in the woods, and illegal housing The surveys, Helms argued, are not really units, as well as counting those who have no place intended “to stop the spread of AIDS. The real to live, has always presented a daunting task. It’s purpose is to compile supposedly scientifi c facts the sort of challenge social researchers tackle with to support the left-wing liberal argument that relish. However, the diffi culty of fi nding the hard- homosexuality is a normal, acceptable life- to-reach and the techniques created for doing so style. . . . As long as I am able to stand on the cannot escape the political net. fl oor of the U.S. Senate,” he added, “I am never Kenneth Prewitt, who directed the Census Bu- going to yield to that sort of thing, because reau from 1998 to 2001, describes some of the po- it is not just another life-style; it is sodomy.” litical aspects of counting heads: — (FAUSTO-STERLING 1992)

Helms was sufﬁ ciently persuasive as to win a 66–34 vote in favor of his amendment in the U.S. Senate. The House rejected the amendment, and it was dropped in conference committee, but government funding for the study was put on hold. Laumann and his colleagues then turned to the private sector and obtained funding for a smaller study, published in 1994 as The Social Organization of Sexuality. Text not available due to copyright restrictions

Politics and the Census There is probably a political dimension to every attempt to study human social behavior. Consider the U. S. decennial census, mandated by the Constitution. The original purpose was to discover the population sizes 86 CHAPTER 3 THE ETHICS AND POLITICS OF SOCIAL RESEARCH

resent either side in court, they are, nonetheless, engaged by only one side to appear, and their testimony tends to support the side of the party who pays for their time. This doesn’t necessarily mean Text not available due to copyright restrictions that these witnesses will lie on behalf of their pa- trons, but the contenders in a lawsuit are under- standably more likely to pay for expert testimony that supports their case than for testimony that attacks it. Thus, as an expert witness, you appear in court In more recent years, concern for undercount- only because your presumably scientifi c and hon- ing the urban poor has become a political issue. est judgment happens to coincide with the inter- The big cities, which have the most to lose from the ests of the party paying you to testify. Once you undercounting, typically vote Democratic rather arrive in court and swear to tell the truth, the than Republican, so you can probably guess which whole truth, and nothing but the truth, however, party supports efforts to improve the counting and you fi nd yourself in a world foreign to the ideals which party is less enthusiastic. By the same toof objective contemplation. Suddenly the norms ken, when social scientists have argued in favor of are those of winning and losing. As an expert wit- replacing the attempt at a total enumeration of the ness, of course, all you have to lose is your respect- population with modern survey sampling meth- ability (and perhaps the chance to earn fees as an ods (see Chapter 7 for more on sampling), they expert witness in the future). Still, such stakes are have enjoyed more support from Democrats, who high enough to create discomfort for most social would stand to gain from such a methodological researchers. shift, than from Republicans, who would stand to I recall one case in federal court when I was lose. Rather than suggesting Democrats support testifying on behalf of some civil service work- science more than do Republicans, this situation ers who had had their cost-of-living allowance offers another example of how the political context (COLA) cut on the basis of research I thought was in which we live and conduct social research often rather shoddy. I was engaged to conduct more affects that research. “scientifi c” research that would demonstrate the injustice worked against the civil servants (Babbie Politics with a Little “p” 1982:232–43). I took the stand, feeling pretty much like a re- Political ideologies often confound social research, spected professor and textbook author. In short but the more personal “politics” of social research order, however, I found I had moved from the runs far deeper still. Social research in relation to academy to the hockey rink. Tests of statistical sig- contested social issues simply cannot remain an- nifi cance and sampling error were suddenly less tiseptically objective—particularly when differing relevant than a slap shot. At one point, an attorney ideologies are pitted against each other in a fi eld of from Washington lured me into casually agree- social science data. ing that I was familiar with a certain professional The same is true when research is invoked in journal. Unfortunately, the journal did not exist. I disputes between people with confl icting inter- was mortifi ed and suddenly found myself shifting ests. For instance, social researchers who have domains. Without really thinking about it, I shifted served as “expert witnesses” in court would prob- from Mr. Rogers’ Neighborhood to the dark alleys ably agree that the scientifi c ideal of a “search for of ninja-professor. I would not be fully satisfi ed un- truth” seems hopelessly naive in a trial or lawsuit. til I, in turn, could mortify the attorney, which I suc- Although expert witnesses technically do not rep- ceeded in doing. THE POLITICS OF SOCIAL RESEARCH 87

Even though the civil servants got their cost- the independence of scientists. As a result, we of-living allowance back, I have to admit I was have enjoyed sustained economic progress and also concerned with how I looked in front of the public health, as well as unequaled leadership courtroom assemblage. I tell you this anecdote to within the global scientifi c community. Re- illustrate the personal “politics” of human interac- cent actions by political appointees, however, tions involving presumably scientifi c and objec- threaten to undermine this legacy by preventing tive research. We need to realize that as human the best available science from informing policy beings social researchers are going to act like hu- decisions that have serious consequences for man beings, and we must take this into account our health, safety, and environment. in assessing their fi ndings. This recognition neither Across a broad range of issues—from child- invalidates their research nor provides an excuse hood lead poisoning and mercury emissions for rejecting fi ndings we happen to dislike, but it to climate change, reproductive health, and does need to be taken into account. nuclear weapons—political appointees have distorted and censored scientifi c fi ndings that contradict established policies. In some cases, Politics in Perspective they have manipulated the underlying science to Although the ethical and the political dimensions align results with predetermined political deci- of research are in principle distinct, they do inter- sions. — (UNION OF CONCERNED SCIENTISTS 2005) sect. Whenever politicians or the public feel that social research is violating ethical or moral stan- There are four main lessons that I hope you will dards, they’ll be quick to respond with remedies take away from this discussion. First, science is of their own. Moreover, the standards they defend not untouched by politics. The intrusion of politics may not be those of the research community. And and related ideologies is not unique to social re- even when researchers support the goals of mea- search; the natural sciences have experienced and sures directed at the way research is done, the continue to experience similar situations. How- means specifi ed by regulations or legislation can ever, social researchers study things that matter hamstring research. to people, things they have fi rm, personal feelings Today, the “politicization of science” is a par- about, and things that affect their lives. Moreover, ticularly hot topic, with charges fl ung from both researchers are human beings, and their feelings sides of the political spectrum. On the one hand, often show through in their professional lives. To we can see renewed objections to the teaching of think otherwise would be naive. evolution while demands for the teaching of cre- Second, science proceeds in the midst of politi- ation science have been replaced by support for cal controversy and hostility. Even when research- intelligent design. In many of these regards, sci- ers get angry and call each other names, or when ence is seen as a threat to religiously based views, the research community comes under attack from and scientists are sometimes accused of hav- the outside, scientifi c inquiry persists. Studies are ing an antireligious agenda. On the other hand, done, reports are published, and new things are a statement by the Union of Concerned Scien- learned. In short, ideological disputes do not bring tists (2005), cosigned by thousands of scientists, science to a halt, but they do make it more chal- illustrates the concern that the concentration lenging—and exciting. of political power in the hands of one party can Third, an awareness of ideological consider- threaten the independent functioning of scientifi c ations enriches the study and practice of social research. research methods. Many of the established characteristics of science, such as intersubjectivity, The United States has an impressive history of function to cancel out or hold in check our human investing in scientifi c research and respecting shortcomings, especially those we are unaware of. 88 CHAPTER 3 THE ETHICS AND POLITICS OF SOCIAL RESEARCH

Otherwise, we might look into the world and never WHAT DO YOU THINK? REVISITED see anything but a refl ection of our personal biases and beliefs. Finally, although researchers should not let their The Nazi medical experiments were outra- own values interfere with the quality and honesty geous in many ways. Some of the experi- of their research, this does not mean that research- ments can only be described as ghoulish ers cannot or should not participate in public de- and sadistic. Often the scientifi c caliber of bates and express both their scientifi c expertise the experiments was shoddy. One could ar- and personal values. You can do scientifi cally ex- gue, however, that some people today suffer cellent research on racial prejudice, all the while and even die from research. We often con- being opposed to prejudice and saying so. Some done these risks by virtue of the benefi ts to would argue that social scientists, because of their humankind expected to follow from the re- scientifi c expertise in the workings of society, have search. Some of the Nazi doctors, no doubt, an obligation to speak out, rather than leaving that salved their own consciences with such role to politicians, journalists, and talk-show hosts. refl ections. Herbert Gans (2002) writes of the need for “public The Nazi medical experiments breached sociologists”: a fundamental ethical norm discussed in this chapter. This is refl ected in the indictments A public sociologist is a public intellectual who of the Nuremberg trials, which charged applies sociological ideas and fi ndings to social several medical personnel in the Nazi war (defi ned broadly) issues about which sociology machine with “plans and enterprises involv- (also defi ned broadly) has something to say. ing medical experiments without the sub- Public intellectuals comment on whatever issues jects’ consent, upon civilians and members show up on the public agenda; public sociolo- of the armed forces of nations then at war gists do so only on issues to which they can with the German Reich [emphasis mine]” apply their sociological insights and fi ndings. (Trials of War Criminals 1949–1953). Even if the most hideous experiments had not been conducted, and even accepting that there is For more on the ethics of Nazi experi- always some risk when human research is ments, see http://www.ushmm.org/ undertaken, it is absolutely unacceptable to research/doctors/indiptx.htm. subject people to risks in research without their informed consent.

Main Points ❏ Researchers agree that participation in research should normally be voluntary. This Introduction norm, however, can confl ict with the scientifi c ❏ Administrative, ethical, and political consider- need for generalizability. ations, in addition to technical scientifi c ones, ❏ Researchers agree that research should not shape social research problems. harm those who participate in it, unless they willingly and knowingly accept the risks of Ethical Issues in Social Research harm, giving their informed consent. ❏ What is ethical and unethical in research is ultimately a matter of what a community of ❏ Whereas anonymity refers to the situation in people agree is right and wrong. which even the researcher cannot identify REVIEW QUESTIONS 89

specifi c information with the individuals it ❏ Even though the norms of science cannot force describes, confi dentiality refers to the situa- individual researchers to give up their personal tion in which the researcher promises to keep values, the intersubjective character of science information about subjects private. The most provides a guard against “scientifi c” fi ndings straightforward way to ensure confi dentiality being the product of bias only. is to destroy identifying information as soon as it’s no longer needed. Key Terms ❏ Many research designs involve a degree of deception of subjects. Because deceiving people anonymity debriefi ng violates common standards of ethical behav- confi dentiality informed consent ior, deception in research requires a strong justifi cation—and even then the justifi cation Review Questions may be challenged. ❏ Social researchers have ethical obligations to 1. Consider the following real and hypotheti- the community of researchers as well as to cal research situations. What is the ethical subjects. These obligations include reporting component in each example? How do you results fully and accurately as well as disclos- feel about it? Do you think the procedures ing errors, limitations, and other shortcomings described are ultimately acceptable or unac- in the research. ceptable? You might fi nd it useful to discuss ❏ Professional associations in several disciplines some of these situations with classmates. publish codes of ethics to guide researchers. a. A psychology instructor asks students in These codes are necessary and helpful, but an introductory psychology class to com- they do not resolve all ethical questions. plete questionnaires that the instructor will analyze and use in preparing a journal Two Ethical Controversies article for publication. ❏ Laud Humphreys’ study of “tearoom” encoun- b. After a fi eld study of deviant behavior ters and Stanley Milgram’s study of obedience during a riot, law enforcement offi cials raised ethical issues that have been debated to demand that the researcher identify those this day. people who were observed looting. Rather than risk arrest as an accomplice after the The Politics of Social Research fact, the researcher complies. ❏ Social research inevitably has a political and c. After completing the fi nal draft of a book ideological dimension. Although science is reporting a research project, the researcher- neutral on political matters, scientists are not. author discovers that 25 of the 2,000 sur- Moreover, much social research inevitably vey interviews were falsifi ed by interview- involves the political beliefs of people outside ers. To protect the bulk of the research, the research community. the author leaves out this information and ❏ Although most researchers agree that politi- publishes the book. cal orientation should not unduly infl uence d. Researchers obtain a list of right-wing research, in practice it can be very diffi cult to radicals they wish to study. They contact separate politics and ideology from the con- the radicals with the explanation that duct of research. Some researchers maintain each has been selected “at random” from that research can and should be an instru- among the general population to take a ment of social action and change. More subtly, sampling of “public opinion.” a shared ideology can affect the way other e. A college instructor who wants to test the researchers receive social research. effect of unfair berating administers an 90 CHAPTER 3 THE ETHICS AND POLITICS OF SOCIAL RESEARCH

hour exam to both sections of a specifi c (1) A peaceful, though illegal, course. The overall performance of the demonstration two sections is essentially the same. The (2) The bombing of a public building dur- grades of one section are artifi cially lowing a time it is sure to be unoccupied ered, however, and the instructor berates (3) The assassination of a public offi cial the students for performing so badly. The instructor then administers the same 2. Review the discussion of the Milgram experi- fi nal exam to both sections and discov- ment on obedience. How would you design a ers that the performance of the unfairly study to accomplish the same purpose while berated section is worse. The hypothesis avoiding the ethical criticisms leveled at is confi rmed, and the research report is Milgram? Would your design be equally valid? published. Would it have the same effect? f. In a study of sexual behavior, the investiga- 3. Suppose a researcher who is personally in tor wants to overcome subjects’ reluctance favor of small families (as a response to the to report what they might regard as shame- problem of overpopulation) wants to con- ful behavior. To get past their reluctance, duct a survey to determine why some people subjects are asked, “Everyone masturbates want many children and others don’t. What now and then; about how much do you personal-involvement problems would the masturbate?” researcher face and how could she or he g. A researcher studying dorm life on campus avoid them? discovers that 60 percent of the residents 4. What ethical issues should the researcher regularly violate restrictions on alcohol in item 3 take into account in designing the consumption. Publication of this fi nd- survey? ing would probably create a furor in the campus community. Because no extensive analysis of alcohol use is planned, the Online Study Resources researcher decides to keep this fi nding quiet. h. To test the extent to which people may try to save face by expressing attitudes on matters they are wholly uninformed about, Go to the researcher asks for subjects’ attitudes http://sociology.wadsworth.com/babbie_basics4e regarding a fi ctitious issue. and click on ThomsonNow for access to this i. A research questionnaire is circulated powerful online study tool. You will get a per- among students as part of their university sonalized study plan based on your responses to registration packet. Although students are a diagnostic pretest. Once you have mastered not told they must complete the question- the material with the help of interactive learning naire, the hope is that they will believe they tools, you can take a posttest to confi rm that you must—thus ensuring a higher completion are ready to move on to the next chapter. rate. j. A researcher pretends to join a radical Website for political group in order to study it and is The Basics of Social Research successfully accepted as a member of the 4th edition inner planning circle. What should the researcher do if the group makes plans for At the book companion website (http://sociology the following? .wadsworth.com/babbie_basics4e) you will fi nd many resources in addition to ThomsonNow ADDITIONAL READINGS 91 to aid you in studying for your exams. For ex- Hamnett, Michael P., Douglas J. Porter, Amarjit Singh, ample, you will fi nd Tutorial Quizzes with feed- and Krishna Kumar. 1984. Ethics, Politics, and Interna- back, Internet Exercises, Flashcards, and Chapter tional Social Science Research. Honolulu: University of Tutorials, as well as Extended Projects, InfoTrac Hawaii Press. Discussions of research ethics typically focus on the interests of the individual participants in College Edition search terms, Social Research in research projects, but this book raises the level of the Cyberspace, GSS Data, Web Links, and primers for discussion to include the rights of whole societies. using various data analysis software such as SPSS Homan, Roger. 1991. The Ethics of Social Research. Lon- and NVivo. don: Longman. A thoughtful analysis of the ethical issues of social science research, by a practicing British Additional Readings social researcher. Lee, Raymond. 1993. Doing Research on Sensitive Topics. Thousand Oaks, CA: Sage. This book examines the Crow, Graham, Rose Wiles, Sue Heath, and Vikki Charles. confl icts between scientifi c research needs and the 2006. “Research Ethics and Data Quality: The Impli- rights of the people involved—with guidelines for cations of Informed Consent.” International Journal dealing with such confl icts. of Social Research Methodology 9 (2): 83–95. Some Sweet, Stephen. 1999. “Using a Mock Institutional Re- researchers worry that increased concerns over view Board to Teach Ethics in Sociological Research.” ethical issues will interfere with the conduct of social Teaching Sociology 27 (Janu ary): 55–59. Though writ- research, whereas others expect quality will improve ten for professors, this article provides some research because of it. These authors examine the question examples that challenge your ethical instincts. and conclude that neither will necessarily occur. PartPart OneTwoOneTwo

ANTHE INTRODUCTIONSTRUCTURING TOOF INQUIRYINQUIRY

14 HumanResearch Inquiry Design and Science 5 Conceptualization, 2 Paradigms,Operationalization, Theory, and ResearchMeasurement

36 TheIndexes, Politics Scales, and EthicsTypologies of Social Research 7 The Logic of Sampling photo credit Kathy Sloane/Photo Researchers, Inc. osing problems properly is often more dif- tarianism. Although such composite measures are P fi cult than answering them. Indeed, a prop- constructed during the analysis of data (see Part 4), erly phrased question often seems to answer itself. the design and execution of data collection must You may have discovered the answer to a question take into account any future uses. simply in the process of making the question clear Finally, we’ll look at how social researchers se- to someone else. lect people or things for observation. Chapter 7, on Part 2 deals with what should be observed; that sampling, addresses the fundamental scientifi c is- is, Part 2 considers the posing of proper scientifi c sue of generalizability. As you’ll see, we can select questions, the structuring of inquiry. Part 3 will de- a few people or things for observation and then scribe some of the specifi c methods of social scien- apply what we observe to a much larger group. For tifi c observation. example, by surveying 2,000 U.S. citizens about Chapter 4 addresses the beginnings of research. whom they favor for president of the United States, It examines some of the purposes of inquiry, units we can accurately predict how tens of millions will of analysis, and the reasons scientists get involved vote. In this chapter, we’ll examine techniques that in research projects. increase the generalizability of what we observe. Chapter 5 deals with the specifi cation of what What you learn in Part 2 will bring you to the it is you want to measure—the processes of con- verge of making controlled social scientifi c obser- ceptualization and operationalization. It looks at vations. Part 3 will then show you how to take that some of the terms that you and I use quite casually next step. in everyday life—prejudice, liberalism, happiness, and so forth—and shows how essential it is to clarify what we really mean by such terms when we do research. This process of clarifi cation is called conceptualization. Once we clarify what we mean by certain terms, we can then measure the referents of those terms. The process of devising steps or operations for measuring what we want to study is called operationalization. Chapter 5 deals with the topic of operationalization in general, paying special attention to the framing of questions for interviews and questionnaires. To complete the introduction to measurement, Chapter 6 breaks with the chronological discussion of how research is conducted. In this chapter, we’ll examine techniques for measuring variables in quantitative research through the combination of several indicators: indexes, scales, and typologies. As an example, we might ask survey respondents fi ve different questions about their attitudes toward gender equality and then combine the answers into a composite measure of gender-based egali-

93 41 HUMANRESEARCH INQUIRY DESIGN AND SCIENCE Photographers Library LTD/eStock Photography/Picture Quest Photo credit Photographers Library LTD/eStock

What You’ll Learn in thisThis ChapterChapter

HereWe’ll you’ll examine see the the wide way variety people of learn research about designs their world available and theto social mistakes researchers, they makeas well along as how the to way. design We’ll a alsostudy—that begin to is, see how what to specify makes exactlysience differentwho or what from is to

otherbe studied ways when, of knowing how, andthings. for what purpose. 2S 1S N L 94 In this chapter . . . WHAT DO YOU THINK?

Introduction In the following let- Three Purposes of Research ter published in a Exploration college newspaper, Description Image not available due to copyright restrictions the Provost objects Explanation to data that had The Logic of Nomothetic Explanation been previously re- Criteria for Nomothetic Causality ported. Nomothetic Causal Analysis and Hypothesis Testing Provost says percentage was wrong False Criteria for Nomothetic Causality I am writing to clarify a misstatement in an editorial in the April 19 The Panther. As recently Necessary and Sufﬁ cient Causes as last fall, the concept behind this statement was presented to your staff. Units of Analysis This current use of erroneous numbers demands correction Individuals The ﬁ gure used in the statement, “With about Groups 52 percent of the faculty being part time . . .” is Organizations absolutely incorrect. Since the thrust of the editorial is Chapman’s Social Interactions ability to live up to its desire to “nurture and help Social Artifacts develop students,” a proper measure of the dif- Units of Analysis in Review ference between full-time faculty presence and that of part-time faculty is how many credits or Faulty Reasoning about Units of Analysis: courses are taught The Ecological Fallacy and Reductionism For the past four years, full-time faculty have taught about 70 percent of the credits in which The Time Dimension students enroll each semester. Cross-Sectional Studies Thus, a large majority of our faculty are here full-time: teaching classes, advising students, at- Longitudinal Studies tending meetings, interacting with students in Approximating Longitudinal Studies the hallways and dining rooms. Examples of Research Strategies Once again, I welcome the opportunity to present the truth. How to Design a Research Project Might I suggest that a future edition of The Panther be devoted to the contributions of part- Getting Started time faculty. Conceptualization Harry L. Hamilton Choice of Research Method Provost Operationalization Sometimes, data seem as though they Population and Sampling dropped out of the sky, making no sense. Observations Which side is correct in this case: the original Data Processing newspaper report or the Provost’s account? Analysis Or are both sides correct? If so, why? See the Application “What Do You Think? Revisited” box toward Research Design in Review the end of the chapter.

95 96 CHAPTER 4 RESEARCH DESIGN

The Research Proposal Hostages’ Experiences with Terrorists Elements of a Research Proposal Firsthand Accounts of Implementing Policies The Ethics of Research Design

Ross goes on to discuss some of the secondary INTRODUCTION materials used by qualitative researchers: “biographies of terrorists, case studies of terrorist organizations, case studies on types of terrorism, Science is dedicated to “fi nding out.” No matter case studies on particular terrorist incidents, and what you want to fi nd out, though, you’ll likely case studies of terrorism in selected regions and discover many ways to go about fi nding it. That’s countries” (2004:27). Quantitative researchers, on true in life generally. Suppose, for example, that the other hand, addressed terrorism in a variety of you want to fi nd out whether a particular automo- ways, including analyses of media coverage, sta- bile—say, the new Burpo-Blasto—would be a good tistical modeling of terrorist events, and the use of car for you. You could, of course, buy one and fi nd various databases relevant to the topic. As you’ll out that way. Or you could talk to a lot of B-B own- see in this chapter, any research topic can be ap- ers or to people who considered buying one and proached from many different directions. didn’t. You might check the classifi ed ads to see if This chapter provides a general introduction to there are a lot of B-Bs being sold cheap. You could research design; the other chapters in Part 2 elabo- read consumer magazines or online evaluations of rate on specifi c aspects of it. In practice, all aspects Burpo-Blastos. A similar situation occurs in scien- of research design are interrelated. As you read tifi c inquiry. through Part 2, the interrelationships among parts Ultimately, scientifi c inquiry comes down to will become clearer. making observations and interpreting what you’ve We’ll start by briefl y examining the main pur- observed, the subjects of Parts 3 and 4 of this book. poses of social research. Then, we’ll consider units Before you can observe and analyze, however, you of analysis—the what or whom you want to study. need a plan. You need to determine what you’re Next we’ll consider alternative ways of handling going to observe and analyze: why and how. That’s time in social research, or how to study a moving what research design is all about. target that changes over time. Although the details vary according to what you With these ideas in hand, we’ll turn to how to wish to study, you face two major tasks in any redesign a research project. This overview of the research design. First, you must specify as clearly as search process serves two purposes: Besides de- possible what it is you want to fi nd out. Second, scribing how you might go about designing a study, you must determine the best way to do it. Interest- it provides a map of the remainder of this book. ingly, if you can handle the fi rst consideration fully, Next, we’ll look at the elements of a research you’ll probably have addressed the second already. proposal. Often you’ll need to detail your inten- As mathematicians say, a properly framed questions before you actually conduct your research, in tion contains the answer. order for you to obtain funding for a major project Let’s say you’re interested in conducting social or perhaps to get your instructor’s approval for a research on terrorism. When Jeffrey Ross (2004) class project. You’ll see that the research proposal addressed this issue, he found the existing stud- provides an excellent opportunity for you to con- ies used a variety of qualitative and quantitative sider all aspects of your research in advance. approaches. Qualitative researchers, for example, Finally, we consider the ethical implications generated original data through of this research design. As you read through this Autobiographies chapter, think about how the practice of social re- Incident Reports and Accounts search in this regard can raise larger issues. THREE PURPOSES OF RESEARCH 97 Earl Babbie Research design involves the creation and integration of many diverse elements.

THREE PURPOSES OF RESEARCH more about the movement: How widespread is it? What levels and degrees of support exist within the community? How is the movement organized? Social research can serve many purposes. Three What kinds of people are active in it? An explor- of the most common and useful purposes are ex- atory study could help you fi nd at least approxi- ploration, description, and explanation. Although mate answers to some of these questions. You most studies have more than one of these pur- might check fi gures with tax-collecting offi cials, poses, examining them separately is useful be- gather and study the literature of the movement, cause each has different implications for other as- attend meetings, and interview leaders. pects of research design. Exploratory studies are also appropriate for more persistent phenomena. Suppose you’re un- Exploration happy with your college’s graduation requirements and want to help change them. You might study Much of social research is conducted to explore the history of such requirements at the college and a topic, that is, to start to familiarize a researcher meet with college offi cials to learn the reasons for with that topic. This approach typically occurs the current standards. You could talk to several when a researcher examines a new interest or students to get a rough idea of their sentiments on when the subject of study itself is relatively new. the subject. Although this last activity would not As an example, let’s suppose that widespread necessarily yield an accurate picture of student taxpayer dissatisfaction with the government opinion, it could suggest what the results of a more erupts into a taxpayers’ revolt. People begin re- extensive study might be. fusing to pay their taxes, and they organize them- Sometimes exploratory research is pursued selves around that issue. You might like to learn through the use of focus groups, or guided small- 98 CHAPTER 4 RESEARCH DESIGN group discussions. This technique is frequently sions with several questions prepared in advance, used in market research; we’ll examine it further each of the interviews followed whatever course in Chapter 10. seemed appropriate in the light of answers given. Exploratory studies are most typically done for This example of exploration illustrates where three purposes: (1) to satisfy the researcher’s cu- social research often begins. Whereas research- riosity and desire for better understanding, (2) to ers working from deductive theories have the key test the feasibility of undertaking a more extensive variables laid out in advance, one of my fi rst tasks study, and (3) to develop the methods to be em- was to identify possibly relevant variables. For ex- ployed in any subsequent study. ample, I noted a channel’s gender, age, education, A while back, for example, I became aware of religious background, regional origins, and previ- the growing popularity of something called “chan- ous participation in things metaphysical. I chose neling,” in which a person known as a channel or most of these variables because they commonly medium enters a trance state and begins speak- affect behavior. ing with a voice or “entity” that claims it originates I also noted differences in the circumstances outside the channel. Some of the voices say they of channeling sessions. Some channels said they come from a spirit world of the dead; some say must go into deep trances, some use light trances, they are from other planets; and still others say and others remain conscious. Most sit down while they exist on dimensions of reality diffi cult to ex- channeling, but others stand and walk about, plain in ordinary human terms. and so forth. Many of these differences became The idea of channeling interested me from sev- apparent to me only in the course of my initial eral perspectives, not the least of which was the observations. methodological question of how to study scientifi - Over the course of this research, I’ve become cally something that violates so much of what we interested in classifying where the entities come take for granted, including scientifi c staples such from and I’ve developed a set of questions to help as space, time, causation, and individuality. me do that. My examination of specifi c topics has Lacking any rigorous theory or precise expecta- also become increasingly focused as I’ve identifi ed tions, I merely set out to learn more. Using some variables that seem worth pursuing: gender, edu- of the techniques of qualitative fi eld research dis- cation, and religion, for example. Note, however, cussed in Chapter 10, I began amassing informa- that I began with a relatively blank slate. tion and forming categories for making sense of Exploratory studies are quite valuable in social what I observed. I read books and articles about scientifi c research. They’re essential whenever a the phenomenon and talked to people who had researcher is breaking new ground, and they al- attended channeling sessions. I then attended most always yield new insights into a topic for channeling sessions myself, observing those who research. Exploratory studies are also a source of attended as well as the channel and entity. Next, grounded theory, as discussed in Chapter 10. I conducted personal interviews with numerous The chief shortcoming of exploratory studies is channels and entities. that they seldom provide satisfactory answers to In most interviews, I began by asking the hu- research questions, although they can hint at the man channels questions about how they fi rst be- answers and can suggest which research meth- gan channeling, what it was like, and why they ods could provide defi nitive answers. The reason continued, as well as standard biographical ques- exploratory studies are seldom defi nitive in them- tions. The channel would then go into a trance, selves has to do with representativeness; that is, whereby the interview continued with the entity the people you study in your exploratory research speaking. “Who are you?” I might ask. “Where do may not be typical of the larger population that you come from?” “How can I tell if you are real or interests you. Once you understand representa- a fake?” Although I went into these interview ses- tiveness, you’ll be able to know whether a given THE LOGIC OF NOMOTHETIC EXPLANATION 99 exploratory study actually answered its research lent episodes among the gangs of different ethnic problem or only pointed the way toward an an- groups. swer. (Representativeness is discussed at length in Reporting the voting intentions of an electorate Chapter 7.) is descriptive, but reporting why some people plan to vote for Candidate A and others for Candidate Description B is explanatory. Reporting why some cities have higher crime rates than others involves explana- Many social scientifi c studies aim at describing sit- tion, as does identifying variables that explain why uations and events. The researcher observes and some cities have higher crime rates than others. A then describes what was observed. Because scien- researcher who sets out to know why an antiabor- tifi c observation is careful and deliberate, however, tion demonstration ended in a violent confronta- scientifi c descriptions are typically more accurate tion with police, as opposed to simply describing and precise than are casual ones. what happened, has an explanatory purpose. For example, the goal of the U.S. Census is to Let’s look at a specifi c case. What factors do you describe accurately and precisely a wide variety of suppose might shape people’s attitudes toward the the population characteristics of the United States legalization of marijuana? To answer this, you as well as areas such as states and counties. Other might fi rst consider whether men and women dif- examples of descriptive studies include the compu- fer in their opinions. An explanatory analysis of the tation of age-gender profi les of populations done 2002 GSS data indicates that 38 percent of men by demographers, the computation of crime rates and 30 percent of women said marijuana should for different cities, and a product-marketing survey be legalized. that describes the people who use, or would use, What about political orientation? The GSS data a particular product. A researcher who carefully show that 55 percent of liberals said marijuana chronicles the events that take place on a labor should be legalized, compared with 29 percent of union picket line serves a descriptive purpose. A moderates and 27 percent of conservatives. Fur- researcher who computes and reports the number ther, 41 percent of Democrats, compared with 34 of times individual legislators voted for or against percent of Independents and 28 percent of Repub- organized labor also fulfi lls a descriptive purpose. licans, supported legalization. Many qualitative studies aim primarily at de- Given these statistics, you might begin to description. An anthropological ethnography, for ex- velop an explanation for attitudes toward mari- ample, may try to detail the particular culture of juana legalization. Further study of gender and some preliterate society. At the same time, such political orientation might then lead to a deeper studies are seldom limited to a merely descriptive explanation of these attitudes. purpose. Researchers usually go on to examine why the observed patterns exist and what they THE LOGIC OF imply. NOMOTHETIC EXPLANATION

Explanation The preceding examination of what factors might The third general purpose of social scientifi c re- cause attitudes about legalizing marijuana illus- search is to explain things. Descriptive studies an- trates nomothetic explanation, as discussed in swer questions of what, where, when, and how; Chapter 1. Recall that in this model, we try to fi nd explanatory studies address questions of why. So a few factors (independent variables) that can ac- when William Sanders (1994) set about describing count for many of the variations in a given phe- the varieties of gang violence, he also wanted to nomenon. This explanatory model stands in con- reconstruct the process that brought about vio- trast to the idiographic model, in which we seek 100 CHAPTER 4 RESEARCH DESIGN a complete, in-depth understanding of a single entation was a cause of attitudes about legalizing case. marijuana. Had the same percentage of liberals In our example, an idiographic approach would and conservatives supported legalization, we could suggest all the reasons that one person was op- hardly say that political orientations caused the at- posed to legalization, involving what her parents, titude. Though this criterion is obvious, it empha- teachers, clergy told her about it, any bad experi- sizes the need to base social research assertions ences experimenting with it, and so forth. When on actual observations rather than assumptions. we understand something idiographically, we feel wereally understand it. When we know all the rea- Time Order Next, we can’t say a causal relation- sons why someone opposed legalizing marijuana, ship exists unless the cause precedes the effect in we couldn’t imagine that person having any other time. Notice that it makes more sense to say that attitude. most children’s religious affi liations are caused by In contrast, a nomothetic approach might sug- those of their parents than to say that parents’ affi l- gest that overall political orientations account for iations are caused by those of their children—even much of the difference of opinion about legalizing though it would be possible for you to change your marijuana. Because this model is inherently proba- religion and for your parents to follow suit. Re- bilistic, it is more open than the idiographic model member, nomothetic explanation deals with “most to misunderstanding and misinterpretation. Let’s cases” but not all. examine what social researchers mean when they In our marijuana example, it would make sense say one variable (nomothetically) causes another. to say that gender causes, to some extent, attitudes Then, we’ll look at what they don’t mean. toward legalization, whereas it would make no sense to say that opinions about marijuana deter- Criteria for Nomothetic Causality mine a person’s gender. Notice, however, that the time order connecting political orientations and at- There are three main criteria for nomothetic causal titudes about legalization is less clear, although we relationships in social research: (1) the variables sometimes reason that general orientations cause must be correlated, (2) the cause takes place before specifi c opinions. And sometimes our analyses the effect, and (3) the variables are nonspurious. involve two or more independent variables that were established at the same time: looking at the CorrelationUnless some actual relationship—or effects of gender and race on voting behavior, for correlation—is found between two variables, example. As we’ll see in the next chapter, the issue we can’t say that a causal relationship exists. Our of time order can be a complex matter. analysis of GSS data suggested that political ori- NonspuriousThe third requirement for a causal relationship is that the effect cannot be explained in terms of some third variable. For example, correlationAn empirical relationship between two there is a correlation between ice cream sales variables such that (1) changes in one are associated with changes in the other or (2) particular attributes of one and deaths due to drowning: the more ice cream variable are associated with particular attributes of the sold, the more drownings, and vice versa. There other. Thus, for example, we say that education and is, however, no direct link between ice cream and income are correlated in that higher levels of education drowning. The third variable at work here is season are associated with higher levels of income. Correlation in or temperature. Most drowning deaths occur dur- and of itself does not constitute a causal relationship between the two variables, but it is one criterion of causality. ing summer—the peak period for ice-cream sales. Here are two more examples of spurious rela- spurious relationshipA coincidental statistical correlation between two variables, shown to be caused by tionships, or ones that aren’t genuine. There is a some third variable. negative relationship between the number of mules THE LOGIC OF NOMOTHETIC EXPLANATION 101

Observed Correlation

Positive (direct) correlation

Shoe size Math skill

Bigger shoe size is associated with greater math skill, and vice versa.

Spurious causal relationships Actual causal relationships

Shoe size Math skill Age

Shoe size Math skill Shoe size Math skill

Neither shoe size nor math skill is a cause The underlying variable of age causes both of the other. bigger shoe size and greater math skill, thus explaining the observed correlation. FIGURE 4-1 An Example of a Spurious Causal Relationship. Finding an empirical correlation between two variables does not necessarily establish a causal relationship. Sometimes the observed correlation is the incidental result of other causal relationships, involving other variables.

and the number of Ph.D.’s in towns and cities: the what the third variable is? In this case, it is the size more mules, the fewer Ph.D.’s and vice versa. Per- of the fi re. haps you can think of another variable that would Thus, when social researchers say there is a explain this apparent relationship. The answer is causal relationship between, say, education and rural versus urban settings. There are more mules racial tolerance, they mean (1) there is a statistical (and fewer Ph.D.’s) in rural areas, whereas the op- correlation between the two variables, (2) a per- posite is true in cities. son’s educational level occurred before their cur- Or, consider the positive correlation between rent level of tolerance or prejudice, and (3) there shoe size and math ability among schoolchildren. is no third variable that can explain away the ob- Here, the third variable that explains the puzzling served correlation as spurious. relationship is age. Older children have bigger feet and more highly developed math skills, on aver- Nomothetic Causal Analysis age, than do younger children. See Figure 4-1 for and Hypothesis Testing an illustration of this spurious relationship. The list goes on. Areas with many storks have The nomothetic model of causal analysis lends high birth rates. Those with few storks have low itself to hypothesis testing (see Chapter 2). To do birth rates. Do storks really deliver babies? Birth this, you would carefully specify the variables you rates are higher in the country than in the city; think are causally related, as well as specifying the more storks live in the country than the city. The manner in which you will measure them. (These third variable here is urban/rural areas. steps will be discussed in detail in the following Finally, the more fi re trucks that put out a fi re, chapter under the terms conceptualization and the more damage to the structure. Can you guess operationalization.) 102 CHAPTER 4 RESEARCH DESIGN

In addition to hypothesizing that two vari- tional cases do not disprove the overall, causal ables will be correlated with each other, you could pattern. specify the strength of the relationship you expect within the study design you’re using. Often this Majority of Cases Causal relationships can be specifi cation will take the form of a level of statis- true even if they do not apply in a majority of cases. tical signifi cance: the probability that a given rela- For example, we say that children who are not su- tionship might have been caused by chance in the pervised after school are more likely to become de- selection of subjects for study (see Chapter 7 for linquent than are those who are supervised; hence, more on this). lack of supervision is a cause of delinquency. This Finally, you could specify the tests for spurious- causal relationship holds true even if only a small ness that any observed relationship must survive. percentage of those not supervised become delin- Not only would you hypothesize, for example, that quent. As long as they are more likely than those increased education will reduce levels of prejudice, who are supervised to be delinquent, we say there but you would specify further that the hypothesized is a causal relationship. relationship will not be the product of, say, political The social scientifi c view of causation may vary orientations. from what you are accustomed to, because people commonly use the term cause to mean something False Criteria for Nomothetic Causality that completely causes another thing. The somewhat different standard used by social researchers Because notions of cause and effect are well en- can be seen more clearly in terms of necessary and trenched in everyday language and logic, it’s im- suffi cient causes. portant to specify some of the things social researchers do not mean when they speak of causal relationships. When they say that one variable NECESSARY AND SUFFICIENT CAUSES causes another, they do not necessarily mean to suggest complete causation, to account for exceptional cases, or to claim that the causation exists in A necessary cause represents a condition that must a majority of cases. be present for the effect to follow. For example, it is necessary for you to take college courses in or- Complete Causation Whereas an idiographic der to get a degree. Take away the courses, and explanation of causation is relatively complete, a the degree never happens. However, simply taking nomothetic explanation is probabilistic and usu- the courses is not a suffi cient cause of getting a ally incomplete. As we’ve seen, social researchers degree. You need to take the right ones and pass may say that political orientations cause attitudes them. Similarly, being female is a necessary condi- toward legalizing marijuana even though not all tion of becoming pregnant, but it is not a suffi cient liberals approve nor all conservatives disapprove. cause. Otherwise, all women would get pregnant. Thus, we say that political orientation is one of the Figure 4-2 illustrates this relationship between causes of the attitude, but not the only one. the variables of gender and pregnancy as a matrix showing the possible outcomes of combining Exceptional Cases In nomothetic explanations, these variables. exceptions do not disprove a causal relationship. A suffi cient cause, on the other hand, represents For example, it is consistently found that women a condition that, if it is present, guarantees the ef- are more religious than men in the United States. fect in question. This is not to say that a suffi cient Thus, gender may be a cause of religiosity, even cause is the only possible cause of a particular if your uncle is a religious zealot or you know a effect. For example, skipping an exam in this course woman who is an avowed atheist. Those excep- would be a suffi cient cause for failing it, though stu- NECESSARY AND SUFFICIENT CAUSES 103

Male Female Pregnant Not pregnant

FIGURE 4-2 Necessary Cause. Being female is a necessary cause of pregnancy; that is, you can’t get pregnant unless you are female.

dents could fail it other ways as well. Thus, a cause Took the exam Didn't take the exam can be sufﬁ cient, but not necessary. Figure 4-3 illustrates the relationship between taking or not F F F taking the exam and either passing or failing it. F F The discovery of a cause that is both necessary F F F F and sufﬁ cient is, of course, the most satisfying out- F F F

come in research. If juvenile delinquency were the Failed the exam effect under examination, it would be nice to dis- A C A B cover a single condition that (1) must be present D A A for delinquency to develop and (2) always results D C B C in delinquency. In such a case, you would surely B C C B A feel that you knew precisely what caused juvenile C B D D D A delinquency. C

Passed the exam A C A Unfortunately, when analyzing the nomothetic relationships among variables, we never discover FIGURE 4-3 Suffi cient Cause. Not taking the single causes that are absolutely necessary and exam is a suffi cient cause of failing it, even though absolutely suffi cient. It is not uncommon, however, there are other ways of failing (such as answering randomly). to fi nd causal factors that are either 100 percent necessary (you must be female to become pregnant) or 100 percent suffi cient (skipping an exam will cause you to fail it). (Anyone with all the same details of your genetic In the idiographic analysis of single cases, you inheritance, upbringing, and subsequent experi- may reach a depth of explanation from which it is ences would have ended up going to college.) At reasonable to assume that things could not have the same time, there could always be other causal turned out differently, suggesting you have deter- paths to the same result. Thus, the idiographic mined the suffi cient causes for a particular result. causes are suffi cient but not necessary. 104 CHAPTER 4 RESEARCH DESIGN

UNITS OF ANALYSIS are its activities? So we might study 52 gangs, reporting that 40 were male and 12 were female, and so forth. The “gang” would be the unit of analysis, In social research, there is virtually no limit to what even though some of the characteristics were drawn or whom can be studied, or the units of analysis. from the components (members) of the gangs. This topic is relevant to all forms of social research, Social researchers perhaps most typically although its implications are clearest in the case of choose individual people as their units of analy- nomothetic, quantitative studies. sis. You might note the characteristics of individual The idea for units of analysis may seem slippery people—gender, age, region of birth, attitudes, and at fi rst, because research—especially nomothetic so forth. You could then combine these descrip- research—often studies large collections of people tions to provide a composite picture of the group or things, or aggregates. It’s important to distin- the individuals represent, whether a street-corner guish between the unit of analysis and the aggre- gang or a whole society. gates that we generalize about. For example, you might note the age and gen- For instance, a researcher may study a class of der of each student enrolled in Political Science people, such as Democrats, college undergradu- 110 and then characterize the group of students as ates, or African American women under 30. But if being 53 percent men and 47 percent women and the researcher is interested in exploring, describ- as having a mean age of 18.6 years. Although the ing, or explaining how different groups of individu- fi nal description would be of the class as a whole, als behave as individuals, the unit of analysis is the the description would be based on characteristics individual, not the group. This is so even though the that members of the class have as individuals. researcher then proceeds to generalize about ag- The same distinction between units of analysis gregates of individuals, as in saying that more Dem- and aggregations occurs in explanatory studies. ocrats than Republicans favor legalizing marijuana. Suppose you wished to discover whether students Think of it this way: Having an attitude about with good study habits received better grades in marijuana is something that can be an attribute Political Science 110 than did students with poor only of an individual, not a group; that is, there is study habits. You would operationalize the variable no one group “mind” that can have an attitude. So study habits and measure this variable, perhaps in even when we generalize about Democrats, we’re terms of hours of study per week. You might then generalizing about an attribute they possess as aggregate students with good study habits and individuals. those with poor study habits and see which group In contrast, we may sometimes want to study received the best grades in the course. The pur- groups, considered as individual “actors” or enti- pose of the study would be to explain why some ties that have attributes as groups. For instance, groups of students do better in the course than do we might want to compare the characteristics of others, but the unit of analysis would still be indi- different types of street gangs. In that case our vidual students. unit of analysis would be gangs (not members of Units of analysis in a study are usually also the gangs), and we might proceed to make generaliza- units of observation. Thus, to study success in a tions about different types of gangs. For example, political science course, we would observe individ- we might conclude that male gangs are more vio- ual students. Sometimes, however, we “observe” lent than female gangs. Each gang (unit of analy- our units of analysis indirectly. For example, sup- sis) would be described in terms of two variables: pose we want to fi nd out whether disagreements (1) What sex are the members? and (2) How violent about the death penalty tend to cause divorce. In this case, we might “observe” individual husbands and wives by asking them about their attitudes units of analysis The what or whom being studied. In social science research, the most typical units of analysis about capital punishment, in order to distinguish are individual people. couples who agree and disagree on this issue. In UNITS OF ANALYSIS 105 this case, our units of observation are individual 4. Of those with an identifi ed motive, 19.5 per- wives and husbands, but our units of analysis (the cent (n 337) are retaliatory. — (KUBRIN AND things we want to study) are couples. WEITZER 2003:163) Units of analysis, then, are those things we ex- In each of these excerpts, the unit of analysis amine in order to create summary descriptions of is homicide (also called killing or murder). Some- all such units and to explain differences among times you can identify the unit of analysis in the them. In most research projects, the unit of anal- description of the sampling methods, as in the fi rst ysis will probably be clear to you. When the unit excerpt. A discussion of classifi cation methods of analysis is not clear, however, it’s essential to might also identify the unit of analysis, as in the determine what it is; otherwise, you cannot de- second excerpt (80 ways to code the homicides). termine what observations are to be made about Often, numerical summaries point the way: 2,161 whom or what. homicides; 19.5 percent (of the homicides). With a Some studies try to describe or explain more little practice you will be able to identify the units than one unit of analysis. In these cases, the re- of analysis in most social research reports, even searcher must anticipate what conclusions she or when more than one is used in a given analysis. he wishes to draw with regard to which units of To explore this topic in more depth, let’s con- analysis. For example, we may want to discover sider several common units of analysis in social what kinds of college students (individuals) are research. most successful in their careers; we may also want to learn what kinds of colleges (organizations) Individuals produce the most successful graduates. Here’s an example that illustrates the com- As mentioned earlier, individual human beings are plexity of units of analysis. Murder is a fairly per- perhaps the most typical units of analysis for social sonal matter: one individual kills another indi- research. We tend to describe and explain social vidual. However, when Charis Kubrin and Ronald groups and interactions by aggregating and ma- Weitzer (2003:157) ask, “Why do these neighbor- nipulating the descriptions of individuals. hoods generate high homicide rates?” the unit Any type of individual can be the unit of analysis of analysis in that phrase is neighborhood. You for social research. This point is more important can probably imagine some kinds of neighbor- than it may seem at fi rst. The norm of generalized hood (such as poor, urban) that would have high understanding in social research should suggest homicide rates and some (such as wealthy, subur- that scientifi c fi ndings are most valuable when they ban) that would have low homicide rates. In this apply to all kinds of people. In practice, however, so- particular conversation, the unit of analysis (neigh- cial researchers seldom study all kinds of people. At borhood) would be categorized in terms of variables the very least, their studies are typically limited to the such as economic level, locale, and homicide rate. people living in a single country, though some com- In their analysis, however, Kubrin and Weitzer parative studies stretch across national boundaries. were also interested in different types of homicide: Often, however, studies are quite circumscribed. in particular, those that occurred in retaliation for Examples of classes of individuals that might some earlier event, such as an assault or insult. be chosen for study include students, gays and Can you identify the unit of analysis common to all lesbians, auto workers, voters, single parents, and of the following excerpts? faculty members. Note that each of these terms implies some population of individuals. Descrip- 1. The sample of killings . . . tive studies with individuals as their units of anal- 2. The coding instrument includes over 80 items ysis typically aim to describe the population that related to the homicide. comprises those individuals, whereas explanatory 3. Of the 2,161 homicides that occurred from studies aim to discover the social dynamics oper- 1985 [to] 1995 . . . ating within that population. 106 CHAPTER 4 RESEARCH DESIGN

As the units of analysis, individuals can be char- the percentage of all families that have a college- acterized in terms of their membership in social educated head of family. In an explanatory study, groupings. Thus, an individual can be described as we might determine whether such families have, belonging to a rich family or to a poor one, or as on average, more or fewer children than do fami- having a college-educated mother or not. We might lies headed by people who have not graduated examine in a research project whether people with from college. In each of these examples, the family college-educated mothers are more likely to attend is the unit of analysis. In contrast, had we asked college than are those with non-college-educated whether college-educated individuals have more mothers, or whether high school graduates in rich or fewer children than do their less-educated families are more likely than those in poor families counterparts, then the individual would have been to attend college. In each case, the unit of analy- the unit of analysis. sis—the “thing” whose characteristics we are seek- Other units of analysis at the group level could be ing to describe or explain—is the individual. We friendship cliques, married couples, census blocks, then aggregate these individuals and make gen- cities, or geographic regions. As with individu- eralizations about the population they belong to. als, each of these terms implies some population. Street gangs implies some population that includes Groups all street gangs, perhaps in a given city. You might then describe this population by generalizing from Social groups can also be units of analysis in social your fi ndings about individual gangs. For instance, research. That is, we may be interested in charac- you might describe the geographic distribution of teristics that belong to one group, considered as a gangs throughout a city. In an explanatory study single entity. If you were to study the members of of street gangs, you might discover whether large a criminal gang to learn about criminals, the indi- gangs are more likely than small ones to engage in vidual (criminal) would be the unit of analysis; but intergang warfare. Thus, you would arrive at con- if you studied all the gangs in a city to learn the dif- clusions about the population of gangs by using ferences, say, between big gangs and small ones, individual groups as your unit of analysis. between “uptown” and “downtown” gangs, and so forth, you would be interested in gangs rather than Organizations their individual members. In this case, the unit of analysis would be the gang, a social group. Formal social organizations can also be the units Here’s another example. Suppose you were in- of analysis in social research. For example, a re- terested in the question of access to computers in searcher might study corporations, by which he different segments of society. You might describe or she implies a population of all corporations. families in terms of total annual income and ac- Individual corporations might be characterized in cording to whether or not they had computers. terms of their number of employees, net annual You could then aggregate families and describe profi ts, gross assets, number of defense contracts, the mean income of families and the percentage percentage of employees from racial or ethnic mi- with computers. You would then be in a position to nority groups, and so forth. We might determine determine whether families with higher incomes whether large corporations hire a larger or smaller were more likely to have computers than were percentage of minority group employees than do those with lower incomes. In this case, the unit of small corporations. Other examples of formal so- analysis would be families. cial organizations suitable as units of analysis in- As with other units of analysis, we can derive clude church congregations, colleges, army divi- the characteristics of social groups from those of sions, academic departments, and supermarkets. their individual members. Thus, we might describe Figure 4-4 provides a graphic illustration of a family in terms of the age, race, or education some different units of analysis and the statements of its head. In a descriptive study, we might fi nd that might be made about them. Units of Analysis Sample Statements

60% of the sample are women

10% of the sample are wearing an eye patch

Individuals 10% of the sample have pigtails

20% of the families have a single parent

50% of the families have two children

20% of the famillies have no children

Families The mean number of children per family is 1.3

20% of the households are occupied by more than one family

30% of the households have holes in their roofs

10% of the households are occupied by aliens Households Notice also that 33% of the families live in multiple-family households with family as the unit of analysis

FIGURE 4-4 Illustrations of Units of Analysis. Units of analysis in social research can be individuals, groups, or even nonhuman entities. 108 CHAPTER 4 RESEARCH DESIGN

Social Interactions the books, they often play insignifi cant roles, remaining both inconspicuous and nameless. Sometimes social interactions are the relevant — (WEITZMAN ET AL. 1972:1128) units of analysis. Instead of studying individual humans, you can study what goes on between them: In a more recent study, Roger Clark, Rachel telephone calls, kisses, dancing, arguments, fi st- Lennon, and Leana Morris (1993) concluded that fi ghts, e-mail exchanges, chat-room discussions, male and female characters are now portrayed and so forth. As you saw in Chapter 2, social inter- less stereotypically than before, observing a clear action is the basis for one of the primary theoretical progress toward portraying men and women in paradigms in the social sciences, and the number nontraditional roles. However, they did not fi nd toof units of analysis that social interactions provide tal equality between the sexes. is nearly infi nite. As these examples suggests, just as people or Even though individuals are usually the actors social groups imply populations, each social ob- in social interactions, there is a difference between ject implies a set of all objects of the same class: (1) comparing the kinds of people who subscribe all books, all novels, all biographies, all introduc- to different Internet service providers (individuals tory sociology textbooks, all cookbooks, all press being the unit of analysis) and (2) comparing the conferences. In a study using books as the units length of chat-room discussions on those same of analysis, an individual book might be character- ISPs (the discussion being the unit of analysis). ized by size, weight, length, price, content, number of pictures, number sold, or description of its Social Artifacts author. Then the population of all books or of a particular kind of book could be analyzed for the Another unit of analysis is the social artifact, or purpose of description or explanation: what kinds any product of social beings or their behavior. One of books sell best and why, for example. class of artifacts includes concrete objects such as Similarly, a social researcher could analyze books, poems, paintings, automobiles, buildings, whether paintings by Russian, Chinese, or U.S. art- songs, pottery, jokes, student excuses for missing ists showed the greatest degree of working-class exams, and scientifi c discoveries. consciousness, taking paintings as the units of For example, Lenore Weitzman and her associ- analysis and describing each, in part, by the na- ates (1972) were interested in learning how gen- tionality of its creator. Or you might examine a der roles are taught. They chose children’s picture newspaper’s editorials regarding a local university, books as their unit of analysis. Specifi cally, they for the purpose of describing, or perhaps explain- examined books that had received the Caldecott ing, changes in the newspaper’s editorial position Medal. Their results were as follows: on the university over time. In this example, individual editorials would be the units of analysis. We found that females were underrepresented Social interactions form another class of social in the titles, central roles, pictures, and sto- artifacts suitable for social research. For example, ries of every sample of books we examined. we might characterize weddings as racially or re- Most children’s books are about boys, men, ligiously mixed or not, as religious or secular in male animals, and deal exclusively with male ceremony, as resulting in divorce or not, or by de- adventures. Most pictures show men singly or scriptions of one or both of the marriage partners in groups. Even when women can be found in (such as “previously married,” “Oakland Raider fan,” “wanted by the FBI”). When a researcher reports that weddings between partners of different social artifact Any product of social beings or their religions are more likely to be performed by secu- behavior. Can be a unit of analysis. lar authorities than are those between partners of UNITS OF ANALYSIS 109 the same religion, the weddings are the units of fact—is irrelevant. The key is to be clear about analysis, not the individuals involved. what your unit of analysis is. When you embark on Other social interactions that might be units of a research project, you must decide whether you’re analysis include friendship choices, court cases, studying marriages or marriage partners, crimes traffi c accidents, divorces, fi stfi ghts, ship launch- or criminals, corporations or corporate executives. ings, airline hijackings, race riots, fi nal exams, Otherwise, you run the risk of drawing invalid con- student demonstrations, and congressional hear- clusions because your assertions about one unit of ings. Congressional hearings, for instance, could analysis are actually based on the examination of be characterized by whether or not they occurred another. We’ll see an example of this issue as we during an election campaign, whether the commit- look at the ecological fallacy in the next section. tee chairs were running for a higher offi ce, whether they had received campaign contributions from Faulty Reasoning about Units interested parties, and so on. Notice that even if of Analysis: The Ecological Fallacy we characterized and compared the hearings in and Reductionism terms of the committee chairs, the hearings themselves—not the individual chairpersons—would be At this point, it’s appropriate to introduce two types our units of analysis. of faulty reasoning: the ecological fallacy and reductionism. Each represents a potential pitfall re- Units of Analysis in Review garding units of analysis, and either can occur in doing research and drawing conclusions from the The examples in this section should suggest the results. nearly infi nite variety of possible units of analysis in social research. Although individual human be- The Ecological Fallacy In this context, ecologi- ings are typical objects of study, many research cal refers to groups or sets or systems: something questions can be answered more appropriately larger than individuals. The ecological fallacy is through the examination of other units of analy- the assumption that something learned about an sis. Indeed, social researchers can study just about ecological unit says something about the individu- anything that bears on social life. als making up that unit. Let’s consider a hypotheti- Moreover, the types of units of analysis named cal illustration of this fallacy. in this section do not begin to exhaust the possi- Suppose we’re interested in learning something bilities. Morris Rosenberg (1968:234–48), for ex- about the nature of electoral support received by ample, speaks of individual, group, organizational, a female political candidate in a recent citywide institutional, spatial, cultural, and societal units election. Let’s assume we have the vote tally for of analysis. Lofl and and colleagues (2006:122–32) each precinct so we can tell which precincts gave speak of practices, episodes, encounters, roles her the greatest support and which the least. As- and social types, social and personal relationships, sume also that we have census data describing groups and cliques, organizations, settlements some characteristics of these precincts. Our anal- and habitats, subcultures and lifestyles as suitable ysis of such data might show that precincts with units of study. The important thing here is to grasp relatively young voters gave the female candidate the logic of units of analysis. Once you do, only a greater proportion of their votes than did pre- your imagination limits the possibilities for fruitful cincts with older voters. We might be tempted to research. Categorizing possible units of analysis may make the concept seem more complicated than it needs to be. What you call a given unit of analysis— ecological fallacy Erroneously basing conclusions a group, a formal organization, or a social arti- about individuals solely on the observation of groups. 110 CHAPTER 4 RESEARCH DESIGN conclude from these fi ndings that young voters are rich without any formal education, that doesn’t more likely to vote for female candidates than are deny the general pattern of higher education relat- older voters—in other words, that age affects sup- ing to higher income. port for the woman. In reaching such a conclusion, The ecological fallacy deals with something else we run the risk of committing the ecological fal- altogether—confusing units of analysis in such a lacy because it may have been the older voters in way that we base conclusions about individuals those “young” precincts who voted for the woman. solely on the observation of groups. Although the Our problem is that we have examined precincts as patterns observed among variables at the level of our units of analysis but wish to draw conclusions groups may be genuine, the danger lies in reason- about voters. ing from the observed attributes of groups to the The same problem would arise if we discov- attributes of the individuals who made up those ered that crime rates were higher in cities having groups, when we have not actually observed large African American populations than in those individuals. with few African Americans. We would not know if the crimes were actually committed by African Reductionism A second type of potentially faulty Americans. Or if we found suicide rates higher in reasoning related to units of analysis is reduction- Protestant countries than in Catholic ones, we still ism. Reductionism involves attempts to explain a could not know for sure that more Protestants than particular phenomenon in terms of limited and/or Catholics committed suicide. lower-order concepts. The reductionist explana- In spite of these hazards, social researchers very tion is not altogether wrong; it is simply too lim- often have little choice but to address a particular ited. Thus, you might attempt to predict this year’s research question through an ecological analysis. winners and losers in the National Basketball As- Perhaps the most appropriate data are simply not sociation by focusing on the abilities of the individ- available. For example, the precinct vote tallies ual players in each team. This is certainly not stu- and the precinct characteristics mentioned in our pid or irrelevant, but the success or failure of teams initial example might be easy to obtain, but we involves more than just the individuals in them; it may not have the resources to conduct a postelec- involves coaching, teamwork, strategies, fi nances, tion survey of individual voters. In such cases, we facilities, fan loyalty, and so forth. To understand may reach a tentative conclusion, recognizing and why some teams do better than others, you would noting the risk of an ecological fallacy. make “team” the unit of analysis, and the quality Although you should be careful not to com- of players would be one variable you would prob- mit the ecological fallacy, don’t let these warnings ably want to use in describing and classifying the lead you into committing what we might call the teams. individualistic fallacy. Some people who approach Thus, different academic disciplines approach social research for the fi rst time have trouble rec- the same phenomenon quite differently. Sociolo- onciling general patterns of attitudes and actions gists tend to consider sociological variables (such with individual exceptions. But generalizations as values, norms, and roles), economists ponder and probabilistic statements are not invalidated economic variables (such as supply and demand by such exceptions. Your knowing a rich Demo- and marginal value), and psychologists examine crat, for example, doesn’t deny the fact that most psychological variables (such as personality types rich people vote Republican—as a general pattern. and traumas). Explaining all or most human be- Similarly, if you know someone who has gotten havior in terms of economic factors is called economic reductionism; explaining all or most human behavior in terms of psychological factors is called psychological reductionism; and so forth. Notice reductionism A fault of some researchers: a strict limitation (reduction) of the kinds of concepts to be con- how this issue relates to the discussion of theoreti- sidered relevant to the phenomenon under study. cal paradigms in Chapter 2. THE TIME DIMENSION 111

For many social scientists, the fi eld of sociobiol- We can choose to make observations more or less ogy is a prime example of reductionism, suggest- at one time or over a long period. ing that all social phenomena can be explained Time plays many roles in the design and execu- in terms of biological factors. Thus, for example, tion of research, quite aside from the time it takes Edward O. Wilson (1975) sought to explain altru- to do research. Earlier we noted that the time se- istic behavior in human beings in terms of our ge- quence of events and situations is critical to deter- netic makeup. In his neo-Darwinian view, Wilson mining causation (a point we’ll return to in Part 4). suggests that humans have evolved in such a way Time also affects the generalizability of research that individuals sometimes need to sacrifi ce them- fi ndings. Do the descriptions and explanations re- selves for the benefi t of the whole species. Some sulting from a particular study accurately represent people might explain such sacrifi ce in terms of ide- the situation of ten years ago, ten years from now, als or warm feelings between humans. However, or only the present? Researchers have two princi- genes are the essential unit in Wilson’s paradigm, pal options for dealing with the issue of time in the producing his famous dictum that human beings design of their research: cross-sectional studies are “only DNA’s way of making more DNA.” and longitudinal studies. Reductionism of any type tends to suggest that particular units of analysis or variables are more Cross-Sectional Studies relevant than others. Suppose we ask what caused the American Revolution. Was it a shared commit- A cross-sectional study involves observations of ment to the value of individual liberty? The eco- a sample, or cross section, of a population or phe- nomic plight of the colonies in relation to Britain? nomenon that are made at one point in time. Ex- The megalomania of the founders? As soon as we ploratory and descriptive studies are often cross- inquire about the single cause, we run the risk of sectional. A single U.S. Census, for instance, is a reductionism. If we were to regard shared values study aimed at describing the U.S. population at a as the cause of the American Revolution, our unit given time. of analysis would be the individual colonist. An Many explanatory studies are also cross-sec- economist, though, might choose the 13 colonies tional. A researcher conducting a large-scale na- as units of analysis and examine the economic tional survey to examine the sources of racial and organizations and conditions of each. A psy- religious prejudice would, in all likelihood, be deal- chologist might choose individual leaders as the ing with a single time frame—taking a snapshot, units of analysis for purposes of examining their so to speak, of the sources of prejudice at a par- personalities. ticular point in history. Like the ecological fallacy, reductionism can Explanatory cross-sectional studies have an occur when we use inappropriate units of analy- inherent problem. Although their conclusions are sis. The appropriate unit of analysis for a given based on observations made at only one time, typi- research question, however, is not always clear. cally they aim at understanding causal processes Social researchers, especially across disciplinary that occur over time. This is akin to determining boundaries, often debate this issue. the speed of a moving object from a high-speed, still photograph. THE TIME DIMENSION Yanjie Bian, for example, conducted a survey of workers in Tianjin, China, to study stratifi cation in contemporary urban Chinese society. In undertak- So far in this chapter, we have regarded research design as a process for deciding what aspects we shall observe, of whom, and for what purpose. Now we must consider a set of time-related options that cross-sectional study A study based on observa- cuts across each of these earlier considerations. tions representing a single point in time. 112 CHAPTER 4 RESEARCH DESIGN ing the survey in 1988, however, he was conscious Many fi eld research projects, involving direct of the important changes brought about by a series observation and perhaps in-depth interviews, are of national campaigns, such as the Great Proletar- naturally longitudinal. Thus, for example, when ian Cultural Revolution, dating from the Chinese Ramona Asher and Gary Fine (1991) studied the Revolution in 1949 (which brought the Chinese life experiences of the wives of alcoholic men, they Communists into power) and continuing into the were in a position to examine the evolution of the present. wives’ troubled marital relationships over time, sometimes even including the reactions of the These campaigns altered political atmospheres subjects to the research itself. and affected people’s work and nonwork activi- In the classic study When Prophecy Fails ties. Because of these campaigns, it is diffi cult to (1956), Leon Festinger, Henry Reicker, and Stanley draw conclusions from a cross-sectional social Schachter set out to learn what happened to a fl y- survey, such as the one presented in this book, ing saucer cult when its predictions of an alien en- about general patterns of Chinese workplaces counter failed to come true. Would the cult mem- and their effects on workers. Such conclusions bers close down the group, or would they become may be limited to one period of time and are all the more committed to their beliefs? A longitudi- subject to further tests based on data collected nal study was required to provide an answer. (They at other times. — (1994:19) redoubled their efforts to get new members.) As you’ll see, this book repeatedly addresses the Longitudinal studies can be more diffi cult for problem of using a “snapshot” to make generaliza- quantitative studies such as large-scale surveys. tions about social life. One solution is suggested Nonetheless, they are often the best way to study by Bian’s fi nal comment—about data collected “at changes over time. There are three special types of other times”: Social research often involves revisit- longitudinal studies that you should know about: ing phenomena and building on the results of ear- trend studies, cohort studies, and panel studies. lier research. Trend Studies A trend study is a type of lon- Longitudinal Studies gitudinal study that examines changes within a population over time. A simple example is a In contrast to cross-sectional studies, a longitu- comparison of U.S. Censuses over a period of dinal study is designed to permit observations of decades, showing shifts in the makeup of the na- the same phenomenon over an extended period. tional population. A similar use of archival data For example, a researcher can participate in and was made by Michael Carpini and Scott Keeter observe the activities of a UFO cult from its incep- (1991), who wanted to know whether contem- tion to its demise. Other longitudinal studies use porary U.S. citizens were better or more poorly records or artifacts to study changes over time. In informed about politics than were citizens of an analyses of newspaper editorials or Supreme Court earlier generation. To fi nd out, they compared the decisions over time, for example, the studies are results of several Gallup polls conducted during longitudinal whether the researcher’s actual ob- the 1940s and 1950s with a 1989 survey that asked servations and analyses were made at one time or several of the same questions tapping political over the course of the actual events under study. knowledge. Overall, the analysis suggested that contemporary citizens were slightly better informed than were earlier generations. In 1989, 74 percent of longitudinal study A study design involving data collected at different points in time. the sample could name the vice president of the trend study A type of longitudinal study in which a United States, compared with 67 percent in 1952. given characteristic of some population is monitored Substantially higher percentages could explain over time. presidential vetoes and congressional overrides THE TIME DIMENSION 113

30 Age

0 1980 1990 2000 Year FIGURE 4-5 A Cohort Study Design. Each of the three groups shown here is a sample representing people who were born in 1960.

of vetoes than could people in 1947. On the other studied in each survey would differ, each sample hand, more of the 1947 sample could identify their would represent the cohort born between 1940 U.S. representative (38 percent) than could the and 1945. 1989 sample (29 percent). Figure 4-5 offers a graphic illustration of a co- An in-depth analysis, however, indicates that hort design. In the example, three studies are being the slight increase in political knowledge resulted compared: One was conducted in 1980, another from the fact that the people in the 1989 sample in 1990, and the third in 2000. Those who were were more highly educated than were those from 20 years old in the 1980 study are compared with earlier samples. When educational levels were those who were 30 in the 1990 study and those taken into account, the researchers concluded that who were 40 in the 2000 study. Although the sub- political knowledge has actually declined within jects being described in each of the three groups specifi c educational groups. are different, each set of subjects represents the same cohort: those who were born in 1960. Cohort Studies In a cohort study, a researcher James Davis (1992) turned to a cohort analysis examines specifi c subpopulations, or cohorts, as in an attempt to understand shifting political ori- they change over time. Typically, a cohort is an age entations during the 1970s and 1980s in the United group, such as people born during the 1950s, but States. Overall, he found a liberal trend on issues it can also be some other time grouping, such as such as race, gender, religion, politics, crime, and people born during the Vietnam War, people who free speech. But did this trend represent people in got married in 1994, and so forth. An example of a general getting a bit more liberal, or did it merely cohort study would be a series of national surveys, refl ect a higher ratio of liberal younger people to conducted perhaps every 20 years, to study the at- conservative older ones? titudes of the cohort born during World War II toward U.S. involvement in global affairs. A sample of people 15–20 years of age might be surveyed in cohort study A study in which some specifi c sub- 1960, another sample of those 35–40 years of age population, or cohort, is studied over time, although data in 1980, and another sample of those 55–60 years may be collected from different members in each set of of age in 2000. Although the specifi c set of people observations. 114 CHAPTER 4 RESEARCH DESIGN

TABLE 4-1 Age and Political Liberalism

Survey Dates 1972 to 1974 1977 to 1980 1982 to 1984 1987 to 1989

Age of cohort 20–24 25–29 30–34 35–39 Percent who would let the Communist speak 72 68 73 73

To answer this question, Davis examined na- sets showing those over 65 voicing less support for tional surveys (from the General Social Survey, education funding than did those under 65. which he helped found) conducted in four periods, Such simplistic analyses, however, leave out an fi ve years apart. In each survey, he grouped the re- important variable: increasing support for educa- spondents into age groups, also fi ve years apart. tional funding in U.S. society over time in general. This strategy allowed him to compare different age The researchers add to this the concept of “gen- groups at any given point in time as well as fol- erational replacement,” meaning that the older re- low the political development of each age group spondents in a survey grew up during a time when over time. there was less support for education in general, One of the questions he examined was whether whereas the younger respondents grew up during a person who admitted to being a Communist a time of greater overall support. should be allowed to speak in the respondents’ A cohort analysis allowed the researchers to communities. Consistently, the younger respon- determine what happened to the attitudes of spe- dents in each period were more willing to let cifi c cohorts over time. Here, for example, are the the Communist speak than were the older ones. percentages of Americans born during the 1940s Among those aged 20–40 in the fi rst set of the sur- who felt educational spending was too low, when vey, for example, 72 percent took this liberal po- members of that cohort were interviewed over sition, compared with 27 percent among respon- time (Plutzer and Berkman 2005:76): dents 80 and older. What Davis found when he examined the youngest cohort over time is shown Percent Who Say in Table 4-1. This pattern of a slight, conservative Educational Funding Year Interviewed Is Too Low shift in the 1970s, followed by a liberal rebound in the 1980s, typifi es the several cohorts Davis ana- 1970s 58 lyzed (J. Davis 1992:269). 1980s 66 In another study, Eric Plutzer and Michael Berk- 1990s 74 man (2005) used a cohort design to reverse a 2000s 79 prior conclusion regarding aging and support for education. Logically, as people grow well beyond As these data indicate, those who were born dur- the child-rearing years, we might expect them to ing the 1940s have steadily increased their sup- reduce their commitment to educational funding. port for educational funding as they have passed Moreover, cross-sectional data support that ex- through and beyond the child-rearing years. pectation. The researchers present several data Panel Studies Though similar to trend and cohort studies, a panel study examines the same set of panel study A type of longitudinal study, in which data people each time. For example, we could interview are collected from the same set of people (the sample or the same sample of voters every month during an panel) at several points in time. election campaign, asking for whom they intended THE TIME DIMENSION 115 to vote. Though such a study would allow us to “the Depression generation,” specifi cally, say, peo- analyze overall trends in voter preferences for dif- ple who were between 20 and 30 in 1932. We could ferent candidates, it would also show the precise study a sample of people 30–40 years old in 1942, patterns of persistence and change in intentions. a new sample of people aged 40–50 in 1952, and For example, a trend study that showed that Can- so forth. A panel study could start with a sample didates A and B each had exactly half of the voters of the whole population or of some special sub- on September 1 and on October 1 as well could set and study those specifi c individuals over time. indicate that none of the electorate had changed Notice that only the panel study would give a full voting plans, that all of the voters had changed picture of the shifts among the various categories their intentions, or something in-between. A panel of affi liations, including “none.” Cohort and trend study would eliminate this confusion by showing studies would uncover only net changes. what kinds of voters switched from A to B and what Longitudinal studies have an obvious advantage kinds switched from B to A, as well as other facts. over cross-sectional ones in providing information Joseph Veroff, Shirley Hatchett, and Elizabeth describing processes over time. But this advan- Douvan (1992) wanted to learn about marital ad- tage often comes at a heavy cost in both time and justment among newlyweds and focused on differ- money, especially in a large-scale survey. Obser- ences between white and African American cou- vations may have to be made at the time events ples. To get subjects for study, they selected a sample are occurring, and the method of observation may of couples who applied for marriage licenses in require many research workers. Wayne County, Michigan, April through June 1986. Panel studies, which offer the most compre- Concerned about the possible impact their re- hensive data on changes over time, face a special search might have on the couples’ marital adjust- problem: panel attrition. Some of the respondents ment, the researchers divided their sample in half studied in the fi rst wave of the survey may not at random: an experimental group and a control participate in later waves. (This is comparable to group (concepts we’ll explore further in Chapter 8). the problem of experimental mortality discussed Couples in the former group were intensively inter- in Chapter 8.) The danger is that those who drop viewed over a four-year period, whereas the latter out of the study may not be typical, thereby dis- group was contacted only briefl y each year. torting the results of the study. Thus, when Carol By studying the same couples over time, the re- S. Aneshensel and colleagues conducted a panel searchers could follow the specifi c problems that study of adolescent girls (comparing Latinas and arose and the way the couples dealt with them. non-Latinas), they looked for and found differ- As a by-product of their research, they found ences in characteristics of survey dropouts among that those studied the most intensely seemed to Latinas born in the United States and those born achieve a somewhat better marital adjustment. in Mexico. These differences needed to be taken The researchers felt that the interviews may have into account to avoid misleading conclusions forced couples to discuss matters they might oth- about differences between Latinas and non-Lati- erwise have buried. nas (Aneshensel et al. 1989).

Comparing the Three Types of Longitudinal Approximating Longitudinal Studies Studies To reinforce the distinctions among trend, cohort, and panel studies, let’s contrast the Longitudinal studies do not always provide a fea- three study designs in terms of the same variable: sible or practical means of studying processes that religious affi liation. A trend study might look at take place over time. Fortunately, researchers of- shifts in U.S. religious affi liations over time, as the ten can draw approximate conclusions about such Gallup Poll does on a regular basis. A cohort study processes even when only cross-sectional data are might follow shifts in religious affi liations among available. Here are some ways to do that. 116 CHAPTER 4 RESEARCH DESIGN

Sometimes cross-sectional data imply processes have something approximating the health history over time on the basis of simple logic. For example, of individuals. Thus, you might conclude that the in the study of student drug use conducted at the average person develops vision problems before University of Hawaii that I mentioned in Chapter hearing problems. You would need to be cautious 2, students were asked to report whether they had in this assumption, however, because the differ- ever tried each of several illegal drugs. The study ences might refl ect societywide trends. For in- found that some students had tried both marijuana stance, improved hearing examinations instituted and LSD, some had tried only one, and others had in the schools might have affected only the young tried neither. Because these data were collected at people in your study. one time, and because some students presumably Asking people to recall their pasts is another would experiment with drugs later on, it would ap- common way of approximating observations over pear that such a study could not tell whether stu- time. Researchers use that method when they dents were more likely to try marijuana or LSD fi rst. ask people where they were born or when they A closer examination of the data showed, how- graduated from high school or whom they voted ever, that although some students reported having for in 1988. Qualitative researchers often conduct tried marijuana but not LSD, there were no stu- in-depth “life history” interviews. For example, C. dents in the study who had tried only LSD. From Lynn Carr (1998) used this technique in a study this fi nding it was inferred—as common sense sug- of “tomboyism.” Her respondents, aged 25 to 40, gested—that marijuana use preceded LSD use. If were asked to reconstruct aspects of their lives the process of drug experimentation occurred in from childhood on, including experiences of iden- the opposite time order, then a study at a given tifying themselves as tomboys. time should have found some students who had The danger in this technique is evident. Some- tried LSD but not marijuana, and it should have times people have faulty memories; sometimes found no students who had tried only marijuana. they lie. When people are asked in postelection Researchers can also make logical inferences polls whom they voted for, the results inevitably whenever the time order of variables is clear. If we show more people voting for the winner than ac- discovered in a cross-sectional study of college stu- tually did so on election day. As part of a series of dents that those educated in private high schools in-depth interviews, such a report can be validated received better college grades than did those edu- in the context of other reported details; however, cated in public high schools, we would conclude we should regard with caution results based on a that the type of high school attended affected col- single question in a survey. lege grades, not the other way around. Thus, even This discussion of how time fi gures into social though our observations were made at only one research suggests several questions you should time, we would feel justifi ed in drawing conclu- confront in your own research projects. In design- sions about processes taking place across time. ing any study, be sure to look at both the explicit Very often, age differences discovered in a and the implicit assumptions you are making cross-sectional study form the basis for inferring about time. Are you interested in describing some processes across time. Suppose you’re interested process that occurs over time, or are you simply in the pattern of worsening health over the course going to describe what exists now? If you want to of the typical life cycle. You might study the results describe a process occurring over time, will you of annual checkups in a large hospital. You could be able to make observations at different points in group health records according to the ages of the process, or will you have to approximate such those examined and rate each age group in terms observations by drawing logical inferences from of several health conditions—sight, hearing, blood what you can observe now? If you opt for a lon- pressure, and so forth. By reading across the age- gitudinal design, which method best serves your group ratings for each health condition, you would research purposes? HOW TO DESIGN A RESEARCH PROJECT 117

Examples of Research Strategies • Using interview and observational fi eld data, I demonstrate how a system of temporary As the preceding discussions have implied, social employment in a participative workplace both research follows many paths. The following short exploited and shaped entry-level workers’ excerpts further illustrate this point. As you read aspirations and occupational goals. (V. Smith them, note both the content of each study and the 1998:411) method used to study the chosen topic. Does the • I collected data [on White Separatist rhetoric] study seem to be exploring, describing, or explain- from several media of public discourse, including (or some combination of these)? What are the ing periodicals, books, pamphlets, transcripts sources of data in each study? Can you identify the from radio and television talk shows, and unit of analysis? Is the dimension of time relevant? newspaper and magazine accounts. (Berbrier If so, how will it be handled? 1998:435) • This case study of unobtrusive mobilizing by • In the analysis that follows, racial and gender Southern California Rape Crisis Center uses inequality in employment and retirement will archival, observational, and interview data to be analyzed, using a national sample of per- explore how a feminist organization worked to sons who began receiving Social Security Old change police, schools, prosecutors, and some Age benefi ts in 1980–81. (Hogan and Perrucci state and national organizations from 1974 to 1998:528) 1994. (Schmitt and Martin 1999:364) • Drawing from interviews with female crack • Using life history narratives, the present study dealers, this paper explores the techniques investigates processes of agency and con- they use to avoid arrest. (Jacobs and Miller sciousness among 14 women who identifi ed 1998:550) themselves as tomboys. (Carr 1998:528) • By drawing on interviews with activists in the former Estonian Soviet Socialist Republic, we HOW TO DESIGN A specify the conditions by which accommoda- RESEARCH PROJECT tive and oppositional subcultures exist and are successfully transformed into social movements. (Johnston and Snow 1998:473) You have now seen some of the options available • This paper presents the results of an ethno- to social researchers in designing projects. I know graphic study of an AIDS service organization there are a lot of pieces, and the relationships located in a small city. It is based on a combi- among them may not be totally clear, so here’s nation of participant observation, interviews a way of pulling the parts together. Let’s assume with participants, and review of organizational you were to undertake research. Where would you records. (Kilburn 1998:89) start? Then, where would you go? • Using interviews obtained during fi eldwork in Although research design occurs at the begin- Palestine in 1992, 1993, and 1994, and employ- ning of a research project, it involves all the steps ing historical and archival records, I argue that of the subsequent project. This discussion, then, Palestinian feminist discourses were shaped provides both guidance on how to start a research and infl uenced by the sociopolitical context project and an overview of the topics that follow in in which Palestinian women acted and with later chapters of this book. which they interacted. (Abdulhadi 1998:649) Figure 4-6 presents a schematic view of the • This article reports on women’s experiences social research process. I present this view reluc- of breastfeeding in public as revealed through tantly, because it may suggest more of a step-by- in-depth interviews with 51 women. (Stearns step order to research than actual practice bears 1999:308) out. Nonetheless, this idealized overview of the INTEREST IDEA THEORY ? ? Y X Y A B E F ? Y ? A B CYD X

CONCEPTUALIZATION CHOICE OF POPULATION RESEARCH METHOD AND SAMPLING Specify the meaning of the concepts and Experiments Whom do we want variables to be studied Survey research to be able to draw Field research conclusions about? Who will be observed Content analysis OPERATIONALIZATION for that purpose? Existing data research How will we actually Comparative research measure the variables Evaluation research under study?

OBSERVATIONS Collecting data for analysis and interpretation

DATA PROCESSING Transforming the data collected into a form appropriate to manipulation and analysis

ANALYSIS Analyzing data and drawing conclusions

APPLICATION Reporting results and assessing their implications

FIGURE 4-6 The Research Process. Here are some of the key elements that we’ll be examining throughout this book: the pieces that make up the whole of social research. HOW TO DESIGN A RESEARCH PROJECT 119 process provides a context for the specifi c details In terms of the options we’ve discussed in this of particular components of social research. Es- chapter, you probably have both descriptive and sentially, it is another and more detailed picture of explanatory interests: What percentage of the stu- the scientifi c process presented in Chapter 2. dent body supports a woman’s right to an abortion At the top of the diagram are interests, ideas, (description), and what causes some to support it and theories, the possible beginning points for a and others to oppose it (explanation)? The units line of research. The letters (A, B, X, Y, and so forth) of analysis are individuals: college students. Let’s represent variables or concepts such as prejudice assume you would be satisfi ed to learn something or alienation. Thus, you might have a general in- about the way things are now. You might then de- terest in fi nding out what causes some people to be cide that a cross-sectional study would suit your more prejudiced than others, or you might want to purposes. Although this would provide you with know some of the consequences of alienation. Al- no direct evidence of processes over time, you ternatively, your inquiry might begin with a specifi c might be able to approximate some longitudinal idea about the way things are. For example, you analyses if you pursued changes in students’ at- might have the idea that working on an assembly titudes over time. line causes alienation. The question marks in the diagram indicate that you aren’t sure things are the Getting Started way you suspect they are—that’s why you’re doing the research. Notice that a theory is represented At the outset of your project, your interests would as a set of complex relationships among several probably be exploratory. At this point, you might variables. choose among several possible activities in explor- The double arrows between “interest,” “idea,” ing student attitudes about abortion rights. To be- and “theory” suggest that researchers often move gin with, you might want to read something about back and forth across these several possible be- the issue. If you have a hunch that attitudes are ginnings. An initial interest may lead to the for- somehow related to college major, you might fi nd mulation of an idea, which may be fi t into a larger out what other researchers have written about that. theory, and the theory may produce new ideas and Appendix A of this book will help you make use of create new interests. your college library. In addition, you would proba- Any or all of these three may suggest the need bly talk to some people who support abortion rights for empirical research. The purpose of such re- and some who do not. You might attend meet- search can be to explore an interest, test a specifi c ings of abortion-related groups. All these activities idea, or validate a complex theory. Whatever the could help prepare you to handle the various deci- purpose, the researcher needs to make a variety sions of research design we are about to examine. of decisions, as indicated in the remainder of the Before designing your study, you must defi ne diagram. the purpose of your project. What kind of study To make this discussion more concrete, let’s will you undertake—exploratory, descriptive, ex- take a specifi c research example. Suppose you’re planatory? Do you plan to write a research paper concerned with the issue of abortion and want to to satisfy a course or thesis requirement? Is your learn why some college students support abortion purpose to gain information that will support you rights and others oppose them. Let’s say you’ve in arguing for or against abortion rights? Do you gone a step further and formed the impression want to write an article for the campus newspaper that students in the humanities and social sciences or an academic journal? In reviewing the previous seem generally more inclined to support the idea research literature regarding abortion rights, you of abortion rights than do those in the natural sci- should note the design decisions other researchers ences. (That kind of thinking often leads people to have made, always asking whether the same deci- design and conduct social research.) sions would satisfy your purpose. 120 CHAPTER 4 RESEARCH DESIGN

Usually, your purpose for undertaking research declared majors or to include students’ intentions can be expressed as a report. A good fi rst step in as well. What will you do with those who have no designing your project is to outline such a report major? (see Chapter 15 for more). Although your fi nal re- In surveys and experiments, such concepts port may not look much like your initial image of must be specifi ed in advance. In less tightly struc- it, this exercise will help you fi gure out which re- tured research, such as open-ended interviews, an search designs are most appropriate. During this important part of the research may involve the dis- step, clearly describe the kinds of statements you covery of different dimensions, aspects, or nuances want to make when the research is complete. Here of concepts. In such cases, the research itself may are some examples of such statements: “Students uncover and report aspects of social life that were frequently mentioned abortion rights in the con- not evident at the outset of the project. text of discussing social issues that concerned them personally.” “X percent of State U. students Choice of Research Method favor a woman’s right to choose an abortion.” “En- gineers are (more/less) likely than sociologists to As we’ll discuss in Part 3, each research method favor abortion rights.” has its strengths and weaknesses, and certain concepts are more appropriately studied by some Conceptualization methods than by others. In our study of attitudes toward abortion rights, a survey might be the most Once you have a well-defi ned purpose and a clear appropriate method: either interviewing students description of the kinds of outcomes you want to or asking them to fi ll out a questionnaire. Surveys achieve, you can proceed to the next step in the are particularly well suited to the study of public design of your study—conceptualization. We often opinion. Of course, you could also make good use talk pretty casually about social science concepts of the other methods presented in Part 3. For ex- such as prejudice, alienation, religiosity, and lib- ample, you might use the method of content anal- eralism, but we need to clarify what we mean by ysis to examine letters to the editor and analyze these concepts in order to draw meaningful con- the different images letter writers have of abortion. clusions about them. Chapter 5 examines this pro- Field research would provide an avenue to under- cess of conceptualization in depth. For now, let’s standing how people interact with one another re- see what it might involve in the case of our hypo- garding the issue of abortion, how they discuss it, thetical example. and how they change their minds. Usually the best If you’re going to study how college students study design uses more than one research method, feel about abortion and why, the fi rst thing you’ll taking advantage of their different strengths. If you have to specify is what you mean by “the right to look back at the brief examples of actual studies at an abortion.” Because support for abortion prob- the end of the preceding section, you’ll see several ably varies according to the circumstances, you’ll instances where the researchers used many meth- want to pay attention to the different conditions ods in a single study. under which people might approve or disapprove of abortion: for example, when the woman’s life is Operationalization in danger, in the case of rape or incest, or simply as a matter of personal choice. Once you’ve specifi ed the concepts to be studied Similarly, you’ll need to specify exact mean- and chosen a research method, the next step is op- ings for all the other concepts you plan to study. erationalization, or deciding on your measurement If you want to study the relationship of opinion techniques (discussed further in Chapters 5 and 6). about abortion to college major, you’ll have to de- The meaning of variables in a study is determined cide whether you want to consider only offi cially in part by how they are measured. Part of the task HOW TO DESIGN A RESEARCH PROJECT 121 here is deciding how the desired data will be col- tal and control groups in a manner that creates lected: direct observation, review of offi cial docu- comparability. ments, a questionnaire, or some other technique. In your hypothetical study of abortion attitudes, If you decided to use a survey to study attitudes the relevant population would be the student toward abortion rights, part of operationalization population of your college. As you’ll discover in is determining the wording of questionnaire items. Chapter 7, however, selecting a sample will re- For example, you might operationalize your main quire you to get more specifi c than that. Will you variable by asking respondents whether they would include part-time as well as full-time students? approve a woman’s right to have an abortion un- Only degree candidates or everyone? International der each of the conditions you’ve conceptualized: students as well as U.S. citizens? Undergraduates, in the case of rape or incest, if her life were threat- graduate students, or both? There are many such ened by the pregnancy, and so forth. You would questions—each of which must be answered in have designed the questionnaire so that it asked terms of your research purpose. If your purpose is respondents to express approval or disapproval for to predict how students would vote in a local ref- each situation. Similarly, you would have specifi ed erendum on abortion, you might want to limit your exactly how respondents would indicate their col- population to those eligible and likely to vote. lege major and what choices to provide those who have not declared a major. Observations

Population and Sampling Having decided what to study among whom by what method, you’re now ready to make observa- In addition to refi ning concepts and measurements, tions—to collect empirical data. The chapters of you must decide whom or what to study. The pop- Part 3, which describe the various research meth- ulation for a study is that group (usually of people) ods, give the different observation techniques ap- about whom we want to draw conclusions. We’re propriate to each. almost never able to study all the members of the To conduct a survey on abortion, you might want population that interests us, however, and we can to print questionnaires and mail them to a sample never make every possible observation of them. In selected from the student body. Alternatively, you every case, then, we select a sample from among could arrange to have a team of interviewers con- the data that might be collected and studied. The duct the survey over the telephone. The relative sampling of information, of course, occurs in ev- advantages and disadvantages of these and other eryday life and often produces biased observations. possibilities are discussed in Chapter 9. (Recall the discussion of “selective observation” in Chapter 1). Social researchers are more deliberate Data Processing in their sampling of what will be observed. Chapter 7 describes methods for selecting Depending on the research method chosen, you’ll samples that adequately refl ect the whole popu- have amassed a volume of observations in a form lation that interests us. Notice in Figure 4-6 that that probably isn’t immediately interpretable. If decisions about population and sampling are re- you’ve spent a month observing a street-corner lated to decisions about the research method to gang fi rsthand, you’ll now have enough fi eld notes be used. Whereas probability sampling techniques to fi ll a book. In a historical study of ethnic diversity would be relevant to a large-scale survey or a at your school, you may have amassed volumes content analysis, a fi eld researcher might need to of offi cial documents, interviews with administra- select only those informants who will yield a bal- tors and others, and so forth. Chapters 13 and 14 anced picture of the situation under study, and an describe some of the ways social scientifi c data are experimenter might assign subjects to experimen- processed for quantitative or qualitative analysis. 122 CHAPTER 4 RESEARCH DESIGN

In the case of a survey, the “raw” observations written report. Perhaps you’ll make oral presen- are typically in the form of questionnaires with tations, such as papers delivered to professional boxes checked, answers written in spaces, and the and scientifi c meetings. Other students would like. The data-processing phase for a survey typi- also be interested in hearing what you’ve learned cally involves the classifi cation (coding) of written- about them. in answers and the transfer of all information to a You may want to go beyond simply reporting computer. what you’ve learned to discussing the implications of your fi ndings. Do your fi ndings say anything Analysis about actions that might be taken in support of policy goals? Both the proponents and the oppo- Once the collected data are in a suitable form, nents of abortion rights would be interested. you’re ready to interpret them for the purpose Finally, be sure to consider what your work sug- of drawing conclusions that refl ect the interests, gests in regard to further research on your subject. ideas, and theories that initiated the inquiry. Chap- What mistakes should be corrected in future stud- ters 13 and 14 describe a few of the many options ies? What avenues—opened up slightly in your available to you in analyzing data. In Figure 4-6, study—should be pursued further? notice that the results of your analyses feed back into your initial interests, ideas, and theories. Often Research Design in Review this feedback represents the beginning of another cycle of inquiry. As this overview shows, research design involves a In the survey of student attitudes about abor- set of decisions regarding what topic is to be stud- tion rights, the analysis phase would pursue both ied, among what population, with what research descriptive and explanatory aims. You might be- methods, for what purpose. Although you’ll want gin by calculating the percentages of students who to consider many ways of studying a subject—and favored or opposed each of the several different use your imagination as well as your knowledge of versions of abortion rights. Taken together, these a variety of methods—research design is the pro- several percentages would provide a good picture cess of focusing your perspective for the purposes of student opinion on the issue. of a particular study. Moving beyond simple description, you might If you’re doing a research project for one of your describe the opinions of subsets of the student courses, many aspects of research design may be body, such as different college majors. Provided specifi ed for you in advance, including the method that your design called for trapping other infor- (such as an experiment) or the topic (as in a course mation about respondents, you could also look on a particular subject). The following summary at men versus women; freshmen, sophomores, assumes that you’re free to choose both your topic juniors, seniors, and graduate students; or other and your research strategy. categories that you’ve included. The description of In designing a research project, you’ll fi nd it subgroups could then lead you into an explanatory useful to begin by assessing three things: your in- analysis. terests, your abilities, and the available resources. Each of these considerations will suggest a large Application number of possible studies. Simulate the beginning of a somewhat con- The fi nal stage of the research process involves ventional research project: Ask yourself what the uses made of the research you’ve conducted you’re interested in understanding. Surely you and the conclusions you’ve reached. To start, have several questions about social behavior and you’ll probably want to communicate your fi nd- attitudes. Why are some people politically lib- ings so that others will know what you’ve learned. eral and others politically conservative? Why are You may want to prepare—and even publish—a some people more religious than others? Why do THE RESEARCH PROPOSAL 123 people join militia groups? Do colleges and uni- pendent replication of research projects is just as versities still discriminate against minority fac- important in the social sciences, although social ulty members? Why would a woman stay in an researchers tend to overlook that. Or, you might abusive relationship? Spend some time thinking want to go beyond replication and study some as- about the kinds of questions that interest and con- pect of the topic that you feel previous researchers cern you. have overlooked. Once you have a few questions you’d be in- Here’s another approach you might take. Sup- terested in answering for yourself, think about pose a topic has been studied previously using fi eld the kind of information needed to answer them. research methods. Can you design an experiment What research units of analysis would provide the that would test the fi ndings those earlier research- most relevant information: college students, col- ers produced? Or, can you think of existing statis- leges, voters, cities, or corporations? This ques- tics that could be used to test their conclusions? tion will probably be inseparable from the ques- Did a mass survey yield results that you would like tion of research topics. Then ask which aspects of to explore in greater detail through on-the-spot the units of analysis would provide the informa- observations and in-depth interviews? The use tion you need in order to answer your research of several different research methods to test the question. same fi nding is sometimes called triangulation, and Once you have some ideas about the kind of you should always keep it in mind as a valuable information relevant to your purpose, ask yourself research strategy. Because each research method how you might go about getting that information. has particular strengths and weaknesses, there is Are the relevant data likely to be available some- always a danger that research fi ndings will refl ect, where already (say, in a government publication), at least in part, the method of inquiry. In the best or would you have to collect them yourself? If you of all worlds, your own research design should think you would have to collect them, how would bring more than one research method to bear on you go about doing it? Would you need to survey a the topic. large number of people or interview a few people in depth? Could you learn what you need to know THE RESEARCH PROPOSAL by attending meetings of certain groups? Could you glean the data you need from books in the library? As you answer these questions, you’ll fi nd Quite often, in the design of a research project, yourself well into the process of research design. you’ll have to lay out the details of your plan for Keep in mind your own research abilities and the someone else’s review or approval. For a course resources available to you. There is little point in project, for example, your instructor might very designing a perfect study that you can’t actually well want to see a “proposal” before you set off carry out. You may want to try a research method to work. Later in your career, if you wanted to un- you have not used before so you can learn from dertake a major project, you might need to obtain it, but be careful not to put yourself at too great a funding from a foundation or government agency, disadvantage. who would most defi nitely want a detailed pro- Once you have a general idea of what you want posal that describes how you would spend their to study and how, carefully review previous re- money. You might respond to a Request for Pro- search in journals and books to see how other re- posals (RFP), which both public and private agen- searchers have addressed the topic and what they cies often circulate in search of someone to do re- have learned about it. Your review of the literature search for them. may lead you to revise your research design: Per- We now turn to a brief discussion of how you haps you’ll decide to use a previous researcher’s might prepare a research proposal. This will give method or even replicate an earlier study. A stan- you one more overview of the whole research pro- dard procedure in the physical sciences, the inde- cess that the rest of this book details. 124 CHAPTER 4 RESEARCH DESIGN

Elements of a Research Proposal our knowledge of the topic, which you propose to remedy. Or a little differently, your review of the Although some funding agencies (or your instruc- literature may point to inconsistencies or disagree- tor, for that matter) may have specifi c requirements among existing fi ndings. In that case, your ments for the elements or structure of a research proposed research will aim to resolve the ambigui- proposal, here are some basic elements you should ties that plague us. I don’t know about you, but I’m include. already excited about the research you’re propos- ing to undertake. Problem or Objective What exactly do you want to study? Why is it worth studying? Does the pro- Subjects for Study Whom or what will you study posed study have practical signifi cance? Does it in order to collect data? Identify the subjects in contribute to the construction of social theories? general, theoretical terms; in specifi c, more concrete terms, identify who is available for study and Literature Review What have others said about how you’ll reach them. Will it be appropriate to se- this topic? What theories address it, and what do lect a sample? If so, how will you do that? If there is they say? What previous research exists? Are there any possibility that your research will affect those consistent fi ndings, or do past studies disagree? you study, how will you insure that the research Does the body of existing research have fl aws that does not harm them? you think you can remedy? Beyond these general questions, the specifi c You’ll fi nd that reading social science research research method you’ll use will further specify the reports requires special skills. If you need to under- matter. If you’re planning to undertake an experi- take a review of the literature at this point in your ment, a survey, or fi eld research, for example, the course, you may want to skip ahead to Chapter techniques for subject selection will vary quite a 15. It will familiarize you with the different types bit. Happily, Chapter 7 of this book discusses sam- of research literature, how to fi nd what you want, pling techniques for both qualitative and quantita- and how to read it. There is a special discussion of tive studies. how to use electronic resources online and how to avoid being misled by information on the Internet. Measurement What are the key variables in your In part, the data-collection method(s) you in- study? How will you defi ne and measure them? Do tend to use in your study will shape your review your defi nitions and measurement methods dupli- of the literature. Reviewing the designs of previ- cate or differ from those of previous research on ous studies using that same technique can give this topic? If you have already developed your mea- you a head start in planning your own study. At the surement device (a questionnaire, for example) or same time, you should focus your search on your will be using something previously developed by research topic, regardless of the methods other others, it might be appropriate to include a copy in researchers have used. So, if you’re planning fi eld an appendix to your proposal. research on, say, interracial marriages, you might gain some useful insights from the fi ndings of sur- Data-Collection Methods How will you actually veys on the topic; further, past fi eld research on collect the data for your study? Will you conduct an interracial marriages could be invaluable in your experiment or a survey? Will you undertake fi eld designing a survey on the topic. research or will you focus on the reanalysis of sta- Because the literature review will appear early tistics already created by others? Perhaps you will in your research proposal, you should write it with use more than one method. an eye to introducing the reader to the topic you’ll address, laying out in a logical manner what has Analysis Indicate the kind of analysis you plan to already been learned on the topic by past research- conduct. Spell out the purpose and logic of your ers, then leading up to the holes or loose ends in analysis. Are you interested in precise description? THE ETHICS OF RESEARCH DESIGN 125

Do you intend to explain why things are the way THE ETHICS OF RESEARCH DESIGN they are? Do you plan to account for variations in some quality: for example, why some students are Designing a research project needs to include seri- more liberal than others? What possible explana- ous considerations of the ethical dimension. To be- tory variables will your analysis consider, and gin, if your study requires the participation of hu- how will you know if you’ve explained variations man subjects, you must determine that the likely adequately? benefi ts of the research will do justice to the time and effort you’ll ask of them. Schedule Providing a schedule for the various You’ll also want to design the study in con- stages of research is often appropriate. Even if you currence with the ethical guidelines discussed in don’t do this for the proposal, do it for yourself. Un- Chapter 3. For example, you should insure that the less you have a timeline for accomplishing the sev- subjects’ privacy and well-being are protected. As eral stages of research and keeping track of how I indicated earlier, having your research design re- you’re doing, you may end up in trouble. viewed by an Institutional Review Board may be appropriate. Budget When you ask someone to cover the costs of your research, you need to provide a budget that specifi es where the money will go. Large, expen- WHAT DO YOU THINK? REVISITED sive projects include budgetary categories such as personnel, equipment, supplies, telephones, and postage. Even if you’ll be paying for your project When the Provost and the student newspa- yourself, you should spend some time anticipating per seemed to disagree over the extent of expenses: offi ce supplies, photocopying, computer part-time faculty teaching, they used differ- disks, telephone calls, transportation, and so on. ent units of analysis. The newspaper said 52 percent of the faculty were part-time; the provost said about 70 percent of the credits Institutional Review Board Depending on the were taught by full-time faculty. The table nature of your research design, you may need to here demonstrates how they could both be submit your proposal to the campus Insitutional right, given that the typical full-time fac- Review Board for approval to insure the protection ulty member teaches three courses, or nine of human subjects. Your instructor can advise you credits, whereas the typical part-time faculty on this. member teaches one course for three cred- As you can see, if you’re interested in con- its. For simplicity, I’ve assumed that there ducting a social research project, it’s a good idea are 100 faculty members. to prepare a research proposal for your own purposes, even if you aren’t required to do so by your Credits Total instructor or a funding agency. If you’re going to Faculty Taught Credits Status Number by Each Taught invest your time and energy in such a project, you should do what you can to insure a return on that Full-time 48 9 432 investment. Part-time 52 3 156 Now that you’ve had a broad overview of social Total 588 research, you can move on to the remaining chapters in this book and learn exactly how to design In this hypothetical illustration, full-time fac- and execute each specifi c step. If you’ve found a ulty taught 432 of the 588 credits, or 73 per- research topic that really interests you, you’ll want cent. As you can see, being clear about what to keep it in mind as you see how you might go the unit of analysis is matters a great deal. about studying it. 126 CHAPTER 4 RESEARCH DESIGN

Main Points Necessary and Sufﬁ cient Causes ❏ Mere association, or correlation, does not in Introduction itself establish causation. A spurious causal ❏ Any research design requires researchers to relationship is an association that in reality is specify as clearly as possible what they want caused by one or more other variables. to ﬁ nd out and then determine the best way to do it. Units of Analysis ❏ Units of analysis are the people or things Three Purposes of Research whose characteristics social researchers ob- ❏ The principal purposes of social research serve, describe, and explain. Typically, the unit include exploration, description, and explana- of analysis in social research is the individual tion. Research studies often combine more person, but it may also be a social group, a for- than one purpose. mal organization, a social interaction, a social ❏ Exploration is the attempt to develop an initial, artifact, or other phenomena such as lifestyles rough understanding of some phenomenon. or social interactions. ❏ ❏ Description is the precise measurement and The ecological fallacy involves conclusions reporting of the characteristics of some popu- drawn from the analysis of groups (such as lation or phenomenon under study. corporations) that are then assumed to apply to individuals (such as the employees of ❏ Explanation is the discovery and reporting corporations). of relationships among different aspects of ❏ the phenomenon under study. Descriptive Reductionism is the attempt to understand a studies answer the question “What’s so?”; complex phenomenon in terms of a narrow set explanatory ones tend to answer the question of concepts, such as attempting to explain the “Why?” American Revolution solely in terms of economics (or political idealism or psychology).

The Logic of Nomothetic Explanation ❏ Both idiographic and nomothetic models of The Time Dimension explanation rest on the idea of causation. The ❏ The research of social processes that occur idiographic model aims at a complete under- over time presents challenges that can be standing of a particular phenomenon, using addressed through cross-sectional studies or all relevant causal factors. The nomothetic longitudinal studies. model aims at a general understanding—not ❏ Cross-sectional studies are based on observa- necessarily complete—of a class of phenom- tions made at one time. Although conclusions ena, using a small number of relevant causal drawn from such studies are limited by this factors. characteristic, researchers can sometimes use ❏ There are three basic criteria for establishing such studies to make inferences about pro- causation in nomothetic analyses: (1) The vari- cesses that occur over time. ables must be empirically associated, or corre- ❏ In longitudinal studies, observations are made lated; (2) the causal variable must occur earlier at many times. Such observations may be in time than the variable it is said to affect; and made of samples drawn from general popu- (3) the observed effect cannot be explained as lations (trend studies), samples drawn from the effect of a different variable. more speciﬁ c subpopulations (cohort studies), REVIEW QUESTIONS 127

or the same sample of people each time (panel spurious relationship units of analysis studies). trend study

How to Design a Research Project Review Questions ❏ Research design starts with an initial interest, idea, or theoretical expectation and proceeds 1. One example in this chapter suggested that through a series of interrelated steps that nar- political orientations cause attitudes toward row the focus of the study so that concepts, legalizing marijuana. Can you make an argu- methods, and procedures are well defi ned. A ment that the time-order is just the opposite good research plan accounts for all these steps of what was assumed? in advance. 2. Here are some examples of real research top- ❏ At the outset, a researcher specifi es the mean- ics. For each excerpt, can you name the unit ing of the concepts or variables to be stud- of analysis? (The answers are at the end of ied (conceptualization), chooses a research this chapter.) method or methods (such as experiments a. Women watch TV more than men because versus surveys), and specifi es the population they are likely to work fewer hours outside to be studied and, if applicable, how it will be the home than men. . . . Black people sampled. watch an average of approximately three- ❏ The researcher operationalizes the proposed quarters of an hour more television per concepts by stating precisely how variables day than white people. (Hughes 1980:290) in the study will be measured. Research then b. Of the 130 incorporated U.S. cities with proceeds through observation, processing the more than 100,000 inhabitants in 1960, data, analysis, and application, such as report- 126 had at least two short-term nonpro- ing the results and assessing their implications. prietary general hospitals accredited by the American Hospital Association. (Turk The Research Proposal 1980:317) ❏ A research proposal provides a preview of why c. The early TM [transcendental meditation] a study will be undertaken and how it will be organizations were small and informal. conducted. Researchers must often get permis- The Los Angeles group, begun in June sion or necessary resources in order to pro- 1959, met at a member’s house where, in- ceed with a project. Even when not required, a cidentally, Maharishi was living. (Johnston proposal is a useful device for planning. 1980:337) d. However, it appears that the nursing The Ethics of Research Design staffs exercise strong infl uence over . . . a ❏ Your research design should indicate how your decision to change the nursing care sys- study will abide by the ethical strictures of tem. . . . Conversely, among those deci- social research. sions dominated by the administration and ❏ It may be appropriate for an Institutional Re- the medical staffs . . . (Comstock 1980:77) view Board to review your research proposal. e. Though 667,000 out of 2 million farmers in the United States are women, women his- Key Terms torically have not been viewed as farmers, but rather, as the farmer’s wife. (Votaw cohort study longitudinal study 1979:8) correlation panel study f. The analysis of community opposition cross-sectional study reductionism to group homes for the mentally handi- ecological fallacy social artifact capped . . . indicates that deteriorating 128 CHAPTER 4 RESEARCH DESIGN

neighborhoods are most likely to organize sonalized study plan based on your responses to in opposition, but that upper-middle class a diagnostic pretest. Once you have mastered neighborhoods are most likely to enjoy pri- the material with the help of interactive learning vate access to local offi cials. (Graham and tools, you can take a posttest to confi rm that you Hogan 1990:513) are ready to move on to the next chapter. g. Some analysts during the 1960s predicted that the rise of economic ambition and po- Website for litical militancy among blacks would foster The Basics of Social Research discontent with the “otherworldly” black 4th edition mainline churches. (Ellison and Sherkat 1990:551) At the book companion website (http://sociology h. This analysis explores whether proposi- .wadsworth.com/babbie_basics4e) you will fi nd tions and empirical fi ndings of contem- many resources in addition to ThomsonNow to porary theories of organizations directly aid you in studying for your exams. For example, apply to both private product producing you will fi nd Tutorial Quizzes with feedback, Inter- organizations (PPOs) and public human net Exercises, Flashcards, and Chapter Tutorials, as service organizations (PSOs). (Schifl ett and well as Extended Projects, InfoTrac College Edition Zey 1990:569) search terms, Social Research in Cyberspace, GSS i. This paper examines variations in job title Data, Web Links, and primers for using various structures across work roles. Analyzing data analysis software such as SPSS and NVivo. 3,173 job titles in the California civil service system in 1985, we investigate how and why lines of work vary in the proliferation Additional Readings of job categories that differentiate ranks, functions, or particular organizational loca- Bart, Pauline, and Linda Frankel. 1986. The Student Sociologist’s Handbook. Morristown, NJ: General tions. (Strang and Baron 1990:479) Learning Press. A handy little reference book to help 3. Review the logic of spuriousness. Can you you get started on a research project. Written from think up an example where an observed rela- the standpoint of a student term paper, this volume tionship between two variables could actually offers a particularly good guide to the periodical be explained away by a third variable? literature of the social sciences that is available in a 4. Make up a research example—different from good library. those discussed in the text—that illustrates a Casley, D. J., and D. A. Lury. 1987. Data Collection in researcher committing the ecological fallacy. Developing Countries. Oxford: Clarendon Press. This book discusses the special problems of research in How would you modify the example to avoid the developing world. this trap? Cooper, Harris M. 1989. Integrating Research: A Guide for Literature Reviews. Thousand Oaks, CA: Sage. The Online Study Resources author leads you through each step in the literature review process. Hunt, Morton. 1985. Profi les of Social Research: The Sci- entifi c Study of Human Interactions. New York: Basic Books. An engaging and informative series of project biographies: James Coleman’s study of segregated Go to schools is presented, as well as several other major http://sociology.wadsworth.com/babbie_basics4e projects that illustrate the elements of social research in actual practice. ThomsonNow and click on for access to this Iversen, Gudmund R. 1991. Contextual Analysis. Thou- powerful online study tool. You will get a per- sand Oaks, CA: Sage. Contextual analysis examines ANSWERS TO UNITS OF ANALYSIS QUIZ, REVIEW QUESTION #2 129

the impact of socioenvironmental factors on individu- Answers to Units of Analysis al behavior. Durkheim’s study of suicide offers a good Quiz, Review Question #2 example of this, identifying social contexts that affect the likelihood of self-destruction. a. Men and women, black and white people Maxwell, Joseph A. 1996. Qualitative Research Design: (individuals) An Interactive Approach. Thousand Oaks, CA: Sage. b. Incorporated U.S. cities (groups) Maxwell covers many of the same topics that this chapter does but with attention devoted speciﬁ cally c. Transcendental meditation organizations to qualitative research projects. (groups) Menard, Scott. 1991. Longitudinal Research. Thousand d. Nursing staffs (groups) Oaks, CA: Sage. The author begins by explaining why researchers conduct longitudinal research, and then e. Farmers (individuals) goes on to detail a variety of study designs as well as f. Neighborhoods (groups) suggestions for the analysis of longitudinal data. g. Blacks (individuals) Miller, Delbert. 1991. Handbook of Research Design and Social Measurement. Thousand Oaks, CA: Sage. A h. Service and production organizations (formal useful reference book for introducing or reviewing organizations) numerous issues involved in design and measure- i. Job titles (artifacts) ment. In addition, the book contains a wealth of practical information relating to foundations, journals, and professional associations. CONCEPTUALIZATION, 51 HUMANOPERATIONALIZATION, INQUIRY AND SCIENCEAND MEASUREMENT Photo credit David M. Grossman/Photo Researchers, Inc.

What You’ll Learn in thisThis ChapterChapter

You’llWe’ll see examine how the the interrelated way people steps learn of aboutconceptualization, their world and operationalization, the mistakes they makeand measurement along the way. allow We’ll researchers also begin toto turnsee whata general makes idea sience for a differentresearch fromtopic

otherinto useful ways andof knowing valid measurements things. in the real world. You’ll also see that an 2S essential part of this process involves transforming the relatively vague terms of 1S N ordinary language into precise objects of study with well-deﬁ ned and measurable L meanings. 130 In this chapter . . . WHAT DO YOU THINK?

Introduction People sometimes Measuring Anything That Exists doubt the social re- Conceptions, Concepts, and Reality searcher’s ability to Conceptions as Constructs Image not available due to copyright restrictions measure and study Conceptualization things that matter. Indicators and Dimensions For many people, The Interchangeability of Indicators for example, reli- Real, Nominal, and Operational Defi nitions gious faith is a deep Creating Conceptual Order and important part of life. Yet both the re- An Example of Conceptualization: ligious and the nonreligious might question The Concept of Anomie the social researcher’s ability to measure how religious a given person or group is Defi nitions in Descriptive and—even more diffi cult—why some people and Explanatory Studies are religious and others are not. Operationalization Choices This chapter will show that social re- Range of Variation searchers can’t say defi nitively who is reli- Variations between the Extremes gious and who is not, nor what percentage A Note on Dimensions of people in a particular population are reli- Defi ning Variables and Attributes gious, but they can to an extent determine Levels of Measurement the causes of religiosity. How can that be? Single or Multiple Indicators See the “What Do You Think? Revisited” box Some Illustrations of Operationalization Choices toward the end of the chapter. Operationalization Goes On and On

Criteria of Measurement Quality Precision and Accuracy real world. This chapter discusses the interrelated Reliability processes of conceptualization, operationaliza- Validity tion, and measurement. Chapter 6 builds on this Who Decides What’s Valid? foundation to discuss types of measurements that Tension between Reliability and Validity are more complex. The Ethics of Measurement Consider a notion such as “satisfaction with college.” I’m sure you know some people who are INTRODUCTION very satisfi ed, some who are very dissatisfi ed, and many who fall between those extremes. Moreover, you can probably place yourself somewhere along This chapter and the next deal with how research- that satisfaction spectrum. Although this prob- ers move from a general idea about what to study ably makes sense to you as a general matter, how to effective and well-defi ned measurements in the would you go about measuring these differences 131 132 CHAPTER 5 CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT among students, so you could place them along satisfaction would surely have increased if you had that spectrum? found yourself being misclassifi ed by the measure- Some of the comments students make in con- ment system. versations (such as “This place sucks”) would tip Your dissatisfaction refl ects an important fact you off as to where they stood. In a more active about social research: Most of the variables we effort, you could think of questions you might ask want to study do not actually exist in the way students (as in “How satisfi ed are you . . . ?”), such that rocks exist. Indeed, they are made up. More- that their answers would indicate their satisfac- over, they seldom have a single, unambiguous tion. Perhaps certain behaviors—such as class at- meaning. tendance, use of campus facilities, or setting the To see what I mean, suppose we want to study dean’s offi ce on fi re—would suggest different levels political party affi liation. To measure this variable, of satisfaction. As you think about ways of mea- we might consult the list of registered voters to suring satisfaction with college, you are engaging note whether the people we were studying were in the subject matter of this chapter. registered as Democrats or Republicans and take We begin by confronting the hidden concern that as a measure of their party affi liation. But we people sometimes have about whether it’s truly could also simply ask someone what party they possible to measure the stuff of life: love, hate, identify with and take their response as our mea- prejudice, religiosity, radicalism, alienation. Over sure. Notice that these two different measurement the next few pages, we’ll see that researchers can possibilities refl ect somewhat different defi nitions measure anything that exists. Once that point has of “political party affi liation.” They might even pro- been established, we’ll turn to the steps involved duce different results: Someone may have regis- in actual measurement. tered as a Democrat years ago but gravitated more and more toward a Republican philosophy over MEASURING ANYTHING THAT EXISTS time. Or someone who is registered with neither political party may, when asked, say she is affi liated with the one she feels the most kinship with. Earlier in this book, I said that one of the two pil- Similar points apply to religious affi liation. Some- lars of science is observation. Because this word times this variable refers to offi cial membership in can suggest a casual, passive activity, scientists a particular church, temple, mosque, and so forth; often use the term measurement instead, meaning other times it simply means whatever religion, if careful, deliberate observations of the real world any, one identifi es oneself with. Perhaps to you it for the purpose of describing objects and events in means something else, such as attendance at re- terms of the attributes composing a variable. ligious services. Like the people in the opening “What Do You The truth is that neither party affi liation nor re- Think?” box, you may have some reservations ligious affi liation has any real meaning, if by “real” about the ability of science to measure the really we mean corresponding to some objective aspect important aspects of human social existence. If of reality. These variables do not exist in nature. you’ve read research reports dealing with some- They are merely terms we have made up and as- thing like liberalism or religion or prejudice, you signed specifi c meanings to for some purpose, may have been dissatisfi ed with the way the re- such as doing social research. searchers measured whatever they were studying. But, you might object, “political affi liation” and You may have felt that they were too superfi cial, “religious affi liation”—and a host of other things missing the aspects that really matter most. Maybe social researchers are interested in, such as preju- they measured religiosity as the number of times dice or compassion—have some reality. After all, a person went to religious services, or liberalism we make statements about them, such as “In Hap- by how people voted in a single election. Your dis- pytown, 55 percent of the adults affi liate with the MEASURING ANYTHING THAT EXISTS 133

Republican Party, and 45 percent of them are Epis- name that represents a collection of apparently re- copalians. Overall, people in Happytown are low lated phenomena that we’ve each observed in the in prejudice and high in compassion.” Even ordi- course of life. In short, we made it up. nary people, not just social researchers, have been Here’s another clue that prejudice isn’t some- known to make statements like that. If these things thing that exists apart from our rough agreement to do not exist in reality, what is it that we’re measur- use the term in a certain way. Each of us develops ing and talking about? our own mental image of what the set of real phe- What indeed? Let’s take a closer look by consid- nomena we’ve observed represents in general and ering a variable of interest to many social research- what these phenomena have in common. When I ers (and many other people as well)—prejudice. say the word prejudice, it evokes a mental image in your mind, just as it evokes one in mine. It’s as Conceptions, Concepts, and Reality though fi le drawers in our minds contained thousands of sheets of paper, with each sheet of paper As you and I wander down the road of life, we have labeled in the upper right-hand corner. A sheet of already observed a lot of things and known they paper in each of our minds has the term prejudice were real through our observations, and we have on it. On your sheet are all the things you’ve been heard reports from other people that have seemed told about prejudice and everything you’ve ob- real. For example: served that seems to be an example of it. My sheet has what I’ve been told about it plus all the things • We personally heard people say nasty things I’ve observed that seem examples of it—and mine about minority groups. isn’t the same as yours. • We heard people said women are inferior to The technical term for those mental images, men. those sheets of paper in our mental fi le drawers, • We read about African Americans being is conception. That is, I have a conception of preju- lynched. dice, and so do you. We can’t communicate these • We read that women and minorities earn less mental images directly, so we use the terms written for the same work. in the upper right-hand corner of our own mental • We learned about “ethnic cleansing” and wars sheets of paper as a way of communicating about in which one ethnic group tried to eradicate our conceptions and the things we observe that are another. related to those conceptions. These terms make With additional experience, we noticed some- it possible for us to communicate and eventually thing more. People who participated in lynching agree on what we will specifi cally mean by those were also quite likely to call African Americans terms. In social research, the process of coming to ugly names. A lot of them, moreover, seemed to an agreement about what terms mean is conceptu- want women to “stay in their place.” Eventually it alization, and the result is called a concept. dawned on us that these several tendencies often I’m sure you’ve heard reference to the many appeared together in the same people and also had words some northern peoples have for “snow,” as something in common. At some point, someone an example of how environment can shape lan- had a bright idea: “Let’s use the word prejudiced for guage. Here’s an exercise you might enjoy. Search people like that. We can use the term even if they the web for “Inuit words for snow.” You’re likely to don’t do all those things—as long as they’re pretty discover wide disagreement on the number of In- much like that.” uit words—ranging from 1 to 400. Several sources, Being basically agreeable and interested in ef- moreover, will suggest that English also has sev- fi ciency, we agreed to go along with the system. eral words for “snow.” Cecil Adams, for example, That’s where “prejudice” came from. We never ob- lists: “snow, slush, sleet, hail, powder, hard pack, served it. We just agreed to use it as a shortcut, a blizzard, fl urries, fl ake, dusting, crust, avalanche, 134 CHAPTER 5 CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT drift, frost, and iceberg” (Adams 2006). This illus- and “cross-species dating”—and you say so. Even- trates the ambiguities with regard to the concepts tually, we set about comparing the entries we have and words we use in everyday communications as on our respective sheets labeled “compassionate.” well as in social research. We then discover that many of our mental images Let’s take another example of a conception. corresponding to that term differ. Suppose that I’m going to meet someone named In the big picture, language and communica- Pat, whom you already know. I ask you what Pat tion work only to the extent that our mental fi le- is like. Now suppose that you’ve seen Pat help lost sheet entries overlap considerably. The similarities children fi nd their parents and put a tiny bird back on those sheets represent the agreements existing in its nest. Pat got you to take turkeys to poor fami- in our society. As we grow up, we’re all taught ap- lies on Thanksgiving and to visit a children’s hos- proximately the same thing when we’re fi rst intro- pital at Christmas. You’ve seen Pat weep through duced to a particular term, though our nationality, a movie about a mother overcoming adversities to gender, race, ethnicity, region, language, or other save and protect her child. As you search through cultural factors may shade our understanding of your mental fi les, you may fi nd all or most of those concepts. phenomena recorded on a single sheet labeled Dictionaries formalize the agreements our so- “compassionate.” You look over the other entries ciety has about such terms. Each person, then, on the page, and you fi nd they seem to provide shapes his or her mental images to correspond an accurate description of Pat. So you say, “Pat is with such agreements. But because all of us have compassionate.” different experiences and observations, no two Now I leaf through my own mental fi le drawer people end up with exactly the same set of entries until I fi nd a sheet marked “compassionate.” I then on any sheet in their fi le systems. If we want to look over the things written on my sheet, and I say, measure “prejudice” or “compassion,” we must “Oh, that’s nice.” I now feel I know what Pat is like, fi rst stipulate what, exactly, counts as prejudice or but my expectations refl ect the entries on my fi le compassion for our purposes. sheet, not yours. Later, when I meet Pat, I happen Returning to the assertion made at the outset to fi nd that my own experiences correspond to the of this chapter, we can measure anything that’s entries I have on my “compassionate” fi le sheet, real. We can measure, for example, whether Pat and I say that you sure were right. But suppose my actually puts the little bird back in its nest, visits observations of Pat contradict the things I have on the hospital on Christmas, weeps at the movie, or my fi le sheet. I tell you that I don’t think Pat is very refuses to contribute to saving the whales. All of compassionate, and we begin to compare notes. those behaviors exist, so we can measure them. You say, “I once saw Pat weep through a movie But is Pat really compassionate? We can’t answer about a mother overcoming adversity to save that question; we can’t measure compassion in and protect her child.” I look at my “compas- any objective sense, because compassion doesn’t sionate sheet” and can’t fi nd anything like that. exist the way those things I just described exist. Looking elsewhere in my fi le, I locate that sort Compassion exists only in the form of the agree- of phenomenon on a sheet labeled “sentimen- ments we have about how to use the term in com- tal.” I retort, “That’s not compassion. That’s just municating about things that are real. sentimentality.” To further strengthen my case, I tell you that Concepts as Constructs I saw Pat refuse to give money to an organization dedicated to saving whales from extinction. Abraham Kaplan (1964) distinguishes three classes “That represents a lack of compassion,” I argue. of things that scientists measure. The fi rst class is You search through your fi les and fi nd saving the direct observables: those things we can observe whales on two sheets—“environmental activism” rather simply and directly, like the color of an ap- MEASURING ANYTHING THAT EXISTS 135 Earl Babbie Conceptualization and operationalization involve the search for ways to measure the variables that interest us.

ple or a check mark on a questionnaire. The sec- To summarize, concepts are constructs derived ond class, indirect observables, requires “relatively by mutual agreement from mental images (con- more subtle, complex, or indirect observations” ceptions). Our conceptions summarize collections (1964:55). When we note a person’s check mark of seemingly related observations and experiences. beside “female” in a questionnaire, we’ve indirectly The observations and experiences are real, at least observed that person’s gender. History books or subjectively, but conceptions, and the concepts minutes of corporate board meetings provide in- derived from them, are only mental creations. The direct observations of past social actions. Finally, terms associated with concepts are merely devices the third class of observables consists of con- created for the purposes of fi ling and communica- structs—theoretical creations that are based on ob- tion. A term like prejudice is, objectively speaking, servations but that cannot be observed directly or only a collection of letters. It has no intrinsic reality indirectly. A good example is intelligence quotient, beyond that. It has only the meaning we agree to or IQ. It is constructed mathematically from obser- give it. vations of the answers given to a large number of Usually, however, we fall into the trap of be- questions on an IQ test. No one can directly or in- lieving that terms for constructs do have intrinsic directly observe IQ. It is no more a “real” character- meaning, that they name real entities in the world. istic of people than is compassion or prejudice. That danger seems to grow stronger when we be- Kaplan (1964:49) defi nes concept as a “family gin to take terms seriously and attempt to use them of conceptions.” A concept is, as Kaplan notes, a precisely. Further, the danger increases in the pres- construct, something we create. Concepts such as ence of experts who appear to know more than we compassion and prejudice are constructs created do about what the terms really mean: Yielding to from your conception of them, my conception of authority is easy in such a situation. them, and the conceptions of all those who have Once we assume that terms like prejudice and ever used these terms. They cannot be observed compassion have real meanings, we begin the tor- directly or indirectly, because they don’t exist. We tured task of discovering what those real meanings made them up. are and what constitutes a genuine measurement 136 CHAPTER 5 CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT of them. Regarding constructs as real is called reifi - term compassionate, I’m probably safe in assum- cation. The reifi cation of concepts in day-to-day life ing that Pat won’t pull the wings off fl ies. A wide is quite common. In science, we want to be quite range of misunderstandings and confl ict—from the clear about what it is we are actually measuring, interpersonal to the international—is the price we but this aim brings a pitfall with it. Settling on the pay for our imprecision, but somehow we muddle “best” way of measuring a variable in a particular through. Science, however, aims at more than study may imply that we’ve discovered the “real” muddling; it cannot operate in a context of such meaning of the concept involved. In fact, concepts imprecision. have no real, true, or objective meanings—only The process through which we specify what we those we agree are best for a particular purpose. mean when we use particular terms in research is Does this discussion imply that compassion, called conceptualization. Suppose we want to prejudice, and similar constructs cannot be mea- fi nd out, for example, whether women are more sured? Interestingly, the answer is no. (And a good compassionate than men. I suspect many people thing, too, or a lot of us social researcher types assume this is the case, but it might be interest- would be out of work.) I’ve said that we can mea- ing to fi nd out if it’s really so. We can’t meaning- sure anything that’s real. Constructs aren’t real in fully study the question, let alone agree on the the way that trees are real, but they do have another answer, without some working agreements about important virtue: They are useful. That is, they help the meaning of compassion. They are “working” us organize, communicate about, and understand agreements in the sense that they allow us to work things that are real. They help us make predictions on the question. We don’t need to agree or even about real things. Some of those predictions even pretend to agree that a particular specifi cation is turn out to be true. Constructs can work this way ultimately the best one. because, although not real or observable in them- Conceptualization, then, produces a specifi c, selves, they have a defi nite relationship to things agreed-on meaning for a concept, for the purposes that are real and observable. The bridge from di- of research. This process of specifying exact mean- rect and indirect observables to useful constructs ing involves describing the indicators we’ll be us- is the process called conceptualization. ing to measure our concept and the different aspects of the concept, called dimensions. CONCEPTUALIZATION Indicators and Dimensions

As we’ve seen, day-to-day communication usu- Conceptualization gives defi nite meaning to a con- ally occurs through a system of vague and general cept by specifying one or more indicators of what agreements about the use of terms. Although you we have in mind. An indicator is a sign of the and I do not agree completely about the use of the presence or absence of the concept we’re studying. Here’s an example. We might agree that visiting children’s hospitals during Christmas and Hanukkah is an indicator of conceptualization The mental process whereby fuzzy compassion. Putting little birds back in their nests and imprecise notions (concepts) are made more specifi c might be agreed on as another indicator, and so and precise. So you want to study prejudice. What do you forth. If the unit of analysis for our study is the in- mean by prejudice? Are there different kinds of prejudice? dividual person, we can then observe the presence What are they? or absence of each indicator for each person under indicator An observation that we choose to consider as a refl ection of a variable we wish to study. Thus, for study. Going beyond that, we can add up the num- example, attending religious services might be considered ber of indicators of compassion observed for each an indicator of religiosity. individual. We might agree on ten specifi c indica- CONCEPTUALIZATION 137 tors, for example, and fi nd six present in our study most always complicates the concepts researchers of Pat, three for John, nine for Mary, and so forth. are interested in. (We’ll return to this issue when Returning to our question about whether men we discuss the validity of measures, toward the or women are more compassionate, we might end of this chapter.) calculate that the women we studied displayed an Whenever we take our concepts seriously and average of 6.5 indicators of compassion, the men set about specifying what we mean by them, we an average of 3.2. On the basis of our quantita- discover disagreements and inconsistencies. Not tive analysis of group difference, we might there- only do you and I disagree, but each of us is likely fore conclude that women are, on the whole, more to fi nd a good deal of muddiness within our own compassionate than men. mental images. If you take a moment to look at Usually, though, it’s not that simple. Imagine what you mean by compassion, you’ll probably you’re interested in understanding a small religious fi nd that your image contains several kinds of cult, particularly their views on various groups: compassion. That is, the entries on your mental fi le gays, nonbelievers, feminists, and others. In fact, sheet can be combined into groups and subgroups, they suggest that anyone who refuses to join their say, compassion toward friends, coreligionists, group and abide by its teachings will “burn in hell.” humans, and birds. You can also fi nd several dif- In the context of your interest in compassion, they ferent strategies for making combinations. For ex- don’t seem to have much. And yet, the group’s lit- ample, you might group the entries into feelings erature often speaks of their compassion for oth- and actions. ers. You want to explore this seeming paradox. The technical term for such groupings is di- To pursue this research interest, you might ar- mension: a specifi able aspect of a concept. For range to interact with cult members, getting to instance, we might speak of the “feeling dimen- know them and learning more about their views. sion” of compassion and its “action dimension.” In You could tell them you were a social researcher a different grouping scheme, we might distinguish interested in learning about their group, or per- “compassion for humans” from “compassion for haps you would just express an interest in learning animals.” Or we might see compassion as help- more without saying why. ing people have what we want for them versus In the course of your conversations with group what they want for themselves. Still differently, we members and perhaps attendance of religious ser- might distinguish compassion as forgiveness from vices, you would put yourself in situations where compassion as pity. you could come to understand what the cult mem- Thus, we could subdivide compassion into sev- bers mean by compassion. You might learn, for ex- eral clearly defi ned dimensions. A complete con- ample, that members of the group were so deeply ceptualization involves both specifying dimensions concerned about sinners burning in hell that they and identifying the various indicators for each. were willing to be aggressive, even violent, to make When Jonathan Jackson set out to measure people change their sinful ways. Within their own “fear of crime,” he considered eleven different paradigm, then, cult members would see beating dimensions: up gays, prostitutes, and abortion doctors as acts • the frequency of worry about becoming a vic- of compassion. tim of three personal crimes and two property Social researchers focus their attention on the crimes in the immediate neighbourhood. . . . meanings that people under study give to words and actions. Doing so can often clarify the behaviors observed: At least now you understand how dimension A specifi able aspect of a concept. “Religios- the cult can see violent acts as compassionate. On ity,” for example, might be specifi ed in terms of a belief the other hand, paying attention to what words dimension, a ritual dimension, a devotional dimension, a and actions mean to the people under study al- knowledge dimension, and so forth. 138 CHAPTER 5 CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT

• estimates of likelihood of falling victim to each Then, as Chirot and Edwards examined his- crime locally; torical instances of genocide, they began conclud- • perceptions of control over the possibility of ing that the motivations for launching genocidal becoming a victim of each crime locally; mayhem differed suffi ciently to represent four dis- • perceptions of the seriousness of the conse- tinct phenomena that were all called “genocide” quences of each crime; (2003:15–18): • beliefs about the incidence of each crime locally; 1. Convenience: Sometimes the attempt to eradi- • perceptions of the extent of social physical cate a group of people serves a function for the incivilities in the neighbourhood; eradicators, such as Julius Caesar’s attempt to • perceptions of community cohesion, including eradicate tribes defeated in battle, fearing they informal social control and trust/social capital would be diffi cult to rule. Or when gold was —(JACKSON 2005:301) discovered on Cherokee land in the South- eastern United States in the early nineteenth Sometimes conceptualization aimed at identi- century, the Cherokee were forcibly relocated fying different dimensions of a variable leads to a to Oklahoma in an event known as the “Trail different kind of distinction. We may conclude that of Tears,” which ultimately killed as many as we’ve been using the same word for meaningfully half of those forced to leave. distinguishable concepts. In the following exam- 2. Revenge: When the Chinese of Nanking bravely ple, the researchers fi nd, fi rst, that “violence” is not resisted the Japanese invaders in the early a suffi cient description of “genocide” and, second, years of World War II, the conquerors felt they that the concept “genocide” itself comprises sev- had been insulted by those they regarded as in- eral distinct phenomena. Let’s look at the process ferior beings. Tens of thousands were slaugh- they went through to come to this conclusion. tered in the “Rape of Nanking” in 1937–1938. When Daniel Chirot and Jennifer Edwards at- 3. Fear: The ethnic cleansing that recently oc- tempted to defi ne the concept of “genocide,” they curred in the former Yugoslavia was at least found that existing assumptions were not precise partly motivated by economic competition and enough for their purposes: worries that the growing Albanian popula- The United Nations originally defi ned it as an at- tion of Kosovo was gaining political strength tempt to destroy “in whole or in part, a national, through numbers. Similarly, the Hutu attempt ethnic, racial, or religious group.” If genocide is to eradicate the Tutsis of Rwanda grew out of distinct from other types of violence, it requires a fear that returning Tutsi refugees would seize its own unique explanation. — (2003:14) control of the country. Often intergroup fears such as these grow out of long histories of Notice the fi nal comment in this excerpt, as it atrocities, often infl icted in both directions. provides an important insight into why researchers 4. Purifi cation: The Nazi Holocaust, probably are so careful in specifying the concepts they study. the most publicized case of genocide, was If genocide, such as the Holocaust, were simply intended as a purifi cation of the “Aryan race.” another example of violence, like assaults and While Jews were the main target, gypsies, homicides, then what we know about violence in homosexuals, and many other groups were general might explain genocide. If it differs from also included. Other examples include the other forms of violence, then we may need a differ- Indonesian witch-hunt against communists ent explanation for it. So, the researchers began by in 1965–1966 and the attempt to eradicate all suggesting that “genocide” was a concept distinct non-Khmer Cambodians under Pol Pot in the from “violence” for their purposes. 1970s. CONCEPTUALIZATION 139

No single theory of genocide could explain these Surprisingly, we can still reach an agreement on various forms of mayhem. Indeed, this act of con- whether men or women are the more compas- ceptualization suggests four distinct phenomena, sionate. How we do that has to do with the inter- each needing a different set of explanations. changeability of indicators. Specifying the different dimensions of a concept, The logic works like this. If we disagree totally then, often paves the way for a more sophisticated on the value of the indicators, one solution would understanding of what we’re studying. To take an- be to study all of them. Suppose that women turn other example, we might observe that women are out to be more compassionate than men on all 100 more compassionate in terms of feelings, and men indicators—on all the indicators you favor and on more so in terms of actions—or vice versa. Which- all of mine. Then we would be able to agree that ever turned out to be the case, we would not be women are more compassionate than men even able to say whether men or women are really more though we still disagree on exactly what compas- compassionate. Our research would have shown sion means in general. that there is no single answer to the question. That The interchangeability of indicators means alone represents an advance in our understanding that if several different indicators all represent, to of reality. some degree, the same concept, then all of them will behave the same way that the concept would behave if it were real and could be observed. Thus, For an excellent sampling of concepts, given a basic agreement about what “compas- variables, and indicators, go to the General Social Survey codebook and explore some sion” is, if women are generally more compas- of the ways the researchers have measured sionate than men, we should be able to observe various concepts:* http://webapp.icpsr that difference by using any reasonable measure .umich.edu/GSS/ of compassion. If, on the other hand, women are more compassionate than men on some indicators but not on others, we should see if the two The Interchangeability of Indicators sets of indicators represent different dimensions of compassion. There is another way that the notion of indicators You have now seen the fundamental logic of can help us in our attempts to understand reality conceptualization and measurement. The discus- by means of “unreal” constructs. Suppose, for the sions that follow are mainly refi nements and ex- moment, that you and I have compiled a list of 100 tensions of what you’ve just read. Before turning to indicators of compassion and its various dimen- a technical elaboration of measurement, however, sions. Suppose further that we disagree widely we need to fi ll out the picture of conceptualization on which indicators give the clearest evidence by looking at some of the ways social research- of compassion or its absence. If we pretty much ers provide the meanings of terms that have stan- agree on some indicators, we could focus our at- dards, consistency, and commonality. tention on those, and we would probably agree on the answer they provided. We would then be able Real, Nominal, and to say that some people are more compassionate Operational Defi nitions than others in some dimension. But suppose we don’t really agree on any of the possible indicators. As we have seen, the design and execution of social research requires us to clear away the confusion over concepts and reality. To this end, logi- *Each time the Internet icon appears, you’ll be given help- cians and scientists have distinguished three kinds ful leads for searching the World Wide Web. of defi nitions: real, nominal, and operational. 140 CHAPTER 5 CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT

The fi rst of these refl ects the reifi cation of terms. As Carl Hempel cautions, IN THE REAL WORLD CONCEPTUALIZATION A “real” defi nition, according to traditional logic, is not a stipulation determining the meaning of some expression but a statement of the By this point in the chapter, you should be “essential nature” or the “essential attributes” gaining an appreciation for the ambiguity of of some entity. The notion of essential nature, language. This creates special challenges for however, is so vague as to render this charac- social researchers, but it is no less signifi cant terization useless for the purposes of rigorous in daily life. George Lakoff, a professor of lin- inquiry.— (1952:6) guistics and cognitive science at the Univer- sity of California–Berkeley, has written widely In other words, trying to specify the “real” meaning about the ways in which language choices of concepts only leads to a quagmire: It mistakes a have shaped political debate in the United construct for a real entity. States (Lakoff 2002). Specifi cally, he sug- The specifi cation of concepts in scientifi c in- gests that conservatives have been generally quiry depends instead on nominal and operational more adept than liberals in this regard. Thus, defi nitions. A nominal defi nition is one that is sim- for example, paying taxes could reasonably ply assigned to a term without any claim that the be seen as a patriotic act, paying your fair defi nition represents a “real” entity. Nominal defi - share of the cost of an orderly society. Avoid- nitions are arbitrary—I could defi ne compassion ing taxes, within this construction, would be as “plucking feathers off helpless birds” if I wanted unpatriotic. Instead of “tax avoidance” or to—but they can be more or less useful. For most “tax evasion,” however, we frequently hear purposes, especially communication, that last defi - calls for “tax relief,” which creates an image nition of compassion would be useless. Most nom- of citizens being unfairly burdened by gov- inal defi nitions represent some consensus, or con- ernment and revolting against that injustice. vention, about how a particular term is to be used. The intellectual act of conceptualization has An operational defi nition, as you may recall from real consequences in your daily life. Chapter 2, specifi es precisely how a concept will be measured—that is, the operations we choose to perform. An operational defi nition is nominal Creating Conceptual Order rather than real, but it achieves maximum clarity about what a concept means in the context of a The clarifi cation of concepts is a continuing pro- given study. In the midst of disagreement and con- cess in social research. Catherine Marshall and fusion over what a term “really” means, we can Gretchen Rossman (1995:18) speak of a “concep- specify a working defi nition for the purposes of an tual funnel” through which a researcher’s interest inquiry. Wishing to examine socioeconomic status becomes increasingly focused. Thus, a general (SES) in a study, for example, we may simply spec- interest in social activism could narrow to “indi- ify that we are going to treat SES as a combination viduals who are committed to empowerment and of income and educational attainment. In this de- social change” and further focus on discovering cision, we rule out other possible aspects of SES: “what experiences shaped the development of fully occupational status, money in the bank, property, committed social activists.” This focusing process lineage, lifestyle, and so forth. Our fi ndings will is inescapably linked to the language we use. then be interesting to the extent that our defi nition In some forms of qualitative research, the clari- of SES is useful for our purpose. See the box “Con- fi cation of concepts is a key element in the collec- ceptualization” for more on this. tion of data. Suppose you were conducting inter- CONCEPTUALIZATION 141 views and observations on a radical political group alization results in a commitment to a specifi c set devoted to combating oppression in U.S. society. of questionnaire items that will represent the con- Imagine how the meaning of oppression would cepts under study. Without that commitment, the shift as you delved more and more deeply into the study could not proceed. members’ experiences and worldviews. For exam- Even in less-structured research methods, how- ple, you might start out thinking of oppression in ever, we need to begin with an initial set of an- physical and perhaps economic terms. The more ticipated meanings that can be refi ned during data you learned about the group, however, the more collection and interpretation. No one seriously you might appreciate the possibility of psychologi- believes we can observe life with no preconcep- cal oppression. tions; for this reason, scientifi c observers must be The same point applies even to contexts where conscious of and explicit about these conceptual meanings might seem more fi xed. In the analysis starting points. of textual materials, for example, social research- Let’s explore initial conceptualization as it ap- ers sometimes speak of the “hermeneutic circle,” a plies to structured inquiries such as surveys and cyclical process of ever-deeper understanding. experiments. Though specifying nominal defi nitions focuses our observational strategy, it does The understanding of a text takes place through not allow us to observe. As a next step we must a process in which the meaning of the separate specify exactly what we are going to observe, how parts is determined by the global meaning of the we will observe it, and how we will interpret vari- text as it is anticipated. The closer determination ous possible observations. All these further speci- of the meaning of the separate parts may even- fi cations make up the operational defi nition of the tually change the originally anticipated meaning concept. of the totality, which again infl uences the mean- In the example of socioeconomic status, we ing of the separate parts, and so on.— (KVALE might decide to measure SES in terms of income 1996:47) and educational attainment. We might then ask Consider the concept “prejudice.” Suppose you survey respondents two questions: needed to write a defi nition of the term. You might 1. What was your total family income during the start out thinking about racial/ethnic prejudice. At past 12 months? some point you would realize you should proba- 2. What is the highest level of school you bly allow for gender prejudice, religious prejudice, completed? antigay prejudice, and the like in your defi nition. Examining each of these specifi c types of prejudice To organize our data, we would probably want would affect your overall understanding of the to specify a system for categorizing the answers general concept. As your general understanding people give us. For income, we might use catego- changed, however, you would likely see each of ries such as “under $5,000,” “$5,000 to $10,000,” the individual forms somewhat differently. and so on. Educational attainment might be simi- The continual refi nement of concepts occurs in larly grouped in categories: less than high school, all social research methods. Often you’ll fi nd your- high school, college, graduate degree. Finally, we self refi ning the meaning of important concepts would specify the way a person’s responses to even as you write up your fi nal report. these two questions would be combined to create Although conceptualization is a continuing a measure of SES. process, it’s vital to address it specifi cally at the be- In this way we would create a working and ginning of any study design, especially rigorously workable defi nition of SES. Although others might structured research designs such as surveys and disagree with our conceptualization and opera- experiments. In a survey, for example, operation- tionalization, the defi nition would have one essen- 142 CHAPTER 5 CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT tial scientifi c virtue: It would be absolutely specifi c act without having any data about the individuals and unambiguous. Even if someone disagreed engaging in it. with our defi nition, that person would have a good At a more general level, Durkheim suggested idea how to interpret our research results, because that suicide also refl ects the extent to which a what we meant by SES—refl ected in our analyses society’s agreements are clear and stable. Noting and conclusions—would be precise and clear. that times of social upheaval and change often Here is a diagram showing the progression of present individuals with grave uncertainties about measurement steps from our vague sense of what what is expected of them, Durkheim suggested a term means to specifi c measurements in a fully that such uncertainties cause confusion, anxiety, structured scientifi c study: and even self-destruction. To describe this societal condition of normlessness, Durkheim chose the Conceptualization term anomie. Durkheim did not make this word ➞➞➞ up. Used in both German and French, it literally Nominal Defi nition meant “without law.” The English term anomy had been used for at least three centuries before Operational Defi nition Durkheim to mean disregard for divine law. How- ever, Durkheim created the social scientifi c con- Measurements in the Real World cept of anomie. Since Durkheim’s time, social scientists have An Example of Conceptualization: found anomie a useful concept, and many have The Concept of Anomie expanded on Durkheim’s use. Robert Merton, in a classic article entitled “Social Structure and Ano- To look at the overall process of conceptualiza- mie” (1938), concluded that anomie results from a tion in research, let’s look briefl y at the history of disparity between the goals and means prescribed a specifi c social scientifi c concept. Researchers by a society. Monetary success, for example, is a studying urban riots often focus on the part played widely shared goal in our society, yet not all indi- by feelings of powerlessness. Social researchers viduals have the resources to achieve it through sometimes use the word anomie in this context. acceptable means. An emphasis on the goal it- This term was fi rst introduced into social science self, Merton suggested, produces normlessness, by Emile Durkheim, the great French sociologist, because those denied the traditional avenues to in his classic 1897 study, Suicide. wealth go about getting it through illegitimate Using only government publications on suicide means. Merton’s discussion, then, could be con- rates in different regions and countries, Durkheim sidered a further conceptualization of the concept produced a work of analytical genius. To determine of anomie. the effects of religion on suicide, he compared the Although Durkheim originally used the concept suicide rates of predominantly Protestant coun- of anomie as a characteristic of societies, as did tries with those of predominantly Catholic ones, Merton after him, other social scientists have used Protestant regions of Catholic countries with Cath- it to describe individuals. To clarify this distinc- olic regions of Protestant countries, and so forth. tion, some scholars have chosen to use anomie in To determine the possible effects of the weather, reference to its original, societal meaning and to he compared suicide rates in northern and south- use the term anomia in reference to the individual ern countries and regions, and he examined the characteristic. In a given society, then, some in- different suicide rates across the months and sea- dividuals experience anomia, and others do not. sons of the year. Thus, he could draw conclusions Elwin Powell, writing 20 years after Merton, pro- about a supremely individualistic and personal vided the following conceptualization of anomia CONCEPTUALIZATION 143

(though using the term anomie) as a characteristic “The Origins of Anomia,” which he prepared for of individuals: this book before his death. This abbreviated history of anomie and ano- When the ends of action become contradic- mia as social scientifi c concepts illustrates several tory, inaccessible or insignifi cant, a condition of points. First, it is a good example of the process anomie arises. Characterized by a general loss through which general concepts become opera- of orientation and accompanied by feelings of tionalized measurements. This is not to say that “emptiness” and apathy, anomie can be simply the issue of how to operationalize anomie/anomia conceived as meaninglessness. — (1958:132) has been resolved once and for all. Scholars will surely continue to reconceptualize and reopera- Powell went on to suggest there were two dis- tionalize these concepts for years to come, con- tinct kinds of anomia and to examine how the two tinually seeking more-useful measures. rose out of different occupational experiences to The Srole scale illustrates another important result at times in suicide. In his study, however, point. Letting conceptualization and operational- Powell did not measure anomia per se; he studied ization be open-ended does not necessarily pro- the relationship between suicide and occupation, duce anarchy and chaos, as you might expect. making inferences about the two kinds of anomia. Order often emerges. For one thing, although we Thus, the study did not provide an operational defi - could defi ne anomia any way we chose—in terms nition of anomia, only a further conceptualization. of, say, shoe size—we would likely defi ne it in ways Although many researchers have offered op- not too different from other people’s mental im- erational defi nitions of anomia, one name stands ages. If you were to use a really offbeat defi nition, out over all. Two years before Powell’s article ap- people would probably ignore you. peared, Leo Srole (1956) published a set of ques- A second source of order is that, as researchers tionnaire items that he said provided a good mea- discover the utility of a particular conceptualization sure of anomia. It consists of fi ve statements that and operationalization of a concept, they’re likely subjects were asked to agree or disagree with: to adopt it, which leads to standardized defi nitions of concepts. Besides the Srole scale, examples 1. In spite of what some people say, the lot of the include IQ tests and a host of demographic and average man is getting worse. economic measures developed by the U.S. Census 2. It’s hardly fair to bring children into the world Bureau. Using such established measures has two with the way things look for the future. advantages: They have been extensively pretested 3. Nowadays a person has to live pretty much for and debugged, and studies using the same scales today and let tomorrow take care of itself. can be compared. If you and I do separate stud- 4. These days a person doesn’t really know who ies of two different groups and use the Srole scale, he can count on. we can compare our two groups on the basis of 5. There’s little use writing to public offi cials anomia. because they aren’t really interested in the Social scientists, then, can measure anything problems of the average man. — (1956:713) that’s real; through conceptualization and operationalization, they can even do a pretty good job In the decades following its publication, the of measuring things that aren’t. Granting that such Srole scale has become a research staple for so- concepts as socioeconomic status, prejudice, com- cial scientists. You’ll likely fi nd this particular op- passion, and anomia aren’t ultimately real, social erationalization of anomia used in many of the scientists can create order in handling them. This research projects reported in academic journals. order is based on utility, however, not on ultimate Srole touches on this in the accompanying box truth. Text not available due to copyright restrictions DEFINITIONS IN DESCRIPTIVE AND EXPLANATORY STUDIES 145

DEFINITIONS IN DESCRIPTIVE ing for work” is defi ned operationally as saying yes AND EXPLANATORY STUDIES in response to an interviewer’s asking, “Have you been looking for a job during the past seven days?” (Seven days is the period most often specifi ed, but As described in Chapter 4, two general purposes for some research purposes it might make more of research are description and explanation. The sense to shorten or lengthen it.) distinction between them has important implica- As you can see, the conclusion of a descriptions for defi nition and measurement. If it seems tive study about the unemployment rate depends that description is simpler than explanation, you directly on how each issue of defi nition is re- may be surprised to learn that defi nitions are more solved. Increasing the period during which people problematic for descriptive research than for ex- are counted as looking for work would add more planatory research. Before we turn to other as- unemployed people to the labor force as defi ned, pects of measurement, you’ll need a basic under- thereby increasing the reported unemployment standing of why this is so (we’ll discuss this point rate. If we follow another convention and speak more fully in Part 4). of the civilian labor force and the civilian unem- It’s easy to see the importance of clear and pre- ployment rate, we are excluding military person- cise defi nitions for descriptive research. If we want nel; that, too, increases the reported unemploy- to describe and report the unemployment rate ment rate, because military personnel would be in a city, our defi nition of “being unemployed” is employed—by defi nition. Thus the descriptive obviously critical. That defi nition will depend on statement that the unemployment rate in a city is 3 our defi nition of another term: the labor force. If percent, or 9 percent, or whatever it might be, de- it seems patently absurd to regard a three-year- pends directly on the operational defi nitions used. old child as being unemployed, it is because such This example is relatively clear because there a child is not considered a member of the labor are several accepted conventions relating to the la- force. Thus, we might follow the U.S. Census Bu- bor force and unemployment. Now, consider how reau’s convention and exclude all people under diffi cult it would be to get agreement about the age 14 from the labor force. defi nitions you would need in order to say, “Forty- This convention alone, however, would not give fi ve percent of the students at this institution are us a satisfactory defi nition, because it would count politically conservative.” Like the unemployment as unemployed such people as high school stu- rate, this percentage would depend directly on the dents, the retired, the disabled, and homemakers. defi nition of what is being measured—in this case, We might follow the Census convention further by political conservatism. A different defi nition might defi ning the labor force as “all persons 14 years of result in the conclusion “Five percent of the stu- age and over who are employed, looking for work, dent body are politically conservative.” or waiting to be called back to a job from which Ironically, defi nitions are less problematic in the they have been laid off or furloughed.” If a student, case of explanatory research. Let’s suppose we’re homemaker, or retired person is not looking for interested in explaining political conservatism. work, such a person would not be included in the Why are some people conservative and others not? labor force. Unemployed people, then, would be More specifi cally, let’s suppose we’re interested in those members of the labor force, as defi ned, who whether conservatism increases with age. What if are not employed. you and I have 25 different operational defi nitions But what does “looking for work” mean? Must a of conservative, and we can’t agree on which defi - person register with the state employment service nition is best? As we saw in the discussion of in- or go from door to door asking for employment? dicators, this is not necessarily an insurmountable Or would it be suffi cient to want a job or be open obstacle to our research. Suppose we found old to an offer of employment? Conventionally, “look- people to be more conservative than young peo- 146 CHAPTER 5 CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT

Text not available due to copyright restrictions

ple in terms of all 25 defi nitions. Clearly, the exact In practice, explanatory research seldom results defi nition wouldn’t matter much. We would con- in fi ndings quite as unambiguous as this example clude that old people are generally more conserva- suggests; nonetheless, the general pattern is quite tive than young people—even though we couldn’t common in actual research. There are consistent agree about exactly what conservative means. patterns of relationships in human social life that OPERATIONALIZATION CHOICES 147 result in consistent research fi ndings. However, bly won’t hurt your research any, either. The same such consistency does not appear in a descriptive decision faces you at the other end of the income situation. Changing defi nitions almost inevitably spectrum. In studies of the general U.S. popula- result in different descriptive conclusions. The tion, a bottom category of $5,000 or less usually box “The Importance of Variable Names” explores works fi ne. this issue in connection with the variable citizen In studies of attitudes and orientations, the participation. question of range of variation has another dimension. Unless you’re careful, you may end up mea- OPERATIONALIZATION CHOICES suring only half an attitude without really meaning to. Here’s an example of what I mean. Suppose you’re interested in people’s attitudes In discussing conceptualization, I frequently have toward expanding the use of nuclear power gen- referred to operationalization, for the two are inti- erators. If you reasonably guess or have experi- mately linked. To recap: Conceptualization is the enced that some people consider nuclear power refi nement and specifi cation of abstract concepts, the greatest thing since the wheel, whereas other and operationalization is the development of spe- people have absolutely no interest in it, it makes cifi c research procedures (operations) that will re- sense to ask people how much they favor expand- sult in empirical observations representing those ing the use of nuclear energy and to give them an- concepts in the real world. swer categories ranging from “Favor it very much” As with the methods of data collection, social to “Don’t favor it at all.” researchers have a variety of choices when opera- This operationalization, however, conceals half tionalizing a concept. Although the several choices the attitudinal spectrum regarding nuclear energy. are intimately interconnected, I’ve separated them Many people have feelings that go beyond simply for the sake of discussion. Realize, however, that not favoring it: They are, with greater or lesser de- operationalization does not proceed through a grees of intensity, actively opposed to it. In this in- systematic checklist. stance, there is considerable variation on the left side of zero. Some oppose it a little, some quite Range of Variation a bit, and others a great deal. To measure the full range of variation, then, you’d want to operation- In operationalizing any concept, researchers must alize attitudes toward nuclear energy with a range be clear about the range of variation that interests from favoring it very much, through no feelings them. The question is, to what extent are we willing one way or the other, to opposing it very much. to combine attributes in fairly gross categories? This consideration applies to many of the vari- Let’s suppose you want to measure people’s ables that social researchers study. Virtually any incomes in a study by collecting the information public issue involves both support and opposi- from either records or interviews. The highest an- tion, each in varying degrees. Political orientations nual incomes people receive run into the millions range from very liberal to very conservative, and of dollars, but not many people earn that much. depending on the people you’re studying, you may Unless you’re studying the very rich, keeping track want to allow for radicals on one or both ends. of extremely high categories probably won’t add Similarly, people are not just more or less religious; much to your study. Depending on whom you some are antireligious. study, you’ll probably want to establish a highest The point is not that you must measure the full income category with a much lower fl oor—maybe range of variation in every case. You should, how- $100,000 or more. Although this decision will ever, consider whether you need to do so, given lead you to throw together people who earn a tril- your particular research purpose. If the difference lion dollars a year with paupers earning a mere between not religious and antireligious isn’t rel- $100,000, they’ll survive it, and that mixing proba- evant to your research, forget it. Someone has de- 148 CHAPTER 5 CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT

ﬁ ned pragmatism as “any difference that makes no of a given study, or why we are making a particu- difference is no difference.” Be pragmatic. lar measurement. I can give you a useful guideline, Finally, decisions about the range of variation though. Whenever you’re not sure how much de- should be governed by the expected distribution tail to pursue in a measurement, go after too much of attributes among the subjects of the study. In rather than too little. When a subject in an in-depth a study of college professors’ attitudes toward interview volunteers that she is 37 years old, re- the value of higher education, you could prob- cord “37” in your notes, not “in her thirties.” When ably stop at no value and not worry about those you’re analyzing the data, you can always combine who might consider higher education dangerous precise attributes into more general categories, but to students’ health. (If you were studying students, you can never separate any variations you lumped however . . . .) together during observation and measurement.

Variations between the Extremes A Note on Dimensions

Degree of precision is a second consideration in We’ve already discussed dimensions as a charac- operationalizing variables. What it boils down to teristic of concepts. When researchers get down is how fi ne you will make the distinctions among to the business of creating operational measures the various possible attributes composing a given of variables, they often discover—or worse, never variable. Does it matter for your purposes whether notice—that they haven’t been exactly clear about a person is 17 or 18 years old, or could you con- which dimensions of a variable they’re really inter- duct your inquiry by throwing them together in a ested in. Here’s an example. group labeled 10 to 19 years old? Don’t answer too Let’s suppose you’re studying people’s attitudes quickly. If you wanted to study rates of voter reg- toward government, and you want to include an istration and participation, you’d defi nitely want examination of how people feel about corruption. to know whether the people you studied were old Here are just a few of the dimensions you might enough to vote. In general, if you’re going to mea- examine: sure age, you must look at the purpose and pro- • Do people think there is corruption in cedures of your study and decide whether fi ne or government? gross differences in age are important to you. In a • How much corruption do they think there is? survey, you’ll need to make these decisions in or- • How certain are they in their judgment of how der to design an appropriate questionnaire. In the much corruption there is? case of in-depth interviews, these decisions will • How do they feel about corruption in govern- condition the extent to which you probe for details. ment as a problem in society? The same thing applies to other variables. If you • What do they think causes it? measure political affi liation, will it matter to your • Do they think it’s inevitable? inquiry whether a person is a conservative Demo- • What do they feel should be done about it? crat rather than a liberal Democrat, or will know- • What are they willing to do personally to elimi- ing just the party suffi ce? In measuring religious nate corruption in government? affi liation, is it enough to know that a person is a • How certain are they that they would be will- Protestant, or do you need to know the denomina- ing to do what they say they would do? tion? Do you simply need to know whether or not a person is married, or will it make a difference to The list could go on and on—how people feel know if he or she has never married or is sepa- about corruption in government has many dimen- rated, widowed, or divorced? sions. It’s essential to be clear about which ones There is, of course, no general answer to such are important in our inquiry; otherwise, you may questions. The answers come out of the purpose measure how people feel about corruption when OPERATIONALIZATION CHOICES 149 you really wanted to know how much they think be able to classify every observation in terms of there is, or vice versa. one and only one attribute. For example, we need Once you’ve determined how you’re going to to defi ne “employed” and “unemployed” in such collect your data (for example, survey, fi eld re- a way that nobody can be both at the same time. search) and decided on the relevant range of varia- That means being able to classify the person who tion, the degree of precision needed between the is working at a job but is also looking for work. extremes of variation, and the specifi c dimensions (We might run across a fully employed mud wres- of the variables that interest you, you may have tler who is looking for the glamour and excitement another choice: a mathematical-logical one. That of being a social researcher.) In this case, we might is, you may need to decide what level of measure- defi ne the attributes so that employed takes prece- ment to use. To discuss this point, we need to take dence over unemployed, and anyone working at a another look at attributes and their relationship to job is employed regardless of whether he or she is variables. looking for something better.

Deﬁ ning Variables and Attributes Levels of Measurement

An attribute, you’ll recall, is a characteristic or Attributes operationalized as mutually exclusive quality of something. “Female” is an example. So and exhaustive may be related in other ways as is “old” or “student.” Variables, on the other hand, well. For example, the attributes composing vari- are logical sets of attributes. Thus, sex is a variable ables may represent different levels of measure- composed of the attributes female and male. ment. In this section, we’ll examine four levels The conceptualization and operationalization of measurement: nominal, ordinal, interval, and processes can be seen as the specifi cation of vari- ratio. ables and the attributes composing them. Thus, in the context of a study of unemployment, employ- Nominal Measures Variables whose attributes ment status is a variable having the attributes em- have only the characteristics of exhaustiveness ployed and unemployed; the list of attributes could and mutual exclusiveness are nominal mea- also be expanded to include the other possibilities sures. Examples include sex, religious affi liation, discussed earlier, such as homemaker. political party affi liation, birthplace, college major, Every variable must have two important quali- and hair color. Although the attributes composing ties. First, the attributes composing it should be each of these variables—as male and female com- exhaustive. For the variable to have any utility in pose the variable —are distinct from one another research, we must be able to classify every obser- (and exhaust the possibilities of gender among vation in terms of one of the attributes composing people), they have no additional structures. Nomi- the variable. We’ll run into trouble if we conceptu- nal measures merely offer names or labels for alize the variable political party affi liation in terms of characteristics. the attributes Republican and Democrat, because Imagine a group of people characterized in some of the people we set out to study will identify terms of one such variable and physically grouped with the Green Party, the Reform Party, or some by the applicable attributes. For example, say other organization, and some (often a large percentage) will tell us they have no party affi liation. We could make the list of attributes exhaustive by nominal measure A variable whose attributes have adding “other” and “no affi liation.” Whatever we only the characteristics of exhaustiveness and mutual exclusiveness. In other words, a level of measurement do, we must be able to classify every observation. describing a variable that has attributes that are merely At the same time, attributes composing a vari- different, as distinguished from ordinal, interval, or ratio able must be mutually exclusive. That is, we must measures. Sex is an example of a nominal measure. 150 CHAPTER 5 CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT we’ve asked a large gathering of people to stand materials, we might eventually be able to arrange together in groups according to the states in which several materials in a row, ranging from the soft- they were born: all those born in Vermont in one est to the hardest. We could never say how hard group, those born in California in another, and so a given material was in absolute terms; we could forth. The variable is birthplace; the attributes are only say how hard in relative terms—which mate- born in California, born in Vermont, and so on. All rials it is harder than and which softer than. the people standing in a given group have at least Let’s pursue the earlier example of grouping the one thing in common and differ from the people people at a social gathering. This time imagine that in all other groups in that same regard. Where the we ask all the people who have graduated from individual groups form, how close they are to one college to stand in one group, all those with only another, or how the groups are arranged in the a high school diploma to stand in another group, room is irrelevant. All that matters is that all the and all those who have not graduated from high members of a given group share the same state school to stand in a third group. This manner of of birth and that each group has a different shared grouping people satisfi es the requirements for ex- state of birth. All we can say about two people in haustiveness and mutual exclusiveness discussed terms of a nominal variable is that they are either earlier. In addition, however, we might logically the same or different. arrange the three groups in terms of the relative amount of formal education (the shared attribute) Ordinal Measures Variables with attributes we each had. We might arrange the three groups in can logically rank-order are ordinal measures. a row, ranging from most to least formal educa- The different attributes of ordinal variables repre- tion. This arrangement would provide a physical sent relatively more or less of the variable. Vari- representation of an ordinal measure. If we knew ables of this type include social class, conservatism, which groups two individuals were in, we could alienation, prejudice, and intellectual sophistication. determine that one had more, less, or the same In addition to saying whether two people are the formal education as the other. same or different in terms of an ordinal variable, Notice in this example that it doesn’t matter how we can also say one is “more” than the other—that close or far apart the educational groups are from is, more conservative, more religious, older, and one another. The college and high school groups so forth. might be 5 feet apart, and the less-than-high- In the physical sciences, hardness is the most school group 500 feet farther down the line. These frequently cited example of an ordinal measure. actual distances don’t have any meaning. The We can say that one material (for example, dia- high school group, however, should be between mond) is harder than another (say, glass) if the for- the less-than-high-school group and the college mer can scratch the latter and not vice versa. By group, or else the rank order will be incorrect. attempting to scratch various materials with other Interval Measures For the attributes composing some variables, the actual distance separating ordinal measure A level of measurement describing those attributes does have meaning. Such vari- a variable with attributes we can rank-order along some ables are interval measures. For these, the logi- dimension. An example is socioeconomic status as com- cal distance between attributes can be expressed posed of the attributes high, medium, low. in meaningful standard intervals. interval measure A level of measurement describing For example, in the Fahrenheit temperature a variable whose attributes are rank-ordered and have scale, the difference, or distance, between 80 de- equal distances between adjacent attributes. The Fahren- heit temperature scale is an example of this, because the grees and 90 degrees is the same as that between distance between 17 and 18 is the same as that between 40 degrees and 50 degrees. However, 80 degrees 89 and 90. Fahrenheit is not twice as hot as 40 degrees, be- OPERATIONALIZATION CHOICES 151 cause in both the Fahrenheit and Celsius scales, would stand (or sit or lie) together, the two-year- “zero” is arbitrary; that is, zero degrees does not olds together, the three-year-olds, and so forth. The really mean total lack of heat. Similarly, minus 30 fact that members of a single group share the same degrees on either scale doesn’t represent 30 de- age and that each different group has a different grees less than no heat. (In contrast, the Kelvin shared age satisfi es the minimum requirements for scale is based on an absolute zero, which does a nominal measure. Arranging the several groups mean a complete lack of heat.) in a line from youngest to oldest meets the addi- About the only interval measures commonly tional requirements of an ordinal measure and lets used in social scientifi c research are constructed us determine if one person is older than, younger measures such as standardized intelligence tests than, or the same age as another. If we space the that have been more or less accepted. The inter- groups equally far apart, we satisfy the additional val separating IQ scores of 100 and 110 may be requirements of an interval measure and will be regarded as the same as the interval between 110 able to say how much older one person is than and 120, by virtue of the distribution of observed another. Finally, because one of the attributes in- scores obtained by many thousands of people who cluded in age represents a true zero (babies car- have taken the tests over the years. But it would be ried by women about to give birth), the phalanx of incorrect to infer that someone with an IQ of 150 hapless party goers also meets the requirements is 50 percent more intelligent than someone with of a ratio measure, permitting us to say that one an IQ of 100. (A person who received a score of 0 person is twice as old as another. (Remember this on a standard IQ test could not be regarded, strictly in case you’re asked about it in a workbook as- speaking, as having no intelligence, although we signment.) Another example of a ratio measure might feel he or she was unsuited to be a college is income, which extends from an absolute zero professor or even a college student. But perhaps a to approximately infi nity, if you happen to be the dean . . . ?) founder of Microsoft. When comparing two people in terms of an in- Comparing two people in terms of a ratio vari- terval variable, we can say they are different from able, then, allows us to determine (1) that they are one another (nominal), and that one is more than different (or the same), (2) that one is more than another (ordinal). In addition, we can say “how the other, (3) how much they differ, and (4) the ra- much” more in terms of the scores themselves. tio of one to another. Figure 5-1 summarizes this discussion by presenting a graphic illustration of Ratio Measures Most of the social scientifi c vari- the four levels of measurement. ables meeting the minimum requirements for interval measures also meet the requirements for ratio Implications of Levels of Measurement Be- measures. In ratio measures, the attributes com- cause it’s unlikely that you’ll undertake the physi- posing a variable, besides having all the structural cal grouping of people just described (try it once, characteristics mentioned previously, are based on and you won’t be invited to many parties), I should a true zero point. The Kelvin temperature scale is draw your attention to some of the practical im- one such measure. Examples from social research plications of the differences that have been distin- include age, length of residence in a given place, guished. These implications appear primarily in number of organizations belonged to, number of times attending religious services during a particular period, number of times married, and number A level of measurement describing of Arab friends. ratio measure a variable with attributes that have all the qualities of Returning to the illustration of methodological nominal, ordinal, and interval measures and in addition party games, we might ask a gathering of people are based on a “true zero” point. Age is an example of a to group themselves by age. All the one-year-olds ratio measure. 152 CHAPTER 5 CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT

Nominal Measure Example: Sex

Female Male

Ordinal Measure Example: Religiosity "How important is religion to you?"

Not very Fairly Very Most important important important important thing in my life

Low High

Interval Measure Example: IQ

95 100 105 110 115

Ratio Measure Example: Income

$0 $10,000 $20,000 $30,000 $40,000 $50,000 FIGURE 5-1 Levels of Measurement. Often you can choose among different levels of measurement— nominal, ordinal, interval, or ratio—carrying progressively more amounts of information.

the analysis of data (discussed in Part 4), but you drawing research conclusions appropriate to the need to anticipate such implications when you’re levels of measurement used in your variables. For structuring any research project. example, you might reasonably plan to determine Certain quantitative analysis techniques re- and report the mean age of a population under quire variables that meet certain minimum levels study (add up all the individual ages and divide by of measurement. To the extent that the variables the number of people), but you should not plan to to be examined in a research project are limited report the mean religious afﬁ liation, because that to a particular level of measurement—say, ordi- is a nominal variable, and the mean requires rational—you should plan your analytical techniques level data. (You could report the modal—the most accordingly. More precisely, you should anticipate common—religious afﬁ liation.) OPERATIONALIZATION CHOICES 153

At the same time, you can treat some variables ally, you cannot convert a lower-level measure to as representing different levels of measurement. a higher-level one. That is a one-way street worth Ratio measures are the highest level, descending remembering. through interval and ordinal to nominal, the low- Typically a research project will tap variables est level of measurement. A variable representing at different levels of measurement. For example, a higher level of measurement—say, ratio—can William and Denise Bielby (1999) set out to exam- also be treated as representing a lower level of ine the world of fi lm and television, using a nomo- measurement—say, ordinal. Recall, for example, thetic, longitudinal approach (take a moment to that age is a ratio measure. If you wished to exam- remind yourself what that means). In what they ine only the relationship between age and some referred to as the “culture industry,” the authors ordinal-level variable—say, self-perceived religios- found that reputation (an ordinal variable) is the ity: high, medium, and low—you might choose to best predictor of screenwriters’ future productivity. treat age as an ordinal-level variable as well. You More interestingly, they found that screenwriters might characterize the subjects of your study as be- who were represented by “core” (or elite) agencies ing young, middle-aged, and old, specifying what were far more likely not only to fi nd jobs (a nomi- age range determined each of these groupings. nal variable) but also to fi nd jobs that paid more Finally, age might be used as a nominal-level vari- (a ratio variable). In other words, the researchers able for certain research purposes. People might found that an agency’s reputation (ordinal) was a be grouped as being born during the depression of key independent variable for predicting a screen- the 1930s or not. Another nominal measurement, writer’s success. The researchers also found that based on birth date rather than just age, would be being older (ratio), being female (nominal), be- the grouping of people by astrological signs. longing to an ethnic minority (nominal), and hav- The level of measurement you’ll seek, then, is ing more years of experience (ratio) were disad- determined by the analytical uses you’ve planned vantageous for a screenwriter. On the other hand, for a given variable, as you keep in mind that some higher earnings from previous years (measured variables are inherently limited to a certain level. in ordinal categories) led to more success in the If a variable is to be used in a variety of ways, re- future. In the researchers’ terms, “success breeds quiring different levels of measurement, the study success” (Bielby and Bielby 1999:80). See the box should be designed to achieve the highest level re- “On to Hollywood” for more on the Bielby study. quired. For example, if the subjects in a study are asked their exact ages, they can later be organized into ordinal or nominal groupings. IN THE REAL WORLD You don’t necessarily need to measure variables ON TO HOLLYWOOD at their highest level of measurement, however. If you’re sure to have no need for ages of people at higher than the ordinal level of measurement, you Say you want to be a Hollywood screen- may simply ask people to indicate their age range, writer. How might you use the results of the such as 20 to 29, 30 to 39, and so forth. In a study Bielby and Bielby (1999) study to enhance of the wealth of corporations, rather than seek your career? Say you didn’t do so well and more precise information, you may use Dun & instead started a school for screenwriters. Bradstreet ratings to rank corporations. Whenever How could the results of the study be used to your research purposes are not altogether clear, plan courses? Finally, how might the results however, seek the highest level of measurement be useful to you if you were a social activist possible. Again, although ratio measures can later committed to fi ghting discrimination in the be reduced to ordinal ones, you cannot convert “culture industry?” an ordinal measure to a ratio one. More gener- 154 CHAPTER 5 CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT

Single or Multiple Indicators erage (GPA). We assign numerical scores to each letter grade, total the points earned by a given stu- With so many alternatives for operationalizing dent, and divide by the number of courses taken to social research variables, you may fi nd yourself obtain a composite measure. (If the courses vary worrying about making the right choices. To coun- in number of credits, we adjust the point values ter this feeling, let me add a dash of certainty and accordingly.) It’s often appropriate to create such stability. composite measures in social research. Many social research variables have fairly obvious, straightforward measures. No matter how you cut it, sex usually turns out to be a matter of male Some Illustrations of or female: a nominal-level variable that can be Operationalization Choices measured by a single observation—either through To bring together all the operationalization choices looking (well, not always) or through asking a available to the social researcher and to show the question (usually). In a study involving the size of potential in those possibilities, let’s look at some families, you’ll want to think about adopted and of the distinct ways you might address various foster children as well as blended families, but it’s research problems. The alternative ways of op- usually pretty easy to fi nd out how many children erationalizing the variables in each case should a family has. For most research purposes, the resi- demonstrate the opportunities that social research dent population of a country is the resident popu- can present to our ingenuity and imaginations. lation of that country—you can fi nd the number in To simplify matters, I have not attempted to dean almanac. A great many variables, then, have scribe all the research conditions that would make obvious single indicators. If you can get one piece one alternative superior to the others, though in of information, you have what you need. a given situation they would not all be equally Sometimes, however, there is no single indi- appropriate. cator that will give you the measure of a chosen variable. As discussed earlier in this chapter, many 1. Are women more compassionate than men? concepts are subject to varying interpretations— a. Select a group of subjects for study, with each with several possible indicators. In these equal numbers of men and women. Pres- cases, you’ll want to make several observations for ent them with hypothetical situations that a given variable. You can then combine the several involve someone’s being in trouble. Ask pieces of information you’ve collected to create a them what they would do if they were composite measurement of the variable in ques- confronted with that situation. What would tion. Chapter 6 is devoted to ways of doing that, so they do, for example, if they came across a here let’s look at just one simple illustration. small child who was lost and crying for his Consider the concept “college performance.” or her parents? Consider any answer that All of us have noticed that some students perform involves helping or comforting the child as well in college courses and others do not. In study- an indicator of compassion. See whether ing these differences, we might ask what charac- men or women are more likely to indicate teristics and experiences are related to high levels they would be compassionate. of performance (many researchers have done just b. Set up an experiment in which you pay a that). How should we measure overall perfor- small child to pretend that he or she is lost. mance? Each grade in any single course is a po- Put the child to work on a busy sidewalk tential indicator of college performance, but it also and observe whether men or women are may not typify the student’s general performance. more likely to offer assistance. Also be The solution to this problem is so fi rmly established sure to count the total number of men and that it is, of course, obvious: the grade point av- women who walk by, because there may OPERATIONALIZATION CHOICES 155

be more of one than the other. If that’s the live in. Look up some recent results in the case, simply calculate the percentage of men library or through your local newspaper. and the percentage of women who help. c. Compare suicide rates in the two states. c. Select a sample of people and do a survey 4. Who are the most popular instructors on your in which you ask them what organizations campus—those in the social sciences, the they belong to. Calculate whether women natural sciences, or the humanities? or men are more likely to belong to those a. If your school has formal student evalu- that seem to refl ect compassionate feel- ations of instructors, review some recent ings. To take account of men who belong results and compute the average ratings of to more organizations than do women in each group. general—or vice versa—do this: For each b. Begin visiting the introductory courses person you study, calculate the percent- given in each group of disciplines and mea- age of his or her organizational member- sure the attendance rate of each class. ships that refl ect compassion. See if men or c. In De cember, select a group of faculty in women have a higher average percentage. each of the three divisions and ask them 2. Are sociology students or accounting students to keep a record of the numbers of holiday better informed about world affairs? greeting cards and presents they receive a. Prepare a short quiz on world affairs and from admiring students. See who wins. arrange to administer it to the students in a sociology class and in an accounting The point of these examples is not necessar- class at a comparable level. If you want to ily to suggest respectable research projects but compare sociology and accounting majors, to illustrate the many ways variables can be op- be sure to ask students what they are erationalized. The box “Measuring College Satis- majoring in. faction” briefl y overviews the preceding steps in b. Get the instructor of a course in world terms of a concept mentioned at the outset of this affairs to give you the average grades of chapter. sociology and accounting students in the course. Operationalization Goes On and On c. Take a petition to sociology and accounting classes that urges that “the United Nations Although I’ve discussed conceptualization and headquarters be moved to New York City.” operationalization as activities that precede data Keep a count of how many in each class collection and analysis—for example, you must sign the petition and how many inform you design questionnaire items before you send out that the UN headquarters is already located a questionnaire—these two processes continue in New York City. throughout any research project, even if the data 3. Do people consider New York or California the have been collected in a structured mass survey. better place to live? As we’ve seen, in less-structured methods such as a. Consulting the Statistical Abstract of the fi eld research, the identifi cation and specifi cation United States or a similar publication, check of relevant concepts is inseparable from the ongo- the migration rates into and out of each ing process of observation. state. See if you can fi nd the numbers mov- As a researcher, always be open to reexamining directly from New York to California and ing your concepts and defi nitions. The ultimate vice versa. purpose of social research is to clarify the nature of b. The national polling companies—Gal- social life. The validity and utility of what you learn lup, Harris, Roper, and so forth—often ask in this regard doesn’t depend on when you fi rst fi g- people what they consider the best state to ured out how to look at things any more than it 156 CHAPTER 5 CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT

matters whether you got the idea from a learned IN THE REAL WORLD textbook, a dream, or your brother-in-law. MEASURING COLLEGE SATISFACTION CRITERIA OF MEASUREMENT QUALITY

Early in this chapter, we considered “college satisfaction” as an example of a concept we This chapter has come some distance. It began may often talk about casually. To study such with the bald assertion that social scientists can a concept, however, we need to engage in measure anything that exists. Then we discovered the processes of conceptualization and op- that most of the things we might want to measure erationalization. I’ll sketch out the process and study don’t really exist. Next we learned that brieﬂ y and you might try your hand at exit’s possible to measure them anyway. Now we panding on my comments. conclude the chapter with a discussion of some What are some of the dimensions of col- of the yardsticks against which we judge our rela- lege satisfaction? Here are a few to get you tive success or failure in measuring things—even started: things that don’t exist.

Academic quality: faculty, courses, majors Physical facilities: classrooms, dorms, Precision and Accuracy cafeteria, grounds To begin, measurements can be made with vary- Athletics and extracurricular activities ing degrees of precision. As we saw in the discus- Costs and availability of fi nancial aid sion of operationalization, precision concerns the Sociability of students, faculty, staff fi neness of distinctions made between the attri- Security, crime on campus butes that compose a variable. The description of What are some more dimensions that might a woman as “43 years old” is more precise than be relevant to students’ satisfaction or dis- “in her forties.” Saying a street-corner gang was satisfaction with their school? formed in the summer of 1996 is more precise than How would you measure each of these saying “during the 1990s.” dimensions? One method would be to ask As a general rule, precise measurements are a sample of students, “How would you rate superior to imprecise ones, as common sense sug- your level of satisfaction with each of the gests. There are no conditions under which im- following?” giving them a list of items simi- precise measurements are intrinsically superior to lar to those listed here and providing a set precise ones. Even so, exact precision is not always of categories for them to use (such as very necessary or desirable. If knowing that a woman is satisfi ed, satisfi ed, dissatisfi ed, very dissatis- in her forties satisfi es your research requirements, fi ed). But suppose you didn’t have the time then any additional effort invested in learning her or money to conduct a survey and were in- precise age is wasted. The operationalization of terested in comparing overall levels of satis- concepts, then, must be guided partly by an un- faction at several schools. What data about derstanding of the degree of precision required. If schools (the unit of analysis) might give you your needs are not clear, be more precise rather the answer you were interested in? Reten- than less. tion rates might be one general indicator of Don’t confuse precision with accuracy, however. satisfaction. Can you think of others? Describing someone as “born in New England” is less precise than “born in Stowe, Vermont”—but suppose the person in question was actually born CRITERIA OF MEASUREMENT QUALITY 157 in Boston. The less-precise description, in this in- In the other set of factories, each worker performs stance, is more accurate, a better refl ection of the many tasks, and small teams of workers complete real world. the whole process. Precision and accuracy are obviously important How should we measure morale? Following qualities in research measurement, and they prob- one strategy, we could observe the workers in each ably need no further explanation. When social re- factory, noticing such things as whether they joke searchers construct and evaluate measurements, with one another, whether they smile and laugh a however, they pay special attention to two techni- lot, and so forth. We could ask them how they like cal considerations: reliability and validity. their work and even ask them whether they think they would prefer their current arrangement or the Reliability other one being studied. By comparing what we observed in the different factories, we might reach In the abstract, reliability is a matter of whether a conclusion about which assembly process pro- a particular technique, applied repeatedly to the duces the higher morale. Notice that I’ve just de- same object, yields the same result each time. Let’s scribed a qualitative measurement procedure. say you want to know how much I weigh. (No, I Now let’s look at some reliability problems don’t know why.) As one technique, say you ask inherent in this method. First, how you and I are two different people to estimate my weight. If the feeling when we do the observing will likely color fi rst person estimates 150 pounds and the other what we see. We may misinterpret what we ob- estimates 300, we have to conclude that the tech- serve. We may see workers kidding each other but nique of having people estimate my weight isn’t think they’re having an argument. We may catch very reliable. them on an off day. If we were to observe the same Suppose, as an alternative, that you use a bath- group of workers several days in a row, we might room scale as your measurement technique. I step arrive at different evaluations on each day. If sev- on the scale twice, and you note the same result eral observers evaluated the same behavior, on the each time. The scale has presumably reported the other hand, they similarly might arrive at different same weight both times, indicating that the scale conclusions about the workers’ morale. provides a more reliable technique for measuring Here’s another, quantitative approach to as- a person’s weight than does asking people to es- sessing morale. Suppose we check the company timate it. records to see how many grievances have been Reliability, however, does not ensure accuracy fi led with the union during some fi xed period. Pre- any more than does precision. Suppose I’ve set sumably this would be an indicator of morale: the my bathroom scale to shave fi ve pounds off my more grievances, the lower the morale. This mea- weight just to make me feel better. Although you surement strategy would appear to be more reli- would (reliably) report the same weight for me able: Counting up the grievances over and over, each time, you would always be wrong. This new we should keep arriving at the same number. element, called bias, is discussed in Chapter 7. For now, just be warned that reliability does not ensure accuracy. reliability That quality of measurement method that Let’s suppose we’re interested in studying mo- suggests that the same data would have been collected rale among factory workers in two different kinds each time in repeated observations of the same phenomenon. In the context of a survey, we would expect that the of factories. In one set of factories, workers have question “Did you attend religious services last week?” specialized jobs, refl ecting an extreme division of would have higher reliability than the question “About labor. Each worker contributes a tiny part to the how many times have you attended religious services in overall process performed on a long assembly line. your life?” This is not to be confused with validity. 158 CHAPTER 5 CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT

If you’re thinking that the number of grievances So how do you create reliable measures? If your doesn’t necessarily measure morale, you’re wor- research design calls for asking people for infor- rying about validity, not reliability. We’ll discuss mation, you can be careful to ask only about things validity in a moment. The point for now is that the the respondents are likely to know the answer to. last method is more like my bathroom scale—it Ask about things relevant to them, and be clear in gives consistent results. what you’re asking. Of course, these techniques In social research, reliability problems crop up don’t solve every possible reliability problem. For- in many forms. Reliability is a concern every time tunately, social researchers have developed sev- a single observer is the source of data, because we eral techniques for cross-checking the reliability of have no certain guard against the impact of that the measures they devise. observer’s subjectivity. We can’t tell for sure how much of what’s reported originated in the situation Test-Retest Method Sometimes it’s appropriate observed and how much came from the observer. to make the same measurement more than once, a Subjectivity is a problem not only with single technique called the test-retest method. If you don’t observers, however. Survey researchers have expect the information being sought to change, known for a long time that different interviewers, then you should expect the same response both because of their own attitudes and demeanors, times. If answers vary, the measurement method get different answers from respondents. Or, if we may, to the extent of that variation, be unreliable. were to conduct a study of newspapers’ editorial Here’s an illustration. positions on some public issue, we might create a In their research on Health Hazard Appraisal team of coders to take on the job of reading hun- (HHA), a part of preventive medicine, Jeffrey Sacks, dreds of editorials and classifying them in terms of W. Mark Krushat, and Jeffrey Newman (1980) their position on the issue. Unfortunately, different wanted to determine the risks associated with coders will code the same editorial differently. Or various background and lifestyle factors, making we might want to classify a few hundred specifi c it possible for physicians to counsel their patients occupations in terms of some standard coding appropriately. By knowing patients’ life situations, scheme, say a set of categories created by the De- physicians could advise them on their potential for partment of Labor or by the Census Bureau. You survival and on how to improve it. This purpose, and I would not place all those occupations in the of course, depended heavily on the accuracy of same categories. the information gathered about each subject in the Each of these examples illustrates problems study. of reliability. Similar problems arise whenever we To test the reliability of their information, Sacks ask people to give us information about them- and his colleagues had all 207 subjects complete selves. Sometimes we ask questions that people a baseline questionnaire that asked about their don’t know the answers to: How many times, if characteristics and behavior. Three months later, any, have you been to religious services this year? a follow-up questionnaire asked the same subjects Sometimes we ask people about things they confor the same information, and the results of the sider totally irrelevant: Are you satisfi ed with Chi- two surveys were compared. Overall, only 15 per- na’s current relationship with Albania? In such cent of the subjects reported the same information cases, people will answer differently at different in both studies. times because they’re making up answers as they Sacks and his colleagues reported the following go. Sometimes we explore issues so complicated research fi ndings: that a person who had a clear opinion in the matter might arrive at a different interpretation of the Almost 10 percent of subjects reported a differ- question when asked a second time. ent height at follow-up examination. Parental CRITERIA OF MEASUREMENT QUALITY 159

age was changed by over one in three subjects. reliability in previous research. If you want to mea- One parent reportedly aged 20 chronologic sure anomia, for example, you might want to fol- years in three months. One in fi ve ex-smok- low Srole’s lead. ers and ex-drinkers have apparent diffi culty in The heavy use of measures, though, does not reliably recalling their previous consumption guarantee their reliability. For example, the Scho- pattern. — (1980:730) lastic Assessment Tests (SATs) and the Minnesota Multiphasic Personality Inventory (MMPI) have Some subjects had erased all traces of previ- been accepted as established standards in their ously reported heart murmurs, diabetes, emphy- respective domains for decades. In recent years, sema, arrest records, and thoughts of suicide. One though, they’ve needed fundamental overhauling subject’s mother, deceased in the fi rst question- to refl ect changes in society, eliminating outdated naire, was apparently alive and well in time for topics and gender bias in wording. the second. One subject had one ovary missing in the fi rst study but present in the second. In another case, an ovary present in the fi rst study was Reliability of Research Workers As we’ve seen, missing in the second study—and had been for ten measurement unreliability can also be generated years! One subject was reportedly 55 years old in by research workers: interviewers and coders, for the fi rst study and 50 years old three months later. example. There are several ways to check on reli- (You have to wonder whether the physician-coun- ability in such cases. To guard against interviewer selors could ever have the impact on their patients unreliability, it is common practice in surveys to that their patients’ memories had.) Thus, test-re- have a supervisor call a subsample of the respon- test revealed that this data-collection method was dents on the telephone and verify selected pieces not especially reliable. of information. Replication works in other situations also. If Split-Half Method As a general rule, it’s always you’re worried that newspaper editorials or oc- good to make more than one measurement of any cupations may not be classifi ed reliably, you could subtle or complex social concept, such as preju- have each independently coded by several coders. dice, alienation, or social class. This procedure Those cases that are classifi ed inconsistently can lays the groundwork for another check on reliabil- then be evaluated more carefully and resolved. ity. Let’s say you’ve created a questionnaire that Finally, clarity, specifi city, training, and practice contains ten items you believe measure prejudice can prevent a great deal of unreliability and grief. If against women. Using the split-half technique, you you and I spent some time reaching a clear agree- would randomly assign those ten items to two sets ment on how to evaluate editorial positions on an of fi ve. As we saw in the discussion of interchange- issue—discussing various positions and reading ability of indicators, each set should provide a good through several together—we could probably do a measure of prejudice against women, and the two good job of classifying them in the same way inde- sets should classify respondents the same way. If pendently. See the box “Pregnant Chads and Voter the two sets of items classify people differently, Intentions” for more on reliability. you most likely have a problem of reliability in your The reliability of measurements is a fundamen- measure of the variable. tal issue in social research, and we’ll return to it more than once in the chapters ahead. For now, however, let’s recall that even total reliability Using Established Measures Another way to doesn’t ensure that our measures measure what help ensure reliability in getting information from we think they measure. Now let’s plunge into the people is to use measures that have proven their question of validity. 160 CHAPTER 5 CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT

Validity IN THE REAL WORLD In conventional usage, validity refers to the extent PREGNANT CHADS AND to which an empirical measure adequately refl ects VOTER INTENTIONS the real meaning of the concept under consideration. Whoops! I’ve already committed us to the Replication in measurement is exactly the view that concepts don’t have real meanings. How issue that was raised in Florida during the can we ever say whether a particular measure 2000 Presidential election in which George adequately refl ects the concept’s meaning, then? W. Bush was declared the winner over Al Ultimately, of course, we can’t. At the same time, Gore. Specifi cally, given the thousands of as we’ve already seen, all of social life, including ballots rejected by the vote-counting ma- social research, operates on agreements about chines, many people questioned the reli- the terms we use and the concepts they repre- ability of this method of measuring votes. sent. There are several criteria of success in mak- Had the election been a survey of voting ing measurements that are appropriate to these intentions, the researchers would have re- agreed-on meanings of concepts. viewed the ballots rejected by the machine First, there’s something called face validity. and sought to make judgments regarding Particular empirical measures may or may not those intentions. Notice that decisions about jibe with our common agreements and our indi- hanging and pregnant “chads” would have vidual mental images concerning a particular con- concerned measurement procedures. Much cept. For example, you and I might quarrel about of the debate hinged on how much each type whether counting the number of grievances fi led of chad refl ected voter intent: What does a with the union is an adequate measure of worker bump in a chad “really” mean? Without morale. Still, we’d surely agree that the number of agreement on what really constituted a vote, grievances has something to do with morale. That researchers would have simply scored bal- is, the measure is valid “on its face,” whether or lots in terms of which candidate was appar- not it’s adequate. If I were to suggest that we mea- ently selected and why—that is, the basis on sure morale by fi nding out how many books the which the researcher’s decision was made workers took out of the library during their off-duty (e.g., “pregnant chad”). Of course, there hours, you’d undoubtedly raise a more serious ob- would fi rst have to be clear operational defi - jection: That measure wouldn’t have much face nitions of what “swinging,” “pregnant,” and validity. other sorts of chads were. It would certainly have been possible to determine the results in terms of the standards used—that is, how validity A term describing a measure that accurately many of each type of vote was counted for refl ects the concept it is intended to measure. For example, your IQ would seem a more valid measure of your each candidate. intelligence than would the number of hours you spend in the library. Though the ultimate validity of a measure can never be proven, we may agree to its relative validity on the basis of face validity, criterion-related validity, content Second, I’ve already pointed to many of the validity, construct validity, internal validation, and external more formally established agreements that defi ne validation. This must not be confused with reliability. some concepts. The Census Bureau, for example, That quality of an indicator that makes face validity has created operational defi nitions of such con- it seem a reasonable measure of some variable. That the frequency of attendance at religious services is some cepts as family, household, and employment sta- indication of a person’s religiosity seems to make sense tus that seem to have a workable validity in most without a lot of explanation. It has face validity. studies using these concepts. CRITERIA OF MEASUREMENT QUALITY 161

Three additional types of validity also specify stitutes evidence of your measure’s construct va- particular ways of testing the validity of measures. lidity. If satisfi ed marriage partners are as likely to The fi rst, criterion-related validity, sometimes cheat on their spouses as are the dissatisfi ed ones, called predictive validity, is based on some exter- however, that would challenge the validity of your nal criterion. For example, the validity of College measure. Board exams is shown in their ability to predict Tests of construct validity, then, can offer a students’ success in college. The validity of a writ- weight of evidence that your measure either does ten driver’s test is determined, in this sense, by the or does not tap the quality you want it to measure, relationship between the scores people get on the but this evidence is not defi nitive proof. Although test and their subsequent driving records. In these I have suggested that tests of construct validity examples, college success and driving ability are are less compelling than tests of criterion validity, the criteria. there is room for disagreement about which kind To test your understanding of criterion-related of test a particular comparison variable (driving validity, see whether you can think of behaviors record, marital fi delity) represents in a given situa- that might be used to validate each of the following tion. It’s less important to distinguish the two types attitudes: of validity tests than to understand the logic of validation that they have in common: If we have Is very religious been successful in measuring some variable, then Supports equality of men and women our measures should relate in some logical way to Supports far-right militia groups other measures. Is concerned about the environment Finally, content validity refers to how much Some possible validators would be, respectively, a measure covers the range of meanings included attends religious services, votes for women can- within a concept. For example, a test of math- didates, belongs to the NRA, and belongs to the ematical ability cannot be limited to addition but Sierra Club. also needs to cover subtraction, multiplication, Sometimes it’s diffi cult to fi nd behavioral criteria division, and so forth. Or, if we’re measuring preju- that can be taken to validate measures as directly dice, do our measurements refl ect all types of prej- as in such examples. In those instances, however, udice, including prejudice against racial and ethnic we can often approximate such criteria by applying groups, religious minorities, women, the elderly, a different test. We can consider how the variable and so on? in question ought, theoretically, to relate to other Figure 5-2 presents a graphic portrayal of the variables. Construct validity is based on the logi- difference between validity and reliability. If you cal relationships among variables. think of measurement as analogous to shooting Suppose, for example, that you want to study repeatedly at the bull’s-eye on a target, you’ll see the sources and consequences of marital satisfac- that reliability looks like a “tight pattern,” regard- tion. As part of your research, you develop a mea- less of where the shots hit, because reliability is sure of marital satisfaction, and you want to assess its validity. In addition to developing your measure, you’ll have developed certain theoretical expectations criterion-related validity The degree to which a measure relates to some external criterion. Also called about the way the variable marital satisfaction re- predictive validity. lates to other variables. For example, you might construct validity The degree to which a measure reasonably conclude that satisfi ed husbands and relates to other variables as expected within a system of wives will be less likely than dissatisfi ed ones to theoretical relationships. cheat on their spouses. If your measure relates to content validity The degree to which a measure cov- marital fi delity in the expected fashion, that con- ers the range of meanings included within a concept. 162 CHAPTER 5 CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT

Reliable but not valid Valid but not reliable Valid and reliable FIGURE 5-2 An Analogy to Validity and Reliability. A good measurement technique should be both valid (measuring what it is intended to measure) and reliable (yielding a given measurement dependably).

a function of consistency. Validity, on the other cally qualitative approach taken by many social hand, is a function of shots being arranged around scientists. We’ll explore this issue more deeply in the bull’s-eye. The failure of reliability in the ﬁ gure Chapter 10. is randomly distributed around the target; the fail- In seeking to understand the way ordinary peo- ure of validity is systematically off the mark. Notice ple make sense of their worlds, ethnomethodolo- that neither an unreliable nor an invalid measure gists have urged all social scientists to pay more is likely to be useful. respect to these natural social processes of conceptualization and shared meaning. At the very Who Decides What’s Valid? least, behavior that may seem irrational from the scientist’s paradigm may make logical sense when Our discussion of validity began with a reminder viewed from the actor’s point of view. that we depend on agreements to determine Clifford Geertz (1973) appropriates the term what’s real, and we’ve just seen some of the ways thick description in reference to the goal of un- social scientists can agree among themselves that derstanding, as deeply as possible, the meanings they have made valid measurements. There is yet that elements of a culture have for those who live another way of looking at validity. within that culture. He recognizes that the outside Social researchers sometimes criticize them- observer will never grasp those meanings fully, selves and one another for implicitly assuming however, and warns that “cultural analysis is inthey are somewhat superior to those they study. trinsically incomplete.” He then elaborates: For example, researchers often seek to uncover motivations that the social actors themselves are There are a number of ways to escape this— unaware of. You think you bought that new Burpo- turning culture into folklore and collecting it, Blasto because of its high performance and good turning it into traits and counting it, turning looks, but we know you’re really trying to achieve it into institutions and classifying it, turning it a higher social status. into structures and toying with it. But they are This implicit sense of superiority would ﬁ t com- escapes. The fact is that to commit oneself to fortably with a totally positivistic approach (the bi- a semiotic concept of culture and an interpre- ologist feels superior to the frog on the lab table), tive approach to the study of it is to commit but it clashes with the more humanistic and typi- oneself to a view of ethnographic assertion as, CRITERIA OF MEASUREMENT QUALITY 163

to borrow W. B. Gallie’s by now famous phrase, rale is much more than a lack of grievances fi led “essentially contestable.” Anthropology, or at with the union; anomia is much more than what least interpretive anthropology, is a science is measured by the fi ve items created by Leo Srole. whose progress is marked less by a perfection of Yet, the more variation and richness we allow for consensus than by a refi nement of debate. What a concept, the more potential for disagreement on gets better is the precision with which we vex how it applies to a particular situation, thus reduc- each other. — (1973:29) ing reliability. To some extent, this dilemma explains the per- Ultimately, social researchers should look to sistence of two quite different approaches to social both colleagues and subjects as sources of agree- research: quantitative, nomothetic, structured tech- ment on the most useful meanings and measure- niques such as surveys and experiments on the one ments of the concepts they study. Sometimes one hand and qualitative, idiographic methods such as source will be more useful, sometimes the other. fi eld research and historical studies on the other. But neither should be dismissed. In the simplest generalization, the former methods tend to be more reliable, the latter more valid. Tension between Reliability and Validity WHAT DO YOU THINK? REVISITED Clearly, we want our measures to be both reliable and valid. Often, however, a tension arises between the criteria of reliability and validity, forcing Can social scientists measure religiosity and a trade-off between the two. determine its causes? As you’ve seen, mak- Recall the example of measuring morale in dif- ing descriptive statements about religiosity ferent factories. The strategy of immersing your- is harder than making explanatory ones. Any self in the day-to-day routine of the assembly line, assertion about who is or is not religious de- observing what goes on, and talking to the work- pends directly on the defi nitions used. By one ers would seem to provide a more valid measure defi nition, nearly all of the population would of morale than would counting grievances. It just be deemed religious; by another defi nition, seems obvious that we’d get a clearer sense of few would be so designated. whether the morale was high or low using this fi rst As the discussion of the interchangeabil- method. ity of indicators suggested, however, we can As I pointed out earlier, however, the counting be more confi dent and defi nitive in speaking strategy would be more reliable. This situation re- about the causes of religiosity. For example, fl ects a more general strain in research measure- it is often reported that U.S. women are ment. Most of the really interesting concepts we more religious than U.S. men. This asser- want to study have many subtle nuances, and it’s tion is based on the observation that women hard to specify precisely what we mean by them. are more religious than men on virtually all Researchers sometimes speak of such concepts as indicators of religiosity: attendance at reli- having a “richness of meaning.” Although scores of gious services, prayer, beliefs, and so forth. books and articles have been written on anomie/ So, even though we may likely disagree on anomia, for example, they still haven’t exhausted which of these indicators is the best or truest its meaning. measure of what we mean by the term reli- Very often, then, specifying reliable operational giosity, women would be more religious than defi nitions and measurements seems to rob con- men regardless of the indicator chosen. cepts of their richness of meaning. Positive mo- 164 CHAPTER 5 CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT

By being forewarned, you’ll be effectively fore- ized one side of the debate as “pro-choice” or as armed against this persistent and inevitable di- “pro-abortion.” If your personal bias made you lemma. If there is no clear agreement on how to want to minimize support for the former position, measure a concept, measure it several different you might be tempted to frame the concept and ways. If the concept has several dimensions, mea- the measurements based on it in terms of people sure each. Above all, know that the concept does being “pro-abortion,” thereby eliminating all those not have any meaning other than what you and who were not especially fond of abortion per se I give it. The only justiﬁ cation for giving any con- but felt a woman should have the right to make cept a particular meaning is utility. Measure con- that choice for herself. To pursue this strategy, cepts in ways that help us understand the world however, would violate accepted research ethics. around us. Consider the choices available to you in con- ceptualizing attitudes toward the American inva- THE ETHICS OF MEASUREMENT sion of Iraq in 2003. Imagine the different levels of support you would “discover” if you framed the position (1) as an unprovoked invasion of a sovereign Measurement decisions can sometimes be judged nation, (2) as a retaliation for the 9/11 attack on the by ethical standards. We have seen that most of World Trade Center, (3) as a defensive act against a the concepts of interest to social researchers are perceived threat, (4) as part of a global war on ter- open to varied meanings. Suppose, for example, rorism, and so forth. There is no one, correct way that you’re interested in sampling public opinion to conceptualize this issue, but it would be unethi- on the abortion issue in the United States. Notice cal to seek to slant the results deliberately through the difference it would make if you conceptual- a biased deﬁ nition of the issue.

Main Points can’t be measured directly, but we can measure the things that our concepts summarize. Introduction ❏ The interrelated processes of conceptualiza- Conceptualization tion, operationalization, and measurement ❏ Conceptualization is the process of specify- allow researchers to move from a general ing observations and measurements that give idea about what they want to study to effective concepts defi nite meaning for the purposes of and well-defi ned measurements in the a research study. real world. ❏ Conceptualization includes specifying the indicators of a concept and describing its Measuring Anything That Exists dimensions. Operational defi nitions specify ❏ Conceptions are mental images we use as how variables relevant to a concept will be summary devices for bringing together ob- measured. servations and experiences that seem to have something in common. We use terms or labels Defi nitions in Descriptive to reference these conceptions. and Explanatory Studies ❏ ❏ Concepts are constructs; they represent the Precise defi nitions are even more important agreed-on meanings we assign to terms. Our in descriptive than in explanatory studies. The concepts don’t exist in the real world, so they degree of precision needed varies with the type and purpose of a study. REVIEW QUESTIONS 165

Operationalization Choices ❏ The yardsticks for assessing a measure’s valid- ❏ Operationalization is an extension of concep- ity include face validity, criterion-related valid- tualization that specifi es the exact procedures ity, construct validity, and content validity. that will be used to measure the attributes of ❏ Creating specifi c, reliable measures often variables. seems to diminish the richness of meaning ❏ Operationalization involves a series of inter- our general concepts have. This problem is related choices: specifying the range of varia- inevitable. The best solution is to use several tion that is appropriate for the purposes of a different measures, tapping the various aspects study, determining how precisely to measure of a concept. variables, accounting for relevant dimensions of variables, clearly defi ning the attributes of The Ethics of Measurement variables and their relationships, and deciding ❏ Conceptualization and measurement must not on an appropriate level of measurement. be guided by bias or preferences for particular ❏ Researchers must choose from four types of research outcomes. measures that capture increasing amounts of information: nominal, ordinal, interval, and Key Terms ratio. The most appropriate level depends on the purpose of the measurement. conceptualization interval measures ❏ A given variable can sometimes be measured construct validity nominal measures at different levels. When in doubt, researchers content validity ordinal measure should use the highest level of measurement criterion-related validity ratio measure appropriate to that variable so they can cap- dimension reliability face validity validity ture the greatest amount of information. indicator ❏ Operationalization begins in the design phase of a study and continues through all phases of the research project, including the analysis Review Questions of data. 1. Pick a social science concept such as liberalism or alienation, then specify that concept so that it could be studied in a research Criteria of Measurement Quality project. How would you specify the indica- ❏ Criteria of the quality of measures include pre- tors you would use as well as the dimensions cision, accuracy, reliability, and validity. you wish to include in and exclude from your ❏ Whereas reliability means getting consistent conceptualization? results from the same measure, validity refers 2. If you wanted to compare two societies on to getting results that accurately refl ect the anomie, or normlessness, what indicators concept being measured. might you look at? For example, what sta- ❏ Researchers can test or improve the reliability tistical indicators might you examine? What of measures through the test-retest method, nonstatistical, qualitative indicators? the split-half method, the use of established 3. What level of measurement—nominal, ordi- measures, and the examination of work per- nal, interval, or ratio—describes each of the formed by research workers. following variables? 166 CHAPTER 5 CONCEPTUALIZATION, OPERATIONALIZATION, AND MEASUREMENT

a. Race (white, African American, Asian, and net Exercises, Flashcards, and Chapter Tutorials, so on) as well as Extended Projects, InfoTrac College b. Order of fi nish in a race (fi rst, second, Edition search terms, Social Research in Cyber- third, and so on) space, GSS Data, Web Links, and primers for using c. Number of children in families various data analysis software such as SPSS and d. Populations of nations NVivo. e. Attitudes toward nuclear energy (strongly approve, approve, disapprove, strongly disapprove) Additional Readings f. Region of birth (Northeast, Midwest, and so on) Bohrnstedt, George W. 1983. “Measurement.” Pp. 70–121 g. Political orientation (very liberal, some- in Handbook of Survey Research, edited by Peter H. what liberal, somewhat conservative, very Rossi, James D. Wright, and Andy B. Anderson. New conservative) York: Academic Press. This essay offers the logical 4. Suppose you’ve been contracted by Wad- and statistical grounding of reliability and validity in sworth Publishing Company to interview a measurement. Carmines, Edward G., and Richard A. Zeller. 1979. group of students and evaluate their level of Reliability and Validity Assessment. Thousand Oaks, satisfaction with this textbook. How would CA: Sage. Carmines and Zeller explore validity and you measure satisfaction in this case? reliability in detail and examine some of the ways to calculate reliability mathematically. Online Study Resources Gould, Julius, and William Kolb. 1964. A Dictionary of the Social Sciences. New York: Free Press. A primary reference to the social scientifi c agreements on various concepts. Although the terms used by social scientists do not have ultimately “true” meanings, this reference book lays out the meanings social scientists Go to have in mind when they use those terms. http://sociology.wadsworth.com/babbie_basics4e Grimes, Michael D. 1991. Class in Twentieth-Century American Sociology: An Analysis of Theories and and click on ThomsonNow for access to this Measurement Strategies. New York: Praeger. This book powerful online study tool. You will get a per- provides an excellent long-term view of conceptual- sonalized study plan based on your responses to ization as the author examines a variety of theo- a diagnostic pretest. Once you have mastered retical views of social class and the measurement the material with the help of interactive learning techniques appropriate to those theories. tools, you can take a posttest to confi rm that you Lazarsfeld, Paul F., and Morris Rosenberg, eds. 1955. The Language of Social Research, Section I. New are ready to move on to the next chapter. York: Free Press. An excellent and diverse classic collection of descriptions of specifi c measurements Website for in past social research. These 14 articles present The Basics of Social Research useful and readable accounts of actual measurement 4th edition operations performed by social researchers as well as more conceptual discussions of measurement At the book companion website (http://sociology in general. .wadsworth.com/babbie_basics4e) you will fi nd Silverman, David. 1993. Interpreting Qualitative Data: many resources in addition to ThomsonNow to Methods for Analyzing Talk, Text, and Interaction, aid you in studying for your exams. For example, Chapter 7. Thousand Oaks, CA: Sage. This deals with you will fi nd Tutorial Quizzes with feedback, Inter- ADDITIONAL READINGS 167

the issues of validity and reliability speciﬁ cally in Wallace, Walter. 1971. The Logic of Science in Sociol- regard to qualitative research. ogy, Chapter 3. Chicago: Aldine-Atherton. A brief U.S. Department of Health and Human Services. 1992. and lucid presentation of concept formation within Survey Measurement of Drug Use. Washington, DC: the context of other research steps. This discussion U.S. Government Printing Ofﬁ ce. An extensive review relates conceptualization to observation on the one of techniques devised and used for measuring vari- hand and to generalization on the other. ous kinds of drug use. 61 HUMANINDEXES, INQUIRY SCALES, ANDAND SCIENCETYPOLOGIES Photo credit Stock Solution/Index Imagery

What You’ll Learn in thisThis ChapterChapter

NowWe’ll we examineconclude the the way discussion people learnof measurement about their begunworld andin Chapter the mistakes 5. Researchers they makeoften needalong to the employ way. We’llmultiple also indicators begin to see to measure what makes a variable sience adequately different from and

othervalidly. ways Indexes, of knowing scales, things.and typologies are useful composite measures made up 2S of several indicators of variables. In this chapter you’ll learn the logic and skills of 1S N constructing such measures. L 168 In this chapter . . . WHAT DO YOU THINK?

Introduction Indexes versus Scales Often, data analysis aims at reducing Index Construction a mass of obser-

Item Selection Image not available due to copyright restrictions vations to a more Examination of Empirical Relationships manageable form. Index Scoring Our use of concepts Handling Missing Data to stand for many Index Validation similar observations The Status of Women: An Illustration is one example. The trick is to have the re- of Index Construction duction represent the original observations well enough to be accurate and useful. Scale Construction Sometimes this sort of reduction can be Bogardus Social Distance Scale accomplished in the analysis of quantita- Thurstone Scales tive data. You could, for example, ask people Likert Scaling to answer fi ve different questions, reduce Semantic Differential each person’s answers to a single number, Guttman Scaling and then use that number to reproduce that Typologies person’s answers. So, if you told me that you had assigned someone a score of 3, I would be able to tell you how he or she answered INTRODUCTION each of the original fi ve questions. How in the world could such a little bit of information communicate so much? As we saw in Chapter 5, many social scientifi c concepts have complex and varied meanings. Making See the “What Do You Think? Revisited” box measurements that capture such concepts can be toward the end of the chapter. a challenge. Recall our discussion of content validity, which concerns whether we have captured all the different dimensions of a concept. To achieve broad coverage of the various di- Multiple indicators are used with quantitative mensions of a concept, we usually need to make data as well. Though you can sometimes construct multiple observations pertaining to it. Thus, for a single questionnaire item that captures the vari- example, Bruce Berg (1989:21) advises in-depth able of interest—“Sex: ❑ Male ❑ Female” is a interviewers to prepare “essential questions,” simple example—other variables are less straight- which are “geared toward eliciting specifi c, desired forward and may require you to use several ques- information.” In addition, the researcher should tionnaire items to measure them adequately. prepare extra questions: “questions roughly equiv- Quantitative data analysts have developed alent to certain essential ones, but worded slightly specifi c techniques for combining indicators into differently.” a single measure. This chapter discusses the con- 169 170 CHAPTER 6 INDEXES, SCALES, AND TYPOLOGIES struction of two types of composite measures of may give us some indication of the political ori- variables—indexes and scales. Although scales entations of that newspaper. Examining several and indexes can be used in any form of social re- editorials would probably give us a better assess- search, they are most common in survey research ment, but the manipulation of several data items and other quantitative methods. A short section at simultaneously could be very complicated. Indexes the end of the chapter considers typologies, which and scales (especially scales) are effi cient data- are relevant to both qualitative and quantitative reduction devices: They allow us to summarize research. several indicators in a single numerical score, Composite measures are frequently used in while sometimes nearly maintaining the specifi c quantitative research, for several reasons. First, details of all the individual indicators. social scientists often wish to study variables that have no clear and unambiguous single indicators. INDEXES VERSUS SCALES Single indicators do suffi ce for some variables, such as age. We can determine a survey respondent’s age simply by asking, “How old are you?” The terms index and scale are commonly used im- Similarly, we can determine a newspaper’s circula- precisely and interchangeably in social research tion merely by looking at the fi gure the newspaper literature. These two types of measures do have reports. In the case of complex concepts, however, some characteristics in common, but in this book researchers can seldom develop single indicators we’ll distinguish between them. However, you before they actually do the research. This is es- should be warned of a growing tendency in the pecially true with regard to attitudes and orienta- literature to use the term scale to refer to both in- tions. Rarely can a survey researcher, for example, dexes and scales, as they are distinguished here. devise single questionnaire items that adequately First, let’s consider what they have in common. tap respondents’ degrees of prejudice, religiosity, Both scales and indexes are ordinal measures of political orientations, alienation, and the like. More variables. Both rank-order the units of analysis in likely, the researcher will devise several items, each terms of specifi c variables such as religiosity, alien- of which provides some indication of the variables. ation, socioeconomic status, prejudice, or intellectual Taken individually, each of these items is likely to sophistication. A person’s score on either a scale or prove invalid or unreliable for many respondents. an index of religiosity, for example, indicates his or A composite measure, however, can overcome her relative religiosity vis-à-vis other people. this problem. Further, both scales and indexes are compos- Second, researchers may wish to employ a ite measures of variables: measurements based rather refi ned ordinal measure of a particular vari- on more than one data item. Thus, a survey re- able—alienation, say—arranging cases in several spondent’s score on an index or scale of religios- ordinal categories from very low to very high, ity is determined by the responses given to several for example. A single data item might not have questionnaire items, each of which provides some enough categories to provide the desired range of indication of religiosity. Similarly, a person’s IQ variation. However, an index or scale formed from score is based on answers to a large number of several items can provide the needed range. test questions. The political orientation of a news- Finally, indexes and scales are effi cient devices paper might be represented by an index or scale for data analysis. If considering a single data item score refl ecting the newspaper’s editorial policy on gives us only a rough indication of a given vari- various political issues. able, considering several data items can give us Despite these shared characteristics, distin- a more comprehensive and more accurate indi- guishing between indexes and scales is useful. In cation. For example, a single newspaper editorial this book we’ll do so through the manner in which INDEXES VERSUS SCALES 171 scores are assigned. We construct an index sim- pate much at all, and those who are somewhere ply by accumulating scores assigned to individual in between. indicators. We might measure prejudice, for exam- The fi rst part of Figure 6-1 illustrates the logic of ple, by counting the number of prejudiced state- indexes. The fi gure shows six different political ac- ments each respondent agreed with. We construct tions. Although you and I might disagree on some a scale, however, by assigning scores to patterns specifi cs, I think we could agree that the six ac- of responses, recognizing that some items refl ect tions represent roughly the same degree of politi- a relatively weak degree of the variable whereas cal activism. others refl ect something stronger. For example, Using these six items, we could construct an agreeing that “Women are different from men” is, index of political activism by giving each person 1 at best, weak evidence of sexism compared with point for each of the actions he or she has taken. If agreeing that “Women should not be allowed to you wrote to a public offi cial and signed a petition, vote.” A scale takes advantage of differences in in- you’d get a total of 2 points. If I gave money to a tensity among the attributes of the same variable candidate and persuaded someone to change his to identify distinct patterns of response. or her vote, I’d get the same score as you. Using Let’s consider this simple example of sexism this approach, we’d conclude that you and I had a bit further. Imagine asking people to agree or the same degree of political activism, even though disagree with the two statements just presented. we had taken different actions. Some might agree with both, some might disagree The second part of Figure 6-1 describes the with both. But suppose I told you someone agreed logic of scale construction. In this case, the actions with one and disagree with the other: Could you clearly represent different degrees of political ac- guess which statement they agreed with and which tivism—ranging from simply voting to running for they did not? I would guess the person in question offi ce. Moreover, it seems safe to assume a pattern agreed that women were different but disagreed of actions in this case. For example, all those who that they should be prohibited from voting. On the contributed money probably also voted. Those other hand, I doubt that anyone would want to who worked on a campaign probably also gave prohibit women from voting and assert that there some money and voted. This suggests that most is no difference between men and women. That people will fall into only one of fi ve idealized action would make no sense. patterns, represented by the number under each Now consider this. The two responses we set of boxes in the fi gure. The discussion of scales, wanted from each person would technically later in this chapter, describes ways of identifying yield four response patterns: agree/agree, agree/ people with the type they most closely represent. disagree, disagree/agree, and disagree/disagree. As you might surmise, scales are generally su- We have just seen, however, that only three of the perior to indexes, because scales take into con- four patterns make any sense or are likely to oc- sideration the intensity with which different items cur. Where indexes score people based on their refl ect the variable being measured. Also, as the responses, scales score people on the basis of re- example in Figure 6-1 shows, scale scores convey sponse patterns: We determine what the logical response patterns are and score people in terms of the pattern their responses most closely resemble. index A type of composite measure that summarizes and rank-orders several specifi c observations and repre- Figure 6-1 provides a graphic illustration of the sents some more general dimension. difference between indexes and scales. Let’s as- scale A type of composite measure composed of several sume we want to develop a measure of political items that have a logical or empirical structure among activism, distinguishing those people who are very them. Examples of scales include Bogardus social distance, active in political affairs, those who don’t partici- Guttman, Likert, and Thurstone scales. 172 CHAPTER 6 INDEXES, SCALES, AND TYPOLOGIES

Index-Construction Logic Here are several types of political actions people Wrote a letter Signed a Gave money may have taken. By and large, the different actions to a political to a represent similar degrees of political activism. public official petition political cause To create an index of overall political activism, we might give people 1 point for each of the actions they’ve taken. Persuaded Gave money Wrote a someone to to a political political letter change her or candidate to the editor his voting plans

Scale-Construction Logic Here are some political actions that represent very different degrees of activism: e.g., running for office represents a higher degree of activism than does simply voting. It seems likely, moreover, that anyone who has taken one of the more demanding actions would have taken all the easier ones as well. To construct a scale of political activism, we might score people according to which of the following “ideal” patterns comes closest to describing them.

Ran for office No No No No Yes

Worked on a No No No Yes Yes political campaign

Contributed money to No No Yes Yes Yes a political campaign

Voted No Yes Yes Yes Yes

0 1 2 3 4 FIGURE 6-1 Indexes versus Scales. Both indexes and scales seek to measure variables such as political activism. Whereas indexes count the number of indicators of the variable, scales take account of the differing intensities of those indicators.

more information than do index scores. Again, be items is a scale simply because it has turned out aware that the term scale is commonly misused that way in an earlier study. to refer to measures that are only indexes. Merely Second, the use of speciﬁ c scaling techniques— calling a given measure a scale instead of an index such as Guttman scaling, to be discussed—does doesn’t make it better. not ensure the creation of a scale. Rather, such There are two other misconceptions about techniques let us determine whether or not a set of scaling that you should know. First, whether the items constitutes a scale. combination of several data items results in a scale An examination of actual social science re- almost always depends on the particular sample of search reports will show that researchers use in- observations under study. Certain items may form dexes much more than scales. Ironically, however, a scale within one sample but not within another. the methodological literature contains little if any For this reason, do not assume that a given set of discussion of index construction, whereas discus- INDEX CONSTRUCTION 173

Item Selection

The ﬁ rst step in creating an index is selecting items for a composite index, which is created to measure some variable.

Face Validity The fi rst criterion for selecting items to be included in an index is face validity (or logical validity). If you want to measure political conservatism, for example, each of your items should appear on its face to indicate conservatism (or its opposite, liberalism). Political party affi liation would be one such item. Another would be an Earl Babbie Composite measures involve the combination of item asking people to approve or disapprove of the elements to create something new. Sometimes it views of a well-known conservative public fi gure. works, sometimes it doesn’t. In constructing an index of religiosity, you might consider items such as attendance of religious services, acceptance of certain religious beliefs, and sions of scale construction abound. There appear frequency of prayer; each of these appears to offer to be two reasons for this disparity. First, indexes some indication of religiosity. are more frequently used because scales are often diffi cult or impossible to construct from the data Unidimensionality The methodological litera- at hand. Second, methods of index construction ture on conceptualization and measurement seem so obvious and straightforward that they stresses the need for unidimensionality in scale and aren’t discussed much. index construction; that is, a composite measure Constructing indexes is not simple, however. should represent only one dimension of a con- The general failure to develop index-construction cept. Thus, items refl ecting religiosity should not techniques has resulted in many bad indexes in so- be included in a measure of political conservatism, cial research. With this in mind, I’ve devoted over even though the two variables might be empiri- half of this chapter to the methods of index con- cally related. struction. With a solid understanding of the logic of this activity, you’ll be better equipped to try con- General or Specifi c Although measures should structing scales. Indeed, a carefully constructed in- tap the same dimension, the general dimension dex may turn out to be a scale. you’re attempting to measure may have many nuances. In the example of religiosity, the indicators INDEX CONSTRUCTION mentioned previously—ritual participation, belief, and so on—represent different types of religiosity. If you want to focus on ritual participation in reli- Let’s look now at four main steps in the construc- gion, you should choose items specifi cally indicat- tion of an index: selecting possible items, examining this type of religiosity: attendance at religious ing their empirical relationships, scoring the index, services and other rituals such as confession, bar and validating it. We’ll conclude this discussion mitzvah, bowing toward Mecca, and the like. If by examining the construction of an index that you want to measure religiosity in a more general provided interesting fi ndings about the status of way, you should include a balanced set of items, women in different countries. representing each of the different types of religios- 174 CHAPTER 6 INDEXES, SCALES, AND TYPOLOGIES ity. Ultimately, the nature of the items included will Bivariate Relationships among Items A bivariate determine how specifi cally or generally the vari- relationship is, simply put, a relationship between able is measured. two variables. Suppose we want to measure respondents’ support for U.S. participation in the Variance In selecting items for an index, you must United Nations. One indicator of different levels of also be concerned with the amount of variance support might be the question “Do you feel the U.S. they provide. If an item is intended to indicate po- fi nancial support of the UN is ❑ Too high ❑ About litical conservatism, for example, you should note right ❑ Too low?” what proportion of respondents would be identi- A second indicator of support for the United fi ed as conservatives by the item. If a given item Nations might be the question “Should the identifi ed no one as a conservative or everyone as United States contribute military personnel to a conservative—for example, if nobody indicated UN peacekeeping actions? ❑ Strongly approve ❑ approval of a radical-right political fi gure—that Mostly approve ❑ Mostly disapprove ❑ Strongly item would not be very useful in the construction disapprove.” of an index. Both of these questions, on their face, seem to To guarantee variance, you have two options. refl ect different degrees of support for the United First, you can select several items that generate re- Nations. Nonetheless, some people might feel the sponses that divide people about equally in terms United States should give more money but not proof the variable; for example, about half conserva- vide troops. Others might favor sending troops but tive and half liberal. Although no single response cutting back on fi nancial support. would justify characterizing a person as very con- If the two items both refl ect degrees of the same servative, a person who responded as a conserva- thing, however, we should expect responses to the tive on all items might be so characterized. two items to generally correspond with one an- The second option is to select items differing in other. Specifi cally, those who approve of military variance. One item might identify about half the support should be more likely to favor fi nancial subjects as conservative, while another might iden- support than would those who disapprove of mili- tify few of the respondents as conservatives. Note tary support. Conversely, those who favor fi nan- that this second option is necessary for scaling, cial support should be more likely to favor military and it is reasonable for index construction as well. support than would those disapproving of fi nancial support. If these expectations are met, we say Examination of there is a bivariate relationship between the two Empirical Relationships items. Here’s another example. Suppose we want to The second step in index construction is to exam- determine the degree to which respondents feel ine the empirical relationships among the items women have the right to an abortion. We might being considered for inclusion. (See Chapter 14 ask (1) “Do you feel a woman should have the right for more.) An empirical relationship is established to an abortion when her pregnancy was the result when respondents’ answers to one question—in a of rape?” and (2) “Do you feel a woman should questionnaire, for example—help us predict how have the right to an abortion if continuing her they will answer other questions. If two items are pregnancy would seriously threaten her life?” empirically related to each other, we can reason- Some respondents might agree with item (1) ably argue that each refl ects the same variable, and and disagree with item (2); others will do just the we can include them both in the same index. There reverse. If both items tap into some general opin- are two types of possible relationships among ion people have about the issue of abortion, then items: bivariate and multivariate. the responses to these two items should be related INDEX CONSTRUCTION 175

Text not available due to copyright restrictions

to each other. Those who support the right to an among the several items being considered for in- abortion in the case of rape should be more likely clusion in an index. Percentage tables or more ad- to support it if the woman’s life is threatened than vanced statistical techniques may be used for this would those who disapproved of abortion in the purpose. How we evaluate the strength of the rela- case of rape. This would be another example of a tionships, however, can be rather subtle. The box bivariate relationship between the two items. “‘Cause’ and ‘Effect’ Indicators” examines some of To determine the relative strengths of relation- these subtleties. ships among the several pairs of items, you should Be wary of items that are not related to one examine all the possible bivariate relationships another empirically: It’s unlikely they measure the 176 CHAPTER 6 INDEXES, SCALES, AND TYPOLOGIES same variable. You should probably drop any item For each of these items, we might conclude that that is not related to several other items. those respondents who chose the second answer At the same time, a very strong relationship are more scientifi cally oriented than respondents between two items presents a different problem. who chose the fi rst answer. Though this compara- If two items are perfectly related to one another, tive conclusion is reasonable, we should not be then only one needs to be included in the index; misled into thinking that respondents who chose because it completely conveys the indications pro- the second answer to a given item are scientists vided by the other, nothing more would be added in any absolute sense. They are simply more sci- by including the other item. (This problem will be- entifi cally oriented than those who chose the fi rst come even clearer in a later section.) answer to the item. Here’s an example to illustrate the testing of To see this point more clearly, let’s examine the bivariate relationships in index construction. I distribution of responses to each item. From the once conducted a survey of medical school faculty fi rst item—greatest teaching contribution—only members to fi nd out about the consequences of about one-third of the respondents appeared sci- a “scientifi c perspective” on the quality of patient entifi cally oriented. That is, a little over one-third care provided by physicians. The primary intent said they could make their greatest teaching con- was to determine whether scientifi cally inclined tribution as medical researchers. In response to doctors treated patients more impersonally than the second item—ultimate medical interests—ap- did other doctors. proximately two-thirds chose the scientifi c answer, The survey questionnaire offered several pos- saying they were more interested in learning about sible indicators of respondents’ scientifi c perspec- basic mechanisms than learning about total pa- tives. Of those, three items appeared to provide tient management. In response to the third item— especially clear indications of whether the doctors reading preferences—about 80 percent chose the were scientifi cally oriented: scientifi c answer. These three questionnaire items can’t tell us 1. As a medical school faculty member, in what how many “scientists” there are in the sample, be- capacity do you feel you can make your great- cause none of them is related to a set of criteria est teaching contribution: as a practicing physi- for what constitutes being a scientist in any abso- cian or as a medical researcher? lute sense. Using the items for this purpose would 2. As you continue to advance your own medical present us with the problem of three quite different knowledge, would you say your ultimate medi- estimates of how many scientists there were in the cal interests lie primarily in the direction of sample. total patient management or the understand- However, these items do provide us with three ing of basic mechanisms? [The purpose of this independent indicators of respondents’ relative in- item was to distinguish those who were mostly clinations toward science in medicine. Each item interested in overall patient care from those separates respondents into the more scientifi c and mostly interested in biological processes.] the less scientifi c. But each grouping of more or 3. In the fi eld of therapeutic research, are you less scientifi c respondents will have a somewhat generally more interested in articles report- different membership from the others. Respon- ing evaluations of the effectiveness of vari- dents who seem scientifi c in terms of one item ous treatments or articles exploring the basic will not seem scientifi c in terms of another. Nev- rationale underlying the treatments? [Similarly, ertheless, to the extent that each item measures I wanted to distinguish those more interested the same general dimension, we should fi nd some in articles dealing with patient care from correspondence among the several groupings. Re- those more interested in biological processes.] spondents who appear scientifi c in terms of one — (BABBIE 1970:27–31) item should be more likely to appear scientifi c in INDEX CONSTRUCTION 177 their response to another item than do those who a. Greatest Teaching Contribution appear nonscientifi c in their response to the fi rst. Physician Researcher In other words, we should fi nd an association or correlation between the responses given to two Total patient management 49% 13% items. Figure 6-2 shows the associations among the responses to the three items. Three bivariate ta- Ultimate Basic 51% 87% mechanisms bles are presented, showing the distribution of re- Medical Interest sponses for each pair of items. An examination of 100% 100% the three bivariate relationships presented in the (268) (159) fi gure supports the suggestion that the three items all measure the same variable: scientifi c orienta- b. Reading Preferences tion. To see why this is so, let’s begin by looking Effectiveness Rationale at the fi rst bivariate relationship in the table. The table shows that faculty who responded that “re- Total patient 68% 30% searcher” was the role in which they could make management their greatest teaching contribution were more

likely to identify their ultimate medical interests as Ultimate Basic 32% 70% “basic mechanisms” (87 percent) than were those mechanisms Medical Interest who answered “physician” (51 percent). The fact 100% 100% that the “physicians” are about evenly split in their (78) (349) ultimate medical interests is irrelevant for our purposes. It is only relevant that they are less scientiﬁ c c. in their medical interests than are the “research- Reading Preferences ers.” The strength of this relationship can be sum- Effectiveness Rationale marized as a 36 percentage-point difference. The same general conclusion applies to the Physician 85% 64% other bivariate relationships. The strength of the relationship between reading preferences and ul- Researcher 15% 36% timate medical interests can be summarized as a Contribution

38 percentage-point difference, and the strength of Greatest Teaching the relationship between reading preferences and 100% 100% (78) (349) the two teaching contributions as a 21 percentage- point difference. In summary, then, each single FIGURE 6-2 Bivariate Relationships among Scientifi c Orientation Items. If several indica- item produces a different grouping of “scientifi c” tors are measures of the same variable, then they and “nonscientifi c” respondents. However, the re- should be empirically correlated with one another. sponses given to each of the items correspond, to a greater or lesser degree, to the responses given to each of the other items. suffi ciently justify including the items in a compos- Initially, the three items were selected on the ite index. Before combining them in a single index, basis of face validity—each appeared to give some we need to examine the multivariate relationships indication of faculty members’ orientations to sci- among the several variables. ence. By examining the bivariate relationship between the pairs of items, we have found support Multivariate Relationships among Items for the expectation that they all measure basically Whereas a bivariate relationship deals with two the same thing. However, that support does not variables at a time, a multivariate one uses more 178 CHAPTER 6 INDEXES, SCALES, AND TYPOLOGIES

Percent Interested in Basic Mechanisms The respondents who gave mixed responses in Greatest Teaching Contribution terms of teaching contributions and reading preferences have an intermediate rank in their concern Physician Researcher for basic mechanisms (58 percent in both cases). Effectiveness 27% 58% This ﬁ gure tells us many things. First, we may of treatments (66) (12) note that the original relationships between pairs of items are not signiﬁ cantly affected by the pres- Rationale

Reading 58% 89% behind ence of a third item. Recall, for example, that the Preferences treatments (219) (130) relationship between teaching contribution and ultimate medical interest was summarized as a 36 FIGURE 6-3 Trivariate Relationships among percentage-point difference. Looking at Figure 6-3, Scientifi c Orientation Items. Indicators of the we see that among only those respondents who same variable should be correlated in a multivariate analysis as well as in bivariate analyses. are most interested in articles dealing with the effectiveness of treatments, the relationship between teaching contribution and ultimate medical inter- than two variables. To present the trivariate re- est is 31 percentage points (58 percent minus 27 lationships among the three variables in our ex- percent: fi rst row). The same is true among those ample, we would fi rst categorize the sample medi- most interested in articles dealing with the ratio- cal school respondents into four groups according nale for treatments (89 percent minus 58 percent: to (1) their greatest teaching contribution and (2) second row). The original relationship between their reading preferences. Figure 6-3 does just that. teaching contribution and ultimate medical inter- The numbers in parentheses indicate the number est is essentially the same as in Figure 6-2, even of respondents in each group. Thus, 66 of the fac- among those respondents judged as scientifi c or ulty members who said they could best teach as nonscientifi c in terms of reading preferences. physicians also said they preferred articles deal- We can draw the same conclusion from the ing with the effectiveness of treatments. Then, for columns in Figure 6-3. Recall that the original re- each of the four groups, we would determine the lationship between reading preferences and ulti- percentage of those who say they are ultimately mate medical interests was summarized as a 38 more interested in basic mechanisms. So, for ex- percentage-point difference. Looking only at the ample, of the 66 faculty mentioned, 27 percent are “physicians” in Figure 6-3, we see that the rela- primarily interested in basic mechanisms, as the tionship between the other two items is now 31 fi gure shows. percentage points. The same relationship is found The arrangement of the four groups is based on among the “researchers” in the second column. a previously drawn conclusion regarding scientifi c The importance of these observations becomes orientations. The group in the upper left corner of clearer when we consider what might have hap- the table is presumably the least scientifi cally ori- pened. In Figure 6-4, hypothetical data tell a much ented, based on greatest teaching contribution and different story than do the actual data in Figure 6-3. reading preference. The group in the lower right As you can see, Figure 6-4 shows that the original corner is presumably the most scientifi cally ori- relationship between teaching contribution and ul- ented in terms of those items. timate medical interest persists, even when read- Recall that expressing a primary interest in ba- ing preferences are introduced into the picture. In sic mechanisms was also taken as an indication of each row of the table, the “researchers” are more scientifi c orientation. As we should expect, then, likely to express an interest in basic mechanisms those in the lower right corner are the most likely than are the “physicians.” Looking down the col- to give this response (89 percent), and those in the umns, however, we note that there is no relation- upper left corner are the least likely (27 percent). ship between reading preferences and ultimate INDEX CONSTRUCTION 179

Percent Interested in Basic Mechanisms from “very conservative” to “not at all conserva- Greatest Teaching Contribution tive” (or “very liberal”). How far to the extremes, then, should the index extend? Physician Researcher In this decision, the question of variance en- Effectiveness 51% 87% ters once more. Almost always, as the possible of treatments (66) (12) extremes of an index are extended, fewer cases are to be found at each end. The researcher who Rationale

Reading 51% 87% behind wishes to measure political conservatism to its Preferences treatments (219) (130) greatest extreme may fi nd there is almost no one in that category. FIGURE 6-4 Hypothetical Trivariate Relation- The fi rst decision, then, concerns the confl icting ship among Scientifi c Orientation Items. This desire for (1) a range of measurement in the index hypothetical relationship would suggest that not all three indicators would contribute effectively to a and (2) an adequate number of cases at each point composite index. in the index. You’ll be forced to reach some kind of compromise between these confl icting desires. The second decision concerns the actual as- medical interest. If we know whether a respondent signment of scores for each particular response. feels he or she can best teach as a physician or as a Basically you must decide whether to give each researcher, knowing the respondent’s reading pref- item in the index equal weight or different weights. erence adds nothing to our evaluation of his or her Although there are no fi rm rules, I suggest—and scientifi c orientation. If something like Figure 6-4 practice tends to support this method—that items resulted from the actual data, we would conclude be weighted equally unless there are compelling that reading preference should not be included in reasons for differential weighting. That is, the bur- the same index as teaching contribution, because den of proof should be on differential weighting; it contributed nothing to the composite index. equal weighting should be the norm. This example used only three questionnaire Of course, this decision must be related to the items. If more were being considered, then more- earlier issue regarding the balance of items cho- complex multivariate tables would be in order, sen. If the index is to represent the composite of constructed of four, fi ve, or more variables. The slightly different aspects of a given variable, then purpose of this step in index construction, again, you should give each aspect the same weight. In is to discover the simultaneous interaction of the some instances, however, you may feel that, say, items in order to determine which should be in- two items refl ect essentially the same aspect, and cluded in the same index. the third refl ects a different aspect. If you wished to have both aspects equally represented by the in- Index Scoring dex, you might decide to give the different item a weight equal to the combination of the two similar When you have chosen the best items for the index, ones. In such a situation, you might want to assign you next assign scores for particular responses, a maximum score of 2 to the different item and a thereby creating a single composite index out of maximum score of 1 to each of the similar ones. the several items. There are two basic decisions to Although the rationale for scoring responses be made in this step. should take such concerns into account, you’ll First, you must decide the desirable range of the typically experiment with different scoring meth- index scores. Certainly a primary advantage of an ods, examining the relative weights given to differ- index over a single item is the range of gradations ent aspects but at the same time worrying about it offers in the measurement of a variable. As noted the range and distribution of cases provided. Ulti- earlier, political conservatism might be measured mately, the scoring method chosen will represent 180 CHAPTER 6 INDEXES, SCALES, AND TYPOLOGIES a compromise among these several demands. Of Sometimes the indexing procedures are contro- course, as in most research activities, such a deci- versial, as evidenced in “What Is the Best College sion is open to revision on the basis of later exami- in the United States?” nations. Validation of the index, to be discussed Although it is often appropriate to examine the shortly, may lead you to recycle your efforts toward relationships among indicators of a variable being constructing a completely different index. measured by an index or scale, you should realize In the example taken from the medical school that the indicators are sometimes independent of faculty survey, I decided to weight the items one another. For example, Stacy De Coster notes equally, because I’d chosen them, in part, because that the indicators of family stress may be indepen- they represented slightly different aspects of the dent of one another, though they contribute to the overall variable scientifi c orientation. On each of same variable: the items, the respondents were given a score of Family Stress is a scale of stressful events within 1 for choosing the “scientifi c” response to the item the family. The experience of any one of these and a score of 0 for choosing the “nonscientifi c” events—parent job loss, parent separation, par- response. Each respondent, then, could receive a ent illness—is independent of the other events. score of 0, 1, 2, or 3. This scoring method provided Indeed, prior research on events utilized in what was considered a useful range of variation— stress scales has demonstrated that the events four index categories—and also provided enough in these scales typically are independent of one cases for analysis in each category. another and reliabilities on the scales [are] low. Here’s a similar example of index scoring, from — (2005:176) a study of work satisfaction. One of the key variables was job-related depression, measured by an If the indicators of a variable are logically re- index composed of the following four items, which lated to one another, on the other hand, that rela- asked workers how they felt when thinking about tionship should be used as a criterion for determin- themselves and their jobs: ing which are the better indicators.

• “I feel downhearted and blue.” • “I get tired for no reason.” Handling Missing Data • “I ﬁ nd myself restless and can’t keep still.” Regardless of your data-collection method, you’ll • “I am more irritable than usual.” frequently face the problem of missing data. In The researchers, Amy Wharton and James a content analysis of the political orientations of Baron, report, “Each of these items was coded: newspapers, for example, you may discover that 4 often, 3 sometimes, 2 rarely, 1 never” a particular newspaper has never taken an edito- (1987:578). They go on to explain how they mea- rial position on one of the issues being studied. In sured other variables examined in the study: an experimental design involving several retests of subjects over time, some subjects may be unable to Job-related self-esteem was based on four items participate in some of the sessions. In virtually ev- asking respondents how they saw themselves in ery survey, some respondents fail to answer some their work: happy/sad; successful/not success- questions (or choose a “don’t know” response). Al- ful; important/not important; doing their best/ though missing data present problems at all stages not doing their best. Each item ranged from 1 to of analysis, they’re especially troublesome in index 7, where 1 indicates a self-perception of not be- construction. However, several methods for dealing happy, successful, important, or doing one’s ing with these problems exist. best. — (1987:578) First, if there are relatively few cases with miss- As you look through the social research litera- ing data, you may decide to exclude them from the ture, you’ll ﬁ nd numerous similar examples of cu- construction of the index and the analysis. (I did mulative indexes being used to measure variables. this in the medical school faculty example.) The ISSUES AND INSIGHTS WHAT IS THE BEST COLLEGE IN THE UNITED STATES?

Each year U.S. News and World Report issues a sciences. Universities are allowed to count special report ranking the nation’s colleges and their research budgets in their per-student universities. Their rankings refl ect an index, cre- expenditures, though students get no direct ated from several items: educational expendi- benefi t from costly research their professors tures per student, graduation rates, selectivity are doing outside of class. (percent accepted of those applying), average In its “best colleges” issue two years ago, SAT scores of fi rst-year students, and similar in- U.S. News made precisely this point, saying dicators of quality. it considered only the rank ordering of per- Typically, Harvard is ranked the number student expenditures, rather than the actual one school in the nation, followed by Yale and amounts, on the grounds that expenditures Princeton. However, the 1999 “America’s Best at institutions with large research programs Colleges” issue shocked educators, prospective and medical schools are substantially higher college students, and their parents. The Cali- than those at the rest of the schools in the fornia Institute of Technology had leaped from category. In other words, just two years ago, ninth place in 1998 to fi rst place a year later. the magazine felt it unfair to give Caltech, Although Harvard, Yale, and Princeton still did MIT, and Johns Hopkins credit for having well, they had been supplanted. What had hap- lots of fancy laboratories that don’t actually pened at Caltech to produce such a remarkable improve undergraduate education. surge in quality? The answer was to be found at U.S. News and Gottlieb reviewed each of the changes in the World Report, not at Caltech. The newsmaga- index and then asked how 1998’s ninth-ranked zine changed the structure of the ranking index Caltech would have done had the revised index- in 1999, which made a big difference in how ing formula been in place a year earlier. His con- schools fared. clusion: Caltech would have been fi rst in 1998 Bruce Gottlieb (1999) gives this example of as well. In other words, the apparent improve- how the altered scoring made a difference. ment was solely a function of how the index was scored. So, how did Caltech come out on top? Well, Composite measures such as scales and in- one variable in a school’s ranking has long dexes are valuable tools for understanding soci- been educational expenditures per student, ety. However, it’s important that we know how and Caltech has traditionally been tops in those measures are constructed and what that this category. But until this year, U.S. News construction implies. considered only a school’s ranking in this So, what’s really the best college in the Unit- category—fi rst, second, etc.—rather than ed States? It depends on how you defi ne “best.” how much it spent relative to other schools. There is no “really best,” only the various social It didn’t matter whether Caltech beat constructions we can create. Harvard by $1 or by $100,000. Two other

schools that rose in their rankings this year Sources: Bruce Gottlieb, “Cooking the School Books: were MIT (from fourth to third) and Johns How U.S. News Cheats in Picking Its ‘Best American Hopkins (from 14th to seventh). All three Colleges,’” Slate, August 31, 1999, http://www .slate.com/crapshoot/99-08-31/crapshoot.asp; have high per-student expenditures and U.S. News and World Report, “America’s Best Col- all three are especially strong in the hard leges,” Au gust 30, 1999. 182 CHAPTER 6 INDEXES, SCALES, AND TYPOLOGIES primary concerns in this instance are whether the and reduce the likelihood that it will relate to other numbers available for analysis will remain suf- variables in ways you may have hypothesized. fi cient and whether the exclusion will result in a If you’re creating an index out of several items, biased sample whenever the index is used in the you can sometimes handle missing data by using analysis. The latter possibility can be examined proportions based on what is observed. Suppose through a comparison—on other relevant vari- your index is composed of six indicators, and you ables—of those who would be included in or ex- have only four observations for a particular sub- cluded from the index. ject. If the subject has earned 4 points out of a pos- Second, you may sometimes have grounds for sible 4, you might assign an index score of 6; if the treating missing data as one of the available re- subject has 2 points (half the possible score on four sponses. For example, if a questionnaire has asked items), you could assign a score of 3 (half the pos- respondents to indicate their participation in vari- sible score on six observations). ous activities by checking “yes” or “no” for each, The choice of a particular method to be used many respondents may have checked some of depends so much on the research situation that the activities “yes” and left the remainder blank. I can’t reasonably suggest a single “best” method In such a case, you might decide that a failure to or rank the several I have described. Excluding all answer meant “no,” and score missing data in this cases with missing data can bias the representa- case as though the respondents had checked the tiveness of the fi ndings, but including such cases “no” space. by assigning scores to missing data can infl uence Third, a careful analysis of missing data may the nature of the fi ndings. The safest and best yield an interpretation of their meaning. In con- method is to construct the index using alternative structing a measure of political conservatism, for methods and see whether the same fi ndings fol- example, you may discover that respondents who low from each. Understanding your data is the fi nal failed to answer a given question were generally goal of analysis anyway. as conservative on other items as were those who gave the conservative answer. As another ex- Index Validation ample, a recent study measuring religious beliefs found that people who answered “don’t know” Up to this point, we’ve discussed all the steps in about a given belief were almost identical to the the selection and scoring of items that result in “disbelievers” in their answers about other beliefs. a composite index purporting to measure some (Note: You should take these examples only as sug- variable. If each of the preceding steps is carried gesting general ways to analyze your own data— out carefully, the likelihood of the index actually not as empirical guides.) Whenever the analysis of measuring the variable is enhanced. To demon- missing data yields such interpretations, then, you strate success, however, we need to validate the may decide to score such cases accordingly. index. Following the basic logic of validation, we There are many other ways of handling the assume that the index provides a measure of some problem of missing data. If an item has several variable; that is, the scores on the index arrange possible values, you might assign the middle value cases in a rank order in terms of that variable. An to cases with missing data; for example, you could index of political conservatism rank-orders people assign a 2 if the values are 0, 1, 2, 3, and 4. For a in terms of their relative conservatism. If the index continuous variable such as age, you could simi- does that successfully, then people scored as rela- larly assign the mean to cases with missing data tively conservative on the index should appear rel- (more on this in Chapter 14). Or, you can supply atively conservative in all other indications of po- missing data by assigning values at random. All litical orientation, such as their responses to other of these are conservative solutions, because any questionnaire items. There are several methods of such changes weaken the “purity” of your index validating an index. INDEX CONSTRUCTION 183

Item Analysis The fi rst step in index validation If the individual item is a good refl ection of is an internal validation called item analysis. In the overall index, we should expect the 1’s and item analysis, you examine the extent to which the 2’s to fi ll in a progression between 0 percent and composite index is related to (or predicts responses 100 percent. More of the 2’s should choose “basic to) the individual items it comprises. Here’s an il- mechanisms” than 1’s. This is not guaranteed by lustration of this step. the way the index was constructed, however; it is In the index of scientifi c orientations among an empirical question—one we answer in an item medical school faculty, for example, index scores analysis. Here’s how this particular item analysis ranged from 0 (most interested in patient care) to turned out. 3 (most interested in research). Now let’s consider one of the items in the index: whether respondents Index of Scientifi c Orientations wanted to advance their own knowledge more 0 1 2 3 with regard to total patient management or more Percent who said in the area of basic mechanisms. The latter were they were more treated as being more scientifi cally oriented than interested in basic the former. The following empty table shows how mechanisms 0 16 91 100 we would examine the relationship between the index and the individual item. As you can see, in accord with our assumption that the 2’s are more scientifi cally oriented than Index of Scientifi c Orientations the 1’s, we fi nd that a higher percentage of the 2’s (91 percent) than the 1’s (16 percent) say “basic 0 1 2 3 mechanisms.” Percent who said An item analysis of the other two components they were more of the index yields similar results, as follows. interested in basic mechanisms ?? ?? ?? ?? Index of Scientifi c Orientations

0 1 2 3 If you take a minute to refl ect on the table, you may see that we already know the numbers that Percent who said go in two of the cells. To get a score of 3 on the they could teach best as medical index, respondents had to say “basic mechanisms” researchers 0 4 14 100 in response to this question and give the “scien- Percent who said tifi c” answers to the other two items as well. Thus, they preferred 100 percent of the 3’s on the index said “basic reading about rationales 0 80 97 100 mechanisms.” By the same token, all the 0’s had to answer this item with “total patient management.” Each of the items, then, seems an appropriate Thus, 0 percent of those respondents said “basic component in the index. Each seems to refl ect the mechanisms.” Here’s how the table looks with the same quality that the index as a whole measures. information we already know. In a complex index containing many items, this step provides a convenient test of the independent Index of Scientifi c Orientations contribution of each item to the index. If a given 0 1 2 3

Percent who said item analysis An assessment of whether each of the they were more items included in a composite measure makes an indepen- interested in basic dent contribution or merely duplicates the contribution of mechanisms 0 ?? ?? 100 other items in the measure. 184 CHAPTER 6 INDEXES, SCALES, AND TYPOLOGIES item is found to be poorly related to the index, it TABLE 6-1 Validation of Scientific Orientation may be assumed that other items in the index can- Index cel out the contribution of that item, and it should Index of Scientific Orientation be excluded from the index. In other words, if the item in question contributes nothing to the index’s Low High 0 1 2 3 power, it should be excluded. Although item analysis is an important fi rst Percent interested in test of the index’s validity, it is scarcely suffi cient. attending scientific lectures at the medical school 34 42 46 65 If the index adequately measures a given variable, Percent who say faculty it should successfully predict other indications of members should have that variable. To test this, we must turn to items experience as medical not included in the index. researchers 43 60 65 89 Percent who would prefer External Validation faculty duties involving People scored as politically research activities only 0 8 32 66 conservative on an index should appear conser- Percent who engaged in vative by other measures as well, such as their research during the responses to other items in a questionnaire. Of preceding academic year 61 76 94 99 course, we’re talking about relative conservatism, because we can’t make an absolute defi nition of what constitutes conservatism. However, those gives a different description of scientifi c orienta- respondents scored as the most conservative on tions overall. For example, the last validating item the index should be the most conservative in an- indicates that the great majority of all faculty were swering other questions. Those scored as the least engaged in research during the preceding year. conservative on the index should be the least con- If this were the only indicator of scientifi c orien- servative on other items. Indeed, the ranking of tation, we would conclude that nearly all faculty groups of respondents on the index should predict were scientifi c. Nevertheless, those scored as more the ranking of those groups in answering other scientifi c on the index are more likely to have en- questions dealing with political orientations. gaged in research than are those who were scored In our example of the scientifi c orientation as relatively less scientifi c. The third validating index, several questions in the questionnaire of- item provides a different descriptive picture: Only fered the possibility of such external validation. a minority of the faculty overall say they would Table 6-1 presents some of these items, which prefer duties limited exclusively to research. (Only provide several lessons regarding index validation. among those scored 3 on the index do a major- First, we note that the index strongly predicts the ity agree with that statement.) Nevertheless, the responses to the validating items in the sense that percentages giving this answer correspond to the the rank order of scientifi c responses among the scores assigned on the index. four groups is the same as the rank order provided by the index itself. That is, the percentages refl ect Bad Index versus Bad Validators Nearly every in- greater scientifi c orientation as you read across dex constructor at some time must face the appar- the rows of the table. At the same time, each item ent failure of external items to validate the index. If the internal item analysis shows inconsistent relationships between the items included in the in- The process of testing the validity external validation dex and the index itself, something is wrong with of a measure, such as an index or scale, by examining its relationship to other, presumed indicators of the same the index. But if the index fails to predict strongly variable. If the index really measures prejudice, for exam- the external validation items, the conclusion to be ple, it should correlate with other indicators of prejudice. drawn is more ambiguous. You must choose be- INDEX CONSTRUCTION 185 tween two possibilities: (1) the index does not ad- The Gender-related Development Index (GDI) equately measure the variable in question, or (2) compared women to men in terms of three indica- the validation items do not adequately measure tors: life expectancy, education, and income. These the variable and thereby do not provide a suffi cient indicators are commonly used in monitoring the test of the index. status of women in the world. The Scandinavian Having worked long and conscientiously on countries of Norway, Sweden, Finland, and Den- the construction of an index, you’ll likely fi nd the mark ranked highest on this measure. second conclusion compelling. Typically, you’ll The second index, the Gender Empowerment feel you have included the best indicators of the Measure (GEM), aimed more at power issues and variable in the index; the validating items are, comprised three different indicators: therefore, second-rate indicators. Nevertheless, you should recognize that the index is purportedly • The proportion of parliamentary seats held by a very powerful measure of the variable; thus, it women should be somewhat related to any item that taps • The proportion of administrative, managerial, the variable even poorly. professional, and technical positions held by When external validation fails, you should reex- women amine the index before deciding that the validat- • A measure of access to jobs and wages ing items are insuffi cient. One way is to examine Once again, the Scandinavian countries ranked the relationships between the validating items and high but were joined by Canada, New Zealand, the the individual items included in the index. If you Netherlands, the United States, and Austria. Hav- discover that some of the index items relate to the ing two different measures of gender equality al- validators and others do not, you’ll have improved lowed the researchers to make more-sophisticated your understanding of the index as it was initially distinctions. For example, in several countries, constituted. most notably Greece, France, and Japan, women There is no cookbook solution to this dilemma; fared relatively well on the GDI but quite poorly it is an agony serious researchers must learn to on the GEM; thus, although they were doing fairly survive. Ultimately, the wisdom of your decision to well in terms of income, education, and life expec- accept an index will be determined by the useful- tancy, they were still denied access to power. And ness of that index in your later analyses. Perhaps whereas the GDI scores were higher in the wealth- you’ll initially decide that the index is a good one ier nations than in the poorer ones, GEM scores and that the validators are defective, but you’ll later showed that women’s empowerment did not seem fi nd that the variable in question (as measured by to depend on national wealth, with many poor, de- the index) is not related to other variables in the veloping countries outpacing some rich, industrial ways you expected. You may then have to com- ones in regard to such empowerment. pose a new index. By examining several different dimensions of the variables involved in their study, the UN re- The Status of Women: searchers also uncovered an aspect of women’s An Illustration of Index Construction earnings that generally goes unnoticed. Popula- tion Communications International (1996:1) sum- For the most part, I’ve talked about index construc- marizes the fi nding nicely: tion in the context of survey research, but other types of research also lend themselves to this kind Every year, women make an invisible contribu- of composite measure. For example, when the tion of eleven trillion U.S. dollars to the global United Nations (1995) set about examining the sta- economy, the UNDP [United Nations Develop- tus of women in the world, they chose to create ment Programme] report says, counting both two indexes, refl ecting two different dimensions. unpaid work and the underpayment of women’s 186 CHAPTER 6 INDEXES, SCALES, AND TYPOLOGIES

work at prevailing market prices. This “under- evaluation” of women’s work not only under- IN THE REAL WORLD mines their purchasing power, says the 1995 ASSESSING WOMEN’S STATUS HDR [Human Development Report], but also reduces their already low social status and af- In our discussion of the Gender Empower- fects their ability to own property and use credit. ment Measure (GEM), we analyze the status Mahbub ul Haq, the principal author of the re- of women in countries around the world. port, says that “if women’s work were accurately How might you use the logic of this analysis reﬂ ected in national statistics, it would shatter to examine and assess the status of women the myth that men are the main breadwinners of in a particular organization, such as the col- the world.” The UNDP report ﬁ nds that women lege you attend or a corporation you’re fa- work longer hours than men in almost every miliar with? country, including both paid and unpaid duties. In developing countries, women do approximately 53% of all work and spend two-thirds of their work time on unremunerated activities. In Scales offer more assurance of ordinality by industrialized countries, women do an average tapping the intensity structures among the indi- of 51% of the total work, and—like their coun- cators. The several items going into a composite terparts in the developing world—perform about measure may have different intensities in terms of two-thirds of their total labor without pay. Men the variable. Many methods of scaling are avail- in industrialized countries are compensated for able. To illustrate the variety of techniques at hand, two-thirds of their work. we’ll look at four scaling procedures, along with a technique called the semantic differential. Al- As you can see, indexes can be constructed though these examples focus on questionnaires, from many different kinds of data for a variety of the logic of scaling, like that of indexing, applies to purposes. (See the box “Assessing Women’s Sta- other research methods as well. tus” for more on this topic.) Now we’ll turn our attention from the construction of indexes to an examination of scaling techniques. Bogardus Social Distance Scale

Let’s suppose you’re interested in the extent to SCALE CONSTRUCTION which U.S. citizens are willing to associate with, say, sex offenders. You might ask the following questions: Good indexes provide an ordinal ranking of cases on a given variable. All indexes are based on this 1. Are you willing to let sex offenders live in your kind of assumption: A senator who voted for seven country? out of ten conservative bills is considered to be 2. Are you willing to let sex offenders live in your more conservative than one who voted for only four community? of them. What an index may fail to take into account, 3. Are you willing to let sex offenders live in your however, is that not all indicators of a variable neighborhood? are equally important or equally strong. The fi rst 4. Would you be willing to let a sex offender live senator might have voted for the seven least con- next door to you? servative bills, whereas the second senator might 5. Would you let your child marry a sex offender? have voted for the four most conservative bills. (The second senator might have considered the These questions increase in terms of how other six bills too liberal and voted against them.) closely the respondents want to associate with sex SCALE CONSTRUCTION 187 offenders. Beginning with the original concern to looking at social distance from the perspective of measure willingness to associate with sex offend- the minority group. Here’s how they framed their ers, you have thus developed several questions questions (1996:19): indicating differing degrees of intensity on this variable. The kinds of items presented constitute a Considering typical Caucasian Americans you Bogardus social distance scale (created by Em- have known, not any specifi c person nor the ory Bogardus). This scale is a measurement tech- worst or the best, circle Y or N to express your nique for determining the willingness of people to opinion. participate in social relations—of varying degrees Y N 5. Do they mind your being a citizen in of closeness—with other kinds of people. this country? The clear differences of intensity suggest a Y N 4. Do they mind your living in the same structure among the items. Presumably, if a person neighborhood? is willing to accept a given kind of association, he Y N 3. Would they mind your living next to or she would be willing to accept all those preced- them? ing it in the list—those with lesser intensities. For Y N 2. Would they mind your becoming a example, the person who is willing to permit sex close friend to them? offenders to live in the neighborhood will surely Y N 1. Would they mind your becoming their accept them in the community and the nation kin by marriage? but may or may not be willing to accept them as As with the original scale, the researchers found next-door neighbors or relatives. This, then, is the that knowing the number of items minority re- logical structure of intensity inherent among the spondents agreed with also told the researchers items. which ones were agreed with—99 percent of the Empirically, one would expect to fi nd the larg- time in this case est number of people accepting co-citizenship and the fewest accepting intermarriage. In this sense, we speak of “easy items” (for example, residence Thurstone Scales in the United States) and “hard items” (for example, intermarriage). More people agree to the easy Often the inherent structure of the Bogardus social items than to the hard ones. With some inevita- distance scale is not appropriate to the variable ble exceptions, logic demands that once a person being measured. Indeed, such a logical structure has refused a relationship presented in the scale, among several indicators is seldom apparent. A he or she will also refuse all the harder ones that Thurstone scale (a format created by Louis Thur- follow it. stone) is an attempt to develop a format for gener- The Bogardus social distance scale illustrates ating groups of indicators of a variable that have at the important economy of scaling as a data-reduc- least an empirical structure among them. tion device. By knowing how many relationships with sex offenders a given respondent will accept, we know which relationships were accepted. Thus, Bogardus social distance scale A measurement a single number can accurately summarize fi ve or technique for determining the willingness of people six data items without a loss of information. to participate in social relations—of varying degrees of Motoko Lee, Stephen Sapp, and Melvin Ray closeness—with other kinds of people. It is an especially effi cient technique in that one can summarize several (1996) noticed an implicit element in the Bogar- discrete answers without losing any of the original details dus social distance scale: It looks at social distance of the data. from the point of view of the majority group in a Thurstone scale A type of composite measure, con- society. These researchers decided to turn the ta- structed in accord with the weights assigned by “judges” bles and create a “reverse social distance” scale: to various indicators of some variables. 188 CHAPTER 6 INDEXES, SCALES, AND TYPOLOGIES

One of the basic formats is that of “equal- of their judgments would depend on their experi- appearing intervals.” A group of judges is given ence with the variable under consideration, pro- perhaps a hundred items felt to be indicators of a fessional researchers might be needed. Moreover, given variable. Each judge is then asked to esti- the meanings conveyed by the several items indi- mate how strong an indicator of a variable each cating a given variable tend to change over time. item is by assigning scores of perhaps 1 to 13. If the Thus, an item might have a given weight at one variable were prejudice, for example, the judges time and quite a different weight later on. To be would be asked to assign the score of 1 to the very effective, a Thurstone scale would have to be up- weakest indicators of prejudice, the score of 13 to dated periodically. the strongest indicators, and intermediate scores to those in between. Likert Scaling Once the judges have completed this task, the researcher examines the scores assigned to each You may sometimes hear people refer to a ques- item to determine which items produced the great- tionnaire item containing response categories est agreement among the judges. Those items on such as “strongly agree,” “agree,” “disagree,” and which the judges disagreed broadly would be re- “strongly disagree” as a Likert scale. This is tech- jected as ambiguous. Among those items produc- nically a misnomer, although Rensis Likert (pro- ing general agreement in scoring, one or more nounced “LICK-ert”) did create this commonly would be selected to represent each scale score used question format. from 1 to 13. The particular value of this format is the un- The items selected in this manner might ambiguous ordinality of response categories. If then be included in a survey questionnaire. Re- respondents were permitted to volunteer or se- spondents who appeared prejudiced on those lect such answers as “sort of agree,” “pretty much items representing a strength of 5 would then be agree,” “really agree,” and so forth, you would expected to appear prejudiced on those having ﬁ nd it impossible to judge the relative strength of lesser strengths, and if some of those respondents agreement intended by the various respondents. did not appear prejudiced on the items with a The Likert format solves this problem. strength of 6, it would be expected that they would Rensis Likert had something more in mind, also not appear prejudiced on those with greater however. He created a method by which this ques- strengths. tion format could be used to determine the relative If the Thurstone scale items were adequately intensity of different items. As a simple example, developed and scored, the economy and effec- suppose we wish to measure prejudice against tiveness of data reduction inherent in the Bogar- women. To do this, we create a set of 20 state- dus social distance scale would appear. A single ments, each of which reﬂ ects that prejudice. One score might be assigned to each respondent (the of the items might be “Women can’t drive as well strength of the hardest item accepted), and that as men.” Another might be “Women shouldn’t be score would adequately represent the responses allowed to vote.” Likert’s scaling technique would to several questionnaire items. And, as is true of demonstrate the difference in intensity between the Bogardus scale, a respondent scored 6 might these items as well as pegging the intensity of the be regarded as more prejudiced than one scored other 18 statements. 5 or less. Let’s suppose we ask a sample of people to Thurstone scaling is not often used in research agree or disagree with each of the 20 statements. today, primarily because of the tremendous ex- Simply giving one point for each of the indica- penditure of energy and time required to have 10 tors of prejudice against women would yield the to 15 judges score the items. Because the quality possibility of index scores ranging from 0 to 20. A SCALE CONSTRUCTION 189

Likert scale goes one step beyond that and cal- way to tap those feelings would be to use a seman- culates the average index score for those agreeing tic differential format. with each of the individual statements. Let’s say To begin, you must determine the dimensions that all those who agreed that women are poorer along which subjects should judge each selection. drivers than are men had an average index score Then you need to fi nd two opposite terms, repre- of 1.5 (out of a possible 20). Those who agreed that senting the polar extremes along each dimension. women should be denied the right to vote might Let’s suppose one dimension that interests you is have an average index score of, say, 19.5—indicat- simply whether subjects enjoyed the piece or not. ing the greater degree of prejudice refl ected in that Two opposite terms in this case could be “enjoy- response. able” and “unenjoyable.” Similarly, you might want As a result of this item analysis, respondents to know whether they regarded the individual se- could be rescored to form a scale: 1.5 points for lections as “complex” or “simple,” “harmonic” or agreeing that women are poorer drivers, 19.5 points “discordant,” and so forth. for saying women shouldn’t vote, and points for Once you have determined the relevant dimen- other responses refl ecting how those items related sions and have found terms to represent the ex- to the initial, simple index. If those who disagreed tremes of each, you might prepare a rating sheet with the statement “I might vote for a woman for each subject would complete for each piece of mu- president” had an average index score of 15, then sic. Figure 6-5 shows what it might look like. the scale would give 15 points to people disagree- On each line of the rating sheet, the subject ing with that statement. would indicate how he or she felt about the piece In practice, Likert scaling is seldom used today. of music: whether it was enjoyable or unenjoyable, The item format devised by Likert, however, is for example, and whether it was “somewhat” that one of the most commonly used formats in con- way or “very much” so. To avoid creating a biased temporary questionnaire design. Typically, it’s now pattern of responses to such items, it’s a good idea used in the creation of simple indexes. With, say, to vary the placement of terms that are likely to fi ve response categories, scores of 0 to 4 or 1 to 5 be related to each other. Notice, for example, that might be assigned, taking the direction of the items “discordant” and “traditional” are on the left side into account (for example, assign a score of 5 to of the sheet, with “harmonic” and “modern” on the “strongly agree” for positive items and to “strongly right. Most likely, those selections scored as “dis- disagree” for negative items). Each respondent would then be assigned an overall score representing the summation of the scores he or she received for responses to the individual items. Likert scale A type of composite measure developed by Rensis Likert in an attempt to improve the levels of measurement in social research through the use of standardized response categories in survey questionnaires to Semantic Differential determine the relative intensity of different items. Likert items are those using such response categories as “strong- Like the Likert format, the semantic differential ly agree,” “agree,” “disagree,” and “strongly disagree.” asks respondents to choose between two opposite Such items may be used in the construction of true Likert positions. Here’s how it works. scales as well as other types of composite measures. Suppose you’re evaluating the effectiveness of semantic differential A questionnaire format in a new music-appreciation lecture on subjects’ ap- which the respondent is asked to rate something in terms of two, opposite adjectives (e.g., rate textbooks as preciation of music. As a part of your study, you “boring” or “exciting”), using qualifi ers such as “very,” want to play some musical selections and have the “somewhat,” “neither,” “somewhat,” and “very” to subjects report their feelings about them. A good bridge the distance between the two opposites. 190 CHAPTER 6 INDEXES, SCALES, AND TYPOLOGIES

Very Much SomewhatNeither Somewhat Very Much

Enjoyable Unenjoyable Simple Complex Discordant Harmonic Traditional Modern

FIGURE 6-5 Semantic Differential: Feelings about Musical Selections. The semantic differential asks respondents to describe something or someone in terms of opposing adjectives.

cordant” would also be scored as “modern” as op- scale construction, however, you would also look posed to “traditional.” for relatively “hard” and “easy” indicators of the Both the Likert and semantic differential for- variable being examined. mats have a greater rigor and structure than do Earlier, when we talked about attitudes regard- other question formats. As I’ve indicated earlier, ing a woman’s right to have an abortion, we dis- these formats produce data suitable to both index- cussed several conditions that can affect people’s ing and scaling. opinions: whether the woman is married, whether her health is endangered, and so forth. These dif- Guttman Scaling fering conditions provide an excellent illustration of Guttman scaling. Researchers today often use the scale developed Here are the percentages of the people in the by Louis Guttman. Like Bogardus, Thurstone, and 2000 GSS sample who supported a woman’s right Likert scaling, Guttman scaling is based on the fact to an abortion, under three different conditions: that some items under consideration may prove Woman’s health is seriously endangered 89% to be more extreme indicators of the variable than Pregnant as a result of rape 81% others. One example should sufﬁ ce to illustrate this Woman is not married 39% pattern. In the earlier example of measuring scientiﬁ c The different percentages supporting abortion orientations among medical school faculty mem- under the three conditions suggest something bers, a simple index was constructed. As it hap- about the different levels of support that each item pens, however, the three items included in the in- indicates. For example, if someone would support dex essentially form a Guttman scale. abortion when the mother’s life is seriously endan- The construction of a Guttman scale would gered, that’s not a very strong indicator of general begin with some of the same steps that initiate support for abortion, because almost everyone index construction. You would begin by examin- agreed with that. Supporting abortion for unmar- ing the face validity of items available for analysis. ried women seems a much stronger indicator of Then, you would examine the bivariate and per- support for abortion in general—fewer than half haps multivariate relations among those items. In the sample took that position. Guttman scaling is based on the notion that anyone who gives a strong indicator of some variable will also give the weaker indicators. In this case, Guttman scale A type of composite measure used to summarize several discrete observations and to represent we would assume that anyone who supported some more-general variable. abortion for unmarried women would also support SCALE CONSTRUCTION 191

TABLE 6-2 Scaling Support for Choice the items do not form a perfect Guttman scale. (It of Abortion would be extremely rare for such data to form a Guttman scale perfectly.) Result Number Women’s of Woman of Recall at this point that one of the chief functions Health Rape Unmarried Cases of scaling is effi cient data reduction. Scales provide a technique for presenting data in a summary form 677 while maintaining as much of the original infor- 607 mation as possible. When the scientifi c orienta- Scale Types 165 tion items were formed into an index in our earlier 147 discussion, respondents were given one point for Total 1,596 each scientifi c response they gave. If these same 42 three items were scored as a Guttman scale, some 5 respondents would be assigned scale scores that Mixed Types 2 would permit the most accurate reproduction of 4 their original responses to all three items. Total 53 In the present example of attitudes regarding favors woman’s right to choose; opposes woman’s right abortion, respondents fi tting into the scale types to choose would receive the same scores as were assigned in the index construction. Persons selecting all it in the case of rape or of the woman’s health be- three pro-choice responses would still be scored ing threatened. Table 6-2 tests this assumption by 3, those who selected pro-choice responses to the presenting the number of respondents who gave two easier items and were opposed on the hardest each of the possible response patterns. item would be scored 2, and so on. For each of the The fi rst four response patterns in the table four scale types, we could predict accurately from compose what we would call the scale types: those their scores all the actual responses given by all patterns that form a scalar structure. Following the respondents. those respondents who supported abortion under The mixed types in the table present a problem, all three conditions (line 1), we see that those with however. The fi rst mixed type () was scored only two pro-choice responses (line 2) have cho- 1 on the index to indicate only one pro-choice re- sen the two easier ones ; those with only one such sponse. But, if 1 were assigned as a scale score, response (line 3) chose the easiest of the three (the we would predict that the 42 respondents in this woman’s health being endangered). And fi nally, group had chosen only the easiest item (approving there are some respondents who opposed abor- abortion when the woman’s life was endangered), tion in all three circumstances (line 4). and we would be making two errors for each such The second part of the table presents those re- respondent: thinking their response pattern was sponse patterns that violate the scalar structure of () instead of (). Scale scores are as- the items. The most radical departures from the signed, therefore, with the aim of minimizing the scalar structure are the last two response patterns: errors that would be made in reconstructing the those who accepted only the hardest item and original responses. those who rejected only the easiest one. Table 6-3 illustrates the index and scale scores The fi nal column in the table indicates the that would be assigned to each of the response number of survey respondents who gave each of patterns in our example. Note that one error is the response patterns. The great majority (1,596, made for each respondent in the mixed types. This or 97 percent) fi t into one of the scale types. The is the minimum we can hope for in a mixed-type presence of mixed types, however, indicates that pattern. In the fi rst mixed type, for example, we 192 CHAPTER 6 INDEXES, SCALES, AND TYPOLOGIES

TABLE 6-3 Index and Scale Scores

Response Number Index Scale Total Pattern of Cases Scores Scores Scale Errors

Scale Types 677 3 3 0 607 2 2 0 165 1 1 0 147 0 0 0 Mixed Types 42 1 2 42 5 2 3 5 2 1 0 2 4 2 3 4 Total Scale errors 53 number of errors Coefﬁ cient of reproducibility 1 number of guesses 53 53 1 1 1,649 3 4,947 0.989 98.9%

This table presents one common method for scoring mixed types, but you should be advised that other methods are also used.

would erroneously predict a pro-choice response commonly used standards in this regard. If the ob- to the easiest item for each of the 42 respondents served reproducibility exceeds the level you’ve set, in this group, making a total of 42 errors. you’ll probably decide to score and use the items The extent to which a set of empirical responses as a scale. form a Guttman scale is determined by the accu- The decision concerning criteria in this regard racy with which the original responses can be re- is, of course, arbitrary. Moreover, a high degree of constructed from the scale scores. For each of the reproducibility does not insure that the scale con- 1,649 respondents in this example, we’ll predict structed in fact measures the concept under con- three questionnaire responses, for a total of 4,947 sideration, although it increases confi dence that predictions. Table 6-3 indicates that we’ll make 53 all the component items measure the same thing. errors using the scale scores assigned. The per- Also, you should realize that a high coeffi cient of centage of correct predictions is called the coef- reproducibility is most likely when few items are fi cient of reproducibility: the percentage of original involved. responses that could be reproduced by knowing One concluding remark with regard to Gutt- the scale scores used to summarize them. In the man scaling: It’s based on the structure observed present example, the coeffi cient of reproducibility among the actual data under examination. This is 4,894/4,947 or 98.9 percent. important point is often misunderstood. It does Except in the case of perfect (100 percent) re- not make sense to say that a set of questionnaire producibility, there is no way of saying that a set items (perhaps developed and used by a previous of items does or does not form a Guttman scale researcher) constitutes a Guttman scale. Rather, in any absolute sense. Virtually all sets of such we can say only that they form a scale within a items approximate a scale. As a general guideline, given body of data being analyzed. Scalability, however, coeffi cients of 90 or 95 percent are the then, is a sample-dependent, empirical matter. Al- TYPOLOGIES 193 though a set of items may form a Guttman scale TYPOLOGIES among one sample of survey respondents, for example, there is no guarantee that this set will form This chapter now ends with a short discussion of such a scale among another sample. In this sense, typology construction and analysis. Recall that in- then, a set of questionnaire items in and of them- dexes and scales are constructed to provide ordi- selves never forms a scale, but a set of empirical nal measures of given variables. We attempt to as- observations may. sign index or scale scores to cases in such a way as This concludes our discussion of indexing and to indicate a rising degree of prejudice, religiosity, scaling. Like indexes, scales are composite mea- conservatism, and so forth. In such cases, we’re sures of a variable, typically broadening the mean- dealing with single dimensions. ing of the variable beyond what might be captured Often, however, the researcher wishes to sum- by a single indicator. Both scales and indexes seek marize the intersection of two or more variables, to measure variables at the ordinal level of mea- thereby creating a set of categories or types, which surement. Unlike indexes, however, scales take we call a typology. You may, for example, wish to advantage of any intensity structure that may be examine the political orientations of newspapers present among the individual indicators. To the ex- separately in terms of domestic issues and foreign tent that such an intensity structure is found and policy. The fourfold presentation in Table 6-4 de- the data from the people or other units of analysis scribes such a typology. comply with the logic of that intensity structure, Newspapers in cell A of the table are conser- we can have confi dence that we have created an vative on both foreign policy and domestic policy; ordinal measure. those in cell D are liberal on both. Those in cells B and C are conservative on one and liberal on the Here are two websites where you can other. further pursue the topic of indexes and As another example, Rodney Coates (2006) cre- scales.* ated a typology of “racial hegemony” from two • Bureau of Labor Statistics, Measure- dimensions: ment Issues in the Consumer Price In- dex: http://www.bls.gov/cpi/cpigm697 1. Political Ideology .htm. The federal government’s Con- a. Democratic sumer Price Index (CPI) is one of those b. Non-Democratic composite measures that affects many 2. Military and Industrial Sophistication people’s lives—determining cost-of- a. Low living increases, for example. This site b. High discusses some aspects of the measure. • Thomas O’Connor, Scales and Indexes: http://faculty.ncwc.edu/toconnor/ He then used the typology to examine mod- 308/308lect05.htm. This website has ern examples of colonial rule, with specifi c refer- an excellent discussion of scales and ence to race relations. The cases he looked at al- indexes in general, provides illustrative lowed him to illustrate and refi ne the typology. He examples, and offers hotlinks useful for pursuing the topic.

typology The classiﬁ cation (typically nominal) of observations in terms of their attributes on two or more variables. The classiﬁ cation of newspapers as liberal-urban, *Each time the Internet icon appears, you’ll be given help- liberal-rural, conservative-urban, or conservative-rural ful leads for searching the World Wide Web. would be an example. 194 CHAPTER 6 INDEXES, SCALES, AND TYPOLOGIES

TABLE 6-4 A Political Typology of Newspapers WHAT DO YOU THINK? REVISITED Foreign Policy

Conservative Liberal If I were to tell you that we had given each Domestic Policy Conservative A B respondent one point for every relation- Liberal C D ship they were to have with sex offenders, and I told you further that a particular respondent had been given a score of 3, you would be able to reproduce each of these points out that such a device represents Weber’s fi ve answers? “ideal type”: 1. Are you willing to let sex offenders live in As stipulated by Weber, idea types represent a your country? YES type of abstraction from reality. These abstrac- 2. Are you willing to let sex offenders live in tions, constructed from the logical extraction your community? YES of elements derived from specifi c examples, 3. Are you willing to let sex offenders live in provide a theoretical model by which and from your neighborhood? YES which we may examine reality. — (2006:87) 4. Would you be willing to let a sex offender Frequently, you arrive at a typology in the course live next door to you? NO of an attempt to construct an index or scale. The 5. Would you let your child marry a sex of- items that you felt represented a single variable fender? NO appear to represent two. You might have been Although this logic is very clear in the case attempting to construct a single index of political of the Bogardus social distance scale, we’ve orientations for newspapers but discovered—em- also seen how social researchers approxi- pirically—that foreign and domestic politics had to mate that structure in creating other types be kept separate. of scales, such as Thurstone and Guttman In any event, you should be warned against a scales, which also take account of differ- diffi culty inherent in typological analysis. When- ing intensities among the indicators of a ever the typology is used as the independent vari- variable. able, there will probably be no problem. In the preceding example, you might compute the percentages of newspapers in each cell that normally endorse Democratic candidates; you could then With a typology, however, you would have to easily examine the effects of both foreign and do- present the distribution of the urban newspapers mestic policies on political endorsements. in your sample among types A, B, C, and D. Then It’s extremely diffi cult, however, to analyze a ty- you would repeat the procedure for the rural ones pology as a dependent variable. If you want to dis- in the sample and compare the two distributions. cover why newspapers fall into the different cells of Let’s suppose that 80 percent of the rural news- typology, you’re in trouble. That becomes apparent papers are scored as type A (conservative on both when we consider the ways you might construct dimensions) as compared with 30 percent of the and read your tables. Assume, for example, that urban ones. Moreover, suppose that only 5 percent you want to examine the effects of community size of the rural newspapers are scored as type B (con- on political policies. With a single dimension, you servative only on domestic issues) as compared could easily determine the percentages of rural with 40 percent of the urban ones. It would be in- and urban newspapers that were scored conserva- correct to conclude from an examination of type B tive and liberal on your index or scale. that urban newspapers are more conservative on MAIN POINTS 195 domestic issues than are rural ones, because 85 Don’t think that typologies should always be percent of the rural newspapers, compared with avoided in social research; often they provide the 70 percent of the urban ones, have this character- most appropriate device for understanding the istic. The relative sparsity of rural newspapers in data. To examine the pro-life orientation in depth, type B is due to their concentration in type A. It you might create a typology involving both abor- should be apparent that an interpretation of such tion and capital punishment. Libertarianism could data would be very diffi cult for anything other than be seen in terms of both economic and social per- description. missiveness. You have been warned, however, In reality, you’d probably examine two such di- against the special diffi culties involved in using ty- mensions separately, especially if the dependent pologies as dependent variables. variable has more categories of responses than does the example given.

Main Points with which a dimension is to be measured, and the amount of variance provided by the Introduction items. ❏ Single indicators of variables seldom capture ❏ If different items are indeed indicators of the all the dimensions of a concept, have suf- same variable, then they should be related ﬁ cient validity to warrant their use, or permit empirically to one another. In constructing an the desired range of variation to allow ordinal index, the researcher needs to examine bivari- rankings. Composite measures, such as scales ate and multivariate relationships among the and indexes, solve these problems by including items. several indicators of a variable in one sum- ❏ Index scoring involves deciding the desirable mary measure. range of scores and determining whether items will have equal or different weights. Indexes versus Scales ❏ Various techniques allow items to be used in ❏ Although both indexes and scales are in- an index in spite of missing data. tended as ordinal measures of variables, scales typically satisfy this intention better than do ❏ Item analysis is a type of internal validation indexes. based on the relationship between individual items in the composite measure and the ❏ Whereas indexes are based on the simple measure itself. External validation refers to the cumulation of indicators of a variable, scales relationships between the composite measure take advantage of any logical or empirical in- and other indicators of the variable—indicators tensity structures that exist among a variable’s not included in the measure. indicators.

Scale Construction Index Construction ❏ Four types of scaling techniques are repre- ❏ The principal steps in constructing an index sented by the Bogardus social distance scale, include selecting possible items, examining a device for measuring the varying degrees their empirical relationships, scoring the index, to which a person would be willing to as- and validating it. sociate with a given class of people; Thur- ❏ Criteria of item selection include face valid- stone scaling, a technique that uses judges to ity, unidimensionality, the degree of speciﬁ city determine the intensities of different indica- 196 CHAPTER 6 INDEXES, SCALES, AND TYPOLOGIES

tors; Likert scaling, a measurement technique Online Study Resources based on the use of standardized response categories; and Guttman scaling, a method of discovering and using the empirical intensity structure among several indicators of a given variable. Guttman scaling is probably the most Go to popular scaling technique in social research http://sociology.wadsworth.com/babbie_basics4e today. and click on ThomsonNow for access to this ❏ The semantic differential is a question format powerful online study tool. You will get a per- that asks respondents to make ratings that lie sonalized study plan based on your responses to between two extremes, such as “very positive” a diagnostic pretest. Once you have mastered and “very negative.” the material with the help of interactive learning tools, you can take a posttest to confi rm that you Typologies are ready to move on to the next chapter. ❏ A typology is a nominal composite measure often used in social research. Typologies can be Website for used effectively as independent variables, but The Basics of Social Research interpretation is diffi cult when they are used as 4th edition dependent variables. At the book companion website (http://sociology .wadsworth.com/babbie_basics4e) you will fi nd Key Terms many resources in addition to ThomsonNow to aid you in studying for your exams. For example, Bogardus social distance scale Likert scale you will fi nd Tutorial Quizzes with feedback, Inter- external validation scale Guttman scale semantic differential net Exercises, Flashcards, and Chapter Tutorials, as index Thurstone scale well as Extended Projects, InfoTrac College Edition item analysis typology search terms, Social Research in Cyberspace, GSS Data, Web Links, and primers for using various data analysis software such as SPSS and NVivo. Review Questions

1. In your own words, what is the difference Additional Readings between an index and a scale? Anderson, Andy B., Alexander Basilevsky, and Derek 2. Suppose you wanted to create an index for P. J. Hum. 1983. “Measurement: Theory and Tech- rating the quality of colleges and universi- niques.” Pp. 231–87 in Handbook of Survey Research, ties. What are three data items that might be edited by Peter H. Rossi, James D. Wright, and Andy included in such an index? B. Anderson. New York: Academic Press. The logic of 3. Why do you suppose Thurstone scales have measurement is analyzed in the context of composite measures. not been used more widely in the social Bobo, Lawrence, and Frederick C. Licari. 1989. “Educa- sciences? tion and Political Tolerance: Testing the Effects of 4. What would be some questionnaire items Cognitive Sophistication and Target Group Effect.” that could measure attitudes toward nuclear Public Opinion Quarterly 53:285–308. The authors use power and that would probably form a Gutt- a variety of techniques for determining how best to man scale? measure tolerance toward different groups in society. ADDITIONAL READINGS 197

Indrayan, A., M. J. Wysocki, A. Chawla, R. Kumar, and N. McIver, John P., and Edward G. Carmines. 1981. Unidi- Singh. 1999. “Three-Decade Trend in Human Devel- mensional Scaling. Thousand Oaks, CA: Sage. Here’s opment Index in India and Its Major States.” Social an excellent way to pursue Thurstone, Likert, and Indicators Research 46 (1): 91–120. The authors use Guttman scaling in further depth. several human development indexes to compare the Miller, Delbert. 1991. Handbook of Research Design and status of different states in India. Social Measurement. Thousand Oaks, CA: Sage. Lazarsfeld, Paul, Ann Pasanella, and Morris Rosenberg, An excellent compilation of frequently used and eds. 1972. Continuities in the Language of Social Re- semistandardized scales. The many illustrations search. New York: Free Press. See especially Section reported in Part 4 of the Miller book may be directly 1. An excellent collection of conceptual discussions adaptable to studies or at least suggestive of modiﬁ ed and concrete illustrations. The construction of com- measures. Studying the several different illustrations, posite measures is presented within the more general moreover, may also give you a better general under- area of conceptualization and measurement. standing of the logic of composite measures. 71 HUMANTHE LOGIC INQUIRY OF SAMPLING AND SCIENCE

Image not available due to copyright restrictions Photo credit

What You’ll Learn in thisThis ChapterChapter

NowWe’ll you’ll examine see how the social way people scientists learn can about select their a few world people and for the study mistakes and discoverthey makethings alongthat apply the way. to hundreds We’ll also of begin millions to see of people what makes not studied. sience different from

other ways of knowing things. 2S 1S N L 198 In this chapter . . . WHAT DO YOU THINK?

Introduction In 1936 the Literary A Brief History of Sampling Digest collected the President Alf Landon voting intentions of President Thomas E. Dewey Image not available due to copyright restrictions two million voters Two Types of Sampling Methods in order to predict whether Franklin Nonprobability Sampling D. Roosevelt or Alf Reliance on Available Subjects Landon would be Purposive or Judgmental Sampling elected president of the United States. Dur- Snowball Sampling ing more recent election campaigns, with Quota Sampling many more voters going to the polls, na- Selecting Informants tional polling ﬁ rms have typically sampled The Theory and Logic of Probability around 2,000 voters across the country. Sampling Which technique do you think is the most Conscious and Unconscious Sampling Bias effective? Why? Representativeness and Probability of Selection See the “What Do You Think? Revisited” box Random Selection toward the end of the chapter. Probability Theory, Sampling Distributions, and Estimates of Sample Error

Populations and Sampling Frames INTRODUCTION Types of Sampling Designs Simple Random Sampling Systematic Sampling One of the most visible uses of survey sampling Stratifi ed Sampling lies in the political polling that the election results Implicit Stratifi cation in Systematic Sampling subsequently test. Whereas some people doubt the Illustration: Sampling University Students accuracy of sample surveys, others complain that Sample Modifi cation political polls take all the suspense out of campaigns by foretelling the result. In recent presiden- Multistage Cluster Sampling tial elections, however, the polls have not removed Multistage Designs and Sampling Error the suspense. Stratifi cation in Multistage Cluster Sampling Going into the 2004 presidential elections, poll- Probability Proportionate to Size (PPS) Sampling sters generally agreed that the election was “too Disproportionate Sampling and Weighting close to call,” a repeat of their experience four years earlier. Table 7-1 reports polls conducted Probability Sampling in Review during the few days preceding the election. Despite The Ethics of Sampling some variations, the overall picture they present is

199 200 CHAPTER 7 THE LOGIC OF SAMPLING

TABLE 7-1 Election Eve Polls Reporting Political polling, like other forms of social re- Percentage of Population Voting for U.S. search, rests on observations. But neither poll- Presidential Candidates, 2004 sters nor other social researchers can observe ev- Date erything that might be relevant to their interests. Poll Begun Bush Kerry A critical part of social research, then, is deciding what to observe and what not. If you want to Fox/OpinDynamics Oct 28 50 50 study voters, for example, which voters should you TIPP Oct 28 53 47 study? CBS/NYT Oct 28 52 48 The process of selecting observations is called ARG Oct 28 50 50 sampling. Although sampling can mean any proce- ABC Oct 28 51 49 dure for selecting units of observation—for exam- Fox/OpinDynamics Oct 29 49 51 ple, interviewing every tenth passerby on a busy Gallup/CNN/USA Oct 29 49 51 street—the key to generalizing from a sample to NBC/WSJ Oct 29 51 49 a larger population is probability sampling, which TIPP Oct 29 51 49 involves the important idea of random selection. Harris Oct 29 52 48 Much of this chapter is devoted to the logic and Democracy Corps Oct 29 49 51 skills of probability sampling. This topic is more Harris Oct 29 51 49 rigorous and precise than some of the other topics CBS Oct 29 51 49 in this book. Whereas social research as a whole Fox/OpinDynamics Oct 30 49 52 is both art and science, sampling leans toward sci- TIPP Oct 30 51 49 ence. Although this subject is somewhat techni- Marist Oct 31 50 50 cal, the basic logic of sampling is not difﬁ cult to GWU Battleground 2004 Oct 31 52 48 understand. In fact, the logical neatness of this Actual vote Nov 2 52 48 topic can make it easier to comprehend than, say, Source: Poll data adapted from the Roper Center, Election 2004 conceptualization. (http://www.ropercenter.uconn.edu/elect_2004/pres_trial_heats Although probability sampling is central to so- .html). Accessed November 16, 2004. I’ve apportioned the undecided and other votes according to the percentages saying cial research today, we’ll also examine a variety they were voting for Bush or Kerry. of nonprobability methods. These methods have their own logic and can provide useful samples for amazingly consistent and pretty well matched the social inquiry. election results. Before we discuss the two major types of sam- Now, how many interviews do you suppose it pling, I’ll introduce you to some basic ideas by way took each of these pollsters to come within a cou- of a brief history of sampling. As you’ll see, the ple of percentage points in estimating the behavior pollsters who correctly predicted the election cliff- of more than 115 million voters? Often fewer than hangers of 2000 and 2004 did so in part because 2,000! In this chapter, we’re going to ﬁ nd out how researchers had learned to avoid some pitfalls that social researchers can achieve such wizardry. earlier pollsters had fallen into. For another powerful illustration of the potency of sampling, look at this graphic portrayal of Presi- A BRIEF HISTORY OF SAMPLING dent George W. Bush’s approval ratings prior to and following the September 11, 2001, terrorist attack on the United States (see Figure 7-1). The Sampling in social research has developed hand data reported by several different polling agencies in hand with political polling. This is the case, no describe the same pattern. doubt, because political polling is one of the few A BRIEF HISTORY OF SAMPLING 201

100 Before After September 11th attack September 11th attack 90

70 Approval rating 60

1/20 2/20 3/20 4/20 5/20 6/20 7/20 8/20 9/20 1/20 2/20 3/20 4/20 5/20 6/20 7/20 8/20 9/20 10/20 11/20 12/20 2001 2002 Date

Key: ABC/Post CBS Harris Ipsos-Reid Pew Bloomberg Fox IBD/CSM NBC/WSJ AmResGp CNN/Time Gallup Zogby Newsweek FIGURE 7-1 Bush Approval: Raw Poll Data. This graph demonstrates how independent polls produce the same picture of reality. This also shows the impact of a national crisis on the president’s popularity: in this case, the 9/11 terrorist attack and President George W. Bush’s popularity. Source: Copyright © 2001, 2002 by drlimerick.com (http://www.pollkatz.homestead.com/ﬁ les/MyHTML2.gif). All rights reserved.

opportunities social researchers have to discover they were planning to vote for in the presidential the accuracy of their estimates. On election day, campaign between Warren Harding and James they fi nd out how well or how poorly they did. Cox. Names were selected for the poll from telephone directories and automobile registration President Alf Landon lists. Based on the postcards sent back, the Digest correctly predicted that Harding would be elected. President Alf Landon? Who’s he? Did you sleep In the elections that followed, the Literary Digest through an entire presidency in your U.S. history expanded the size of its poll and made correct pre- class? No—but Alf Landon would have been presi- dictions in 1924, 1928, and 1932. dent if a famous poll conducted by the Literary Di- In 1936 the Digest conducted its most ambitious gest had proved to be accurate. The Literary Digest poll: Ten million ballots were sent to people listed was a popular newsmagazine published between in telephone directories and on lists of automobile 1890 and 1938. In 1920 Digest editors mailed post- owners. Over two million people responded, giv- cards to people in six states, asking them whom ing the Republican contender, Alf Landon, a stun- 202 CHAPTER 7 THE LOGIC OF SAMPLING ning 57 to 43 percent landslide over the incumbent, poor voted predominantly for Roosevelt’s New President Franklin Roosevelt. The editors modestly Deal recovery program. The Digest’s poll may or cautioned, may not have correctly represented the voting intentions of telephone subscribers and automobile We make no claim to infallibility. We did not owners. Unfortunately for the editors, it decidedly coin the phrase “uncanny accuracy” which has did not represent the voting intentions of the popu- been so freely applied to our Polls. We know lation as a whole. only too well the limitations of every straw vote, however enormous the sample gathered, however scientifi c the method. It would be a You may be able to fi nd the Literary Digest in your library. You can fi nd traces of it by miracle if every State of the forty-eight behaved searching the web. As an alternative, go to on Election Day exactly as forecast by the Poll. http://www.eBay.com and see how many — (LITERARY DIGEST 1936a:6) old issues are available for sale.* Two weeks later, the Digest editors knew the limitations of straw polls even better: The voters gave Roosevelt a second term in offi ce by the President Thomas E. Dewey largest landslide in history, with 61 percent of the The 1936 election also saw the emergence of a vote. Landon won only 8 electoral votes to Roos- young pollster whose name would become synon- evelt’s 523. ymous with public opinion. In contrast to the Liter- The editors were puzzled by their unfortunate ary Digest, George Gallup correctly predicted that turn of luck. Part of the problem surely lay in the 22 Roosevelt would beat Landon. Gallup’s success in percent return rate garnered by the poll. The edi- 1936 hinged on his use of something called quota tors asked, sampling, which we’ll examine later in the chapter. Why did only one in fi ve voters in Chicago to For now, it’s enough to know that quota sampling whom the Digest sent ballots take the trouble to is based on a knowledge of the characteristics of reply? And why was there a preponderance of the population being sampled: what proportion Republicans in the one-fi fth that did reply? . . . are men, what proportion are women, what pro- We were getting better cooperation in what we portions are of various incomes, ages, and so on. have always regarded as a public service from Quota sampling selects people to match a set of Republicans than we were getting from Demo- these characteristics: the right number of poor, crats. Do Republicans live nearer to mailboxes? white, rural men; the right number of rich, African American, urban women; and so on. The quotas Do Democrats generally disapprove of straw are based on those variables most relevant to the polls? — (LITERARY DIGEST 1936b:7) study. In the case of Gallup’s poll, the sample selec- Actually, there was a better explanation—what tion was based on levels of income; the selection is technically called the sampling frame used by the procedure ensured the right proportion of respon- Digest. In this case the sampling frame consisted of dents at each income level. telephone subscribers and automobile owners. In Gallup and his American Institute of Public the context of 1936, this design selected a dispro- Opinion used quota sampling to good effect in portionately wealthy sample of the voting popula- 1936, 1940, and 1944—correctly picking the presi- tion, especially coming on the tail end of the worst economic depression in the nation’s history. The *Each time the Internet icon appears, you’ll be given help- sample effectively excluded poor people, and the ful leads for searching the World Wide Web. NONPROBABILITY SAMPLING 203 dential winner each time. Then, in 1948, Gallup Today, probability sampling remains the primary and most political pollsters suffered the embar- method of selecting large, representative samples rassment of picking Governor Thomas Dewey of for social research, including national political New York over the incumbent, President Harry polls. At the same time, probability sampling can Truman. The pollsters’ embarrassing miscue con- be impossible or inappropriate in many research tinued right up to election night. A famous photo- situations. Accordingly, before turning to the logic graph shows a jubilant Truman—whose followers’ and techniques of probability sampling, we’ll fi rst battle cry was “Give ‘em hell, Harry!”—holding aloft take a look at techniques for nonprobability sam- a newspaper with the banner headline “Dewey De- pling and how they’re used in social research. feats Truman.” Several factors accounted for the pollsters’ failure in 1948. First, most pollsters stopped polling in NONPROBABILITY SAMPLING early October despite a steady trend toward Tru- man during the campaign. In addition, many voters were undecided throughout the campaign, and Social research is often conducted in situations they went disproportionately for Truman when that do not permit the kinds of probability samples they stepped into the voting booth. used in large-scale social surveys. Suppose you More important, Gallup’s failure rested on the wanted to study homelessness: There is no list of unrepresentativeness of his samples. Quota sam- all homeless individuals, nor are you likely to cre- pling—which had been effective in earlier years— ate such a list. Moreover, as you’ll see, there are was Gallup’s undoing in 1948. This technique re- times when probability sampling would not be ap- quires that the researcher know something about propriate even if it were possible. Many such situa- the total population (of voters in this instance). For tions call for nonprobability sampling. national political polls, such information came pri- In this section, we’ll examine four types of non- marily from census data. By 1948, however, World probability sampling: reliance on available sub- War II had produced a massive movement from jects, purposive or judgmental sampling, snowball the country to cities, radically changing the char- sampling, and quota sampling. We’ll conclude acter of the U.S. population from what the 1940 with a brief discussion of techniques for obtaining census showed, and Gallup relied on 1940 cen- information about social groups through the use sus data. City dwellers, moreover, tended to vote of informants. Democratic; hence, the overrepresentation of rural voters in his poll had the effect of underestimating the number of Democratic votes. Reliance on Available Subjects Relying on available subjects, such as stopping Two Types of Sampling Methods people at a street corner or some other location, is an extremely risky sampling method; even so, it’s By 1948 some academic researchers had already used all too frequently. Clearly, this method does been experimenting with a form of sampling based on probability theory. This technique involves the selection of a “random sample” from a list contain- nonprobability sampling Any technique in which ing the names of everyone in the population being samples are selected in some way not suggested by sampled. By and large, the probability-sampling probability theory. Examples include reliance on available methods used in 1948 were far more accurate than subjects as well as purposive (judgmental), snowball, and quota-sampling techniques. quota sampling. 204 CHAPTER 7 THE LOGIC OF SAMPLING not permit any control over the representative- tion than were other physicians, but we can’t say ness of a sample. It’s justifi ed only if the researcher for sure. Although such studies can provide useful wants to study the characteristics of people pass- insights, we must take care not to overgeneralize ing the sampling point at specifi ed times or if less from them. risky sampling methods are not feasible. Even when this method is justifi ed on grounds of fea- Purposive or Judgmental Sampling sibility, researchers must exercise great caution in generalizing from their data. Also, they should alert Sometimes it’s appropriate to select a sample on readers to the risks associated with this method. the basis of knowledge of a population, its ele- University researchers frequently conduct sur- ments, and the purpose of the study. This type of veys among the students enrolled in large lecture sampling is called purposive sampling (or judg- classes. The ease and frugality of this method ex- mental sampling). In the initial design of a ques- plains its popularity, but it seldom produces data of tionnaire, for example, you might wish to select any general value. It may be useful for pretesting a the widest variety of respondents to test the broad questionnaire, but such a sampling method should applicability of questions. Although the study fi nd- not be used for a study purportedly describing stu- ings would not represent any meaningful popula- dents as a whole. tion, the test run might effectively uncover any pe- Consider this report on the sampling design in culiar defects in your questionnaire. This situation an examination of knowledge and opinions about would be considered a pretest, however, rather nutrition and cancer among medical students and than a fi nal study. family physicians: In some instances, you may wish to study a small subset of a larger population in which many The fourth-year medical students of the Univer- members of the subset are easily identifi ed, but the sity of Minnesota Medical School in Minneapolis enumeration of them all would be nearly impos- comprised the student population in this study. sible. For example, you might want to study the The physician population consisted of all physi- leadership of a student protest movement; many of cians attending a “Family Practice Review and the leaders are visible, but it would not be feasible Update” course sponsored by the University of to defi ne and sample all leaders. In studying all or a Minnesota Department of Continuing Medical sample of the most visible leaders, you may collect Education. — (COOPER-STEPHENSON AND THEOLOGIDES data suffi cient for your purposes. 1981:472) Or let’s say you want to compare left-wing and right-wing students. Because you may not be able After all is said and done, what will the results of to enumerate and sample from all such students, this study represent? They do not provide a mean- you might decide to sample the memberships of ingful comparison of medical students and family left- and right-leaning groups, such as the Green physicians in the United States or even in Minne- Party and the Young Americans for Freedom. Al- sota. Who were the physicians who attended the though such a sample design would not provide a course? We can guess that they were probably good description of either left-wing or right-wing more concerned about their continuing educa- students as a whole, it might suffi ce for general comparative purposes. Field researchers are often particularly inter- purposive sampling A type of nonprobability sam- ested in studying deviant cases—cases that do not pling in which the units to be observed are selected on the fi t into patterns of mainstream attitudes and be- basis of the researcher’s judgment about which ones will be the most useful or representative. Also called judgmen- haviors—in order to improve their understanding tal sampling. of the more usual pattern. For example, you might NONPROBABILITY SAMPLING 205 gain important insights into the nature of school snowball sampling to discover a network of to- spirit, as exhibited at a pep rally, by interviewing bacco policy makers in Australia: both those at the people who did not appear to be caught up in the core of the network and those on the periphery. emotions of the crowd or by interviewing students who did not attend the rally at all. Selecting devi- Quota Sampling ant cases for study is another example of purposive study. Quota sampling is the method that helped George Gallup avoid disaster in 1936—and set up the di- Snowball Sampling saster of 1948. Like probability sampling, quota sampling addresses the issue of representative- Another nonprobability-sampling technique, which ness, although the two methods approach the is- some consider to be a form of accidental sam- sue quite differently. pling, is called snowball sampling. This proce- Quota sampling begins with a matrix, or table, dure is appropriate when the members of a special describing the characteristics of the target popula- population are diffi cult to locate, such as homeless tion. Depending on your research purposes, you individuals, migrant workers, or undocumented may need to know what proportion of the popula- immigrants. In snowball sampling, the researcher tion is male and what proportion female as well as collects data on the few members of the target what proportions of each gender fall into various population he or she can locate, then asks those age categories, educational levels, ethnic groups, individuals to provide the information needed and so forth. In establishing a national quota sam- to locate other members of that population whom ple, you might need to know what proportion of they happen to know. “Snowball” refers to the the national population is urban, Eastern, male, process of accumulation as each located subject under 25, white, working class, and the like, and suggests other subjects. Because this procedure all the possible combinations of these attributes. also results in samples with questionable repre- Once you’ve created such a matrix and assigned sentativeness, it’s used primarily for exploratory a relative proportion to each cell in the matrix, you purposes. proceed to collect data from people having all the Suppose you wish to learn a community orga- characteristics of a given cell. You then assign to nization’s pattern of recruitment over time. You all the people in a given cell a weight appropriate might begin by interviewing fairly recent recruits, to their portion of the total population. When all asking them who introduced them to the group. the sample elements are so weighted, the overall You might then interview the people named, ask- data should provide a reasonable representation of ing them who introduced them to the group. You the total population. might then interview those people named, ask- Although quota sampling resembles probability ing, in part, who introduced them. Or, in studying sampling, it has several inherent problems. First, a loosely structured political group, you might ask one of the participants who he or she believes to be the most infl uential members of the group. You snowball sampling A nonprobability-sampling might interview those people and, in the course of method, often employed in fi eld research, whereby each the interviews, ask who they believe to be the most person interviewed may be asked to suggest additional people for interviewing. infl uential. In each of these examples, your sample A type of nonprobability sampling would “snowball” as each of your interviewees quota sampling in which units are selected into a sample on the basis of suggested other people to interview. prespecifi ed characteristics, so that the total sample will In another example, Karen Farquharson (2005) have the same distribution of characteristics assumed to provides a detailed discussion of how she used exist in the population being studied. 206 CHAPTER 7 THE LOGIC OF SAMPLING the quota frame (the proportions that different cells of respondents as people who provide informa- represent) must be accurate, and it is often diffi cult tion about themselves, allowing the researcher to to get up-to-date information for this purpose. The construct a composite picture of the group those Gallup failure to predict Truman as the presidential respondents represent, an informant is a member victor in 1948 stemmed partly from this problem. of the group who can talk directly about the group Second, the selection of sample elements within a per se. given cell may be biased even though its propor- Especially important to anthropologists, infor- tion of the population is accurately estimated. In- mants are important to other social researchers structed to interview fi ve people who meet a given, as well. If you wanted to learn about informal so- complex set of characteristics, an interviewer may cial networks in a local public housing project, for still avoid people living at the top of seven-story example, you would do well to locate individuals walk-ups, having particularly run-down homes, or who could understand what you were looking for owning vicious dogs. and help you fi nd it. In recent years, some researchers have at- When Jeffrey Johnson (1990) set out to study a tempted to combine probability and quota sam- salmon-fi shing community in North Carolina, he pling methods, but the effectiveness of this effort used several criteria to evaluate potential infor- remains to be seen. At present, you should treat mants. Did their positions allow them to interact quota sampling warily if your purpose is statistical regularly with other members of the camp, for ex- description. ample, or were they isolated? (He found that the At the same time, the logic of quota sampling carpenter had a wider range of interactions than can sometimes be applied usefully to a fi eld re- did the boat captain.) Was their information about search project. In the study of a formal group, for the camp limited to their specifi c jobs, or did it example, you might wish to interview both leaders cover many aspects of the operation? These and and nonleaders. In studying a student political other criteria helped determine how useful the po- organization, you might want to interview radi- tential informants might be. cal, moderate, and conservative members of that Usually, you’ll want to select informants who group. You may be able to achieve suffi cient rep- are somewhat typical of the groups you’re study- resentativeness in such cases by using quota sam- ing. Otherwise, their observations and opinions pling to ensure that you interview both men and may be misleading. Interviewing only physicians women, both younger and older people, and so will not give you a well-rounded view of how a forth. community medical clinic is working, for example. Along the same lines, an anthropologist who in- Selecting Informants terviews only men in a society where women are sheltered from outsiders will get a biased view. When fi eld research involves the researcher’s at- Similarly, although informants fl uent in English are tempt to understand some social setting—a ju- convenient for English-speaking researchers from venile gang or local neighborhood, for exam- the United States, they do not typify the members ple—much of that understanding will come from of many societies or even many subgroups within a collaboration with some members of the group English-speaking countries. being studied. Whereas social researchers speak Simply because they’re the ones willing to work with outside investigators, informants will almost always be somewhat “marginal” or atypical within informant Someone well versed in the social phenomenon that you wish to study and who is willing to tell you their group. Sometimes this is obvious. Other what he or she knows about it. Not to be confused with a times, however, you’ll learn about their marginal- respondent. ity only in the course of your research. THE THEORY AND LOGIC OF PROBABILITY SAMPLING 207

contexts in which those actions are embedded. — (LOFLAND ET AL. 2006:15)

In other words, nonprobability sampling does have its uses, particularly in qualitative research projects. But researchers must take care to acknowledge the limitations of nonprobability sampling, especially regarding accurate and precise representations of populations. This point will

Earl Babbie become clearer as we discuss the logic and tech- With so many possible informants, how can the niques of probability sampling. researcher begin to choose?

To see some practical implications of choosing and using informants, visit the In Johnson’s study, the county agent identifi ed website of Canada’s Community Adapta- one fi sherman who seemed squarely in the main- tion and Sustainable Livelihoods (CASL) stream of the community. Moreover, he was co- Program: http://www.iisd.ca/casl operative and helpful to Johnson’s research. The /CASLGuide/KeyInformEx.htm. more Johnson worked with the fi sherman, however, the more he found the man to be a marginal member of the fi shing community. THE THEORY AND LOGIC OF First, he was a Yankee in a southern town. Sec- PROBABILITY SAMPLING ond, he had a pension from the Navy [so he was not seen as a “serious fi sherman” by others in Although appropriate to some research purposes, the community]. . . . Third, he was a major Re- nonprobability-sampling methods cannot guaran- publican activist in a mostly Democratic village. tee that the sample we observed is representative Finally, he kept his boat in an isolated anchor- of the whole population. When researchers want age, far from the community harbor. — (JOHNSON precise, statistical descriptions of large popula- 1990:56) tions—for example, the percentage of the popula- Informants’ marginality may not only bias the tion who are unemployed, plan to vote for Candi- view you get but also limit their access (and hence date X, or feel a rape victim should have the right to yours) to the different sectors of the community an abortion—they turn to probability sampling. you wish to study. All large-scale surveys use probability-sampling These comments should give you some sense methods. of the concerns involved in nonprobability sam- Although the application of probability sampling, typically used in qualitative research proj- pling involves a somewhat sophisticated use of ects. I conclude with the following injunction: statistics, the basic logic of probability sampling is not diffi cult to understand. If all members of a Your overall goal is to collect the richest possible data. By rich data, we mean a wide and diverse range of information collected over a relatively probability sampling The general term for samples prolonged period of time in a persistent and sys- selected in accord with probability theory, typically involv- tematic manner. Ideally, such data enable you to ing some random-selection mechanism. Specifi c types of grasp the meanings associated with the actions probability sampling include EPSEM, PPS, simple random of those you are studying and to understand the sampling, and systematic sampling. 208 CHAPTER 7 THE LOGIC OF SAMPLING

50 a sample that should adequately reﬂ ect variations 44 44 that exist in the population. 40 Conscious and 30 Unconscious Sampling Bias

At ﬁ rst glance, it may look as though sampling is 20 pretty straightforward. To select a sample of 100

Number of people university students, you might simply interview the 10 6 6 fi rst 100 students you fi nd walking around campus. Although untrained researchers often use this kind 0 White African White African of sampling method, it runs a high risk of introduc- women American men American ing biases into the samples. women men In connection with sampling, bias simply means FIGURE 7-2 A Population of 100 Folks. Typically, that those selected are not typical or representa- sampling aims to refl ect the characteristics and tive of the larger populations they have been cho- dynamics of large populations. For the purpose sen from. This kind of bias does not have to be in- of some simple illustrations, let’s assume our total tentional. In fact, it’s virtually inevitable when you population has only 100 members. pick people by the seat of your pants. Figure 7-3 illustrates what can happen when researchers simply select people who are convenient population were identical in all respects—all de- for study. Although women make up 50 percent of mographic characteristics, attitudes, experiences, our micropopulation, the people closest to the re- behaviors, and so on—there would be no need for searcher (in the lower right corner) happen to be careful sampling procedures. In this extreme case 70 percent women, and although the population is of perfect homogeneity, in fact, any single case 12 percent African American, none were selected would suffi ce as a sample to study characteristics into the sample. of the whole population. Beyond the risks inherent in simply studying In fact, of course, the human beings who com- people who are convenient, other problems can pose any real population are quite heterogeneous, arise. To begin, the researcher’s personal leanings varying in many ways. Figure 7-2 offers a simpli- may affect the sample to the point where it does fi ed illustration of a heterogeneous population: not truly represent the student population. Sup- The 100 members of this small population differ pose you’re a little intimidated by students who by gender and race. We’ll use this hypothetical mi- look particularly “cool,” feeling they might ridicule cropopulation to illustrate various aspects of prob- your research effort. You might consciously or un- ability sampling. consciously avoid interviewing such people. Or, The fundamental idea behind probability sam- you might feel that the attitudes of “super-straight- pling is this: In order to provide useful descriptions looking” students would be irrelevant to your re- of the total population, a sample of individuals search purposes and so avoid interviewing them. from a population must contain essentially the Even if you sought to interview a “balanced” same variations that exist in the population. This group of students, you wouldn’t know the exact isn’t as simple as it might seem, however. Let’s take proportions of different types of students making a minute to look at some of the ways researchers up such a balance, and you wouldn’t always be might go astray. Then, we’ll see how probability able to identify the different types just by watching sampling provides an effi cient method for selecting them walk by. THE THEORY AND LOGIC OF PROBABILITY SAMPLING 209

African White American Men The Sample Women

FIGURE 7-3 A Sample of Convenience: Easy, but Not Representative. Selecting and observing those people who are most readily at hand is the simplest method, perhaps, but it’s unlikely to provide a sample that accurately reﬂ ects the total population.

Further, even if you made a conscientious ef- Ironically, the failure of such polls to represent fort to interview, say, every tenth student entering all opinions equally was inadvertently acknowl- the university library, you could not be sure of a edged by Phillip Perinelli (1986), a staff manager representative sample, because different types of of AT&T Communications’ DIAL-IT 900 Service, students visit the library with different frequencies. which offers a call-in poll facility to organizations. Your sample would overrepresent students who Perinelli attempted to counter criticisms by say- visit the library more often than do others. ing, “The 50-cent charge assures that only inter- Similarly, the “public opinion” call-in polls— ested parties respond and helps assure also that in which radio stations or newspapers ask people no individual ‘stuffs’ the ballot box.” We cannot to call specifi ed telephone numbers to register determine general public opinion while consider- their opinions—cannot be trusted to represent ing “only interested parties.” This excludes those general populations. At the very least, not every- who don’t care 50-cents’ worth, as well as those one in the population will even be aware of the who recognize that such polls are not valid. Both poll. This problem also invalidates polls by maga- types of people may have opinions and may even zines and newspapers who publish coupons for vote on election day. Perinelli’s assertion that the readers to complete and mail in. Even among 50-cent charge will prevent ballot stuffi ng actually those who are aware of such polls, not all will ex- means that only those who can afford it will en- press an opinion, especially if doing so will cost gage in ballot stuffi ng. them a stamp, an envelope, or a telephone charge. The possibilities for inadvertent sampling bias Similar considerations apply to polls taken over are endless and not always obvious. Fortunately the Internet. several techniques can help us avoid bias. 210 CHAPTER 7 THE LOGIC OF SAMPLING

Representativeness and Probability of Selection IN THE REAL WORLD REPRESENTATIVE SAMPLING Although the term representativeness has no precise, scientiﬁ c meaning, it carries a common- Representativeness applies to many areas of sense meaning that makes it useful here. For our life, not just survey sampling. Consider qual- purpose, a sample is representative of the popula- ity control, for example. Imagine running a tion from which it is selected if the aggregate char- company that makes light bulbs. You want acteristics of the sample closely approximate those to be sure that they actually light up, but you same aggregate characteristics in the population. can’t test them all. You could, however, de- If, for example, the population contains 50 percent vise a method of selecting a sample of bulbs women, then a sample must contain “close to” 50 drawn from different times in the production percent women to be representative. Later, we’ll day, on different machines, in different fac- discuss “how close” in detail. See the box “Repre- tories, and so forth. sentative Sampling” for more on this. Sometimes the concept of representa- Note that samples need not be representative tive sampling serves as a protection against in all respects; representativeness concerns only overgeneralization, discussed in Chapter 1. those characteristics that are relevant to the sub- Suppose you go to a particular restaurant stantive interests of the study. However, you may and don’t like the food or service. You’re not know in advance which characteristics are ready to cross it off your list of dining possi- relevant. bilities, but then you think about it—perhaps A basic principle of probability sampling is that you hit them on a bad night. Perhaps the chef a sample will be representative of the population had just discovered her boyfriend in bed with from which it is selected if all members of the pop- that “witch” from the Saturday wait staff and ulation have an equal chance of being selected in her mind wasn’t on her cooking. Or perhaps the sample. (We’ll see shortly that the size of the the “witch” was serving your table and kept sample selected also affects the degree of repre- looking over her shoulder to see if anyone sentativeness.) Samples that have this quality are with a meat cleaver was bursting out of the often labeled EPSEM samples (EPSEM stands for kitchen. In short, your ﬁ rst experience might “equal probability of selection method”). Later we’ll not have been representative. discuss variations of this principle, which forms the basis of probability sampling.

Moving beyond this basic principle, we must representativeness That quality of a sample of realize that samples—even carefully selected EP- having the same distribution of characteristics as the population from which it was selected. By implication, de- SEM samples—seldom if ever perfectly represent scriptions and explanations derived from an analysis of the the populations from which they are drawn. Nev- sample may be assumed to represent similar ones in the ertheless, probability sampling offers two special population. Representativeness is enhanced by probability advantages. sampling and provides for generalizability and the use of First, probability samples, although never per- inferential statistics. fectly representative, are typically more represen- EPSEM (equal probability of selection tative than other types of samples, because the bi- method) A sample design in which each member of a population has the same chance of being selected into the ases previously discussed are avoided. In practice, sample. a probability sample is more likely than a nonprob- THE THEORY AND LOGIC OF PROBABILITY SAMPLING 211 ability sample to be representative of the popula- (Americans as of when?). Translating the abstract tion from which it is drawn. “adult New Yorkers” into a workable population Second, and more important, probability theory would require a specifi cation of the age defi ning permits us to estimate the accuracy or represen- adult and the boundaries of New York. Specifying tativeness of the sample. Conceivably, an unin- the term college student would include a consider- formed researcher might, through wholly haphaz- ation of full- and part-time students, degree can- ard means, select a sample that nearly perfectly didates and nondegree candidates, undergraduate represents the larger population. The odds are and graduate students, and so forth. against doing so, however, and we would be un- A study population is that aggregation of ele- able to estimate the likelihood that he or she has ments from which the sample is actually selected. achieved representativeness. The probability sam- As a practical matter, researchers are seldom in a pler, on the other hand, can provide an accurate position to guarantee that every element meeting estimate of success or failure. Shortly we’ll see ex- the theoretical defi nitions laid down actually has actly how this estimate can be achieved. a chance of being selected in the sample. Even I’ve said that probability sampling ensures that where lists of elements exist for sampling pur- samples are representative of the population we poses, the lists are usually somewhat incomplete. wish to study. As we’ll see in a moment, probabil- Some students are always inadvertently omitted ity sampling rests on the use of a random-selection from student rosters. Some telephone subscribers procedure. To develop this idea, though, we need have unlisted numbers. to give more-precise meaning to two important Often, researchers decide to limit their study terms: element and population.* populations more severely than indicated in the An element is that unit about which informa- preceding examples. National polling fi rms may tion is collected and that provides the basis of limit their national samples to the 48 adjacent analysis. Typically, in survey research, elements states, omitting Alaska and Hawaii for practical are people or certain types of people. However, reasons. A researcher wishing to sample psychol- other kinds of units can constitute the elements ogy professors may limit the study population to of social research: Families, social clubs, or cor- those in psychology departments, omitting those porations might be the elements of a study. In a in other departments. Whenever the population given study, elements are often the same as units under examination is altered in such fashion, you of analysis, though the former are used in sample must make the revisions clear to your readers. selection and the latter in data analysis. Up to now we’ve used the term population to Random Selection mean the group or collection that we’re interested in generalizing about. More formally, a popula- With these defi nitions in hand, we can defi ne the tion is the theoretically specifi ed aggregation of ultimate purpose of sampling: to select a set of study elements. Whereas the vague term Ameri- elements from a population in such a way that cans might be the target for a study, the delinea- tion of the population would include the defi nition of the element Americans (for example, citizen- element That unit of which a population is composed ship, residence) and the time referent for the study and which is selected in a sample. Distinguished from units of analysis, which are used in data analysis. population The theoretically specifi ed aggregation of *I would like to acknowledge a debt to Leslie Kish and his the elements in a study. excellent textbook Survey Sampling. Although I’ve modifi ed some of the conventions used by Kish, his presentation is study population That aggregation of elements from easily the most important source of this discussion. which a sample is actually selected. 212 CHAPTER 7 THE LOGIC OF SAMPLING descriptions of those elements accurately portray this danger. Second, and more important, random the total population from which the elements are selection offers access to the body of probability selected. Probability sampling enhances the likeli- theory, which provides the basis for estimating the hood of accomplishing this aim and also provides characteristics of the population as well as esti- methods for estimating the degree of probable mates of the accuracy of samples. Let’s now ex- success. amine probability theory in greater detail. Random selection is the key to this process. In random selection, each element has an equal Probability Theory, chance of selection independent of any other event Sampling Distributions, in the selection process. Flipping a coin is the most and Estimates of Sample Error frequently cited example: Provided that the coin is perfect (that is, not biased in terms of coming up Probability theory is a branch of mathematics that heads or tails), the “selection” of a head or a tail provides the tools researchers need to devise sam- is independent of previous selections of heads or pling techniques that produce representative sam- tails. No matter how many heads turn up in a row, ples and to statistically analyze the results of their the chance that the next fl ip will produce “heads” sampling. More formally, probability theory pro- is exactly 50–50. Rolling a perfect set of dice is an- vides the basis for estimating the parameters of a other example. population. A parameter is the summary descrip- Such images of random selection, though use- tion of a given variable in a population. The mean ful, seldom apply directly to sampling methods in income of all families in a city is a parameter; so is social research. More typically, social researchers the age distribution of the city’s population. When use tables of random numbers or computer pro- researchers generalize from a sample, they’re us- grams that provide a random selection of sampling ing sample observations to estimate population units. A sampling unit is that element or set of parameters. Probability theory enables them both elements considered for selection in some stage of to make these estimates and to arrive at a judg- sampling. In Chapter 9, on survey research, we’ll ment of how likely the estimates will accurately see how computers are used to select random represent the actual parameters in the population. telephone numbers for interviewing, a technique So, for example, probability theory allows pollsters called random-digit dialing. to infer from a sample of 2,000 voters how a popu- There are two reasons for using random-se- lation of 100 million voters is likely to vote—and to lection methods. First, this procedure serves as a specify exactly what the probable margin of error check on conscious or unconscious bias on the in the estimates is. part of the researcher. The researcher who selects Probability theory accomplishes these seem- cases on an intuitive basis might very well select ingly magical feats by way of the concept of sam- cases that would support his or her research ex- pling distributions. A single sample selected from a pectations or hypotheses. Random selection erases population will give an estimate of the population parameter. Other samples would give the same or slightly different estimates. Probability theory tells us about the distribution of estimates that would random selection A sampling method in which each element has an equal chance of selection independent of be produced by a large number of such samples. any other event in the selection process. The logic of sampling error can be applied to sampling unit That element or set of elements consid- different kinds of measurements: mean income or ered for selection in some stage of sampling. mean age, for example. Measurements expressed parameter The summary description of a given variable as percentages, however, provide the simplest in- in a population. troduction to this general concept. THE THEORY AND LOGIC OF PROBABILITY SAMPLING 213

To start, suppose we were to select—at random—a sample of only one person from the ten. Our ten possible samples thus consist of the ten cases shown in Figure 7-4. The ten dots shown on the graph in Figure 7-5 represent these ten samples. Because we’re taking samples of only one, they also represent the “means” we would get as estimates of the population. The distribution of the dots on the graph is called the sampling distribution. Obviously, it wouldn’t be a very good idea to select a sample of only one, because we’ll very likely miss the true mean of $4.50 by quite a bit. Now suppose if we take a sample of two. As Earl Babbie shown in Figure 7-6, increasing the sample size How would researchers conduct a random sample improves our estimations. There are now 45 pos- of this subdivision? What are the pitfalls they would sible samples: [$0 $1], [$0 $2], . . . [$7 $8], [$8 $9]. need to avoid? Moreover, some of those samples produce the same means. For example, [$0 $6], [$1 $5], and To see how this works, we’ll look at two ex- [$2 $4] all produce means of $3. In Figure 7-6, the amples of sampling distributions, beginning with three dots shown above the $3 mean represent a simple example in which our population consists those three samples. of just ten cases. Moreover, the 45 samples are not evenly distributed, as they were when the sample size was The Sampling Distribution of Ten Cases Sup- only one. Rather, they cluster somewhat around pose there are ten people in a group, and each has the true value of $4.50. Only two possible samples a certain amount of money in his or her pocket. deviate by as much as $4 from the true value ([$0 To simplify, let’s assume that one person has no $1] and [$8 $9]), whereas fi ve of the samples give money, another has one dollar, another has two the true estimate of $4.50; another eight samples dollars, and so forth up to the person with nine miss the mark by only 50 cents (plus or minus). dollars. Figure 7-4 presents the population of ten Now suppose we select even larger samples. people.* What do you suppose that will do to our estimates Our task is to determine the average amount of the mean? Figure 7-7 presents the sampling dis- of money one person has: specifi cally, the mean tributions of samples of 3, 4, 5, and 6. number of dollars. If you simply add up the money The progression of sampling distributions is shown in Figure 7-4, you’ll fi nd that the total is $45, clear. Every increase in sample size improves the so the mean is $4.50. Our purpose in the rest of this distribution of estimates of the mean. The limit- exercise is to estimate that mean without actually ing case in this procedure, of course, is to select a observing all ten individuals. We’ll do that by se- sample of ten. There would be only one possible lecting random samples from the population and sample (everyone) and it would give us the true using the means of those samples to estimate the mean of $4.50. As we’ll see shortly, this principle mean of the whole population. applies to actual sampling of meaningful populations. The larger the sample selected, the more ac-

*I want to thank Hanan Selvin for suggesting this method curate it is, as an estimation of the population from of introducing probability sampling. which it was drawn. FIGURE 7-4 A Population of 10 People with $0–$9. Let’s imagine a population of only 10 people with differing amounts of money in their pockets—ranging from $0 to $9.

10 10 9 9 True mean = $4.50 8 8 7 7 6 6 True mean = $4.50 5 5 4 4 (Total = 10) (Total = 45) 3 3 Number of samples 2 Number of samples 2 1 1 0 0 $0$1 $2 $3 $4 $5 $6 $7 $8 $9 $0$1 $2 $3 $4 $5 $6 $7 $8 $9 Estimate of mean Estimate of mean (Sample size = 1) (Sample size = 2) FIGURE 7-5 The Sampling Distribution of FIGURE 7-6 The Sampling Distribution of Samples of 1. In this simple example the mean Samples of 2. After merely increasing our sample amount of money these people have is $4.50 size to 2, the possible samples provide somewhat ($45/10). If we picked 10 different samples of better estimates of the mean. We couldn’t get 1 person each, our “estimates” of the mean either $0 or $9, and the estimates are beginning to would range all across the board. cluster around the true value of the mean: $4.50. a. Samples of 3 b. Samples of 4 True mean = $4.50 True mean = $4.50 20 20

18 18 • • 16 16 ••••• ••••• 14 14 •••••• • ••••••• •• ••••••• ••

(Total = 120) 12 (Total = 210) 12 ••••••• •• 10 •••• 10 •••• •••• •• • •••• •• ••••••••••• •• 8 ••••• •• • 8 ••••••••••• •• •••••••••• ••••••••••• •• 6 ••••• •• •• • 6 •••••• ••• • •• • • • •••••• •• •• •• ••••••••• ••• • • • • • 4 ••••• •• ••• •• •• 4 ••••••••• ••• • • • • • Number of samples •••••••• •• •• •• •• Number of samples •••••• •• ••• • •• • • • • • 2 •••••••••••••••••• 2 ••• •••• •• ••• • •• • • • • •• • ••••••••• •• •• •• •• •• •• • • • ••••• •••• •• •••• •• •••• •• •••• • 0 0 $0$1 $2 $3 $4 $5 $6 $7 $8 $9 $0$1 $2 $3 $4 $5 $6 $7 $8 $9 Estimate of mean Estimate of mean (Sample size = 3) (Sample size = 4)

c. Samples of 5 d. Samples of 6 True mean = $4.50 True mean = $4.50

20 •• 20 •••• 18 •••••• 18 • •••••• • 16 •••••• •• 16 ••••• •••••• •• ••••• 14 •••••••• •• 14 •••••• • •••••••• •• ••••••• •• •••••••• •• ••••••• •• (Total = 252) 12 (Total = 210) 12 •••• •••• •• •• ••••••• •• 10 •••• •••• •• •• 10 •••• •••• •• • ••••• •••• •• ••• ••••••••••• •• 8 ••••• •••• •• ••• 8 ••••••••••• •• •••••• •••• •• •••• ••••••••••• •• 6 •••••• •••• •• •••• 6 •••••• •••• •• ••• •••••••••• •••• •••• ••••••••• •••• •••• 4 •••••••••• •••• •••• 4 ••••••••• •••• •••• Number of samples •••••• •• •••• •• •••• •• Number of samples •••••• •• •••• •• •••• • 2 ••• •••• •• •••• •• •••• •• • 2 ••• •••• •• •••• •• •••• •• •••••••• •••• •• •••• •• •••• •• •••• • ••••• •••• •• •••• •• •••• •• •• 0 0 $0$1 $2 $3 $4 $5 $6 $7 $8 $9 $0$1 $2 $3 $4 $5 $6 $7 $8 $9 Estimate of mean Estimate of mean (Sample size = 5) (Sample size = 6) FIGURE 7-7 The Sampling Distributions of Samples of 3, 4, 5, and 6. As we increase the sample size, the possible samples cluster ever more tightly around the true value of the mean. The chance of extremely inaccurate estimates is reduced at the two ends of the distribution, and the percentage of the samples near the true value keeps increasing. 216 CHAPTER 7 THE LOGIC OF SAMPLING

Sampling Distribution and Estimates of Sam- pling Error Let’s turn now to a more realistic 050100 sampling situation involving a much larger popu- Percent of students approving of the student code lation and see how the notion of sampling distri- FIGURE 7-8 Range of Possible Sample Study bution applies. Assume that we wish to study the Results. Shifting to a more realistic example, let’s student population of State University (SU) to de- assume that we want to sample student attitudes termine the percentage of students who approve concerning a proposed conduct code. Let’s assume that 50 percent of the whole student body ap- or disapprove of a student conduct code proposed proves and 50 percent disapproves—though the by the administration. The study population will be researcher doesn’t know that. the aggregation of, say, 20,000 students contained in a student roster: the sampling frame. The elements will be the individual students at SU. We’ll Sample 2 (51%) select a random sample of, say, 100 students for the purposes of estimating the entire student body. Sample 1 (48%) Sample 3 (52%) The variable under consideration will be attitudes toward the code, a binomial variable comprising • •• the attributes approve and disapprove. (The logic of probability sampling applies to the examination 050100 of other types of variables, such as mean income, Percent of students approving of the student code but the computations are somewhat more compli- FIGURE 7-9 Results Produced by Three cated. Consequently, this introduction focuses on Hypothetical Studies. Assuming a large student binomials.) body, let’s suppose that we selected three different The horizontal axis of Figure 7-8 presents all samples, each of substantial size. We would not necessarily expect those samples to perfectly possible values of this parameter in the popula- reﬂ ect attitudes in the whole student body, but tion—from 0 percent to 100 percent approval. The they should come reasonably close. midpoint of the axis—50 percent—represents half the students approving of the code and the other half disapproving. sure their approval or disapproval of the student To choose our sample, we give each student on code. Perhaps 51 students in the second sample the student roster a number and select 100 ran- approve of the code. We place another dot in the dom numbers from a table of random numbers. appropriate place on the x axis. Repeating this pro- Then we interview the 100 students whose num- cess once more, we may discover that 52 students bers have been selected and ask whether they ap- in the third sample approve of the code. prove or disapprove of the student code. Suppose Figure 7-9 presents the three different sample this operation gives us 48 students who approve statistics representing the percentages of students of the code and 52 who disapprove. This summary in each of the three random samples who ap- description of a variable in a sample is called a proved of the student code. The basic rule of ran- statistic. We present this statistic by placing a dot dom sampling is that such samples, drawn from on the x axis at the point representing 48 percent. a population, give estimates of the parameter that Now let’s suppose we select another sample of exists in the total population. Each of the random 100 students in exactly the same fashion and mea- samples, then, gives us an estimate of the percentage of students in the total student body who approve of the student code. Unhappily, however, we statistic The summary description of a variable in a have selected three samples and now have three sample, used to estimate a population parameter. separate estimates. THE THEORY AND LOGIC OF PROBABILITY SAMPLING 217

• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 80 • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 60 • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •

Number of samples 40 • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 20 • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 0 050100 Percent of students approving of the student code FIGURE 7-10 The Sampling Distribution. If we were to select a large number of good samples, we would expect them to cluster around the true value (50 percent), but given enough such samples, a few would fall far from the mark.

To retrieve ourselves from this problem, let’s clustered around the true value. To put it another draw more and more samples of 100 students each, way, probability theory enables us to estimate the question each of the samples concerning their ap- sampling error—the degree of error to be ex- proval or disapproval of the code, and plot the new pected for a given sample design. This formula sample statistics on our summary graph. In draw- contains three factors: the parameter, the sample ing many such samples, we discover that some of size, and the standard error (a measure of sam- the new samples provide duplicate estimates, as in pling error): the illustration of ten cases. Figure 7-10 shows the sampling distribution of, say, hundreds of samples. P Q s ͱ This is often referred to as a normal curve. n Note that by increasing the number of samples selected and interviewed, we have also increased The symbols P and Q in the formula equal the the range of estimates provided by the sampling population parameters for the binomial: If 60 per- operation. In one sense we have increased our di- cent of the student body approve of the code and lemma in attempting to guess the parameter in the 40 percent disapprove, P and Q are 60 percent and population. Probability theory, however, provides 40 percent, respectively, or 0.6 and 0.4. Note that certain important rules regarding the sampling dis- Q 1 P and P 1 Q. The symbol n equals tribution presented in Figure 7-10. the number of cases in each sample, and s is the First, if many independent random samples standard error. are selected from a population, the sample statis- Let’s assume that the population parameter in tics provided by those samples will be distributed the student example is 50 percent approving of around the population parameter in a known way. the code and 50 percent disapproving. Recall that Thus, although Figure 7-10 shows a wide range of we’ve been selecting samples of 100 cases each. estimates, more of them fall near 50 percent than When these numbers are put into the formula, elsewhere in the graph. Probability theory tells us, then, that the true value is in the vicinity of 50 sampling error The degree of error to be expected percent. in probability sampling. The formula for determining Second, probability theory gives us a formula sampling error contains three factors: the parameter, the for estimating how closely the sample statistics are sample size, and the standard error. 218 CHAPTER 7 THE LOGIC OF SAMPLING we fi nd that the standard error equals 0.05, or ment, we note that the standard error will increase 5 percent. as a function of an increase in the quantity P times In probability theory, the standard error is a Q. Note further that this quantity reaches its maxi- valuable piece of information because it indicates mum in the situation of an even split in the popu- the extent to which the sample estimates will be lation. If P 0.5, PQ 0.25; if P 0.6, PQ 0.24; distributed around the population parameter. (If if P 0.8, PQ 0.16; if P 0.99, PQ 0.0099. By you’re familiar with the standard deviation in sta- extension, if P is either 0.0 or 1.0 (either 0 percent tistics, you may recognize that the standard error, or 100 percent approve of the student code), the in this case, is the standard deviation of the sam- standard error will be 0. If everyone in the popula- pling distribution.) Specifi cally, probability theory tion has the same attitude (no variation), then ev- indicates that certain proportions of the sample es- ery sample will give exactly that estimate. timates will fall within specifi ed increments—each The standard error is also a function of the equal to one standard error—from the population sample size—an inverse function. As the sample parameter. Approximately 34 percent (0.3413) size increases, the standard error decreases. As of the sample estimates will fall within one stan- the sample size increases, more and more sam- dard error increment above the population pa- ples will be clustered nearer to the true value. An- rameter, and another 34 percent will fall within other general guideline is evident in the formula: one standard error below the parameter. In our Because of the square root formula, the standard example, the standard error increment is 5 per- error is reduced by half if the sample size is qua- cent, so we know that 34 percent of our samples drupled. In our present example, samples of 100 will give estimates of student approval between produce a standard error of 5 percent; to reduce 50 percent (the parameter) and 55 percent (one the standard error to 2.5 percent, we must increase standard error above); another 34 percent of the the sample size to 400. samples will give estimates between 50 percent All of this information is provided by established and 45 percent (one standard error below the probability theory in reference to the selection of parameter). Taken together, then, we know that large numbers of random samples. (If you’ve taken roughly two-thirds (68 percent) of the samples will a statistics course, you may know this as the Cen- give estimates within 5 percentage points of the tral Tendency Theorem.) If the population param- parameter. eter is known and many random samples are se- Moreover, probability theory dictates that lected, we can predict how many of the sampling roughly 95 percent of the samples will fall within estimates will fall within specifi ed intervals from plus or minus two standard errors of the true value, the parameter. and 99.9 percent of the samples will fall within plus Recognize that this discussion illustrates only or minus three standard errors. In our present ex- the logic of probability sampling; it does not de- ample, then, we know that only one sample out of scribe the way research is actually conducted. a thousand would give an estimate lower than 35 Usually, we don’t know the parameter: The very percent approval or higher than 65 percent. reason we conduct a sample survey is to estimate The proportion of samples falling within one, that value. Moreover, we don’t actually select large two, or three standard errors of the parameter numbers of samples: We select only one sample. is constant for any random sampling procedure Nevertheless, the preceding discussion of prob- such as the one just described, providing that a ability theory provides the basis for inferences large number of samples are selected. The size about the typical social research situation. Know- of the standard error in any given case, however, ing what it would be like to select thousands of is a function of the population parameter and the samples allows us to make assumptions about the sample size. If we return to the formula for a mo- one sample we do select and study. THE THEORY AND LOGIC OF PROBABILITY SAMPLING 219

Confi dence Levels and Confi dence Intervals appropriate sample size for a study. Once you’ve Whereas probability theory specifi es that 68 per cent decided on the degree of sampling error you can of that fi ctitious large number of samples would tolerate, you’ll be able to calculate the number of produce estimates falling within one standard er- cases needed in your sample. Thus, for example, ror of the parameter, we can turn the logic around if you want to be 95 percent confi dent that your and infer that any single random sample has a 68 study fi ndings are accurate within 5 percentage percent chance of falling within that range. This points of the population parameters, you should observation leads us to the two key components of select a sample of at least 400. (Appendix E is a sampling-error estimates: confi dence level and convenient guide in this regard.) confi dence interval. We express the accuracy This, then, is the basic logic of probability sam- of our sample statistics in terms of a level of confi - pling. Random selection permits the researcher to dence that the statistics fall within a specifi ed inter- link fi ndings from a sample to the body of probabil- val from the parameter. For example, we may say ity theory so as to estimate the accuracy of those we are 95 percent confi dent that our sample statis- fi ndings. All statements of accuracy in sampling tics (for example, 50 percent favor the new student must specify both a confi dence level and a confi - code) are within plus or minus 5 percentage points dence interval. The researcher must report that he of the population parameter. As the confi dence or she is x percent confi dent that the population interval is expanded for a given statistic, our con- parameter lies between two specifi c values. In this fi dence increases. For example, we may say that example, I have demonstrated the logic of sam- we are 99.9 percent confi dent that our statistic falls pling error using a variable analyzed in percent- within three standard errors of the true value. ages. A different statistical procedure would be re- Although we may be confi dent (at some level) quired to calculate the standard error for a mean, of being within a certain range of the parameter, for example, but the overall logic is the same. we’ve already noted that we seldom know what Notice that nowhere in this discussion of sam- the parameter is. To resolve this problem, we sub- ple size and accuracy of estimates did we consider stitute our sample estimate for the parameter in the size of the population being studied. This is the formula; that is, lacking the true value, we sub- because the population size is almost always irrel- stitute the best available guess. evant. A sample of 2,000 respondents drawn prop- The result of these inferences and estimations erly to represent Vermont voters will be no more is that we can estimate a population parameter accurate than a sample of 2,000 drawn properly and also the expected degree of error on the ba- to represent all voters in the United States, even sis of one sample drawn from a population. Be- though the Vermont sample would be a substan- ginning with the question “What percentage of the tially larger proportion of that small state’s voters student body approves of the student code?” you than would the same number chosen to represent could select a random sample of 100 students and the nation’s voters. The reason for this counter- interview them. You might then report that your intuitive fact is that the equations for calculating best estimate is that 50 percent of the student body sampling error all assume that the populations be- approves of the code and that you are 95 percent confi dent that between 40 and 60 percent (plus or minus two standard errors) approve. The range confi dence level The estimated probability that a from 40 to 60 percent is the confi dence interval. population parameter lies within a given confi dence inter- (At the 68 percent confi dence level, the confi dence val. Thus, we might be 95 percent confi dent that between interval would be 45–55 percent.) 35 and 45 percent of all voters favor Candidate A. The logic of confi dence levels and confi dence in- confi dence interval The range of values within tervals also provides the basis for determining the which a population parameter is estimated to lie. 220 CHAPTER 7 THE LOGIC OF SAMPLING ing sampled are infi nitely large, so every sample would equal 0 percent of the whole. IN THE REAL WORLD Of course, this is not literally true in practice. MEDIA SUCCESS WITH THE A sample of 2,000 represents only 0.68 percent CONCEPT OF SAMPLING ERROR of the Vermonters who voted for president in the 2000 election, and a sample of 2,000 U.S. voters The mass media have become increasingly represents a mere 0.002 percent of the national sophisticated in reporting the concept of electorate. Both of these proportions are small sampling error over the years. Notice how enough to approach the situation with infi nitely this report by CNN (2006) distinguishes mar- large populations. gins of error within the whole sample and Further, unless a sample represents, say, 5 per- subsamples: cent or more of the population it is drawn from, that proportion is irrelevant. In those rare cases of A CNN poll released Monday found large proportions being selected, a “fi nite popula- Bush’s approval rating was 34 percent— tional correction” can be calculated to adjust the an uptick of 2 percentage points from confi dence intervals. Simply subtract the propor- the most recent CNN poll in late April. tion from 1.0 and multiply the result times the The president’s disapproval rating sampling error. As you can see, with proportions was 58 percent, down 2 points from the close to zero, this will make no difference. If, on previous poll. the other hand, your sample were half of the popu- The poll, also done by Opinion Re- lation, the sampling error would be cut in half by search Corp., was based on interviews this procedure. In the extreme, if you included the of 1,021 adults. Both shifts are within the whole population in your sample, the sample-to- poll’s sampling error of plus or minus 3 population proportion would be 1.0, and you would percentage points. multiple the calculated standard error by 0.0—sug- More than half of those who disap- gesting there was no sampling error, which would, proved of Bush’s job performance— of course, be the case. (How cool is that?) See the 56 percent—said the war in Iraq was box “Media Success with the Concept of Sampling the reason. Error” for more on this topic. Thirteen percent said the recent Two cautions are in order before we conclude increase in gas prices had fueled their this discussion of the basic logic of probability sam- displeasure. Twenty-six percent gave pling. First, the survey uses of probability theory other reasons. as discussed here are technically not wholly jus- Because that question was asked only tifi ed. The theory of sampling distribution makes of those who disapproved, it had a differ- assumptions that almost never apply in survey ent sampling error—4 percentage points. conditions. The exact proportion of samples con- Source: CNN, 2006 (http://www.cnn.com/2006/ tained within specifi ed increments of standard POLITICS/05/10/congress.poll/index.html). errors, for example, mathematically assumes an Accessed May 11, 2006. infi nitely large population, an infi nite number of samples, and sampling with replacement—that is, every sampling unit selected is “thrown back into the pot” and could be selected again. Sec- I offer these cautions to provide perspective on ond, our discussion has greatly oversimplifi ed the the uses of probability theory in sampling. Social inferential jump from the distribution of several researchers often appear to overestimate the preci- samples to the probable characteristics of one sion of estimates produced by the use of probabil- sample. ity theory. As I’ll mention elsewhere in this chapter POPULATIONS AND SAMPLING FRAMES 221 and throughout the book, variations in sampling grade in public and parochial schools in Yakima techniques and nonsampling factors may further County, Washington. — (PETERSEN AND MAYNARD reduce the legitimacy of such estimates. For exam- 1981:92) ple, those selected in a sample who fail or refuse to The sample at Time 1 consisted of 160 names participate further detract from the representative- drawn randomly from the telephone directory of ness of the sample. Lubbock, Texas. — (TAN 1980:242) Nevertheless, the calculations discussed in this The data reported in this paper . . . were section can be extremely valuable to you in under- gathered from a probability sample of adults standing and evaluating your data. Although the aged 18 and over residing in households in the calculations do not provide estimates as precise as 48 contiguous United States. Personal interviews some researchers might assume, they can be quite with 1,914 respondents were conducted by the valid for practical purposes. They are unquestion- Survey Research Center of the University of ably more valid than less rigorously derived esti- Michigan during the fall of 1975. — (Jackman mates based on less rigorous sampling methods. and Senter 1980:345) Most important, being familiar with the basic logic In each example I’ve italicized the actual sampling underlying the calculations can help you react sen- frames. sibly both to your own data and to those reported Properly drawn samples provide information by others. appropriate for describing the population of elements composing the sampling frame—nothing POPULATIONS AND SAMPLING FRAMES more. I emphasize this point in view of the all- too-common tendency for researchers to select samples from a given sampling frame and then The preceding section introduced the theoretical make assertions about a population similar to, but model for social research sampling. Although as not identical to, the population defi ned by the sam- students, research consumers, and researchers we pling frame. need to understand that theory, appreciating the For example, take a look at this report, which less-than-perfect conditions that exist in the fi eld is discusses the drugs most frequently prescribed by no less important. In this section we’ll look at one U.S. physicians: aspect of fi eld conditions that requires a compromise with idealized theoretical conditions and as- Information on prescription drug sales is not sumptions: the congruence of or disparity between easy to obtain. But Rinaldo V. DeNuzzo, a populations of sampling frames. professor of pharmacy at the Albany College Simply put, a sampling frame is the list or quasi of Pharmacy, Union University, Albany, NY, has list of elements from which a probability sample is been tracking prescription drug sales for 25 selected. If a sample of students is selected from years by polling nearby drugstores. He pub- a student roster, the roster is the sampling frame. lishes the results in an industry trade magazine, If the primary sampling unit for a complex popu- MM&M. lation sample is the census block, the list of cen- DeNuzzo’s latest survey, covering 1980, is sus blocks composes the sampling frame—in the based on reports from 66 pharmacies in 48 com- form of a printed booklet, a magnetic tape fi le, a munities in New York and New Jersey. Unless CD-ROM, or some other medium. Here are some reports of sampling frames appearing in research sampling frame That list or quasi list of units compos- journals: ing a population from which a sample is selected. If the sample is to be representative of the population, it is The data for this research were obtained from a essential that the sampling frame include all (or nearly all) random sample of parents of children in the third members of the population. 222 CHAPTER 7 THE LOGIC OF SAMPLING

there is something peculiar about that part of members; factory workers; fraternity or sorority the country, his fi ndings can be taken as repre- members; members of social, service, or political sentative of what happens across the country. clubs; and members of professional associations. — (MOSKOWITZ 1981:33) The preceding comments apply primarily to local organizations. Often, statewide or national or- What is striking in the excerpt is the casual ganizations do not have a single membership list. comment about whether there is anything pecu- There is, for example, no single list of Episcopalian liar about New York and New Jersey. There is. The church members. However, a slightly more com- lifestyle in these two states hardly typifi es the plex sample design could take advantage of local lifestyles in the other 48. We cannot assume that church membership lists by fi rst sampling churches residents in these large, urbanized, Eastern sea- and then subsampling the membership lists of board states necessarily have the same drug-use those churches selected. (More about that later.) patterns that residents of Mississippi, Nebraska, or Other lists of individuals may be especially rel- Vermont have. evant to the research needs of a particular study. Does the survey even represent prescription Government agencies maintain lists of registered patterns in New York and New Jersey? To determine voters, for example, that might be used if you that, we would have to know something about the wanted to conduct a preelection poll or an in-depth way the 48 communities and the 66 pharmacies examination of voting behavior—but you must in- were selected. We should be wary in this regard, sure that the list is up-to-date. Similar lists contain in view of the reference to “polling nearby drug- the names of automobile owners, welfare recipi- stores.” As we’ll see, there are several methods for ents, taxpayers, business permit holders, licensed selecting samples that ensure representativeness, professionals, and so forth. Although it may be and unless they’re used, we shouldn’t generalize diffi cult to gain access to some of these lists, they from the study fi ndings. provide excellent sampling frames for specialized A sampling frame, then, must be consonant research purposes. with the population we wish to study. In the sim- The sampling elements in a study need not plest sample design, the sampling frame is a list be individual persons. Lists of other types of ele- of the elements composing the study population. ments also exist: universities, businesses of vari- In practice, though, existing sampling frames often ous types, cities, academic journals, newspapers, defi ne the study population rather than the other unions, political clubs, professional associations, way around. That is, we often begin with a pop- and so forth. ulation in mind for our study; then we search for Telephone directories are frequently used for possible sampling frames. Having examined and “quick and dirty” public opinion polls. Undeniably evaluated the frames available for our use, we de- they’re easy and inexpensive to use—no doubt cide which frame presents a study population most the reason for their popularity. And, if you want to appropriate to our needs. make assertions about telephone subscribers, the Studies of organizations are often the simplest directory is a fairly good sampling frame. (Realize, from a sampling standpoint because organizations of course, that a given directory will include neither typically have membership lists. In such cases, the new subscribers nor those who have requested list of members constitutes an excellent sampling unlisted numbers. Sampling is further complicated frame. If a random sample is selected from a mem- by the directories’ inclusion of nonresidential list- bership list, the data collected from that sample ings.) Unfortunately, telephone directories are all may be taken as representative of all members—if too often used as a listing of a city’s population or all members are included in the list. of its voters. Of the many defects in this reason- Populations that can be sampled from good or- ing, the chief one involves a social-class bias. Poor ganizational lists include elementary school, high people are less likely to have telephones; rich peo- school, and university students and faculty; church ple may have more than one line. TYPES OF SAMPLING DESIGNS 223

The class bias inherent in telephone directory Because social research literature gives surpris- samples often remains hidden. Preelection polls ingly little attention to the issues of populations conducted in this fashion are sometimes quite ac- and sampling frames, I’ve devoted special atten- curate, perhaps because of the class bias evident tion to them here by providing a summary of the in voting itself: Poor people are less likely than main guidelines to remember: rich people to vote. Frequently, then, these two bi- 1. Findings based on a sample can be taken as ases nearly coincide, so that the results of a tele- representing only the aggregation of elements phone poll may come very close to the fi nal elec- that compose the sampling frame. tion outcome. Unhappily, you never know for sure 2. Often, sampling frames do not truly include all until after the election. Sometimes, as in the case the elements their names might imply. Omis- of the 1936 Literary Digest poll, you may discover sions are almost inevitable. Thus, a fi rst concern that the voters have not acted according to the ex- of the researcher must be to assess the extent pected class biases. The ultimate disadvantage of of the omissions and to correct them if pos- this method, then, is the researcher’s inability to sible. (Of course, the researcher may feel that estimate the degree of error to be expected in the he or she can safely ignore a small number of sample fi ndings. omissions that cannot easily be corrected.) The growth in popularity of cell phones has be- 3. To be generalized even to the population com- come a new source of concern for survey research- posing the sampling frame, all elements must ers, because cell phone numbers are typically not have equal representation in the frame. Typi- included in phone surveys. Those who use cell cally, each element should appear only once. phones exclusively, moreover, tend to be younger Elements that appear more than once will have and, in 2004, they were more likely to vote for John a greater probability of selection, and the sam- Kerry than were older voters. Scott Keeter (2006) ple will, overall, overrepresent those elements. found, however, that researchers who weighted their results in terms of age avoided bias in this Other, more practical matters relating to popu- respect. lations and sampling frames will be treated else- Street directories and tax maps are often used where in this book. For example, the form of the for easy samples of households, but they may also sampling frame—such as a list in a publication, suffer from incompleteness and possible bias. For 3-by-5 card fi le, CD-ROM, or magnetic tape—can example, in strictly zoned urban regions, illegal affect how easy it is to use. And ease of use may housing units tend not to appear on offi cial re- often take priority over scientifi c considerations: cords. As a result, such units might not be selected, An “easier” list may be chosen over a “harder” one, and sample fi ndings might not be representative even though the latter is more appropriate to the of those units, which are often poorer and more target population. Every researcher should care- overcrowded than average. fully weigh the relative advantages and disadvan- Though the preceding comments apply to the tages of such alternatives. United States, the situation is quite different in some other countries. In Japan, for example, the TYPES OF SAMPLING DESIGNS government maintains quite accurate population registration lists. Moreover, citizens are required by law to update their information, such as changes Up to this point, we’ve focused on simple ran- in residence or births and deaths in the household. dom sampling (SRS). Indeed, the body of statis- As a consequence, you can select simple random tics typically used by social researchers assumes samples of the Japanese population more easily such a sample. As you’ll see shortly, however, you than in the United States. Such a registration list in have several options in choosing your sampling the United States would confl ict directly with this method, and you’ll seldom if ever choose simple country’s norms regarding individual privacy. random sampling. There are two reasons for this. 224 CHAPTER 7 THE LOGIC OF SAMPLING

First, with all but the simplest sampling frame, sim- Figure 7-11 offers a graphic illustration of simple random sampling is not feasible. Second, and ple random sampling. Note that the members of probably surprisingly, simple random sampling our hypothetical micropopulation have been num- may not be the most accurate method available. bered from 1 to 100. Moving to Appendix B, we Let’s turn now to a discussion of simple random decide to use the last two digits of the fi rst column sampling and the other available options. and to begin with the third number from the top. This yields person number 30 as the fi rst one se- Simple Random Sampling lected into the sample. Number 67 is next, and so forth. (Person 100 would have been selected if “00” As noted, simple random sampling is the basic had come up in the list.) sampling method assumed in the statistical computations of social research. Because the mathe- Systematic Sampling matics of random sampling are especially complex, we’ll detour around them in favor of describing the Simple random sampling is seldom used in prac- ways of employing this method in the fi eld. tice. As you’ll see, it’s not usually the most effi cient Once a sampling frame has been properly es- method, and it can be laborious if done manually. tablished, to use simple random sampling the re- Typically, simple random sampling requires a list of searcher assigns a single number to each element elements. When such a list is available, research- in the list, not skipping any number in the process. ers usually employ systematic sampling instead. A table of random numbers (Appendix B) is then In systematic sampling, every kth element in used to select elements for the sample. The box “Us- the total list is chosen (systematically) for inclusion ing a Table of Random Numbers” explains its use. in the sample. If the list contained 10,000 elements If your sampling frame is in a machine-readable and you wanted a sample of 1,000, you would se- form, such as computer disk or magnetic tape, a lect every tenth element for your sample. To en- simple random sample can be selected automati- sure against any possible human bias in using this cally by computer. (In effect, the computer pro- method, you should select the fi rst element at ran- gram numbers the elements in the sampling frame, dom. Thus, in the preceding example, you would generates its own series of random numbers, and begin by selecting a random number between one prints out the list of elements selected.) and ten. The element having that number is included in the sample, plus every tenth element following it. This method is technically referred to as a systematic sample with a random start. Two terms simple random sampling A type of probability sampling in which the units composing a population are are frequently used in connection with systematic assigned numbers. A set of random numbers is then gen- sampling. The sampling interval is the standard erated, and the units having those numbers are included distance between elements selected in the sample: in the sample. ten in the preceding sample. The sampling ratio systematic sampling A type of probability sampling is the proportion of elements in the population that in which every kth unit in a list is selected for inclusion are selected: 1/10 in the example. in the sample—for example, every 25th student in the college directory of students. You compute k by dividing population size the size of the population by the desired sample size; k sampling interval sample size is called the sampling interval. Within certain constraints, systematic sampling is a functional equivalent of simple sample size random sampling and usually easier to do. Typically, the sampling ratio population size fi rst unit is selected at random. sampling interval The standard distance (k) between In practice, systematic sampling is virtually elements selected from a population for a sample. identical to simple random sampling. If the list of sampling ratio The proportion of elements in the elements is indeed randomized before sampling, population that are selected to be in a sample. one might argue that a systematic sample drawn TYPES OF SAMPLING DESIGNS 225

Micropopulation

12345678910111213141516171819202122232425

26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75

76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100

Appendix B: Table of Random Numbers The Sample 10480 15011 01536 22368 46573 25595 241 30 48360 22527 421 67 93093 06243 375 70 39975 81837

779 21 06907 11008 30 67 70 21 62 995 62 72905 56420 963 01 91977 05463 895 79 14342 63661 854 75 36857 53342

289 18 69578 88231 1 79751853 635 53 40961 48235 09429 93969 52636 FIGURE 7-11 A Simple Random Sample. Having numbered everyone in the population, we can use a table of random numbers to select a representative sample from the overall population. Anyone who’s number is chosen from the table is in the sample.

from that list is in fact a simple random sample. There is one danger involved in systematic By now, debates over the relative merits of simple sampling. The arrangement of elements in the list random sampling and systematic sampling have can make systematic sampling unwise. Such an been resolved largely in favor of the latter, simpler arrangement is usually called periodicity. If the list method. Empirically, the results are virtually identi- of elements is arranged in a cyclical pattern that cal. And, as you’ll see in a later section, systematic coincides with the sampling interval, a grossly bi- sampling, in some instances, is slightly more ac- ased sample can be drawn. Here are two examples curate than simple random sampling. that illustrate this danger. 226 CHAPTER 7 THE LOGIC OF SAMPLING

ISSUES AND INSIGHTS USING A TABLE OF RANDOM NUMBERS

In social research, it’s often appropriate to select 3. Now turn to the fi rst page of Appendix B. a set of random numbers from a table such as Notice there are several rows and columns the one in Appendix B. Here’s how to do that. of fi ve-digit numbers, and there are several Suppose you want to select a simple random pages. The table represents a series of ran- sample of 100 people (or other units) out of a dom numbers in the range from 00001 population totaling 980. to 99999. To use the table for your hypothetical sample, you have to answer these 1. To begin, number the members of the popu- questions: lation: in this case, from 1 to 980. Now the problem is to select 100 random numbers. a. How will you create three-digit numbers Once you’ve done that, your sample will out of fi ve-digit numbers? consist of the people having the numbers b. What pattern will you follow in moving you’ve selected. (Note: It’s not essential to through the table to select your numbers? actually number them, as long as you’re c. Where will you start? sure of the total. If you have them in a list, for example, you can always count through Each of these questions has several satis- the list after you’ve selected the numbers.) factory answers. The key is to create a plan 2. The next step is to determine the number and follow it. Here’s an example. of digits you’ll need in the random num- 4. To create three-digit numbers from fi ve- bers you select. In our example, there are digit numbers, let’s agree to select fi ve-digit 980 members of the population, so you’ll numbers from the table but consider only need three-digit numbers to give everyone the left-most three digits in each case. If we a chance of selection. (If there were 11,825 picked the fi rst number on the fi rst page— members of the population, you’d need to 10480—we would only consider the 104. select fi ve-digit numbers.) Thus, we want (We could agree to take the digits farthest to to select 100 random numbers in the range the right, 480, or the middle three digits, 048, from 001 to 980. and any of these plans would work.) The

In a classic study of soldiers during World War II, the sample is drawn from a list of apartments ar- the researchers selected a systematic sample from ranged in numerical order (for example, 101, 102, unit rosters. Every tenth soldier on the roster was 103, 104, 201, 202, and so on), there is a danger of selected for the study. The rosters, however, were the sampling interval coinciding with the number arranged in a table of organizations: sergeants ﬁ rst, of apartments on a ﬂ oor or some multiple thereof. then corporals and privates, squad by squad. Each Then the samples might include only northwest- squad had ten members. As a result, every tenth corner apartments or only apartments near the person on the roster was a squad sergeant. The elevator. If these types of apartments have some systematic sample selected contained only ser- other particular characteristic in common (for ex- geants. It could, of course, have been the case that ample, higher rent), the sample will be biased. The no sergeants were selected for the same reason. same danger would appear in a systematic sample As another example, suppose we select a of houses in a subdivision arranged with the same sample of apartments in an apartment building. If number of houses on a block. TYPES OF SAMPLING DESIGNS 227

key is to make a plan and stick with it. For have 99 more to go. Moving down the sec- convenience, let’s use the left-most three ond column, we select 069, 729, 919, 143, digits. 368, 695, 409, 939, and so forth, continuing 5. We can also choose to progress through in the same column onto the next page. At the tables any way we want: down the col- the bottom of column 2 (the second page of umns, up them, across to the right or to the the table), we select number 017 and con- left, or diagonally. Again, any of these plans tinue to the top of column 3: 015, 255, and will work just fi ne so long as we stick to so on. it. For convenience, let’s agree to move 8. See how easy it is? But trouble could lie down the columns. When we get to the bot- ahead. Say we need more than our 100 tom of one column, we’ll go to the top numbers. When we reach column 5, we are of the next. speeding along, selecting 816, 309, 763, 078, 6. Now, where do we start? You can close 061, 277, 988 . . . Wait a minute! There are your eyes and stick a pencil into the table only 980 students in the senior class. How and start wherever the pencil point lands. can we pick number 988? The solution is (I know it doesn’t sound scientifi c, but it simple: Ignore it. Any time you come across works.) Or, if you’re afraid you’ll hurt the a number that lies outside your range, skip it book or miss it altogether, close your eyes and continue on your way: 188, 174, and so and make up a column number and a row forth. The same solution applies if the same number. (“I’ll pick the number in the fi fth number comes up more than once. If you row of column 2.”) Start with that select 399 again, for example, just ignore it number. the second time. 7. Let’s suppose we decide to start with the 9. That’s it. You keep up the procedure until fi fth number in column 2. If you look on the you’ve selected 100 random numbers. fi rst page of Appendix B, you’ll see that the Returning to your list, your sample consists starting number is 39975. We have selected of person number 399, person number 69, 399 as our fi rst random number, and we person number 729, and so forth.

In considering a systematic sample from a list, Stratifi ed Sampling then, you should carefully examine the nature of So far we have discussed two methods of sam- that list. If the elements are arranged in any par- ple selection from a list: random and systematic. ticular order, you should fi gure out whether that Stratifi cation is not an alternative to these meth- order will bias the sample to be selected, then you ods; rather, it represents a possible modifi cation of should take steps to counteract any possible bias their use. (for example, take a simple random sample from cyclical portions). Usually, however, systematic sampling is supe- stratifi cation The grouping of the units composing a population into homogeneous groups (or strata) before rior to simple random sampling, in convenience if sampling. This procedure, which may be used in conjunc- nothing else. Problems in the ordering of elements tion with simple random, systematic, or cluster sampling, in the sampling frame can usually be remedied improves the representativeness of a sample, at least in quite easily. terms of the variables used for stratifi cation. 228 CHAPTER 7 THE LOGIC OF SAMPLING

Simple random sampling and systematic sam- neous on the stratifi cation variables, they may be pling both ensure a degree of representativeness homogeneous on other variables as well. Because and permit an estimate of the error present. Strati- age is related to college class, a sample stratifi ed fi ed sampling is a method for obtaining a greater by class will be more representative in terms of degree of representativeness by decreasing the age as well, compared with an unstratifi ed sample. probable sampling error. To understand this Because occupational aspirations still seem to be method, we must return briefl y to the basic theory related to gender, a sample stratifi ed by gender will of sampling distribution. be more representative in terms of occupational Recall that sampling error is reduced by two aspirations. factors in the sample design. First, a large sample The choice of stratifi cation variables typically produces a smaller sampling error than does a depends on what variables are available. Gender small sample. Second, a homogeneous popula- can often be determined in a list of names. Univer- tion produces samples with smaller sampling er- sity lists are typically arranged by class. Lists of fac- rors than does a heterogeneous population. If ulty members may indicate their departmental af- 99 percent of the population agrees with a certain fi liation. Government agency fi les may be arranged statement, it’s extremely unlikely that any prob- by geographic region. Voter registration lists are ability sample will greatly misrepresent the extent arranged according to precinct. of agreement. If the population is split 50–50 on the In selecting stratifi cation variables from among statement, then the sampling error will be much those available, however, you should be con- greater. cerned primarily with those that are presumably Stratifi ed sampling is based on this second factor related to variables you want to represent accu- in sampling theory. Rather than selecting a sample rately. Because gender is related to many variables from the total population at large, the researcher and is often available for stratifi cation, it is often ensures that appropriate numbers of elements are used. Education is related to many variables, but it drawn from homogeneous subsets of that popu- is often not available for stratifi cation. Geographic lation. To get a stratifi ed sample of university stu- location within a city, state, or nation is related to dents, for example, you would fi rst organize your many things. Within a city, stratifi cation by geo- population by college class and then draw appro- graphic location usually increases representative- priate numbers of freshmen, sophomores, juniors, ness in social class, ethnic group, and so forth. and seniors. In a nonstratifi ed sample, representa- Within a nation, it increases representativeness in tion by class would be subject to the same sam- a broad range of attitudes as well as in social class pling error as would other variables. In a sample and ethnicity. stratifi ed by class, the sampling error on this vari- When you’re working with a simple list of all able is reduced to zero. elements in the population, two methods of strati- More-complex stratifi cation methods are also fi cation predominate. In one method, you sort the possible. In addition to stratifying by class, you population elements into discrete groups based on might also stratify by gender, by GPA, and so forth. whatever stratifi cation variables are being used. In this fashion you might be able to ensure that On the basis of the relative proportion of the popu- your sample would contain the proper numbers lation represented by a given group, you select— of male sophomores with a 3.5 average, of female randomly or systematically—several elements sophomores with a 4.0 average, and so forth. from that group constituting the same proportion The ultimate function of stratifi cation, then, is of your desired sample size. For example, if sopho- to organize the population into homogeneous sub- more men with a 4.0 average compose 1 percent sets (with heterogeneity between subsets) and to of the student population and you desire a sample select the appropriate number of elements from of 1,000 students, you would select 10 sophomore each. To the extent that the subsets are homoge- men with a 4.0 average. TYPES OF SAMPLING DESIGNS 229

Random start

12345678910111213141516171819202122232425

26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75

76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100

The Sample

3 13 23 33 43 53 63 73 83 93 FIGURE 7-12 A Stratifi ed, Systematic Sample with a Random Start. A stratifi ed, systematic sample involves two stages. First the members of the population are gathered into homogeneous strata; this simple example merely uses gender as a stratifi cation variable but more could be used. Then every kth (in this case, every 10th) person in the stratifi ed arrangement is selected into the sample.

The other method is to group students as de- up our micropopulation according to gender and scribed and then put those groups together in a race. Then, beginning with a random start of “3,” continuous list, beginning with all male freshmen we’ve taken every tenth person thereafter, result- with a 4.0 average and ending with all female se- ing in a list of 3, 13, 23, . . . , 93. niors with a 1.0 or below. You would then select a Stratifi ed sampling ensures the proper repre- systematic sample, with a random start, from the sentation of the stratifi cation variables; this, in entire list. Given the arrangement of the list, a syste- turn, enhances the representation of other vari- matic sample would select proper numbers (within ables related to them. Taken as a whole, then, a an error range of 1 or 2) from each subgroup. (Note: stratifi ed sample is more likely than a simple ran- A simple random sample drawn from such a com- dom sample to be more representative on several posite list would cancel out the stratifi cation.) variables. Although the simple random sample is Figure 7-12 offers a graphic illustration of strati- still regarded as somewhat sacred, it should now fi ed, systematic sampling. As you can see, we lined be clear that you can often do better. 230 CHAPTER 7 THE LOGIC OF SAMPLING

Implicit Stratifi cation of other information such as fi eld of study, class, in Systematic Sampling age, and gender. The computer database, however, contained I mentioned that systematic sampling can, under information on all people who could, by any con- certain conditions, be more accurate than simple ceivable defi nition, be called students, many of random sampling. This is the case whenever the whom seemed inappropriate to the purposes of the arrangement of the list creates an implicit strati- study. As a result, researchers needed to defi ne the fi cation. As already noted, if a list of university study population in a somewhat more restricted students is arranged by class, then a systematic fashion. The fi nal defi nition included those 15,225 sample provides a stratifi cation by class whereas a day-program degree candidates registered for the simple random sample would not. fall semester on the Manoa campus of the univer- In a study of students at the University of Ha- sity, including all colleges and departments, both waii, after stratifi cation by school class, the stu- undergraduate and graduate students, and both dents were arranged by their student identifi cation U.S. and foreign students. The computer program numbers. These numbers, however, were their so- used for sampling, therefore, limited consideration cial security numbers. The fi rst three digits of the to students fi tting this defi nition. social security number indicate the state in which the number was issued. As a result, within a class, Stratifi cation The sampling program also per- students were arranged by the state in which they mitted stratifi cation of students before sample se- were issued a social security number, providing a lection. The researchers decided that stratifi cation rough stratifi cation by geographic origin. by college class would be suffi cient, although the An ordered list of elements, therefore, may be students might have been further stratifi ed within more useful to you than an unordered, random- class, if desired, by gender, college, major, and so ized list. I’ve stressed this point in view of the un- forth. fortunate belief that lists should be randomized before systematic sampling. Only if the arrangement Sample Selection Once the students had been presents the problems discussed earlier should the arranged by class, a systematic sample was se- list be rearranged. lected across the entire rearranged list. The sample size for the study was initially set at 1,100. To Illustration: achieve this sample, the sampling program was set Sampling University Students for a 1/14 sampling ratio. The program generated Let’s put these principles into practice by looking at a random number between 1 and 14; the student an actual sampling design used to select a sample having that number and every 14th student thereof university students. The purpose of the study after was selected in the sample. was to survey, with a mail-out questionnaire, a Once the sample had been selected, the com- representative cross section of students attending puter was instructed to print students’ names and the main campus of the University of Hawaii. The mailing addresses on self-adhesive mailing labels. following sections describe the steps and decisions These labels were then simply transferred to enve- involved in selecting that sample. lopes for mailing the questionnaires.

Study Population and Sampling Frame The Sample Modifi cation obvious sampling frame available for use in this sample selection was the computerized fi le main- This initial design of the sample had to be modifi ed. tained by the university administration. The fi le Before the mailing of questionnaires, the research- contained students’ names, local and permanent ers discovered that unexpected expenses in the addresses, social security numbers, and a variety production of the questionnaires made it impos- MULTISTAGE CLUSTER SAMPLING 231 sible to cover the costs of mailing to all 1,100 stu- bers for study. (For an example, see Glock, Ringer, dents. As a result, one-third of the mailing labels and Babbie 1967.) were systematically selected (with a random start) Another typical situation concerns sampling for exclusion from the sample. The fi nal sample for among population areas such as a city. Although the study was thereby reduced to 733 students. there is no single list of a city’s population, citizens I mention this modifi cation in order to illus- reside on discrete city blocks or census blocks. Re- trate the frequent need to alter a study plan in searchers can therefore select a sample of blocks midstream. Because the excluded students were initially, create a list of people living on each of the systematically omitted from the initial systematic selected blocks, and take a subsample of the peo- sample, the remaining 733 students could still be ple on each block. taken as reasonably representing the study popu- In a more complex design, researchers might lation. The reduction in sample size did, of course, (1) sample blocks, (2) list the households on each increase the range of sampling error. selected block, (3) sample the households, (4) list the people residing in each household, and (5) MULTISTAGE CLUSTER SAMPLING sample the people within each selected household. This multistage sample design leads ultimately to a selection of a sample of individuals but does not The preceding sections have dealt with reason- require the initial listing of all individuals in the ably simple procedures for sampling from lists of city’s population. elements. Such a situation is ideal. Unfortunately, Multistage cluster sampling, then, involves however, much interesting social research re- the repetition of two basic steps: listing and sam- quires the selection of samples from populations pling. The list of primary sampling units (churches, that cannot easily be listed for sampling purposes: blocks) is compiled and, perhaps, stratifi ed for the population of a city, state, or nation; all univer- sampling. Then a sample of those units is selected. sity students in the United States; and so forth. In The selected primary sampling units are then listed such cases the sample design must be much more and perhaps stratifi ed. The list of secondary sam- complex. Such a design typically involves the ini- pling units is then sampled, and so forth. tial sampling of groups of elements—clusters—fol- The listing of households on even the selected lowed by the selection of elements within each of blocks is, of course, a labor-intensive and costly the selected clusters. activity—one of the elements making face-to- Cluster sampling may be used when it’s ei- face household surveys quite expensive. Vincent ther impossible or impractical to compile an ex- Iannacchione, Jennifer Staab, and David Redden haustive list of the elements composing the target (2003) report some initial success using postal population, such as all church members in the mailing lists for this purpose. Although the lists are United States. Often, however, the population el- not perfect, they may be close enough to warrant ements are already grouped into subpopulations, the signifi cant savings in cost. and a list of those subpopulations either exists or Multistage cluster sampling makes possible can be created practically. For example, church those studies that would otherwise be impossible. members in the United States belong to discrete churches, which are either listed or could be. Fol- lowing a cluster-sample format, then, researchers cluster sampling A multistage sampling in which would sample the list of churches in some manner natural groups (clusters) are sampled initially, with the (for example, a stratifi ed, systematic sample). Next, members of each selected group being subsampled afterward. For example, you might select a sample of U.S. they would obtain lists of members from each of colleges and universities from a directory, get lists of the the selected churches. Each of the lists would then students at all the selected schools, then draw samples of be sampled, to provide samples of church mem- students from each. 232 CHAPTER 7 THE LOGIC OF SAMPLING

ISSUES AND INSIGHTS SAMPLING IRAN

Whereas most of the examples given in this 5. The western provinces including western textbook are taken from its country of origin, and eastern Azarbaijan, Zanjan, Ghazvin the United States, the basic methods of sam- and Ardebil pling would apply in other national settings 6. The eastern provinces including Khorasan as well. At the same time, researchers may and Semnan need to make modiﬁ cations appropriate to lo- 7. The northern provinces including Gilan, cal conditions. In selecting a national sample Mazandran and Golestan of Iran, for example, Hamid Abdollahyan and 8. Systan Taghi Azadarmaki (2000:21) from the Uni- 9. Kurdistan versity of Tehran began by stratifying the nation on the basis of cultural differences, divid- Within each of these cultural areas, the researching the country into nine cultural zones as ers selected samples of census blocks and, on follows: each selected block, a sample of households. Their sample design made provisions for getting 1. Tehran the proper numbers of men and women as re- 2. Central region including Isfahan, Arak, spondents within households and provisions for Qum, Yazd and Kerman replacing those households where no one was 3. The southern provinces including at home. Hormozgan, Khuzistan, Bushehr and Fars Source: Hamid Abdollahyan and Taghi Azadarmaki, 4. The marginal western region including Sampling Design in a Survey Research: The Sampling Practice in Iran, paper presented to the meetings of Lorestan, Charmahal and Bakhtiari, the American Sociological Association, Au gust 12– Kogiluyeh and Eelam 16, 2000, Washington, DC.

Specifi c research circumstances often call for spe - elements selected within a given cluster will rep- cial designs, as the box “Sampling Iran” demon- resent all the elements in that cluster only within strates. a range of sampling error. Thus, for example, a researcher runs a certain risk of selecting a sample Multistage Designs of disproportionately wealthy city blocks, plus a and Sampling Error sample of disproportionately wealthy households within those blocks. The best solution to this Although cluster sampling is highly effi cient, the problem lies in the number of clusters selected price of that effi ciency is a less accurate sample. initially and the number of elements within each A simple random sample drawn from a popula- cluster. tion list is subject to a single sampling error, but Typically, researchers are restricted to a total a two-stage cluster sample is subject to two sam- sample size; for example, you may be limited to pling errors. First, the initial sample of clusters will conducting 2,000 interviews in a city. Given this represent the population of clusters only within broad limitation, however, you have several op- a range of sampling error. Second, the sample of tions in designing your cluster sample. At the ex- MULTISTAGE CLUSTER SAMPLING 233 tremes you could choose one cluster and select The general guideline for cluster design, then, 2,000 elements within that cluster, or you could is to maximize the number of clusters selected select 2,000 clusters with one element selected while decreasing the number of elements within within each. Of course, neither approach is advis- each cluster. However, this scientifi c guideline able, but a broad range of choices lies between must be balanced against an administrative con- them. Fortunately, the logic of sampling distribu- straint. The effi ciency of cluster sampling is based tions provides a general guideline for this task. on the ability to minimize the listing of population Recall that sampling error is reduced by two fac- elements. By initially selecting clusters, you need tors: an increase in the sample size and increased only list the elements composing the selected homogeneity of the elements being sampled. clusters, not all elements in the entire popula- These factors operate at each level of a multistage tion. Increasing the number of clusters, however, sample design. A sample of clusters will best rep- goes directly against this effi ciency factor. A small resent all clusters if a large number are selected number of clusters may be listed more quickly and and if all clusters are very much alike. A sample more cheaply than a large number. (Remember of elements will best represent all elements in a that all the elements in a selected cluster must be given cluster if a large number are selected from listed even if only a few are to be chosen in the the cluster and if all the elements in the cluster are sample.) very much alike. The fi nal sample design will refl ect these two With a given total sample size, however, if the constraints. In effect, you’ll probably select as number of clusters is increased, the number of el- many clusters as you can afford. Lest this is- ements within a cluster must be decreased, and sue be left too open-ended at this point, here is vice versa. In the fi rst case, the representativeness one general guideline. Population researchers of the clusters is increased at the expense of more conventionally aim at the selection of 5 house- poorly representing the elements composing each holds per census block. If a total of 2,000 house- cluster. Fortunately, homogeneity can be used to holds are to be interviewed, you would aim at ease this dilemma. 400 blocks with 5 household interviews on each. Typically, the elements composing a given Figure 7-13 presents a graphic overview of this natural cluster within a population are more ho- process. mogeneous than are all elements composing the Before turning to other, more detailed proce- total population. The members of a given church dures available to cluster sampling, let me reiterate are more alike than are all church members; the that this method almost inevitably involves a loss residents of a given city block are more alike than of accuracy. The manner in which this appears, are the residents of a whole city. As a result, rela- however, is somewhat complex. First, as noted tively few elements may be needed to represent a earlier, a multistage sample design is subject to a given natural cluster adequately, although a larger sampling error at each stage. Because the sample number of clusters may be needed to represent size is necessarily smaller at each stage than the adequately the diversity found among the clusters. total sample size, the sampling error at each stage This fact is most clearly seen in the extreme case will be greater than would be the case for a single- of very different clusters composed of identical stage random sample of elements. Second, sam- elements within each. In such a situation, a large pling error is estimated on the basis of observed number of clusters would adequately represent all variance among the sample elements. When those its members. Although this extreme situation never elements are drawn from among relatively homo- exists in reality, it’s closer to the truth in most cases geneous clusters, the estimated sampling error will than its opposite: identical clusters composed of be too optimistic and must be corrected in the light grossly divergent elements. of the cluster sample design. 234 CHAPTER 7 THE LOGIC OF SAMPLING

1st St.

Stage One: Identify blocks and select a sample. 2nd St. (Selected blocks are shaded.)

3rd St. Sage Ave. Boxer Ave. Bridge Ave. Thyme Ave. Parsley Ave. Robinson Ave. Rosemary Ave.

4th St.

5th St.

Stage Two: Go to each selected block and list all households in order. (Example of one listed block.)

1. 491 Rosemary Ave. 16. 408 Thyme Ave. 2. 487 Rosemary Ave. 17. 424 Thyme Ave. 3. 473 Rosemary Ave. 18. 446 Thyme Ave. 4. 455 Rosemary Ave. 19. 458 Thyme Ave. 5. 437 Rosemary Ave. Stage Three: For 20. 480 Thyme Ave. 6. 423 Rosemary Ave. each list, select 21. 498 Thyme Ave. 7. 411 Rosemary Ave. sample of households. 22. 1186 5th St. 8. 403 Rosemary Ave. (In this example, every 23. 1174 5th St. sixth household has 9. 1101 4th St. been selected starting 24. 1160 5th St. 10. 1123 4th St. with #5, which was 25. 1140 5th St. 11. 1137 4th St. selected at random.) 26. 1122 5th St. 12. 1157 4th St. 27. 1118 5th St. 13. 1169 4th St. 28. 1116 5th St. 14. 1187 4th St. 29. 1104 5th St. 15. 402 Thyme Ave. 30. 1102 5th St. FIGURE 7-13 Multistage Cluster Sampling. In multistage cluster sampling, we begin by selecting a sample of the clusters (in this case, city blocks). Then, we make a list of the elements (households, in this case) and select a sample of elements from each of the selected clusters.

Stratifi cation in Multistage Cluster Sampling techniques can be used to refi ne and improve the sample being selected. Thus far, we’ve looked at cluster sampling as The basic options here are essentially the same though a simple random sample were selected as those in single-stage sampling from a list. In seat each stage of the design. In fact, stratifi cation lecting a national sample of churches, for example, MULTISTAGE CLUSTER SAMPLING 235 you might initially stratify your list of churches by individual steps in sampling. That is, each house- denomination, geographic region, size, rural or hold has a 1/10 chance of its block being selected urban location, and perhaps by some measure of and a 1/10 chance of that specifi c household being social class. selected if the block is one of those chosen. Each Once the primary sampling units (churches, household, in this case, has a 1/10 1/10 1/100 blocks) have been grouped according to the chance of selection overall. Because each house- relevant, available stratifi cation variables, either hold would have the same chance of selection, the simple random or systematic sampling techniques sample so selected should be representative of all can be used to select the sample. You might select households in the city. a specifi ed number of units from each group, or There are dangers in this procedure, however. stratum, or you might arrange the stratifi ed clus- In particular, the variation in the size of blocks ters in a continuous list and systematically sample (measured in numbers of households) presents a that list. problem. Let’s suppose that half the city’s popu- To the extent that clusters are combined into lation resides in 10 densely packed blocks fi lled homogeneous strata, the sampling error at this with high-rise apartment buildings, and suppose stage will be reduced. The primary goal of stratifi - that the rest of the population lives in single-fam- cation, as before, is homogeneity. ily dwellings spread out over the remaining 900 Stratifi cation could, of course, take place at blocks. When we fi rst select our sample of 1/10 of each level of sampling. The elements listed within the blocks, it’s quite possible that we’ll miss all of a selected cluster might be stratifi ed before the the 10 densely packed high-rise blocks. No matter next stage of sampling. Typically, however, this is what happens in the second stage of sampling, our not done. (Recall the assumption of relative homo- fi nal sample of households will be grossly unrepre- geneity within clusters.) sentative of the city, comprising only single-family dwellings. Probability Proportionate Whenever the clusters sampled are of greatly to Size (PPS) Sampling differing sizes, it’s appropriate to use a modifi ed sampling design called PPS (probability pro- This section introduces you to a more sophisti- portionate to size). This design guards against cated form of cluster sampling, one that is used the problem I’ve just described and still produces a in many large-scale survey sampling projects. In fi nal sample in which each element has the same the preceding discussion, I talked about selecting a chance of selection. random or systematic sample of clusters and then As the name suggests, each cluster is given a a random or systematic sample of elements within chance of selection proportionate to its size. Thus, each cluster selected. Notice that this produces an a city block with 200 households has twice the overall sampling scheme in which every element chance of selection as one with only 100 house- in the whole population has the same probability holds. Within each cluster, however, a fi xed num- of selection. ber of elements is selected, say, 5 households per Let’s say we’re selecting households within a block. Notice how this procedure results in each city. If there are 1,000 city blocks and we initially household having the same probability of selection select a sample of 100, that means that each block overall. has a 100/1,000 or 0.1 chance of being selected. If we next select 1 household in 10 from those re- This siding on the selected blocks, each household has PPS (probability proportionate to size) refers to a type of multistage cluster sample in which clus- a 0.1 chance of selection within its block. To cal- ters are selected, not with equal probabilities (see EPSEM) culate the overall probability of a household being but with probabilities proportionate to their sizes—as selected, we simply multiply the probabilities at the measured by the number of units to be subsampled. 236 CHAPTER 7 THE LOGIC OF SAMPLING

Let’s look at households of two different city noted that the various sampling procedures result blocks. Block A has 100 households, whereas in an equal chance of selection—even though the Block B has only 10. In PPS sampling, we would ultimate selection probability is the product of sev- give Block A ten times as good a chance of be- eral partial probabilities. ing selected as Block B. So if, in the overall sam- More generally, however, a probability sample is ple design, Block A has a 1/20 chance of being one in which each population element has a known selected, that means Block B would only have a nonzero probability of selection—even though dif- 1/200 chance. Notice that this means that all the ferent elements may have different probabilities. If households on Block A would have a 1/20 chance controlled probability-sampling procedures have of having their block selected; Block B households been used, any such sample may be representative have only a 1/200 chance. of the population from which it is drawn if each If Block A is selected and we’re taking 5 house- sample element is assigned a weight equal to the holds from each selected block, then the house- inverse of its probability of selection. Thus, where holds on Block A have a 5/100 chance of being all sample elements have had the same chance of selected into the block’s sample. Because we can selection, each is given the same weight: 1. This is multiply probabilities in a case like this, we see that called a self-weighting sample. every household on Block A had an overall chance Sometimes it’s appropriate to give some cases of selection equal to 1/20 5/100 5/2000 more weight than others, a process called weight- 1/400. ing. Disproportionate sampling and weighting If Block B happens to be selected, on the other come into play in two basic ways. First, you may hand, its households stand a much better chance of sample subpopulations disproportionately to en- being among the 5 chosen there: 5/10. When this sure suffi cient numbers of cases from each for is combined with their relatively poorer chance of analysis. For example, a given city may have a sub- having their block selected in the fi rst place, how- urban area containing one-fourth of its total popu- ever, they end up with the same chance of selec- lation. Yet you might be especially interested in a tion as those on Block A: 1/200 5/10 5/2000 detailed analysis of households in that area and 1/400. may feel that one-fourth of this total sample size Further refi nements to this design make it a very would be too few. As a result, you might decide effi cient and effective method for selecting large to select the same number of households from the cluster samples. For now, however, it’s enough to suburban area as from the remainder of the city. understand the basic logic involved. Households in the suburban area, then, are given a disproportionately better chance of selection than Disproportionate those located elsewhere in the city. Sampling and Weighting As long as you analyze the two area samples separately or comparatively, you need not worry Ultimately, a probability sample is representative about the differential sampling. If you want to of a population if all elements in the population combine the two samples to create a composite have an equal chance of selection in that sample. picture of the entire city, however, you must take Thus, in each of the preceding discussions, we’ve the disproportionate sampling into account. If n is the number of households selected from each area, then the households in the suburban area weighting Assigning different weights to cases that had a chance of selection equal to n divided by were selected into a sample with different probabilities one-fourth of the total city population. Because of selection. In the simplest scenario, each case is given a the total city population and the sample size are weight equal to the inverse of its probability of selection. When all cases have the same chance of selection, no the same for both areas, the suburban-area house- weighting is necessary. holds should be given a weight of 1/4n, and the MULTISTAGE CLUSTER SAMPLING 237 remaining households should be given a weight of ated,” we know that the female response is based 3/4n. This weighting procedure could be simpli- on a substantial number of cases. That’s good. fi ed by merely giving a weight of 3 to each of the There are problems, however. households selected outside the suburban area. To begin with, subscriber surveys are always (This procedure gives a proportionate representa- problematic. In this case, the best the research- tion to each sample element. The population fi g- ers can hope to talk about is “what subscribers to ure would have to be included in the weighting if Harvard Business Review think.” In a loose way, it population estimates were desired.) might make sense to think of that population as Here’s an example of the problems that can be representing the more sophisticated portion of created when disproportionate sampling is not corporate management. Unfortunately, the over- accompanied by a weighting scheme. When the all response rate was 25 percent. Although that’s Harvard Business Review decided to survey its sub- quite good for subscriber surveys, it’s a low re- scribers on the issue of sexual harassment at work, sponse rate in terms of generalizing from prob- it seemed appropriate to oversample women be- ability samples. cause female subscribers were vastly outnumbered Beyond that, however, the disproportionate by male subscribers. Here’s how G. C. Collins and sample design creates a further problem. When the Timothy Blodgett explained the matter: authors state that 73 percent of respondents favor company policies against harassment (Collins We also skewed the sample another way: to and Blodgett, 1981:78), that fi gure is undoubtedly ensure a representative response from women, too high, because the sample contains a dispro- we mailed a questionnaire to virtually every portionately high percentage of women, who are female subscriber, for a male/female ratio of more likely to favor such policies. Further, when 68% to 32%. This bias resulted in a response of the researchers report that top managers are more 52% male and 44% female (and 4% who gave likely to feel that claims of sexual harassment are no indication of gender)—compared to HBR’s exaggerated than are middle- and lower-level U.S. subscriber proportion of 93% male and 7% managers (1981:81), that fi nding is also suspect. female. — (1981:78) As the researchers report, women are dispropor- Notice a couple of things in this quotation. tionately represented in lower management. That First, it would be nice to know a little more about alone might account for the apparent differences what “virtually every female” means. Evidently, the among levels of management regarding harass- authors of the study didn’t send questionnaires ment. In short, the failure to take account of the to all female subscribers, but there’s no indica- oversampling of women confounds all survey re- tion of who was omitted and why. Second, they sults that do not separate the fi ndings by gender. didn’t use the term representative in its normal The solution to this problem would have been to social science usage. What they mean, of course, weight the responses by gender, as described ear- is that they wanted to get a substantial or “large lier in this section. enough” response from women, and oversam- In the 2000 and 2004 election campaign polls, pling is a perfectly acceptable way of accomplish- survey weighting became a controversial topic, as ing that. some polling agencies weighted their results on By sampling more women than a straightfor- the basis of party affi liation and other variables, ward probability sample would have produced, the whereas others did not. Weighting in this instance authors were able to “select” enough women (812) involved assumptions regarding the differential to compare with the men (960). Thus, when they participation of Republicans and Democrats in report, for example, that 32 percent of the women opinion polls and on election day—plus a determi- and 66 percent of the men agree that “the amount nation of how many Republicans and Democrats of sexual harassment at work is greatly exagger- there were. This will likely remain a topic of de- 238 CHAPTER 7 THE LOGIC OF SAMPLING bate among pollsters and politicians in the years WHAT DO YOU THINK? REVISITED to come.

Contrary to common sense, we have seen Alan Reifman has created a website devoted to a discussion of weighting: that the number of people selected in a sam- http://www.hs.ttu.edu/hdfs3390/ ple, while important, is less important than weighting.htm. how people are selected. The Literary Digest mailed ballots to ten million people and received two million from voters around the PROBABILITY SAMPLING IN REVIEW country. However, the people they selected for their enormous sample—auto owners and telephone subscribers—were not rep- Much of this chapter has been devoted to the resentative of the population in 1936, in the key sampling method used in controlled sur- aftermath of the Great Depression. Overall, vey research: probability sampling. In each of the sample was wealthier than was the vot- the variations examined, we’ve seen that ele- ing population at large. Because rich people ments are chosen for study from a population on are more likely than the general public to a basis of random selection with known nonzero vote Republican, the Literary Digest tallied probabilities. the voting intentions of a disproportionate Depending on the fi eld situation, probability number of Republicans. sampling can be either very simple or extremely The probability-sampling techniques used diffi cult, time consuming, and expensive. What- today allow researchers to select smaller, ever the situation, however, it remains the most ef- more representative samples. Even a couple fective method for the selection of study elements. of thousand respondents, properly selected, There are two reasons for that. can accurately predict the behavior of a hun- First, probability sampling avoids researchers’ dred million voters. conscious or unconscious biases in element selection. If all elements in the population have an equal (or unequal and subsequently weighted) chance of selection, there is an excellent chance detail how social researchers have found ways to that the sample so selected will closely represent deal with this issue. the population of all elements. Second, probability sampling permits estimates THE ETHICS OF SAMPLING of sampling error. Although no probability sample will be perfectly representative in all respects, controlled selection methods permit the researcher to The key purpose of the sampling techniques dis- estimate the degree of expected error. cussed in this chapter is to allow researchers to In this lengthy chapter, we’ve taken on a basic make relatively few observations but gain an ac- issue in much social research: selecting observa- curate picture of a large population. Quantitative tions that will tell us something more general than studies using probability sampling should result the specifi cs we’ve actually observed. This issue in a statistical profi le, based on the sample, that confronts fi eld researchers, who face more action closely mirrors the profi le that would have been and more actors than they can observe and record gained from observing the whole population. In fully, as well as political pollsters who want to pre- addition to using legitimate sampling techniques, dict an election but can’t interview all voters. As researchers should be careful to point out the pos- we proceed through the book, we’ll see in greater sibility of errors: sampling error, fl aws in the sam- MAIN POINTS 239 pling frame, nonresponse error, or anything else ber of that population. Although this is a legitimate that might make the results misleading. and valuable approach, readers may mistake the Sometimes, more typically in qualitative stud- display of differences to refl ect the distribution ies, the purpose of sampling may be to tap into of characteristics in the population. As such, the the breadth of variation within a population rather researcher should ensure that the reader is not than focusing on the “average” or “typical” mem- misled.

Main Points ❏ The most carefully selected sample will never provide a perfect representation of the popula- Introduction tion from which it was selected. There will ❏ Social researchers must select observations always be some degree of sampling error. that will allow them to generalize to people ❏ By predicting the distribution of samples with and events not observed. Often this involves respect to the target parameter, probability- sampling, a selection of people to observe. sampling methods make it possible to estimate ❏ Understanding the logic of sampling is essen- the amount of sampling error expected in a tial to doing social research. given sample. ❏ The expected error in a sample is expressed A Brief History of Sampling in terms of confi dence levels and confi dence ❏ Sometimes you can and should select prob- intervals. ability samples using precise statistical techniques, but at other times nonprobability Populations and Sampling Frames techniques are more appropriate. ❏ A sampling frame is a list or quasi list of the Nonprobability Sampling members of a population. It is the resource ❏ Nonprobability-sampling techniques include used in the selection of a sample. A sample’s reliance on available subjects, purposive representativeness depends directly on the (judgmental sampling), snowball sampling, extent to which a sampling frame contains all and quota sampling. In addition, research- the members of the total population that the ers studying a social group may make use of sample is intended to represent. informants. Each of these techniques has its uses, but none of them ensures that the result- Types of Sampling Designs ing sample is representative of the population ❏ Several sampling designs are available to being sampled. researchers. ❏ Simple random sampling is logically the most The Theory and Logic fundamental technique in probability sam- of Probability Sampling pling, but it is seldom used in practice. ❏ Probability-sampling methods provide an excellent way of selecting representative ❏ Systematic sampling involves the selection of samples from large, known populations. These every kth member from a sampling frame. This methods counter the problems of conscious method is more practical than simple random and unconscious sampling bias by giving each sampling and, with a few exceptions, is func- element in the population a known (nonzero) tionally equivalent. probability of selection. ❏ Stratifi cation, the process of grouping the ❏ The key to probability sampling is random members of a population into relatively homo- selection. geneous strata before sampling, improves the 240 CHAPTER 7 THE LOGIC OF SAMPLING

representativeness of a sample by reducing the informant sampling frame degree of sampling error. nonprobability sampling sampling interval parameter sampling ratio Multistage Cluster Sampling population sampling unit ❏ Multistage cluster sampling is a relatively PPS (probability simple random sampling complex sampling technique that is frequently proportionate to size) snowball sampling used when a list of all the members of a popu- probability sampling statistic purposive sampling stratifi cation lation does not exist. Typically, researchers quota sampling study population must balance the number of clusters and the random selection systematic sampling size of each cluster to achieve a given sample representativeness weighting size. Stratifi cation can be used to reduce the sampling error sampling error involved in multistage cluster sampling. Review Questions ❏ Probability proportionate to size (PPS) is a special, effi cient method for multistage cluster 1. Review the discussion of the 1948 Gallup poll sampling. that predicted that Thomas Dewey would defeat Harry Truman for president. What are ❏ If the members of a population have unequal some ways Gallup could have modifi ed his probabilities of selection into the sample, quota-sample design to avoid the error? researchers must assign weights to the different observations made, in order to provide a 2. Using Appendix B of this book, select a representative picture of the total population. simple random sample of 10 numbers in the Basically, the weight assigned to a particular range from 1 to 9,876. What is each step in sample member should be the inverse of its the process? probability of selection. 3. What are the steps involved in selecting a multistage cluster sample of students taking fi rst- Probability Sampling in Review year English in U.S. colleges and universities? ❏ Probability sampling remains the most effec- 4. In Chapter 9, we’ll discuss surveys conducted tive method for the selection of study elements on the Internet. Can you anticipate possible for two reasons: It allows researchers to avoid problems concerning sampling frames, rep- biases in element selection and it permits resentativeness, and the like? Do you see any estimates of error. solutions? The Ethics of Sampling ❏ Probability sampling always carries a risk of Online Study Resources error; researchers must inform readers of any errors that might make results misleading. ❏ When nonprobability-sampling methods are used to obtain the breadth of variations in a Go to population, researchers must take care not to mislead readers into confusing variations with http://sociology.wadsworth.com/babbie_basics4e what’s typical in the population. and click on ThomsonNow for access to this powerful online study tool. You will get a per- Key Terms sonalized study plan based on your responses to a diagnostic pretest. Once you have mastered cluster sampling element the material with the help of interactive learning confi dence interval EPSEM (equal probability tools, you can take a posttest to confi rm that you confi dence level of selection method) are ready to move on to the next chapter. ADDITIONAL READINGS 241

Website for topics we’ve discussed in this chapter but in a more The Basics of Social Research statistical context. It demonstrates the links between 4th edition probability sampling and statistical analyses. Kalton, Graham. 1983. Introduction to Survey Sampling. At the book companion website (http://sociology Thousand Oaks, CA: Sage. Kalton goes into more .wadsworth.com/babbie_basics4e) you will fi nd of the mathematical details of sampling than the present chapter does but without attempting to be as many resources in addition to ThomsonNow to defi nitive as Kish, described next. aid you in studying for your exams. For example, Kish, Leslie. 1965. Survey Sampling. New York: Wiley. you will fi nd Tutorial Quizzes with feedback, Inter- Unquestionably the defi nitive work on sampling in net Exercises, Flashcards, and Chapter Tutorials, as social research. Kish’s coverage ranges from the sim- well as Extended Projects, InfoTrac College Edition plest matters to the most complex and mathematical, search terms, Social Research in Cyberspace, GSS both highly theoretical and downright practical. Eas- Data, Web Links, and primers for using various ily readable and diffi cult passages intermingle as Kish data analysis software such as SPSS and NVivo. exhausts everything you could want or need to know about each aspect of sampling. Sudman, Seymour. 1983. “Applied Sampling.” Pp. 145–94 Additional Readings in Handbook of Survey Research, edited by Peter H. Rossi, James D. Wright, and Andy B. Anderson. New Frankfort-Nachmias, Chava, and Anna Leon-Guerrero. York: Academic Press. An excellent, practical guide to 1997. Social Statistics for a Diverse Society. Thousand survey sampling. Oaks, CA: Pine Forge Press. See Chapter 11 especially. This statistics textbook covers many of the PartPart ThreeOneThreeOne

MODESAN OF OSERVATIONINTRODUCTION TO INQUIRY

8 Experiments

19 Human Survey Inquiry Research and Science 10 Qualitative Field 2 Paradigms,Research Theory, and Research 11 Unobtrusive 3 TheResearch Politics and Ethics of 12 Social Evaluation Research Research photo credit Paul Conklin/PhotoEdit aving explored the structuring of inquiry in Chapter 12, on evaluation research, looks at a H depth, we’re now ready to dive into the vari- rapidly growing subfi eld in social science, involv- ous observational techniques available to social ing the application of experimental and quasi- scientists. experimental models to the testing of social in- Experiments are usually thought of in connec- terventions in real life. You might use evaluation tion with the physical sciences. In Chapter 8 we’ll research, for example, to test the effectiveness of see how social scientists use experiments. This is a drug rehabilitation program or the effi ciency of the most rigorously controllable of the methods a new school cafeteria. In the same chapter, we’ll we’ll examine. Understanding experiments is also look briefl y at social indicators as a way of assess- a useful way to enhance your understanding of the ing broader social processes. general logic of social scientifi c research. Before we turn to the actual descriptions of Chapter 9 will describe survey research, one of the several methods, two points should be made. the most popular methods in social science. This First, you’ll probably discover that you’ve been us- type of research involves collecting data by asking these scientifi c methods casually in your daily ing people questions—either in self-administered life for as long as you can remember. You use questionnaires or through interviews, which can some form of fi eld research every day. You employ be conducted face-to-face or over the telephone. a crude form of content analysis every time you Chapter 10, on fi eld research, examines perhaps judge an author’s motivation or orientation from the most natural form of data collection used by her or his writings. You engage in at least casual social researchers: the direct observation of social experiments frequently. The chapters in Part 3 will phenomena in natural settings. Some researchers show you how to improve your use of these meth- go beyond mere observation to participate in what ods so as to avoid the pitfalls of casual, uncon- they’re studying, because they want a more inti- trolled observation. mate view and fuller understanding of it. Second, none of the data-collection methods Chapter 11 discusses three forms of unobtru- described in the following chapters is appropriate sive data collection that take advantage of some to all research topics and situations. I’ll try to give of the data available all around us. For example, you some ideas, early in each chapter, regarding content analysis is a method of collecting social when a given method might be appropriate. Still, data by carefully specifying and counting social ar- I could never anticipate all the possible research tifacts such as books, songs, speeches, and paint- topics that might interest you some day. In general, ings. Without making any personal contact with you should always try to use a variety of techniques others, you can use this method to examine a wide in the study of any topic. Because each method has variety of social phenomena. The analysis of exist- its weaknesses, the use of several methods can ing statistics offers another way of studying people help fi ll in any gaps; if the different, independent without having to talk to them. Governments and approaches to the topic all yield the same conclu- a variety of private organizations regularly compile sion, you’ve achieved a form of replication. great masses of data, which you often can use with little or no modifi cation to answer properly posed questions. Finally, historical documents provide a valuable resource for social scientifi c analysis.

243 81 HUMANEXPERIMENTS INQUIRY AND SCIENCE Photo credit Jeff Greenburg/PhotoEdit

What You’ll Learn in thisThis ChapterChapter

ThisWe’ll chapter examine examines the way the people experimental learn about method, their a world mode and of observation the mistakes that they makeenables along researchers the way. to We’ll probe also causal begin relationships. to see what makes Many experimentssience different in social from

otherresearch ways are of conducted knowing things.under the controlled conditions of a laboratory, but 2S experimenters can also take advantage of natural occurrences to study the effects 1S N of events in the social world. L 244 In this chapter . . . WHAT DO YOU THINK?

Introduction The impact of Topics Appropriate to Experiments the observer raises The Classical Experiment many serious ques- Independent and Dependent Variables Image not available due to copyright restrictions tions regarding the Pretesting and Posttesting usefulness of ex- Experimental and Control Groups periments in social The Double-Blind Experiment research. How can the manipulation of Selecting Subjects people in a controlled, experimental envi- Probability Sampling ronment tell us anything about “natural” Randomization human behavior? After all is said and done, Matching doesn’t an experiment simply tell us how Matching or Randomization? people behave when they participate in an Variations on Experimental Design experiment? Preexperimental Research Designs See the “What Do You Think? Revisited” box Validity Issues in Experimental Research toward the end of the chapter. An Illustration of Experimentation

Web-Based Experiments

“Natural” Experiments derstanding more commonly used techniques. Of course, this chapter will also present some of the Strengths and Weaknesses of the inventive ways social scientists have conducted Experimental Method experiments and will demonstrate some basic ex- Ethics and Experiments perimental techniques. At the most basic level, experiments involve (1) taking action and (2) observing the consequences INTRODUCTION of that action. Social researchers typically select a group of subjects, do something to them, and observe the effect of what was done. This chapter addresses the controlled experiment: It’s worth noting that experiments are often a research method commonly associated with the used in nonscientiﬁ c human inquiry. In preparing natural sciences. Although this is not the approach a stew, for example, we add salt, taste, add more most commonly used in the social sciences, Part 3 salt, and taste again. In defusing a bomb, we clip begins with this method because it illustrates fun- the red wire, observe whether the bomb explodes, damental elements in the logic of explanatory re- clip the blue wire, and . . . search. If you can grasp the logic of the controlled We also experiment copiously in our attempt experiment, you’ll ﬁ nd it a useful backdrop for un- to develop generalized understanding about the

245 246 CHAPTER 8 EXPERIMENTS world we live in. All skills are learned through experimentation: eating, walking, riding a bicycle, and so forth. This chapter will discuss some ways social researchers use experiments to develop generalized understandings. We’ll see that, like other methods available to the social researcher, experimenting has its special strengths and weaknesses. Image not available due to copyright restrictions

TOPICS APPROPRIATE TO EXPERIMENTS

Experiments are more appropriate for some topics and research purposes than others. Experiments are especially well suited to research projects involving relatively limited and well-defi ned concepts and propositions. In terms of the traditional image of science, discussed earlier in this book, the experimental model is especially appropriate for handling the task successfully. By observing differ- hypothesis testing. Because experiments focus on ences in the way groups organize themselves and determining causation, they’re also better suited to operate under these varying conditions, we could explanatory than to descriptive purposes. learn a great deal about the nature of small group Let’s assume, for example, that we want to dis- interaction and the factors that infl uence it. For cover ways of reducing prejudice against African example, attorneys sometimes present evidence Americans. We hypothesize that learning about in different ways to different mock juries, to see the contribution of African Americans to U.S. his- which method is the most effective. (You’ll see tory will reduce prejudice, and we decide to test more on this later in the chapter.) this hypothesis experimentally. To begin, we might We typically think of experiments as being test a group of experimental subjects to determine conducted in laboratories. Indeed, most of the ex- their levels of prejudice against African Ameri- amples in this chapter involve such a setting. This cans. Next, we might show them a documentary need not be the case, however. Increasingly, social fi lm depicting the many important ways African researchers are using the World Wide Web as a ve- Americans have contributed to the scientifi c, liter- hicle for conducting experiments. Further, some- ary, political, and social development of the nation. times we can construct what are called natural ex- Finally, we would measure our subjects’ levels of periments: “experiments” that occur in the regular prejudice against African Americans to determine course of social events. The latter portion of this whether the fi lm has actually reduced prejudice. chapter deals with such research. Experimentation has also been successful in the study of small group interaction. Thus, we THE CLASSICAL EXPERIMENT might bring together a small group of experimental subjects and assign them a task, such as making recommendations for popularizing car pools. In both the natural and the social sciences, the We would observe, then, how the group organizes most conventional type of experiment involves itself and deals with the problem. Over the course three major pairs of components: (1) independent of several such experiments, we might systemati- and dependent variables, (2) pretesting and post- cally vary the nature of the task or the rewards for testing, and (3) experimental and control groups. THE CLASSICAL EXPERIMENT 247

This section looks at each of these components might be elements in the operational defi nition and the way they’re put together in the execution of interaction with African Americans in a small of an experiment. group setting. Conventionally, in the experimental model, de- Independent and Dependent Variables pendent and independent variables must be operationally defi ned before the experiment begins. Essentially, an experiment examines the effect of However, as you’ll see in connection with survey an independent variable on a dependent variable. research and other methods, it’s sometimes appro- Typically, the independent variable takes the form priate to make a wide variety of observations dur- of an experimental stimulus, which is either pres- ing data collection and then determine the most ent or absent. That is, the stimulus is a dichoto- useful operational defi nitions of variables during mous variable, having two attributes—present or later analyses. Ultimately, however, experimen- not present. In this typical model, the experimenter tation, like other quantitative methods, requires compares what happens when the stimulus is specifi c and standardized measurements and present to what happens when it is not. observations. In the example concerning prejudice against African Americans, prejudice is the dependent vari- Pretesting and Posttesting able and exposure to African American history is the independent variable. The researcher’s hypothesis In the simplest experimental design, pretesting suggests that prejudice depends, in part, on a lack occurs fi rst, whereby subjects are measured in of knowledge of African American history. The terms of a dependent variable. Then the subjects purpose of the experiment is to test the validity of are exposed to a stimulus representing an inde- this hypothesis by presenting some subjects with pendent variable. Finally, in posttesting, they are an appropriate stimulus, such as a documentary remeasured in terms of the dependent variable. fi lm. In other words, the independent variable is Any differences between the fi rst and last mea- the cause and the dependent variable is the effect. surements on the dependent variable are then at- Thus, we might say that watching the fi lm caused a tributed to the independent variable. change in prejudice or that reduced prejudice was In the example of prejudice and exposure to Af- an effect of watching the fi lm. rican American history, we would begin by pretest- The independent and dependent variables ap- ing the extent of prejudice among our experimen- propriate to experimentation are nearly limitless. tal subjects. Using a questionnaire asking about Moreover, a given variable might serve as an in- attitudes toward African Americans, for example, dependent variable in one experiment and as a we could measure the extent of prejudice exhibited dependent variable in another. For example, preju- by each individual subject and the average prejudice is the dependent variable in the previous ex- dice level of the whole group. After exposing the ample, but it might be the independent variable in subjects to the African American history fi lm, we an experiment examining the effect of prejudice on could administer the same questionnaire again. voting behavior. Responses given in this posttest would permit us To be used in an experiment, both independent and dependent variables must be operationally de- pretesting The measurement of a dependent variable fi ned. Such operational defi nitions might involve a among subjects before they are exposed to a stimulus variety of observational methods. Responses to a representing an independent variable. questionnaire, for example, might be the basis for posttesting The remeasurement of a dependent defi ning prejudice. Speaking to or ignoring African variable among subjects after they’ve been exposed to a Americans, or agreeing or disagreeing with them, stimulus representing an independent variable. 248 CHAPTER 8 EXPERIMENTS to measure the later extent of prejudice for each Experimental Control subject and the average prejudice level of the Group Group group as a whole. If we discovered a lower level of Measure Measure prejudice during the second administration of the Compare: dependent dependent Same? questionnaire, we might conclude that the fi lm had variable variable indeed reduced prejudice. In the experimental examination of attitudes such as prejudice, we face a special practical prob- Administer lem relating to validity. As you may already have experimental stimulus (film) imagined, the subjects might respond differently to the questionnaires the second time even if their attitudes remain unchanged. During the fi rst admin- Remeasure Remeasure Compare: dependent dependent istration of the questionnaire, the subjects may be Different? unaware of its purpose. By the second measure- variable variable ment, they may have fi gured out that the research- FIGURE 8-1 Diagram of Basic Experimental ers are interested in measuring their prejudice. Design. The fundamental purpose of an experi- Because no one wishes to seem prejudiced, the ment is to isolate the possible effect of an indepen- subjects may “clean up” their answers the second dent variable (called the stimulus in experiments) time around. Thus, the fi lm will seem to have re- on a dependent variable. Members of the experimental group(s) are exposed to the stimulus and duced prejudice although, in fact, it has not. those in the control group(s) are not. This is an example of a more general problem that plagues many forms of social research: The very act of studying something may change jects. To begin, we give each group a question- it. The techniques for dealing with this problem in naire designed to measure their prejudice against the context of experimentation will be discussed African Americans. Then we show the fi lm only in various places throughout the chapter. The fi rst to the experimental group. Finally, we administer technique involves the use of control groups. a posttest of prejudice to both groups. Figure 8-1 illustrates this basic experimental design. Using a control group allows the researcher to Experimental and Control Groups detect any effects of the experiment itself. If the Laboratory experiments seldom, if ever, involve posttest shows that the overall level of prejudice ex- only the observation of an experimental group hibited by the control group has dropped as much to which a stimulus has been administered. In ad- as that of the experimental group, then the ap- dition, the researchers observe a control group, parent reduction in prejudice must be a function which does not receive the experimental stimulus. of the experiment or of some external factor rather In the example of prejudice and African Ameri- than a function of the fi lm. If, on the other hand, can history, we might examine two groups of sub- prejudice is reduced only in the experimental group, this reduction would seem to be a conse- experimental group In experimentation, a group quence of exposure to the fi lm, because that’s the of subjects to whom an experimental stimulus is only difference between the two groups. Alterna- administered. tively, if prejudice is reduced in both groups but to control group In experimentation, a group of subjects a greater degree in the experimental group than to whom no experimental stimulus is administered and in the control group, that, too, would be grounds who resemble the experimental group in all other respects. The comparison of the control group and the experimental for assuming that the fi lm reduced prejudice. group at the end of the experiment points to the effect of The need for control groups in social research the experimental stimulus. became clear in connection with a series of stud- THE CLASSICAL EXPERIMENT 249 ies of employee satisfaction, conducted by F. J. tual drug will improve more than those receiving Roethlisberger and W. J. Dickson (1939) in the the placebo. late 1920s and early 1930s. These two research- In social scientifi c experiments, control groups ers were interested in discovering what kinds of provide an important guard against not only the changes in working conditions would improve effects of the experiments themselves but also employee satisfaction and productivity. To pursue the effects of any events outside the laboratory this objective, they studied working conditions in during the experiments. In the example of the the telephone “bank wiring room” of the Western study of prejudice, suppose that a popular Afri- Electric Works in the Chicago suburb of Hawthorne, can American leader is assassinated in the middle Illinois. of, say, a week-long experiment. Such an event To the researchers’ great satisfaction, they dis- might horrify the experimental subjects, requiring covered that improving the working conditions in- them to examine their own attitudes toward Afri- creased satisfaction and productivity consistently. can Americans, with the result of reduced preju- As the workroom was brightened up through bet- dice. Because such an effect should happen about ter lighting, for example, productivity went up. equally for members of the control and experimen- When lighting was further improved, productivity tal groups, a greater reduction of prejudice among went up again. the experimental group would, again, point to the To further substantiate their scientifi c conclu- impact of the experimental stimulus: the docu- sion, the researchers then dimmed the lights. mentary fi lm. Whoops—productivity again improved! Sometimes an experimental design will require At this point it became evident then that the more than one experimental or control group. In wiring-room workers were responding more to the case of the documentary fi lm, for example, we the attention given them by the researchers than might also want to examine the impact of reading to the improved working conditions. As a result of a book on African American history. In that case, this phenomenon, often called the Hawthorne ef- we might have one group see the fi lm and read fect, social researchers have become more sensi- the book; another group only see the movie; still tive to and cautious about the possible effects of another group only read the book; and the con- experiments themselves. In the wiring-room study, trol group do neither. With this kind of design, we the use of a proper control group—one that was could determine the impact of each stimulus sepa- studied intensively without any other changes in rately, as well as their combined effect. the working conditions—would have pointed to the existence of this effect. The Double-Blind Experiment The need for control groups in experimentation has been nowhere more evident than in medical Like patients who improve when they merely think research. Time and again, patients who participate they’re receiving a new drug, sometimes experi- in medical experiments have appeared to improve, menters tend to prejudge results. In medical re- but it has been unclear how much of the improve- search, the experimenters may be more likely to ment has come from the experimental treatment “observe” improvements among patients receiving and how much from the experiment. In testing the experimental drug than among those receiving the effects of new drugs, then, medical research- the placebo. (This would be most likely, perhaps, ers frequently administer a placebo—a “drug” with for the researcher who developed the drug.) A no relevant effect, such as sugar pills—to a control double-blind experiment eliminates this possi- group. Thus, the control-group patients believe they, like the experimental group, are receiving an double-blind experiment An experimental design experimental drug. Often, they improve. If the new in which neither the subjects nor the experimenters know drug is effective, however, those receiving the ac- which is the experimental group and which is the control. 250 CHAPTER 8 EXPERIMENTS bility, because neither the subjects nor the experi- In relation to the norm of generalizability in menters know which is the experimental group science, this tendency clearly represents a poten- and which is the control. In the medical case, those tial defect in social research. Simply put, college researchers responsible for administering the drug undergraduates do not typify the public at large. and for noting improvements would not be told There is a danger, therefore, that we may learn which subjects were receiving the drug and which much about the attitudes and actions of college the placebo. Conversely, the researcher who knew undergraduates but not about social attitudes and which subjects were in which group would not ad- actions in general. minister the experiment. However, this potential defect is less signifi - In social scientifi c experiments, as in medical cant in explanatory research than in descriptive ones, the danger of experimenter bias is further research. True, having noted the level of prejudice reduced to the extent that the operational defi ni- among a group of college undergraduates in our tions of the dependent variables are clear and pretesting, we would have little confi dence that precise. For example, medical researchers would the same level existed among the public at large. be less likely to unconsciously bias their reading On the other hand, if we found that a documentary of a patient’s temperature than they would to bias fi lm reduced whatever level of prejudice existed their assessment of how lethargic the patient was. among those undergraduates, we would have Similarly, the small group researcher would be less more confi dence—without being certain—that it likely to misperceive which subject spoke, or to would have a similar effect in the community at whom he or she spoke, than whether the subject’s large. Social processes and patterns of causal re- comments sounded cooperative or competitive, a lationships appear to be more generalizable and more subjective judgment that’s diffi cult to defi ne more stable than specifi c characteristics such as an in precise behavioral terms. individual’s level of prejudice. As I’ve indicated several times, seldom can we Aside from the question of generalizability, the devise operational defi nitions and measurements cardinal rule of subject selection and experimen- that are wholly precise and unambiguous. This tation concerns the comparability of experimen- is another reason why employing a double-blind tal and control groups. Ideally, the control group design in social research experiments might be represents what the experimental group would appropriate. have been like if it had not been exposed to the experimental stimulus. The logic of experiments SELECTING SUBJECTS requires, therefore, that experimental and control groups be as similar as possible. There are several ways to accomplish this. In Chapter 7 we discussed the logic of sampling, which involves selecting a sample that is repre- Probability Sampling sentative of some populations. Similar considerations apply to experiments. Because most social The discussions of the logic and techniques of prob- researchers work in colleges and universities, it ability sampling in Chapter 7 provide one method seems likely that most social research laboratory for selecting two groups that are similar to each experiments are conducted with college under- other. Beginning with a sampling frame composed graduates as subjects. Typically, the experimenter of all the people in the population under study, the asks students enrolled in his or her classes to par- researcher might select two probability samples. If ticipate in experiments or advertises for subjects these samples each resemble the total population in a college newspaper. Subjects may or may not from which they’re selected, they’ll also resemble be paid for participating in such experiments. (See each other. Chapter 3 for more on the ethical issues involved Recall also, however, that the degree of resem- in this situation.) blance (representativeness) achieved by probabil- SELECTING SUBJECTS 251 ity sampling is largely a function of the sample and one as the control, there would be no reason size. As a general guideline, probability samples of to assume that the two subjects are similar to each less than 100 are not likely to be representative, other. With larger numbers of subjects, however, and social scientifi c experiments seldom involve randomization makes good sense. that many subjects in either experimental or control groups. As a result, then, probability sampling Matching is seldom used in experiments to select subjects from a larger population. Researchers do, how- Another way to achieve comparability between ever, use the logic of random selection when they the experimental and control groups is through assign subjects to groups. matching. This process is similar to the quota- sampling methods discussed in Chapter 7. If 12 Randomization of our subjects are young white men, we might assign 6 of those at random to the experimental Having recruited, by whatever means, a total group and the other 6 to the control group. If 14 are group of subjects, the experimenter may randomly middle-aged African American women, we might assign those subjects to either the experimental assign 7 to each group. or the control group. Such randomization might The overall matching process could be most ef- be accomplished by numbering all of the subjects fi ciently achieved through the creation of a quota serially and selecting numbers by means of a ran- matrix constructed of all the most relevant charac- dom number table, or the experimenter might as- teristics. Figure 8-2 provides a simplifi ed illustration sign the odd-numbered subjects to the experimen- of such a matrix. In this example, the experimenter tal group and the even-numbered subjects to the has decided that the relevant characteristics are control group. race, age, and sex. Ideally, the quota matrix is con- Let’s return again to the basic concept of prob- structed to result in an even number of subjects in ability sampling. If we recruit 40 subjects (in re- each cell of the matrix. Then, half the subjects in sponse to a newspaper advertisement, for ex- each cell go into the experimental group and half ample), there’s no reason to believe that the 40 into the control group. subjects represent the entire population from Alternatively, we might recruit more subjects which they’ve been drawn. Nor can we assume than our experimental design requires. We might that the 20 subjects randomly assigned to the ex- then examine many characteristics of the large ini- perimental group represent that larger population. tial group of subjects. Whenever we discover a pair We can have greater confi dence, however, that the of quite similar subjects, we might assign one at 20 subjects randomly assigned to the experimental random to the experimental group and the other to group will be reasonably similar to the 20 assigned the control group. Potential subjects who are un- to the control group. like anyone else in the initial group might be left Following the logic of our earlier discussions out of the experiment altogether. of sampling, we can see our 40 subjects as a Whatever method we employ, the desired re- population from which we select two probability sult is the same. The overall average description samples—each consisting of half the population. of the experimental group should be the same as Because each sample refl ects the characteristics of the total population, the two samples will mirror each other. randomization A technique for assigning experimental subjects to experimental and control groups randomly. As we saw in Chapter 7, our assumption of In connection with experiments, the proce- similarity in the two groups depends in part on the matching dure whereby pairs of subjects are matched on the basis number of subjects involved. In the extreme case, of their similarities on one or more variables, and one if we recruited only two subjects and assigned, by member of the pair is assigned to the experimental group the fl ip of a coin, one as the experimental subject and the other to the control group. 252 CHAPTER 8 EXPERIMENTS

Men Women

African African White White American American

Under 30 years 8 12 10 16

30 to 50 years 18 3014 28

Over 50 years 12 20 12 22

Experimental group Control group 6 6 7 7 etc. etc. FIGURE 8-2 Quota Matrix Illustration. Sometimes the experimental and control groups are created by fi nding pairs of matching subjects and assigning one to the experimental group and the other to the control group. that of the control group. For example, they should the two groups exhibit the same overall level of have about the same average age, the same gen- prejudice. der composition, the same racial composition, and so forth. This test of comparability should be used Matching or Randomization? whether the two groups are created through probability sampling or through randomization. When assigning subjects to the experimental and Thus far, I’ve referred to the “relevant” vari- control groups, you should be aware of two ar- ables without saying clearly what those variables guments in favor of randomization over match- are. Of course, I can’t give a defi nite answer to this ing. First, you may not be in a position to know in question, any more than I could specify in Chap- advance which variables will be relevant for the ter 7 which variables should be used in stratifi ed matching process. Second, most of the statistics sampling. Which variables are relevant ultimately used to analyze the results of experiments assume depends on the nature and purpose of an experi- randomization. Failure to design your experiment ment. As a general rule, however, the control and that way, then, makes your later use of those sta- experimental groups should be comparable in tistics less meaningful. terms of those variables most likely to be related On the other hand, randomization only makes to the dependent variable under study. In a study sense if you have a fairly large pool of subjects, so of prejudice, for example, the two groups should that the laws of probability sampling apply. With be alike in terms of education, ethnicity, and age, only a few subjects, matching would be a better among other characteristics. In some cases, more- procedure. over, we may delay assigning subjects to experi- Sometimes researchers can combine match- mental and control groups until we have initially ing and randomization. When conducting an ex- measured the dependent variable. Thus, for exam- periment in the educational enrichment of young ple, we might administer a questionnaire measur- adolescents, for example, Milton Yinger and his ing subjects’ prejudice and then match the experi- colleagues (1977) needed to assign a large num- mental and control groups to assure ourselves that ber of students, aged 13 and 14, to several differ- VARIATIONS ON EXPERIMENTAL DESIGN 253

see the box “Daily Experimentation” to consider IN THE REAL WORLD how the tools of experimentation can affect a wide DAILY EXPERIMENTATION range of everyday activities.

I mentioned at the outset of this chapter that VARIATIONS ON EXPERIMENTAL DESIGN we engage in experimentation all the time in our everyday lives, whether trying to lose weight, develop better study habits, or de- In their classic book on research design, Donald bug a computer problem. However, we of- Campbell and Julian Stanley (1963) describe 16 ten do this a bit sloppily. For example, you different experimental and quasi-experimental de- might try three different fi xes for the com- signs. This section describes some of these varia- puter problem at the same time. If it happens tions to help show the potential for experimenta- to work, you won’t know which action did tion in social research. the trick and won’t really understand what the problem was. Or you may fi nd yourself Preexperimental Research Designs trying the same solution repeatedly because you can’t remember if you already tried it. To begin, Campbell and Stanley discuss three “pre- Perhaps your study of social science experi- experimental” designs, not to recommend them ments will help you make such nonresearch but because they’re frequently used in less-than- experimentation more rigorous and effi cient. professional research. In the fi rst such design—the Keeping records will help, for example. one-shot case study—a single group of subjects is measured on a dependent variable following the administration of some experimental stimulus. Suppose, for example, that we show the previ- ent experimental and control groups to ensure the ously mentioned African American history fi lm to comparability of students composing each of the a group of people and then administer a question- groups. They achieved this goal by the following naire that seems to measure prejudice against Af- method. rican Americans. Suppose further that the answers Beginning with a pool of subjects, the research- given to the questionnaire seem to represent a ers fi rst created strata of students nearly identi- low level of prejudice. We might be tempted to cal to one another in terms of some 15 variables. conclude that the fi lm reduced prejudice. Lacking From each of the strata, students were randomly a pretest, however, we can’t be sure. Perhaps the assigned to the different experimental and control questionnaire doesn’t really represent a very sen- groups. In this fashion, the researchers actually sitive measure of prejudice, or perhaps the group improved on conventional randomization. Essen- we’re studying was low in prejudice to begin with. tially, they used a stratifi ed sampling procedure In either case, the fi lm might have made no differ- (recall Chapter 7), except that they employed far ence, though our experimental results might mis- more stratifi cation variables than are typically used lead us into thinking it did. in, say, survey sampling. The second preexperimental design discussed Thus far I’ve described the classical experi- by Campbell and Stanley adds a pretest for the ex- ment—the experimental design that best repre- perimental group but lacks a control group. This sents the logic of causal analysis in the laboratory. design—which the authors call the one-group pre- In practice, however, social researchers use a great test-posttest design—suffers from the possibility variety of experimental designs. In the next sec- that some factor other than the independent vari- tion, we’ll look at some of these approaches. To able might cause a change between the pretest cap our general discussion of experimental design, and posttest results, such as the assassination of a 254 CHAPTER 8 EXPERIMENTS respected African American leader. Thus, although The one-group pretest-posttest design offers we can see that prejudice has been reduced, we somewhat better evidence that exercise produces can’t be sure the fi lm caused that reduction. weight loss. Specifi cally, we’ve ruled out the possi- To round out the possibilities for preexperi- bility that the man was thin before beginning to ex- mental designs, Campbell and Stanley point out ercise. However, we still have no assurance that it that some research is based on experimental and was his exercising that caused him to lose weight. control groups but has no pretests. They call this Finally, the static-group comparison eliminates design the static-group comparison. For example, the problem of our questionable defi nition of what we might show the African American history fi lm constitutes trim or overweight body shapes. In this to one group but not to another and then measure case, we can compare the shapes of the man who prejudice in both groups. If the experimental group exercises and the one who does not. This design, had less prejudice at the conclusion of the experi- however, reopens the possibility that the man who ment, we might assume the fi lm was responsible. exercises was thin to begin with. But unless we had randomized our subjects, we would have no way of knowing that the two groups had the same degree of prejudice initially; perhaps Validity Issues in Experimental Research the experimental group started out with less. At this point I want to present in a more system- Figure 8-3 graphically illustrates these three atic way the factors that affect experimental re- preexperimental research designs, using a differ- search—those I’ve already discussed as well as ad- ent research question: Does exercise cause weight ditional factors. First we’ll look at what Campbell reduction? To make the several designs clearer, and Stanley call the sources of internal invalidity, the fi gure shows individuals rather than groups, reviewed and expanded in a follow-up book by but the same logic pertains to group comparisons. Thomas Cook and Donald Campbell (1979). Then Let’s review the three preexperimental designs in we’ll consider the problem of generalizing experi- this new example. mental results to the “real” world, referred to as The one-shot study design represents a com- external invalidity. Having examined these, we’ll be mon form of logical reasoning in everyday life. in a position to appreciate the advantages of some Asked whether exercise causes weight reduction, of the more sophisticated experimental and quasi- we may bring to mind an example that would experimental designs social science researchers seem to support the proposition: someone who sometimes use. exercises and is thin. There are problems with this reasoning, however. Perhaps the person was thin Sources of Internal Invalidity The problem of long before beginning to exercise. Or, perhaps he internal invalidity refers to the possibility that became thin for some other reason, such as eating the conclusions drawn from experimental results less or getting sick. The observations shown in the may not accurately refl ect what has gone on in diagram do not guard against these other possi- the experiment itself. The threat of internal inva- bilities. Moreover, the observation that the man in lidity is present whenever anything other than the the diagram is in trim shape depends on our intui- experimental stimulus can affect the dependent tive idea of what constitutes trim and overweight variable. body shapes. All told, this is very weak evidence Campbell and Stanley (1963:5–6) and Cook and for testing the relationship between exercise and Campbell (1979:51–55) point to several sources of weight loss. internal invalidity. Here are eight, to illustrate this concern: internal invalidity Refers to the possibility that the conclusions drawn from experimental results may not ac- 1. History. During the course of the experiment, curately refl ect what went on in the experiment itself. historical events may occur that confound the VARIATIONS ON EXPERIMENTAL DESIGN 255

Comparison One-Shot Case Study

A man who exercises Some intuitive is observed to be in standard of trim shape what constitutes a trim shape Time 1 Time 2 Time 3

One-Group Pretest-Posttest Design Comparison An overweight man who exercises is later observed to be in trim shape

Time 1 Time 2 Time 3

Static-Group Comparison A man who exercises is observed to be in trim shape whereas one who doesn’t is observed to be overweight Comparison

Time 1 Time 2 Time 3 FIGURE 8-3 Three Preexperimental Research Designs. These preexperimental designs anticipate the logic of true experiments but leave themselves open to errors of interpretation. Can you see the errors that might be made in each of these designs? The various risks are solved by the addition of control groups, pretesting, and posttesting.

experimental results. The assassination of an of the experiment. In a long-term experiment, African American leader during the course the fact that the subjects grow older (and of an experiment on reducing anti–African wiser?) can have an effect. In shorter experi- American prejudice is one example. ments, they can grow tired, sleepy, bored, or 2. Maturation. People are continually growing and hungry or change in other ways that affect changing, and such changes affect the results their behavior in the experiment. 256 CHAPTER 8 EXPERIMENTS

3. Testing. Often the process of testing and re- that extremely tall people as a group are likely testing infl uences people’s behavior, thereby to have children shorter than themselves, and confounding the experimental results. Suppose extremely short people as a group are likely to we administer a questionnaire to a group as have children taller than themselves. There is a a way of measuring their prejudice. Then we danger, then, that changes occurring by virtue administer an experimental stimulus and re- of subjects starting out in extreme positions measure their prejudice. As we saw earlier, by will be attributed erroneously to the effects of the time we conduct the posttest, the subjects the experimental stimulus. will probably have gotten more sensitive to the 6. Selection biases. We discussed selection bias issue of prejudice and will be more thoughtful earlier when we examined different ways in their answers. In fact, they may have fi gured of selecting subjects for experiments and out that we’re trying to fi nd out how preju- assigning them to experimental and control diced they are, and, because few people like groups. Comparisons have no meaning unless to appear prejudiced, they may give answers the groups are comparable at the start of an that they think we want or that will make them experiment. look good. 7. Experimental mortality. Although some social 4. Instrumentation. The process of measurement experiments could, I suppose, kill subjects, in pretesting and posttesting brings in some experimental mortality refers to a more general of the issues of conceptualization and opera- and less extreme problem. Often, experimental tionalization discussed earlier in the book. For subjects will drop out of the experiment before example, if we use different measures of the it’s completed, and this can affect statistical dependent variable (say, different question- comparisons and conclusions. In the classical naires about prejudice), how can we be sure experiment involving an experimental and a they’re comparable to one another? Perhaps control group, each with a pretest and posttest, prejudice will seem to decrease simply be- suppose that the bigots in the experimental cause the pretest measure was more sensitive group are so offended by the African American than the posttest measure. Or if the measure- history fi lm that they leave before it’s over. ments are being made by the experimenters, Those subjects sticking around for the posttest their standards or abilities may change over will have been less prejudiced to start with, the course of the experiment. so the group results will refl ect a substantial 5. Statistical regression. Sometimes it’s appro- “decrease” in prejudice. priate to conduct experiments on subjects 8. Demoralization. On the other hand, feelings of who start out with extreme scores on the deprivation within the control group may result dependent variable. If you were testing a new in their giving up. In educational experiments, method for teaching math to hard-core failures demoralized control-group subjects may stop in math, you would want to conduct your studying, act up, or get angry. experiment on people who previously have done extremely poorly in math. But consider These, then, are some of the sources of inter- for a minute what’s likely to happen to the nal invalidity in experiments, as cited by Camp- math achievement of such people over time bell, Stanley, and Cook. Aware of these pitfalls, without any experimental interference. They’re experimenters have devised designs aimed at starting out so low that they can only stay at handling them. The classical experiment, coupled the bottom or improve: They can’t get worse. with proper subject selection and assignment, ad- Even without any experimental stimulus, then, dresses each of these problems. Let’s look again the group as a whole is likely to show some at that study design, presented graphically in Fig- improvement over time. Referring to a regres- ure 8-4, as it applies to our hypothetical study of sion to the mean, statisticians often point out prejudice. VARIATIONS ON EXPERIMENTAL DESIGN 257

Pretest Stimulus Posttest

Compare

Experimental Group

Compare

Control Group

FIGURE 8-4 The Classical Experiment: Using an African American History Film to Reduce Prejudice. This diagram illustrates the basic structure of the classical experiment as a vehicle for testing the impact of a ﬁ lm on prejudice. Notice how the control group, the pretesting, and the posttesting function.

If we use the experimental design shown in Fig- assignment of subjects. Experimental mortality is ure 8-4, we should expect two fi ndings from our Af- more complicated to handle, but the data provided rican American history fi lm experiment. For the ex- in this study design offer several ways to deal with perimental group, the level of prejudice measured it. Pretest measurements would let us discover any in their posttest should be less than was found in differences in the dropouts of the experimental their pretest. In addition, when the two posttests and control groups. Slight modifi cations to the de- are compared, less prejudice should be found in sign—administering a placebo (such as a fi lm hav- the experimental group than in the control group. ing nothing to do with African Americans) to the This design also guards against the problem control group, for example—can make the prob- of history, in that anything occurring outside the lem even easier to manage. Finally, demoraliza- experiment that might affect the experimental tion can be watched for and taken into account in group should also affect the control group. Conse- evaluating the results of the experiment. quently, the two posttest results should still differ. The same comparison guards against problems of Sources of External Invalidity Internal invalidity maturation as long as the subjects have been ran- accounts for only some of the complications faced domly assigned to the two groups. Testing and in- by experimenters. In addition, there are problems strumentation can’t be problems, because both the of what Campbell and Stanley call external in- experimental and control groups are subject to the validity, which relates to the generalizability of same tests and experimenter effects. If the subjects experimental fi ndings to the “real” world. Even if have been assigned to the two groups randomly, statistical regression should affect both equally, even if people with extreme scores on prejudice external invalidity Refers to the possibility that (or whatever the dependent variable is) are being conclusions drawn from experimental results may not be studied. Selection bias is ruled out by the random generalizable to the “real” world. 258 CHAPTER 8 EXPERIMENTS the results of an experiment provide an accurate 1 gauge of what happened during that experiment, Stimulus Group 1 Pretest Posttest do they really tell us anything about life in the wilds (film) of society? 3 Campbell and Stanley describe four forms of 2 Group 2 this problem; I’ll present one of them to you as an (control) Pretest No stimulus Posttest illustration. The generalizability of experimental fi ndings is jeopardized, as the authors point out, if there’s an interaction between the testing situation Group 3 No Stimulus Posttest pretest (film) and the experimental stimulus (1963:18). Here’s an example of what they mean. 4 Group 4 No No Staying with the study of prejudice and the Af- Posttest rican American history fi lm, let’s suppose that our (control) pretest stimulus experimental group—in the classical experiment— TIME has less prejudice in its posttest than in its pretest and that its posttest shows less prejudice than that Expected Findings In Group 1, posttest prejudice should be less than of the control group. We can be confi dent that the 1 pretest prejudice. fi lm actually reduced prejudice among our experi- In Group 2, prejudice should be the same in the mental subjects. But would it have the same effect 2 pretest and the posttest. if the fi lm were shown in theaters or on television? The Group 1 posttest should show less prejudice We can’t be sure, because the fi lm might only be 3 than the Group 2 posttest. effective when people have been sensitized to the The Group 3 posttest should show less prejudice issue of prejudice, as the subjects may have been 4 than the Group 4 posttest. in taking the pretest. This is an example of interaction between the testing and the stimulus. The FIGURE 8-5 The Solomon Four-Group Design. The classical experiment runs the risk that pretest- classical experimental design cannot control for ing will have an effect on subjects, so the Solomon that possibility. Fortunately, experimenters have Four Group Design adds experimental and control devised other designs that can. groups that skip the pretest. The Solomon four-group design (D. Campbell and Stanley 1963:24–25) addresses the problem of testing interaction with the stimulus. As the name suggests, it involves four groups of subjects, as- 2. In Group 2, prejudice should be the same in the signed randomly from a pool. Figure 8-5 presents pretest and the posttest. this design graphically. 3. The Group 1 posttest should show less preju- Notice that Groups 1 and 2 in Figure 8-5 com- dice than the Group 2 posttest. pose the classical experiment. Group 3 is adminis- 4. The Group 3 posttest should show less preju- tered the experimental stimulus without a pretest, dice than the Group 4 posttest. and Group 4 is only posttested. This latest experimental design permits four meaningful compari- Notice that fi nding (4) rules out any interaction sons. If the African American history fi lm really between the testing and the stimulus. Remember reduces prejudice—unaccounted for by the prob- that these comparisons are meaningful only if sub- lem of internal invalidity and unaccounted for by jects have been assigned randomly to the different an interaction between the testing and the stimu- groups, thereby providing groups of equal preju- lus—we should expect four fi ndings: dice initially, even though their preexperimental prejudice is only measured in Groups 1 and 2. 1. In Group 1, posttest prejudice should be less There is a side benefi t to this research design, than pretest prejudice. as the authors point out. Not only does the Solo- AN ILLUSTRATION OF EXPERIMENTATION 259 mon four-group design rule out interactions between testing and the stimulus, it also provides IN THE REAL WORLD data for comparisons that will reveal the amount MOCK JURIES of such interaction that occurs in the classical experimental design. This knowledge would al- One dramatic application of small group re- low a researcher to review and evaluate the search that also poses the problem of exter- value of any prior research that used the simpler nal invalidity is the use of “mock juries.” In design. this application, people are asked to pretend The last experimental design I’ll mention here they are jurists in a real trial. Lawyers try is what Campbell and Stanley (1963:25–26) call various strategies on different mock juries the posttest-only control-group design; it consists to determine which strategy will be the most of the second half—Groups 3 and 4—of the Solo- effective in a real trial. This is a meaningless mon design. As the authors argue persuasively, exercise, however, unless the effects pro- with proper randomization, only Groups 3 and duced in the “laboratory” of mock juries can 4 are needed for a true experiment that controls be replicated in actual court cases. for the problems of internal invalidity as well as for the interaction between testing and stimulus. With randomized assignment to experimental and control groups (which distinguishes this design AN ILLUSTRATION OF from the static-group comparison discussed ear- EXPERIMENTATION lier), the subjects will be initially comparable on the dependent variable—comparable enough to satisfy the conventional statistical tests used to evalu- Experiments have been used to study a wide variety ate the results—so it’s not necessary to measure of topics in the social sciences. Some experiments them. Indeed, Campbell and Stanley suggest that have been conducted within laboratory situations; the only justifi cation for pretesting in this situation others occur out in the “real world”—these are re- is tradition. Experimenters have simply grown ac- ferred to as fi eld experiments. The following dis- customed to pretesting and feel more secure with cussion will give you a glimpse of both. We’ll begin research designs that include it. Be clear, how- with an example of a fi eld experiment. ever, that this point applies only to experiments in In George Bernard Shaw’s well-loved play, Pyg- which subjects have been assigned to experimen- malion—the basis for the musical My Fair Lady— tal and control groups randomly, because that’s Eliza Doolittle speaks of the powers others have what justifi es the assumption that the groups are in determining our social identity. Here’s how equivalent—without actually measuring them to she distinguishes between the ways she’s treated fi nd out. by her tutor, Professor Higgins, and by Higgins’s This discussion has introduced the intricacies of friend, Colonel Pickering: experimental design, its problems, and some so- You see, really and truly, apart from the things lutions. Of course, researchers use a great many anyone can pick up (the dressing and the proper other possible experimental designs as well. Some way of speaking, and so on), the difference involve more than one stimulus and combinations between a lady and a fl ower girl is not how she of stimuli. Others involve several tests of the de- behaves, but how she’s treated. I shall always pendent variable over time and the administra- be a fl ower girl to Professor Higgins, because tion of the stimulus at different times for different he always treats me as a fl ower girl, and always groups. If you’re interested in pursuing this topic, you might look at the Campbell and Stanley book. Also see the box “Mock Juries” for more on exter- fi eld experiment A formal experiment conducted nal invalidity. outside the laboratory, in a natural setting. 260 CHAPTER 8 EXPERIMENTS

will, but I know I can be a lady to you, because ress made by the “spurters” was simply a result you always treat me as a lady, and always will. of the teachers’ expecting the improvement and — (ACT V) paying more attention to those students, encouraging them, and rewarding them for achieve- The sentiment Eliza expresses here is basic ments. (Notice the similarity between this situation social science, addressed more formally by soci- and the Hawthorne effect, discussed earlier in this ologists such as Charles Horton Cooley (“looking- chapter.) glass self”) and George Herbert Mead (“the gener- The Rosenthal-Jacobson study attracted a lot alized other”). The basic point is that who we think of popular as well as scientifi c attention. Subse- we are—our self-concept—and how we behave quent experiments have focused on specifi c as- is largely a function of how others see and treat pects of what has become known as the attribu- us. Further, the way others perceive us is largely tion process, or the expectations communication conditioned by their expectations. If they’ve been model. This research, largely conducted by psy- told we’re stupid, for example, they’re likely to chologists, parallels research primarily by sociolo- see us that way—and we may come to see our- gists, which takes a slightly different focus and is selves that way and actually act stupidly. “Label- often gathered under the label expectations-states ing theory” addresses the phenomenon of people theory. The former studies focus on situations in acting in accord with the ways they are perceived which the expectations of a dominant individual and labeled by others. These theories have served affect the performance of subordinates—as in as the premise for numerous movies, such as the the case of a teacher and students, or a boss and 1983 fi lm Trading Places, in which Eddie Murphy employees. The sociological research has tended and Dan Ackroyd play a derelict converted into a to focus more on the role of expectations among stockbroker and vice versa. equals in small, task-oriented groups. In a jury, The tendency to see in others what we’ve been for example, how do jurors initially evaluate each led to expect takes its name from Shaw’s play and other, and how do those initial assessments affect is called the Pygmalion effect. This effect is nicely their later interactions? suited to controlled experiments. In one of the best-known experiments on this topic, Robert Search the web for “Pygmalion effect” Rosenthal and Lenore Jacobson (1968) adminis- to learn more about this phenomenon, tered what they called a “Harvard Test of Infl ected including attempts to fi nd practical applica- Acquisition” to students in a West Coast school. tions of it.* Subsequently, they met with the students’ teachers to present the results of the test. In particular, Rosenthal and Jacobson identifi ed certain students Here’s an example of a laboratory experiment as very likely to exhibit a sudden spurt in academic conducted to examine the way our perceptions abilities during the coming year, based on the re- of our abilities and those of others affect our will- sults of the test. ingness to accept the other person’s ideas. Mar- When IQ test scores were compared later, the tha Foschi, G. Keith Warriner, and Stephen Hart researchers’ predictions proved accurate. The stu- were particularly interested in the role “standards” dents identifi ed as “spurters” far exceeded their played in that respect: classmates during the following year, suggesting that the predictive test was a powerful one. In fact, In general terms, by “standards” we mean how the test was a hoax! The researchers had made well or how poorly a person has to perform in their predictions randomly among both good and poor students. What they told the teachers did not *Each time the Internet icon appears, you’ll be given help- really refl ect students’ test scores at all. The prog- ful leads for searching the World Wide Web. AN ILLUSTRATION OF EXPERIMENTATION 261

order for an ability to be attributed or denied partners via computer terminals but would not be him/her. In our view, standards are a key vari- allowed to see each other.) If you were assigned a able affecting how evaluations are processed score of 14, you would be told your partner had a and what expectations result. For example, score of 6; if you were assigned 6, you would be depending on the standards used, the same told your partner had 14. level of success may be interpreted as a major This procedure meant that you would enter accomplishment or dismissed as unimportant. the teamwork phase of the experiment believing — (1985:108–9) either (1) you had done better than your partner or (2) you had done worse than your partner. This To begin examining the role of standards, the information constituted part of the “standard” you researchers designed an experiment involving four would be operating under in the experiment. In experimental groups and a control. Subjects were addition, half of each group was told that a score told that the experiment involved something called of between 12 and 20 meant that the subject defi - “pattern recognition ability,” which was an innate nitely had pattern recognition ability; the other ability some people had and others didn’t. The re- subjects were told that a score of 14 wasn’t really searchers said that subjects would be working in high enough to prove anything defi nite. Thus, you pairs on pattern recognition problems. would emerge from this with one of the following In fact, of course, there’s no such thing as pat- beliefs: tern recognition ability. The object of the experiment was to determine how information about 1. You are defi nitely better at pattern recognition this supposed ability affected subjects’ subsequent than your partner. behavior. 2. You are possibly better than your partner. The fi rst stage of the experiment was to “test” 3. You are possibly worse than your partner. each subject’s pattern recognition abilities. If you 4. You are defi nitely worse than your partner. had been a subject in the experiment, you would have been shown a geometrical pattern for 8 sec- The control group for this experiment was told onds, followed by two more patterns, each of nothing about their own abilities or their partners’. which was similar to but not the same as the fi rst In other words, they had no expectations. one. Your task would be to choose which of the The fi nal step in the experiment was to set the subsequent set had a pattern closest to the fi rst one “teams” to work. As before, each subject would you saw. You would be asked to do this 20 times, be given an initial pattern, followed by a compari- and a computer would print out your “score.” Half son pair to choose from. When you entered your the subjects would be told they had gotten 14 cor- choice in this round, however, you would be told rect; the other half would be told they had only what your partner had answered; you would then gotten 6 correct—regardless of which patterns be asked to choose again. In your fi nal choice, they matched with which. Depending on the luck you could either stick with your original choice or of the draw, you would think you had done either switch. The “partner’s” choice was, of course, cre- quite well or quite badly. Notice, however, that you ated by the computer, and, as you can guess, there wouldn’t really have any standard for judging your were often disagreements in the teams: 16 out of performance—maybe getting 4 correct would be 20 times, in fact. considered a great performance. The dependent variable in this experiment was At the same time you were given your score, the extent to which subjects would switch their however, you would also be given your “partner’s choices to match those of their partners. The re- score,” although both the “partners” and their searchers hypothesized that the defi nitely bet- “scores” were also computerized fi ctions. (Subjects ter group would switch least often, followed by were told they would be communicating with their the probably better group, followed by the control 262 CHAPTER 8 EXPERIMENTS group, followed by the probably worse group, fol- I’ve mentioned the case of jury deliberations. How lowed by the defi nitely worse group, who would about all forms of prejudice and discrimination? Or, switch most often. consider how expectation states fi gure into job in- The number of times subjects in the fi ve groups terviews or meeting your heartthrob’s parents. If you switched their answers follows. Realize that each think about it, you’ll probably see other situations had 16 opportunities to do so. These data indicate where these laboratory concepts apply in real life. that each of the researchers’ expectations was correct—with the exception of the comparison between the possibly worse and defi nitely worse WEB-BASED EXPERIMENTS groups. The latter group was in fact the more likely to switch, but the difference was too small to be Increasingly, researchers are using the World Wide taken as a confi rmation of the hypothesis. Web as a vehicle for conducting experiments. Be- cause representative samples are not essential in Mean Number most experiments, researchers can often use vol- Group of Switches unteers who respond to invitations online. Here Defi nitely better 5.05 are two sites you might visit to get a better idea of Possibly better 6.23 this form of experimentation. Control group 7.95 Possibly worse 9.23 • Online Social Psychology Studies: http://www Defi nitely worse 9.28 .socialpsychology.org/expts.htm. This website offers hotlinks to numerous professional and student research projects on such topics as “in- In more detailed analyses, the researchers terpersonal relations,” “beliefs and attitudes,” found that the same basic pattern held for both and “personality and individual differences.” In men and women, though it was somewhat clearer addition, the site offers resources for conduct- for women than for men. Here are the actual data: ing web experiments. • Small World Phenomenon: http://smallworld Mean Number of Switches .columbia.edu/. This study, by a group of sociologists at Columbia University, is seeking Women Men to research the much misunderstood* con- Defi nitely better 4.50 5.66 cept of “six degrees of separation.” Based on Possibly better 6.34 6.10 the idea that you know someone who knows Control group 7.68 8.34 someone who knows someone (and so forth), Possibly worse 9.36 9.09 this experiment seeks to determine the aver- Defi nitely worse 10.00 8.70 age number of steps it takes to get from one person to another around the world. Because specifi c research efforts like this one sometimes seem extremely focused in their scope, you might wonder about their relevance to any- *This is often expressed as something like “There are no thing. As part of a larger research effort, however, more than six degrees of separation between any two people on earth,” but that’s not exactly accurate. In 1967 studies like this one add concrete pieces to our un- Stanley Milgram selected 300 people at random in Omaha derstanding of more general social processes. and asked them to contact a specifi c person in Boston, It’s worth taking a minute or so to consider some through a chain of personal contacts. The average number of steps required was indeed six, but only 20 percent suc- of the life situations where “expectation states” ceeded. The Columbia study is an attempt to examine the might have very real and important consequences. matter in a more rigorously controlled manner. “NATURAL” EXPERIMENTS 263

“NATURAL” EXPERIMENTS IN THE REAL WORLD WARTIME MORALE Although people tend to equate the terms experiment and laboratory experiment, we have seen that One example of a natural experiment comes experiments are sometimes conducted outside the from the annals of social research concern- lab (fi eld experiments) and can be conducted on ing World War II. After the war ended, social the web. Other important social scientifi c experi- researchers undertook retrospective sur- ments occur outside controlled settings altogether, veys of wartime morale among civilians in often in the course of normal social events. Some- several German cities. Among other things, times nature designs and executes experiments they wanted to determine the effect of mass that we can observe and analyze; sometimes so- bombing on the morale of civilians. They cial and political decision makers serve this natu- compared the reports of wartime morale of ral function. residents in heavily bombed cities and cit- Imagine, for example, that a hurricane has ies that received relatively little bombing. struck a particular town. Some residents of the Bombing had not reduced morale. town suffer severe fi nancial damages, whereas others escape relatively lightly. What, we might ask, are the behavioral consequences of suffering a natural disaster? Are those who suffer most cal analysis of research designs, our study falls more likely to take precautions against future di- into the “static-group comparison” category, sasters than are those who suffer least? To answer considered one of the weak research designs. these questions, we might interview residents of However, the weaknesses are potential and the town some time after the hurricane. We might their actual presence depends on the unique question them regarding their precautions before circumstances of each study. — (1981:474) the hurricane and the ones they’re currently tak- The foundation of this study was a survey of the ing, comparing the people who suffered greatly people who had been working at Three Mile Island from the hurricane with those who suffered rela- on March 28, 1979, when the cooling system failed tively little. In this fashion, we might take advan- in the number 2 reactor and began melting the tage of a natural experiment, which we could not uranium core. The survey was conducted fi ve to have arranged even if we’d been perversely willing six months after the accident. Among other things, to do so. See the box “Wartime Morale” for another the survey questionnaire measured workers’ atti- example of a natural experiment. tudes toward working at nuclear power plants. If Because in natural experiments the researcher they had measured only the TMI workers’ attitudes must take things pretty much as they occur, such after the accident, the researchers would have had experiments raise many of the validity problems no idea whether attitudes had changed as a con- discussed earlier. Thus when Stanislav Kasl, Ru- sequence of the accident. But they improved their pert Chisolm, and Brenda Eskenazi (1981) chose study design by selecting another, nearby—seem- to study the impact that the Three Mile Island (TMI) ingly comparable—nuclear power plant (abbrevi- nuclear accident in Pennsylvania had on plant ated as PB) and surveyed workers there as a con- workers, they had to be especially careful in the trol group: hence their reference to a static-group study design: comparison. Disaster research is necessarily opportunistic, Even with an experimental and a control group, quasi-experimental, and after-the-fact. In the the authors were wary of potential problems in terminology of Campbell and Stanley’s classi- their design. In particular, their design was based 264 CHAPTER 8 EXPERIMENTS on the idea that the two sets of workers were ward African Americans. Their opportunity arose equivalent to one another, except for the single in 1979, when a sequel—Roots: The Next Genera- fact of the accident. The researchers could have tion—was televised. Although it would have been assumed this if they had been able to assign work- nice (from a researcher’s point of view) to assign ers to the two plants randomly, but of course they random samples of Americans either to watch or couldn’t. Instead, they compared characteristics of not watch the show, this wasn’t possible. Instead, the two groups to see whether they were equiva- the researchers selected four samples in Washing- lent. Ultimately, the researchers concluded that the ton State and mailed questionnaires (before the two sets of workers were very much alike, and the broadcast) that measured attitudes toward African plant the employees worked at was merely a func- Americans. Following the last episode of the show, tion of where they lived. respondents were called and asked how many, if Even granting that the two sets of workers any, episodes they had watched. Subsequently, were equivalent, the researchers faced another questionnaires were sent to respondents, remea- problem of comparability. They could not contact suring their attitudes toward African Americans. all the workers who had been employed at TMI at By comparing attitudes before and after for the time of the accident. The researchers discuss both those who watched the show and those who the problem as follows: didn’t, the researchers reached several conclusions. For example, they found that people with al- One special attrition problem in this study was ready egalitarian attitudes were much more likely the possibility that some of the no-contact non- to watch the show than were those who were respondents among the TMI subjects, but not PB more prejudiced toward African Americans: a self- subjects, had permanently left the area because selection phenomenon. Comparing the before and of the accident. This biased attrition would, after attitudes of those who watched the show, most likely, attenuate the estimated extent of the moreover, suggested that the show itself had little impact. Using the evidence of disconnected or or no effect. Those who watched it were no more “not in service” telephone numbers, we estimate egalitarian afterward than they had been before. this bias to be negligible (1 percent). — (KASL, This example anticipates the subject of Chapter CHISOLM, AND ESKENAZI 1981:475) 12, evaluation research, which can be seen as a The TMI example points both to the special special type of natural experiment. As you’ll see, problems involved in natural experiments and to evaluation research involves taking the logic of ex- the possibility of taking those problems into ac- perimentation into the fi eld to observe and evalu- count. Social research generally requires ingenu- ate the effects of stimuli in real life. Because this is ity and insight; natural experiments call for a little an increasingly important form of social research, more than the average. an entire chapter is devoted to it. Earlier in this chapter, we used a hypothetical example of studying whether an African Ameri- STRENGTHS AND WEAKNESSES can history fi lm reduced prejudice. Sandra Ball- OF THE EXPERIMENTAL METHOD Rokeach, Joel Grube, and Milton Rokeach (1981) were able to address that topic in real life through a natural experiment. In 1977 the television dra- Experiments are the primary tool for studying matization of Alex Haley’s Roots, a historical saga causal relationships. However, like all research about African Americans, was presented by ABC methods, experiments have both strengths and on eight consecutive nights. It garnered the larg- weaknesses. est audiences in television history at that time. The chief advantage of a controlled experiment Ball-Rokeach and her colleagues wanted to know lies in the isolation of the experimental variable’s if Roots changed white Americans’ attitudes to- impact over time. This is seen most clearly in terms ETHICS AND EXPERIMENTS 265 of the basic experimental model. A group of exper- WHAT DO YOU THINK? REVISITED imental subjects are found, at the outset of the experiment, to have a certain characteristic; following the administration of an experimental stimulus, As we’ve seen, the impact of the experi- they are found to have a different characteristic. To ment itself on subjects’ responses is a ma- the extent that subjects have experienced no other jor concern in social research. Several ele- stimuli, we may conclude that the change of char- ments of experimental designs address this acteristics is caused by the experimental stimulus. concern. First, the use of control groups al- Further, because individual experiments are of- lows researchers to account for any effects ten rather limited in scope, requiring relatively little of the experiment that are not related to the time and money and relatively few subjects, we of- stimulus. Second, the Solomon four-group ten can replicate a given experiment several times design tests for the possible impact of pre- using many different groups of subjects. (This isn’t tests on the dependent variable. And, fi nally, always the case, of course, but it’s usually easier so-called natural experiments are done in to repeat experiments than, say, surveys.) As in all real-life situations, imposing an experimen- other forms of scientifi c research, replication of re- tal template over naturally occurring events. search fi ndings strengthens our confi dence in the Thus, although the impact of the observer validity and generalizability of those fi ndings. can affect experimental results negatively, The greatest weakness of laboratory experi- researchers have developed methods for ad- ments lies in their artifi ciality. Social processes that dressing it. occur in a laboratory setting might not necessarily occur in more natural social settings. For example, an African American history fi lm might genuinely reduce prejudice among a group of experimental subjects. This would not necessarily mean, how- First, experiments almost always involve de- ever, that the same fi lm shown in neighborhood ception. In most cases, explaining the purpose of movie theaters throughout the country would re- the experiment to subjects would probably cause duce prejudice among the general public. Artifi ci- them to behave differently—trying to look good, ality is not as much of a problem, of course, for for example. It’s important, therefore, to determine natural experiments as for those conducted in the (1) whether a particular deception is essential to laboratory. the experiment and (2) whether the value of what In discussing several of the sources of internal may be learned from the experiment justifi es the and external invalidity mentioned by Campbell, ethical violation. Stanley, and Cook, we saw that we can create Second, experiments typically intrude on the experimental designs that logically control such lives of the subjects. Experimental researchers problems. This possibility points to one of the great commonly put them in unusual situations and ask advantages of experiments: They lend themselves them to undergo unusual experiences. Even when to a logical rigor that is often much more diffi cult to they do not physically injure the subjects (Don’t do achieve in other modes of observation. that, by the way.), psychological damage to subjects may occur, as some of the examples in this chapter illustrate. As with the matter of deception, ETHICS AND EXPERIMENTS then, researchers must balance the potential value of the research against the potential damage to As you’ve seen, many important ethical issues subjects. come up in the conduct of social science experiments. I’ll mention only two here. 266 CHAPTER 8 EXPERIMENTS

Main Points ❏ Experiments also face problems of external invalidity, in that experimental ﬁ ndings might Introduction not reﬂ ect real life. ❏ In experiments, social researchers typically se- ❏ The interaction of testing with the stimulus is lect a group of subjects, do something to them, an example of external invalidity that the clas- and observe the effect of what was done. sical experiment does not guard against. Topics Appropriate to Experiments ❏ The Solomon four-group design and other ❏ Experiments provide an excellent vehicle for variations on the classical experiment can the controlled testing of causal processes. safeguard against external invalidity.

❏ The Classical Experiment Campbell and Stanley suggest that, given ❏ The classical experiment tests the effect of proper randomization in the assignment an experimental stimulus (the independent of subjects to the experimental and control variable) on a dependent variable through the groups, there is no need for pretesting in pretesting and posttesting of experimental and experiments. control groups. An Illustration of Experimentation ❏ It is generally less important that a group of ❏ The Pygmalion effect is one phenomenon that experimental subjects be representative of researchers must account for in experimental some larger population than that experimental design. and control groups be similar to each other. Web-Based Experiments ❏ A double-blind experiment guards against ❏ The World Wide Web has become an increas- experimenter bias because neither the experi- ingly common vehicle for performing social menter nor the subject knows which subjects science experiments. are in the control and experimental groups. “Natural” Experiments Selecting Subjects ❏ Natural experiments often occur in the course ❏ Probability sampling, randomization, and of social life in the real world, and social matching are all methods of achieving compa- researchers can implement them in somewhat rability in the experimental and control groups. the same way they would design and conduct Randomization is the generally preferred laboratory experiments. method. In some designs, it can be combined with matching. Strengths and Weaknesses of the Experimental Method Variations on Experimental Design ❏ Like all research methods, experiments have ❏ Campbell and Stanley describe three forms of strengths and weaknesses. Their primary preexperiments: the one-shot case study, the weakness is artifi ciality: What happens in an one-group pretest-posttest design, and the experiment may not refl ect what happens in static-group comparison. the outside world. Their strengths include the ❏ Campbell and Stanley list, among others, eight isolation of the independent variable, which sources of internal invalidity in experimental permits causal inferences; the relative ease of design: history, maturation, testing, instrumen- replication; and scientifi c rigor. tation, statistical regression, selection biases, experimental mortality, and demoralization. Ethics and Experiments The classical experiment with random assign- ❏ Experiments typically involve deceiving ment of subjects guards against each of these. subjects. ADDITIONAL READINGS 267

❏ By their intrusive nature, experiments open the Website for possibility of inadvertently causing damage to The Basics of Social Research subjects. 4th edition

Key Terms At the book companion website (http://sociology .wadsworth.com/babbie_basics4e) you will fi nd control group internal invalidity many resources in addition to ThomsonNow to double-blind experiment matching aid you in studying for your exams. For example, experimental group posttesting you will fi nd Tutorial Quizzes with feedback, Inter- external invalidity pretesting net Exercises, Flashcards, and Chapter Tutorials, as fi eld experiment randomization well as Extended Projects, InfoTrac College Edition search terms, Social Research in Cyberspace, GSS Review Questions Data, Web Links, and primers for using various 1. What are some examples of internal invalid- data analysis software such as SPSS and NVivo. ity? Pick four of the eight sources discussed in the book and make up your own examples to Additional Readings illustrate each. 2. Think of a recent natural disaster you’ve wit- Campbell, Donald, and Julian Stanley. 1963. Experimental and Quasi-Experimental Designs for Research. Chicago: nessed or read about. What research question Rand McNally. An excellent analysis of the logic and might be studied by treating that disaster as a methods of experimentation in social research. This natural experiment? Take two or three para- book is especially useful in its application of the logic graphs to outline how the study might be done. of experiments to other social research methods. 3. Say you want to evaluate a new operating Though fairly old, this book has attained the status of system or other software. How might you set a classic and is still frequently cited. up an experiment to see what people really Cook, Thomas D., and Donald T. Campbell. 1979. Quasi- think of it? Keep in mind the use of control Experimentation: Design and Analysis Issues for Field Settings. Chicago: Rand McNally. An expanded and groups and the placebo effect. updated version of Campbell and Stanley. 4 Think of a recent, highly publicized trial. How Jones, Stephen R. G. 1990. “Worker Independence might the attorneys have used mock juries and Output: The Hawthorne Studies Reevaluated.” to evaluate different strategies for presenting American Sociological Review 55:176–90. This article evidence? reviews these classical studies and questions the traditional interpretation (which was presented in Online Study Resources this chapter). Martin, David W. 1996. Doing Psychology Experiments. 4th ed. Monterey, CA: Brooks/Cole. With thorough explanations of the logic behind research methods, often in a humorous style, this book emphasizes ideas of particular importance to the beginning researcher, Go to such as getting an idea for an experiment or review- http://sociology.wadsworth.com/babbie_basics4e ing the literature. Ray, William J. 2000. Methods toward a Science of Behav- and click on ThomsonNow for access to this ior and Experience. 6th ed. Belmont, CA: Wadsworth. powerful online study tool. You will get a per- A comprehensive examination of social science sonalized study plan based on your responses to research methods, with a special emphasis on ex- a diagnostic pretest. Once you have mastered perimentation. This book is especially strong in the the material with the help of interactive learning philosophy of science. tools, you can take a posttest to confi rm that you are ready to move on to the next chapter. 91 HUMANSURVEY RESEARCHINQUIRY AND SCIENCE

Image not available due to copyright restrictions Photo credit

What You’ll Learn in thisThis ChapterChapter

HereWe’ll you’ll examine learn aboutthe way many people of the learn methods about researcherstheir world anduse theto collect mistakes data they makethrough along surveys—from the way. We’ll mail also questionnaires begin to see towhat personal makes interviews sience different to online from

othersurveys ways conducted of knowing over things. the Internet. You’ll also learn how to select an appropriate 2S method and how to implement it effectively. 1S N L 268 In this chapter . . . WHAT DO YOU THINK?

Introduction All of us get tele- Topics Appropriate for Survey Research phone calls asking Guidelines for Asking Questions us to participate in

Choose Appropriate Question Forms Image not available due to copyright restrictions a “survey,” but some Make Items Clear are actually tele- Avoid Double-Barreled Questions marketing calls. Respondents Must Be Competent to Answer Caller: The ﬁ rst ques- Respondents Must Be Willing to Answer tion is “Who was the Questions Should Be Relevant ﬁ rst president of the United States?” Short Items Are Best You: Crazy Horse. Avoid Negative Items Caller: Close enough. You have just won . . . Avoid Biased Items and Terms This is obviously not legitimate. But subtler Questionnaire Construction techniques can entangle you in a phony General Questionnaire Format sales pitch before you know what’s happen- Formats for Respondents ing. When you get a call announcing that Contingency Questions you’ve been selected for a “survey,” how can Matrix Questions you tell whether it’s genuine? Ordering Items in a Questionnaire Questionnaire Instructions See the “What Do You Think? Revisited” box Pretesting the Questionnaire toward the end of the chapter. A Sample Questionnaire

Self-Administered Questionnaires Mail Distribution and Return Strengths and Weaknesses of Survey Monitoring Returns Research Follow-up Mailings Acceptable Response Rates Secondary Analysis A Case Study Ethics and Survey Research Interview Surveys The Role of the Survey Interviewer INTRODUCTION General Guidelines for Survey Interviewing Coordination and Control Surveys are a very old research technique. In the Telephone Surveys Old Testament, for example, we ﬁ nd the following: Positive and Negative Factors Computer-Assisted Telephone Interviewing (CATI) After the plague the Lord said to Moses and to Response Rates in Interview Surveys Eleazar the son of Aaron, the priest, “Take a census of all the congregation of the people of New Technologies and Survey Research Israel, from twenty old and upward.” — (NUMBERS 26:1–2) Comparison of the Different Survey Methods 269 270 CHAPTER 9 SURVEY RESEARCH

Ancient Egyptian rulers conducted censuses to TOPICS APPROPRIATE help them administer their domains. Jesus was FOR SURVEY RESEARCH born away from home because Joseph and Mary were journeying to Joseph’s ancestral home for a Surveys may be used for descriptive, explanatory, Roman census. and exploratory purposes. They are chiefl y used A little-known survey was attempted among in studies that have individual people as the units French workers in 1880. A German political soci- of analysis. Although this method can be used for ologist mailed questionnaires to workers to deter- other units of analysis, such as groups or inter- mine the extent of their exploitation by employers. actions, some individual persons must serve as The rather lengthy questionnaire included items respondents or informants. Thus, we could un- such as these: dertake a survey in which divorces were the unit Does your employer or his representative resort of analysis, but we would need to administer the to trickery in order to defraud you of a part of survey questionnaire to the participants in the di- your earnings? vorces (or to some other respondents). If you are paid piece rates, is the quality of Survey research is probably the best method the article made a pretext for fraudulent deduc- available to the social researcher who is interested tions from your wages? in collecting original data for describing a population too large to observe directly. Careful prob- The survey researcher in this case was not ability sampling provides a group of respondents George Gallup but Karl Marx ([1880] 1956:208). whose characteristics may be taken to refl ect those Although 25,000 questionnaires were mailed out, of the larger population, and carefully constructed there is no record of any being returned. standardized questionnaires provide data in the Today, survey research is a frequently used same form from all respondents. mode of observation in the social sciences. In a Surveys are also excellent vehicles for measur- typical survey, the researcher selects a sample of ing attitudes and orientations in a large popula- respondents and administers a standardized question. Public opinion polls—for example, Gallup, tionnaire to each person in the sample. Chapter 7 Harris, Roper, and Yankelovich—are well-known discussed sampling techniques in detail. This chap- examples of this use. Indeed, polls have become ter discusses how to prepare a questionnaire and so prevalent that at times the public seems unsure describes the various options for administering it what to think of them. Pollsters are criticized by so that respondents answer questions adequately. those who don’t think (or want to believe) that polls The chapter also briefl y discusses secondary are accurate (candidates who are “losing” in polls analysis, the analysis of survey data collected by often tell voters not to trust the polls). But polls are someone else. This use of survey results has be- also criticized for being too accurate—for example, come an important aspect of survey research in when exit polls on election day are used to predict recent years, and it’s especially useful for students a winner before the actual voting is complete. See and others with scarce research funds. The chap- the box “Political Polling” for more on this topic. ter closes with a look at the ethical implications of The general attitude toward public opinion re- survey research. search is further complicated by scientifi cally un- Let’s begin here by looking at the kinds of topics sound “surveys” that nonetheless capture people’s that researchers can appropriately study through attention because of the topics they cover and/or survey research. their “fi ndings.” A good example is the “Hite Re- ports” on human sexuality. Although enjoying considerable attention in the popular press, Shere Hite respondent A person who provides data for analysis by was roundly criticized by the research community responding to a survey questionnaire. for her data-collection methods. For example, a GUIDELINES FOR ASKING QUESTIONS 271

that the fi rst question was “How would you like to IN THE REAL WORLD make thousands of dollars a week right there in POLITICAL POLLING your own home?” Unfortunately, a few unscrupu- lous telemarketers prey on the general coopera- One of the most popular uses of survey re- tion people have given to survey researchers. search is for political polling: horse races By the same token, political parties and chari- among candidates, approval ratings of pub- table organizations have begun conducting phony lic offi cials, and opinion polling on issues in “surveys.” Often under the guise of collecting pub- the political arena. Often you’ll hear people lic opinion about some issue, callers ultimately ask complain that such polls play too promi- respondents for a monetary contribution. nent a role in determining the actions of Recent political campaigns have produced an- offi ce holders and offi ce seekers. However, other form of bogus survey, called the “push poll.” even more often, you’ll hear political fi gures Here’s what the American Association for Public declare that they don’t pay attention to the Opinion Polling had to say in condemning this polls, that they don’t take them seriously. Do practice: you think that either of these points is true? A “push poll” is a telemarketing technique in What in the polling process itself might con- which telephone calls are used to canvass po- tribute to these opinions? tential voters, feeding them false or misleading “information” about a candidate under the pretense of taking a poll to see how this “information” affects voter preferences. In fact, the intent Mark Blumenthal has prepared an excellent is not to measure public opinion but to manipu- overview of political polling* at http:// www.pollster.com/blogs/. late it—to “push” voters away from one candidate and toward the opposing candidate. Such polls defame selected candidates by spreading 1987 Hite report was based on questionnaires com- false or misleading information about them. The pleted by women around the country—but which intent is to disseminate campaign propaganda women? Hite reported that she distributed some under the guise of conducting a legitimate pub- 100,000 questionnaires through various organiza- lic opinion poll. — (BEDNARZ 1996) tions, and around 4,500 were returned. Now, 4,500 In short, the labels “survey” and “poll” are some- and 100,000 are large numbers in the context of times misused. Done properly, however, survey survey sampling. However, given Hite’s research research can be a useful tool of social inquiry. De- methods, her 4,500 respondents didn’t necessarily signing useful (and trustworthy) survey research represent U.S. women any more than the Literary begins with formulating good questions. Let’s turn Digest’s enormous 1936 sample represented the to that topic now. U.S. electorate when their two million sample ballots indicated that Alf Landon would bury FDR in a landslide. GUIDELINES FOR ASKING QUESTIONS Sometimes, people use the pretense of survey research for quite different purposes, as the “What Do You Think” box suggested. For example, In social research, variables are often operation- you may have received a telephone call indicat- alized when researchers ask people questions as ing you’ve been selected for a survey, only to fi nd a way of getting data for analysis and interpretation. Sometimes the questions are asked by an interviewer; sometimes they are written down *Each time the Internet icon appears, you’ll be given helpful leads for searching the World Wide Web. and given to respondents for completion. In other 272 CHAPTER 9 SURVEY RESEARCH cases, several general guidelines can help re- may remember, Rensis Likert greatly formalized searchers frame and ask questions that serve as this procedure through the creation of the Likert excellent operationalizations of variables while scale, a format in which respondents are asked avoiding pitfalls that can result in useless or even to strongly agree, agree, disagree, or strongly dis- misleading information. agree, or perhaps strongly approve, approve, and Surveys include the use of a questionnaire— so forth. an instrument specifi cally designed to elicit infor- You can use both questions and statements mation that will be useful for analysis. Although profi tably. Using both in a given questionnaire gives some of the specifi c points to follow are more ap- you more fl exibility in the design of items and can propriate to structured questionnaires than to the make the questionnaire more interesting as well. more open-ended questionnaires used in qualitative, in-depth interviewing, the underlying logic Open-Ended and Closed-Ended Questions In is valuable whenever we ask people questions in asking questions, researchers have two options. order to gather data. They can ask open-ended questions, in which case the respondent is asked to provide his or her Choose Appropriate Question Forms own answer to the question. For example, the respondent may be asked, “What do you feel is the Let’s begin with some of the options available to most important issue facing the United States you in creating questionnaires. These options in- today?” and be provided with a space to write in clude using questions or statements and choosing the answer (or be asked to report it verbally to an open-ended or closed-ended questions. interviewer). As we’ll see in Chapter 10, in-depth, qualitative interviewing relies almost exclusively Questions and Statements Although the term on open-ended questions. However, they are also questionnaire suggests a collection of questions, a used in survey research. typical questionnaire often presents as many state- In the case of closed-ended questions, the ments as questions. Often, the researcher wants to respondent is asked to select an answer from determine the extent to which respondents hold a among a list provided by the researcher. Closed- particular attitude or perspective. If you can sum- ended questions are quite popular in survey re- marize the attitude in a fairly brief statement, you search because they provide a greater uniformity can present that statement and ask respondents of responses and are more easily processed than whether they agree or disagree with it. As you open-ended ones. Open-ended responses must be coded before they can be processed for computer analysis, as questionnaire A document containing questions and will be discussed in Chapter 14. This coding pro- other types of items designed to solicit information ap- cess often requires the researcher to interpret the propriate for analysis. Questionnaires are used primarily in survey research but also in experiments, fi eld research, and meaning of responses, opening the possibility of other modes of observation. misunderstanding and researcher bias. There is open-ended questions Questions for which the also a danger that some respondents will give respondent is asked to provide his or her own answers. answers that are essentially irrelevant to the re- In-depth, qualitative interviewing relies almost exclusively searcher’s intent. Closed-ended responses, on the on open-ended questions. other hand, can often be transferred directly into a closed-ended questions Survey questions in which computer format. the respondent is asked to select an answer from among a The chief shortcoming of closed-ended ques- list provided by the researcher. These are popular in survey research because they provide a greater uniformity of tions lies in the researcher’s structuring of re- responses and are more easily processed than open-ended sponses. When the relevant answers to a given questions. question are relatively clear, there should be no GUIDELINES FOR ASKING QUESTIONS 273 problem. In other cases, however, the researcher’s The possibilities for misunderstanding are end- structuring of responses might overlook some less, and no researcher is immune (Polivka and important responses. In asking about “the most Rothgeb 1993). One of the most established re- important issue facing the United States,” for ex- search projects in the United States is the Census ample, his or her checklist of issues might omit Bureau’s ongoing “Current Population Survey” or certain issues that respondents would have said CPS, which measures, among other critical data, were important. the nation’s unemployment rate. A part of the The construction of closed-ended questions measurement of employment patterns focuses should be guided by two structural requirements. on a respondent’s activities during “last week,” by First, the response categories should be exhaus- which the Census Bureau means Sunday through tive: They should include all the possible responses Saturday. Studies undertaken to determine the ac- that might be expected. Often, researchers ensure curacy of the survey found that more than half the this by adding a category such as “Other (Please respondents took “last week” to include only Mon- specify: ______).” Second, the answer catego- day through Friday. By the same token, whereas ries must be mutually exclusive: The respondent the Census Bureau defi nes “working full-time” as should not feel compelled to select more than one. 35 or more hours a week, the same evaluation (In some cases, you may wish to solicit multiple studies showed that some respondents used the answers, but these can create diffi culties in data more traditional defi nition of 40 hours per week. processing and analysis later on.) To ensure that As a consequence, the wording of these questions your categories are mutually exclusive, carefully in the CPS was modifi ed in 1994 to specify the Cen- consider each combination of categories, ask- sus Bureau’s defi nitions. ing yourself whether a person could reasonably Similarly, the use of the term Native American to choose more than one answer. In addition, it’s mean American Indian often produces an overrep- useful to add an instruction to the question asking resentation of that ethnic group in surveys. Clearly, the respondent to select the one best answer, but many respondents understand the term to mean this technique cannot serve as a substitute for a “born in the United States.” carefully constructed set of responses. Avoid Double-Barreled Questions Make Items Clear Frequently, researchers ask respondents for a sin- It should go without saying that questionnaire items gle answer to a question that actually has multiple should be clear and unambiguous, but the broad parts. That seems to happen most often when the proliferation of unclear and ambiguous questions researcher has personally identifi ed with a com- in surveys makes the point worth emphasizing. Of- plex question. For example, you might ask respon- ten we can become so deeply involved in the topic dents to agree or disagree with the statement “The under examination that opinions and perspectives United States should abandon its space program are clear to us but not to our respondents, many and spend the money on domestic programs.” Al- of whom have paid little or no attention to the though many people would unequivocally agree topic. Or, if we have only a superfi cial understand- with the statement and others would unequivo- ing of the topic, we may fail to specify the intent of cally disagree, still others would be unable to an- a question suffi ciently. The question “What do you swer. Some would want to abandon the space pro- think about the proposed peace plan?” may evoke gram and give the money back to the taxpayers. in the respondent a counterquestion: “Which pro- Others would want to continue the space program posed peace plan?” Questionnaire items should be but also put more money into domestic programs. precise so that the respondent knows exactly what These latter respondents could neither agree nor the researcher is asking. disagree without misleading you. 274 CHAPTER 9 SURVEY RESEARCH

As a general rule, whenever the word and ap- survive under authoritarianism. During the pears in a question or questionnaire statement, Cultural Revolution between 1966 and 1976, for check whether you’re asking a double-barreled example, because of the radical political agenda question. See the box “Double-Barreled and Be- and political intensity throughout the country, it yond” for some imaginative variations on this was almost impossible to use survey techniques theme. to collect valid and reliable data inside China about the Chinese people’s life experiences, Respondents Must Be characteristics, and attitudes towards the Com- Competent to Answer munist regime. — (1994:19–20)

In asking respondents to provide information, you Sometimes, U.S. respondents may say they’re should continually ask yourself whether they can undecided when, in fact, they have an opinion but do so reliably. In a study of child rearing, you might think they’re in a minority. Under that condition, ask respondents to report the age at which they they may be reluctant to tell a stranger (the inter- fi rst talked back to their parents. Quite aside from viewer) what that opinion is. Given this problem, the problem of defi ning talking back to parents, it’s the Gallup Organization, for example, has used a doubtful that most respondents would remember “secret ballot” format, which simulates actual elec- with any degree of accuracy. tion conditions, in that the “voter” enjoys complete As another example, student government lead- anonymity. In an analysis of the Gallup Poll elec- ers occasionally ask their constituents to indicate tion data from 1944 to 1988, Andrew Smith and how students’ fees ought to be spent. Typically, re- G. F. Bishop (1992) found that this technique sub- spondents are asked to indicate the percentage of stantially reduced the percentage of respondents available funds that should be devoted to a long who said they were undecided about how they list of activities. Without a fairly good knowledge would vote. of the nature of those activities and the costs in- This problem is not limited to survey research, volved in them, the respondents cannot provide however. Richard Mitchell faced a similar problem meaningful answers. Administrative costs, for ex- in his fi eld research among U.S. survivalists: ample, will receive little support although they may Survivalists, for example, are ambivalent about be essential to the program as a whole. concealing their identities and inclinations. One group of researchers examining the driv- They realize that secrecy protects then from the ing experience of teenagers insisted on asking an ridicule of a disbelieving majority, but enforced open-ended question concerning the number of separatism diminishes opportunities for recruit- miles driven since receiving a license, even though ment and information exchange. . . . consultants argued that few drivers could estimate “Secretive” survivalists eschew telephones, such information with any accuracy. In response, launder their mail through letter exchanges, use some teenagers reported driving hundreds of nicknames and aliases, and carefully conceal thousands of miles. their addresses from strangers. Yet once I was invited to group meetings, I found them coop- Respondents Must erative respondents. — (1991:100) Be Willing to Answer

Often, we would like to learn things from people Questions Should Be Relevant that they are unwilling to share with us. For exam- Similarly, questions asked in a questionnaire ple, Yanjie Bian indicates that it has often been dif- should be relevant to most respondents. When at- ﬁ cult to get candid answers from people in China. titudes are requested on a topic that few respon- [Here] people are generally careful about what dents have thought about or really care about, they say on nonprivate occasions in order to the results are not likely to be useful. Of course, ISSUES AND INSIGHTS DOUBLE-BARRELED AND BEYOND

Even established, professional researchers have U.S. Will War Is sometimes created double-barreled questions Not Probable Go to but Not War Is and worse. Consider this question, asked of War Inevitable Inevitable U.S. citizens in April 1986, at a time when the U.S. will not country’s relationship with Libya was at an es- invade Libya 1 2 3 pecially low point. Some observers suggested U.S. will invade the United States might end up in a shooting Libya but it war with the small North African nation. The would be wrong 4 5 Harris Poll sought to fi nd out what U.S. public U.S. will invade Libya and it opinion was. would be right 6 7 If Libya now increases its terrorist acts against the U.S. and we keep infl icting more The examination of prognoses about the Libyan damage on Libya, then inevitably it will situation is not the only example of double- all end in the U.S. going to war and fi nally barreled questions sneaking into public opinion invading that country which would be research. Here are some statements the Harris wrong. Poll asked people to agree or disagree with in an attempt to gauge U.S. public opinion about then Respondents were given the opportunity of Soviet General Secretary Gorbachev: answering “Agree,” “Disagree,” or “Not sure.” Notice the elements contained in the complex He looks like the kind of Russian leader who statement: will recognize that both the Soviets and the Americans can destroy each other with 1. Will Libya increase its terrorist acts against nuclear missiles so it is better to come to the U.S.? verifi able arms control agreements. 2. Will the U.S. infl ict more damage on He seems to be more modern, enlight- Libya? ened, and attractive, which is a good sign for 3. Will the U.S. inevitably or otherwise go to the peace of the world. war against Libya? Even though he looks much more mod- 4. Would the U.S. invade Libya? ern and attractive, it would be a mistake to 5. Would that be right or wrong? think he will be much different from other These several elements offer the possibility of Russian leaders. numerous points of view—far more than the How many elements can you identify in each of three alternatives offered respondents to the the statements? How many possible opinions survey. Even if we were to assume hypotheti- could people have in each case? What does a cally that Libya would “increase its terrorist at- simple “agree” or “disagree” really mean in such tacks” and the United States would “keep infl ict- cases? ing more damage” in return, you might have any one of at least seven distinct expectations about Sources: Reported in World Opinion Update, Oc tober the outcome: 1985 and May 1986, respectively. 276 CHAPTER 9 SURVEY RESEARCH because the respondents might express attitudes sizable portion of the respondents will read over even though they have never given any thought to the word not and answer on that basis. Thus, the issue, researchers run the risk of being misled. some will agree with the statement when they’re This point is illustrated occasionally when rein favor of recognition, and others will agree when searchers ask for responses relating to fi ctitious they oppose it. And you may never know which people and issues. In one political poll I conducted, are which. I asked respondents whether they were familiar Similar considerations apply to other “negative” with each of 15 political fi gures in the community. words. In a study of support for civil liberties, for As a methodological exercise, I made up a name: example, respondents were asked whether they Tom Sakumoto. In response, 9 percent of the re- felt “the following kinds of people should be pro- spondents said they were familiar with him. Of hibited from teaching in public schools” and were those respondents familiar with him, about half represented with a list including such items as a ported seeing him on television and reading about Communist, a Ku Klux Klansman, and so forth. him in the newspapers. The response categories “yes” and “no” were given When you obtain responses to fi ctitious issues, beside each entry. A comparison of the responses you can disregard those responses. But when the to this item with other items refl ecting support issue is real, you may have no way of telling which for civil liberties strongly suggested that many re- responses genuinely refl ect attitudes and which respondents gave the answer “yes” to indicate will- fl ect meaningless answers to an irrelevant question. ingness for such a person to teach, rather than to Ideally, we would like respondents simply to indicate that such a person should be prohibited report that they don’t know, have no opinion, or from teaching. (A later study in the series giving are undecided in those instances where that is the as answer categories “permit” and “prohibit” pro- case. Unfortunately, however, they often make up duced much clearer results.) answers. In 1993 a national survey commissioned by the American Jewish Committee produced shock- Short Items Are Best ing results: One American in fi ve believed that the Holocaust—in which six million Jews were report- In the interests of being unambiguous and pre- edly killed—never happened; further, one in three cise and of pointing to the relevance of an issue, Americans expressed some doubt that it had oc- researchers tend to create long and complicated curred. This research fi nding suggested that the items. That should be avoided. Respondents are Holocaust Revisionist movement in America was often unwilling to study an item in order to under- powerfully infl uencing public opinion (“1 in 5 in stand it. The respondent should be able to read an New Survey” 1993). item quickly, understand its intent, and select or In the aftermath of this shocking news, re- provide an answer without diffi culty. In general, searchers reexamined the actual question that assume that respondents will read items quickly had been asked: “Does it seem possible or does it and give quick answers. Accordingly, provide clear, seem impossible to you that the Nazi extermina- short items that respondents will not misinterpret tion of the Jews never happened?” On refl ection, under those conditions. it seemed clear that the complex, double-negative question could have confused some respondents. Avoid Negative Items A new survey was commissioned and asked, “Does it seem possible to you that the Nazi exter- The appearance of a negation in a questionnaire mination of the Jews never happened, or do you item paves the way for easy misinterpretation. feel certain that it happened?” In the follow-up sur- Asked to agree or disagree with the statement vey, only 1 percent of the respondents believed the “The United States should not recognize Cuba,” a Holocaust never happened, and another 8 percent GUIDELINES FOR ASKING QUESTIONS 277 said they weren’t sure (“Poll on Doubt of Holocaust More Support Less Support Is Corrected” 1993). “Assistance to the poor” ”Welfare” “Halting rising crime rate” ”Law enforcement” Avoid Biased Items and Terms “Dealing with drug addiction” ”Drug rehabilitation” “Solving problems of big ”Assistance to big Recall from our discussion of conceptualization cities” cities” and operationalization in Chapter 5 that there are “Improving conditions of blacks” ”Assistance to blacks” no ultimately true meanings for any of the concepts “Protecting social security” ”Social security” we typically study in social science. Prejudice has no ultimately correct defi nition; whether a given In 1986, for example, 62.8 percent of the respon- person is prejudiced depends on our defi nition of dents said too little money was being spent on that term. The same general principle applies to “assistance to the poor,” while in a matched survey the responses we get from people completing a that year, only 23.1 percent said we were spending questionnaire. too little on “welfare.” The meaning of someone’s response to a ques- In this context, be wary of what researchers tion depends in large part on its wording. This is call the social desirability of questions and answers. true of every question and answer. Some ques- Whenever we ask people for information, they an- tions seem to encourage particular responses swer through a fi lter of what will make them look more than do other questions. In the context of good. This is especially true if they’re interviewed questionnaires, bias refers to any property of face-to-face. Thus, for example, a particular man questions that encourages respondents to answer may feel that things would be a lot better if women in a particular way. were kept in the kitchen, excluded from voting, Most researchers recognize the likely effect of forced to be quiet in public, and so forth. Asked a question that begins, “Don’t you agree with the whether he supports equal rights for women, how- President of the United States that . . . “ and no ever, he may want to avoid looking like a chauvin- reputable researcher would use such an item. Un- ist. Recognizing that his views are out of step with happily, the biasing effect of items and terms is far current thinking, he may choose to say “yes.” subtler than this example suggests. The best way to guard against this problem is The mere identifi cation of an attitude or posi- to imagine how you would feel giving each of the tion with a prestigious person or agency can bias answers you intend to offer to respondents. If you responses. The item “Do you agree or disagree would feel embarrassed, perverted, inhumane, with the recent Supreme Court decision that . . . “ stupid, irresponsible, or otherwise socially disad- would have a similar effect. Such wording may not vantaged by any particular response, give serious produce consensus or even a majority in support thought to how willing others will be to give those of the position identifi ed with the prestigious per- answers. son or agency, but it will likely increase the level The biasing effect of particular wording is often of support over what would have been obtained diffi cult to anticipate. In both surveys and experi- without such identifi cation. ments, it’s sometimes useful to ask respondents to Sometimes the impact of different forms of consider hypothetical situations and say how they question wording is relatively subtle. For example, think they would behave. Because those situations when Kenneth Rasinski (1989) analyzed the re- often involve other people, the names used can sults of several General Social Survey studies of attitudes toward government spending, he found that the way programs were identifi ed affected the bias That quality of a measurement device that tends to amount of public support they received. Here are result in a misrepresentation, in a particular direction, of some comparisons: what is being measured. 278 CHAPTER 9 SURVEY RESEARCH affect responses. For example, researchers have As a general rule, a questionnaire should be long known that male names for the hypothetical spread out and uncluttered. If a self-administered people may produce different responses than do questionnaire is being designed, inexperienced female names. Research by Joseph Kasof (1993) researchers tend to fear that their questionnaire points to the importance of what the specifi c will look too long; as a result, they squeeze several names are: whether they generally evoke positive questions onto a single line, abbreviate questions, or negative images in terms of attractiveness, age, and use as few pages as possible. These efforts intelligence, and so forth. Kasof’s review of past are ill-advised and even dangerous. Putting more research suggests there has been a tendency to than one question on a line will cause some re- use more positively valued names for men than for spondents to miss the second question altogether. women. Some respondents will misinterpret abbreviated As in all other research, carefully examine the questions. More generally, respondents who fi nd purpose of your inquiry and construct items that they have spent considerable time on the fi rst page will be most useful to it. You should never be mis- of what seemed a short questionnaire will be more led into thinking there are ultimately “right” and demoralized than respondents who quickly com- “wrong” ways of asking the questions. When in plete the fi rst several pages of what initially seemed doubt about the best question to ask, moreover, a rather long form. Moreover, the latter will have remember that you should ask more than one made fewer errors and will not have been forced to question for a given variable. reread confusing, abbreviated questions. Nor will These, then, are some general guidelines for they have been forced to write a long answer in a writing questions to elicit data for analysis and tiny space. interpretation. Next we look at how to construct Similar problems can arise for interviewers in a questionnaires. face-to-face or telephone interview. Like respondents with a self-administered questionnaire, in- QUESTIONNAIRE CONSTRUCTION terviewers may miss questions, lose their place, and generally become frustrated and fl ustered. Interview questionnaires need to be laid out in a Questionnaires are used in connection with many way that supports the interviewer’s work, includ- modes of observation in social research. Although ing special instructions and guidelines. structured questionnaires are essential to and The desirability of spreading out questions most directly associated with survey research, they in the questionnaire cannot be overemphasized. are also widely used in experiments, fi eld research, Squeezed-together questionnaires are disastrous, and other data-collection activities. For this rea- whether completed by the respondents them- son, questionnaire construction can be an impor- selves or administered by trained interviewers. tant practical skill for researchers. As we discuss The processing of such questionnaires is another the established techniques for constructing ques- nightmare. tionnaires, let’s begin with some issues concerning questionnaire format. Formats for Respondents

General Questionnaire Format In one of the most common types of questionnaire items, the respondent is expected to check one The format of a questionnaire is just as important response from a series. In my experience, boxes as the nature and wording of the questions asked. adequately spaced apart provide the best format An improperly laid out questionnaire can lead refor this purpose. Modern word processing makes spondents to miss questions, confuse them about the use of boxes a practical technique; setting the nature of the data desired, and even lead them boxes in type can also be accomplished easily and to throw the questionnaire away. neatly. You can approximate boxes by using brack- QUESTIONNAIRE CONSTRUCTION 279 ets: [ ], but if you’re creating a questionnaire on a Did you happen to vote in the last presidential computer, you should take the few extra minutes election? to use genuine boxes that will give your question- 1. Yes naire a more professional look. Here are some 2. No easy examples: 3. Don't know ❏ ❍ ٗ Have you ever felt you were the victim of Rather than providing boxes to be checked, you sexual discrimination? might print a code number beside each response 1. Yes and ask the respondent to circle the appropriate 2. No number (see Figure 9-1). This method has the 3. Don't know added advantage of specifying the code number to be entered later in the processing stage (see Chap- FIGURE 9-1 Circling the Answer ter 14). If numbers are to be circled, however, you should provide clear and prominent instructions to the respondent, because many will be tempted to There are several formats for contingency cross out the appropriate number, which makes questions. The one shown in Figure 9-2 is prob- data processing even more diffi cult. (Note that the ably the clearest and most effective. Note two key technique can be used more safely when interview- elements in this format. First, the contingency ers administer the questionnaires, because the in- question is isolated from the other questions by terviewers themselves record the responses.) being set off to the side and enclosed in a box. Sec- ond, an arrow connects the contingency question Contingency Questions to the answer on which it is contingent. In the illustration, only those respondents answering “yes” Quite often in questionnaires, certain questions are expected to answer the contingency ques- will be relevant to some of the respondents and ir- tion. The rest of the respondents should simply relevant to others. In a study of birth control meth- skip it. ods, for instance, you would probably not want to Note that the questions shown in Figure 9-2 ask men if they take birth control pills. could have been dealt with in a single question. This sort of situation often arises when re- The question might have read, “How many times, searchers wish to ask a series of questions about if any, have you smoked marijuana?” The response a certain topic. You may want to ask whether categories, then, might have read: “Never,” “Once,” your respondents belong to a particular organiza- “2 to 5 times,” and so forth. This single question tion and, if so, how often they attend meetings, would apply to all respondents, and each would whether they have held offi ce in the organization, fi nd an appropriate answer category. Such a ques- and so forth. Or, you might want to ask whether tion, however, might put some pressure on respon- respondents have heard anything about a certain dents to report having smoked marijuana, because political issue and then learn the attitudes of those the main question asks how many times they have who have heard of it. Each subsequent question in series such as these is called a contingency question: Whether it is to be asked and answered is contingent on contingency question A survey question intended responses to the fi rst question in the series. The for only some respondents, determined by their responses to some other question. For example, all respondents proper use of contingency questions can facilitate might be asked whether they belong to the Cosa Nostra, the respondents’ task in completing the question- and only those who said yes would be asked how often naire, because they are not faced with trying to an- they go to company meetings and picnics. The latter swer questions irrelevant to them. would be a contingency question. 280 CHAPTER 9 SURVEY RESEARCH smoked it, even though it allows for those excep- 23. Have you ever smoked marijuana? tional cases who have never smoked marijuana Yes even once. (The emphases used in the previous sentence give a fair indication of how respondents No might read the question.) The contingency ques- If yes: About how many times have tion format illustrated in Figure 9-2 should reduce you smoked marijuana? the subtle pressure on respondents to report hav- Once ing smoked marijuana. 2 to 5 times Used properly, even rather complex sets of contingency questions can be constructed without 6 to 10 times confusing the respondent. Figure 9-3 illustrates a 11 to 20 times more complicated example. More than 20 times Sometimes a set of contingency questions is long enough to extend over several pages. Sup- FIGURE 9-2 Contingency Question pose you’re studying the political activities of col- Format. Contingency questions offer a structure lege students, and you wish to ask a large number for exploring subject areas logically in some depth. of questions of those students who have voted in a national, state, or local election. You could separate out the relevant respondents with an 24. Have you ever been abducted by aliens? initial question such as “Have you ever voted in a Yes national, state, or local election?” but it would be confusing to place the contingency questions in a No box stretching over several pages. It would make If yes: Did they let you steer the ship? more sense to enter instructions in parentheses after each answer telling respondents to answer Yes or skip the contingency questions. Figure 9-4 pro- No vides an illustration of this method. In addition to these instructions, place an in- If yes: How fast did you go? struction at the top of each page containing only Warp speed the contingency questions. For example, you might Weenie speed say, “This page is only for respondents who have voted in a national, state, or local election.” Clear FIGURE 9-3 Contingency Table. Sometimes it instructions such as these spare respondents the will be appropriate for certain kinds of respondents frustration of reading and puzzling over questions to skip over inapplicable questions. To avoid confu- that are irrelevant to them and increase the likeli- sion, you should provide clear instructions to that end. hood of responses from only those for whom the questions are relevant.

Matrix Questions 13. Have you ever voted in a national, state, or local election? Quite often you’ll want to ask several questions Yes (Please answer questions 14–25.) that have the same set of answer categories. This No (Please skip questions 14–25. Go directly is typically the case whenever the Likert response to question 26 on page 8.) categories are used. In such cases, a matrix of items and answers is often possible, as illustrated FIGURE 9-4 Instructions for Skipping in Figure 9-5. Questions QUESTIONNAIRE CONSTRUCTION 281

17. Beside each of the statements presented below, please indicate whether you Strongly Agree (SA), Agree (A), Disagree (D), Strongly Disagree (SD), or are Undecided (U).

SA A D SD U a. What this country needs is more law and order...... b. The police should be disarmed in America...... c. During riots, looters should be shot on sight...... etc.

FIGURE 9-5 Matrix Question Format. Matrix questions offer an efﬁ cient format for presenting a set of closed-ended questionnaire items that have the same response categories.

This format offers several advantages over statements representing different orientations and other formats. First, it uses space effi ciently. Sec- by making all statements short and clear. ond, respondents will probably complete such a set of questions more quickly than other formats Ordering Items in a Questionnaire would allow. In addition, this format may increase the comparability of responses given to different The order in which questionnaire items are pre- questions—for the respondent as well as for the resented can also affect responses. First, the appear- searcher. Because respondents can quickly review ance of one question can affect the answers given their answers to earlier items in the set, they might to later ones. For example, if several questions choose between, say, “strongly agree” and “agree” have been asked about the dangers of terrorism to on a given statement by comparing the strength the United States and then a question asks respon- of their agreement with their earlier responses in dents to list (open-ended) things that they believe the set. represent dangers to the United States, terrorism This format, however, holds some inherent will receive more citations than would otherwise dangers. Its advantages may encourage you to be the case. In this situation, asking the open- structure an item so that the responses fi t into the ended question fi rst is best. matrix format, when a different, more idiosyncratic Similarly, if respondents are asked to assess set of responses might be more appropriate. Also, their overall religiosity (“How important is your re- the matrix question format can foster a response- ligion to you in general?”), their responses to later set among some respondents: They may develop questions concerning specifi c aspects of religiosity a pattern of, say, agreeing with all the statements. will be aimed at consistency with the prior assess- This would be especially likely if the set of statement. The converse is true as well. If respondents ments began with several that indicated a par- are fi rst asked specifi c questions about different ticular orientation (for example, a liberal political aspects of their religiosity, their subsequent overall perspective) with only a few later ones represent- assessment will refl ect the earlier answers. The or- ing the opposite orientation. Respondents might der of responses within a question can also make assume that all the statements represent the same a difference (Bishop and Smith 2001). orientation and, reading quickly, misread some The impact of item order is not uniform among of them, thereby giving the wrong answers. This respondents. When J. Edwin Benton and John Daly problem can be reduced somewhat by alternating (1991) conducted a local government survey, they 282 CHAPTER 9 SURVEY RESEARCH found that the less-educated respondents were opens, the interviewer must gain rapport quickly. more infl uenced by the order of questionnaire After a short introduction to the study, the inter- items than were those with more education. viewer can best begin by enumerating the mem- Some researchers attempt to overcome this bers of the household, getting demographic data effect by randomizing the order of items. This ef- about each. Such items are easily answered and fort is usually futile. In the fi rst place, a random- generally nonthreatening. Once the initial rapport ized set of items will probably strike respondents has been established, the interviewer can then as chaotic and worthless. The random order also move into more sensitive matters. An interview makes answering more diffi cult because respon- that began with the question “Do you believe in dents must continually switch their attention from witchcraft?” would probably end rather quickly. one topic to another. Finally, even a randomized ordering of items will have the effect discussed Questionnaire Instructions previously—except that researchers will have no control over the effect. Every questionnaire, whether it is to be completed The safest solution is sensitivity to the problem. by respondents or administered by interviewers, Although you cannot avoid the effect of item order, should contain clear instructions and introductory try to estimate what that effect will be so that you comments where appropriate. can interpret results meaningfully. If the order of It’s useful to begin every self-administered ques- items seems especially important in a given study, tionnaire with basic instructions for completing it. you might construct more than one version of the Although many people these days have experience questionnaire with different orderings of the items with forms and questionnaires, begin by telling in each. You will then be able to determine the ef- them exactly what you want: that they are to indi- fects by comparing responses to the various ver- cate their answers to certain questions by placing sions. At the very least, you should pretest your a check mark or an X in the box beside the appro- questionnaire in the different forms. (We’ll discuss priate answer or by writing in their answer when pretesting in a moment.) asked to do so. If many open-ended questions are The desired ordering of items differs between used, respondents should receive some guidelines interviews and self-administered questionnaires. about whether brief or lengthy answers are ex- In the latter, it’s usually best to begin the question- pected. If you wish to encourage your respondents naire with the most interesting set of items. The to elaborate on their responses to closed-ended potential respondents who glance casually over questions, that should be noted. the fi rst few items should want to answer them. If a questionnaire has subsections—political at- Perhaps the items will ask for attitudes they’re ach- titudes, religious attitudes, background data—in- ing to express. At the same time, the initial items troduce each with a short statement concerning should not be threatening. (It might be a bad idea its content and purpose. For example, “In this sec- to begin with items about sexual behavior or drug tion, we would like to know what people consider use.) Requests for duller, demographic data (age, the most important community problems.” De- sex, and the like) should generally go at the end mographic items at the end of a self-administered of a self-administered questionnaire. Placing these questionnaire might be introduced thus: “Finally, items at the beginning, as many inexperienced re- we would like to know just a little about you so we searchers are tempted to do, gives the question- can see how different types of people feel about naire the initial appearance of a routine form, and the issues we have been examining.” the person receiving it might not be motivated to Short introductions such as these help the re- complete it. spondent make sense of the questionnaire. They Just the opposite is generally true for interview make the questionnaire seem less chaotic, espe- surveys. When the potential respondent’s door fi rst cially when it taps a variety of data. And they help QUESTIONNAIRE CONSTRUCTION 283 put the respondent in the proper frame of mind for Pretesting the Questionnaire answering the questions. No matter how carefully researchers design a Some questions may require special instruc- data-collection instrument such as a question- tions to facilitate proper answering. This is es- naire, there is always the possibility—indeed the pecially true if a given question varies from the certainty—of error. They will always make some general instructions pertaining to the whole ques- mistake: an ambiguous question, one that people tionnaire. The following three examples illustrate cannot answer, or some other violation of the rules this situation. just discussed. Despite attempts to provide mutually exclusive The surest protection against such errors is to answers in closed-ended questions, often more pretest the questionnaire in full or in part. Give than one answer will apply for respondents. If the questionnaire to the ten people in your bowl- you want a single answer, you should make this ing league, for example. It’s not usually essential perfectly clear in the question. An example would that the pretest subjects compose a representative be “From the list below, please check the primary sample, although you should use people for whom reason for your decision to attend college.” Often the questionnaire is at least relevant. the main question can be followed by a parentheti- By and large, it’s better to ask people to com- cal note: “Please check the one best answer.” If, on plete the questionnaire than to read through it the other hand, you want the respondent to check looking for errors. All too often, a question seems as many answers as apply, you should make this to make sense on a fi rst reading but proves impos- clear. sible to answer. When the respondent needs to rank-order a Stanley Presser and Johnny Blair (1994) describe set of answer categories, the instructions should several different pretesting strategies and report indicate this, and a different type of answer for- on the effectiveness of each. They also provide mat should be used (for example, blanks instead data on the cost of the various methods. You might of boxes). These instructions should indicate how want to review their work before sending out your many answers are to be ranked (for example: all; fi rst or next questionnaire. only the fi rst and second; only the fi rst and last; the There are many more tips and guidelines for most important and least important). These in- questionnaire construction, but covering them all structions should also spell out the order of rank- would take a book in itself. Now I’ll complete this ing (for example, “Place a 1 beside the most impor- discussion with an excerpt taken from a real ques- tant item, a 2 beside the next most important, and tionnaire, showing how some of these comments so forth”). Rank-ordering of responses is often diffi nd substance in practice. fi cult for respondents, however, because they may have to read and reread the list several times, so this technique should only be used in those situ- A Sample Questionnaire ations where no other method will produce the desired result. Figure 9-6 is part of a questionnaire used by the In multiple-part matrix questions, it’s useful to University of Chicago’s National Opinion Research give special instructions unless the same format is Center in its General Social Survey. The question- used throughout the questionnaire. Sometimes re- naire deals with people’s attitudes toward the gov- spondents will be expected to check one answer in ernment and is designed to be self-administered, each column of the matrix; in other questionnaires though most of the GSS is conducted in face-to- they’ll be expected to check one answer in each face interviews. row. Whenever the questionnaire contains both You may notice several mysterious-looking formats, it’s useful to add an instruction clarifying numbers in the right-hand margins of this sample which is expected in each case. survey. These refl ect a critical aspect of question- 10. Here are some things the government might do for the economy. Circle one number for each action to show whether you are in favor of it or against it.

1. Strongly in favor of 2. In favor of 3. Neither in favor of nor against 4. Against 5. Strongly against

PLEASE CIRCLE A NUMBER a. Control of wages by legislation ...... 1234528/ b. Control of prices by legislation...... 1 2 3 4 5 29/ c. Cuts in government spending...... 1 2 3 4 5 30/ d. Government ﬁnancing of projects to create new jobs...... 1 2 3 4 5 31/ e. Less government regulation of business...... 1 2 3 4 5 32/ f. Support for industry to develop new products and technology...... 1 2 3 4 5 33/ g. Supporting declining industries to protect jobs...... 1 2 3 4 5 34/ h. Reducing the work week to create more jobs...... 1 2 3 4 5 35/ 11. Listed below are various areas of government spending. Please indicate whether you would like to see more or less government spending in each area. Remember that if you say “much more,” it might require a tax increase to pay for it.

1. Spend much more 2. Spend more 3. Spend the same as now 4. Spend less 5. Spend much less 8. Can’t choose

PLEASE CIRCLE A NUMBER a. The environment ...... 12345836/ b. Health ...... 12345837/ c. The police and law enforcement ...... 12345838/ d. Education...... 1 2 3 4 5 8 39/ e. The military and defense...... 1 2 3 4 5 8 40/ f. Retirement benefits...... 1 2 3 4 5 8 41/ g. Unemployment benefits...... 1 2 3 4 5 8 42/ h. Culture and the arts...... 1 2 3 4 5 8 43/ 12. If the government had to choose between keeping down inflation or keeping down unemployment, to which do you think it should give highest priority? Keeping down inflation ...... 144/ Keeping down unemployment ...... 2 Can’t choose ...... 8 13. Do you think that labor unions in this country have too much power or too little power? Far too much power ...... 145/ Too much power ...... 2 About the right amount of power...... 3 Too little power ...... 4 Far too little power ...... 5 Can’t choose ...... 8

FIGURE 9-6 A Sample Questionnaire. This questionnaire excerpt is from the General Social Survey, a major source of data for analysis by social researchers around the world. 14. How about business and industry, do they have too much power or too little power? Far too much power ...... 146/ Too much power ...... 2 About the right amount of power...... 3 Too little power ...... 4 Far too little power ...... 5 Can’t choose ...... 8 15. And what about the federal government, does it have too much power or too little power? Far too much power ...... 147/ Too much power ...... 2 About the right amount of power...... 3 Too little power ...... 4 Far too little power ...... 5 Can’t choose ...... 8 16. In general, how good would you say labor unions are for the country as a whole? Excellent ...... 148/ Very good ...... 2 Fairly good ...... 3 Not very good ...... 4 Not good at all ...... 5 Can’t choose ...... 8 17. What do you think the government’s role in each of these industries should be?

1. Own it 2. Control prices and proﬁts but not own it 3. Neither own it nor control its prices and proﬁts 8. Can’t choose

PLEASE CIRCLE A NUMBER a. Electric power ...... 123849/ b. The steel industry ...... 123850/ c. Banking and insurance ...... 123851/ 18. On the whole, do you think it should or should not be the government’s responsibility to . . .

1. Deﬁnitely should be 2. Probably should be 3. Probably should not be 4. Deﬁnitely should not be 8. Can’t choose

PLEASE CIRCLE A NUMBER a. Provide a job for everyone who wants one ...... 1234852/ b. Keep prices under control ...... 1234853/ c. Provide health care for the sick ...... 1234854/ d. Provide a decent standard of living for the old ...... 1234855/

FIGURE 9-6 A Sample Questionnaire (continued ) 286 CHAPTER 9 SURVEY RESEARCH naire design: precoding. Because the information up the questionnaires and check them for com- collected by questionnaires is typically transformed pleteness. Or, research workers can hand deliver into some type of computer format, it’s usually ap- questionnaires with a request that the respondents propriate to include data-processing instructions mail the completed questionnaires to the research on the questionnaire itself. These instructions in- offi ce. dicate where specifi c pieces of information will be On the whole, when a research worker either stored in the machine-readable data fi les. delivers the questionnaire, picks it up, or both, the completion rate seems higher than it is in straight- SELF-ADMINISTERED QUESTIONNAIRES forward mail surveys. Additional experimentation with this technique will likely point to other ways to improve completion rates while reducing So far we’ve discussed how to formulate questions costs. The remainder of this section, however, is and how to design effective questionnaires. As im- devoted specifi cally to the mail survey, which re- portant as these tasks are, the labor will be wasted mains the typical form of the self-administered unless the questionnaire produces useful data— questionnaire. which means that respondents have actually completed the questionnaire. We turn now to the major Mail Distribution and Return methods of getting responses to questionnaires. I’ve referred several times in this chapter to in- The basic method for collecting data through the terviews versus self-administered questionnaires. mail has been to send a questionnaire accom- Along these lines, there are three main methods of panied by a letter of explanation and a self-ad- administering survey questionnaires to a sample dressed, stamped envelope. The respondent is ex- of respondents: self-administered questionnaires, pected to complete the questionnaire, put it in the in which respondents are asked to complete the envelope, and return it. If, by any chance, you’ve questionnaire themselves; surveys administered by received such a questionnaire and failed to return interviewers in face-to-face encounters; and sur- it, it would be valuable to recall the reasons you veys conducted by telephone. This section and the had for not returning it and keep them in mind any next two discuss each of these methods in turn. time you plan to send questionnaires to others. The most common form of self-administered A common reason for not returning question- questionnaire is the mail survey. However, several naires is that it’s too much trouble. To overcome other techniques are used as well. At times, it may this problem, researchers have developed several be appropriate to administer a questionnaire to a ways to make returning them easier. For instance, group of respondents gathered at the same place a self-mailing questionnaire requires no return en- at the same time. A survey of students taking in- velope: When the questionnaire is folded a particu- troductory psychology might be conducted in this lar way, the return address appears on the outside. manner during class. High school students might The respondent therefore doesn’t have to worry be surveyed during homeroom period. about losing the envelope. Some recent experimentation has been con- More-elaborate designs are available also. The ducted with regard to the home delivery of ques- university student questionnaire to be described tionnaires. A research worker delivers the ques- later in this chapter was bound in a booklet with tionnaire to the home of sample respondents and a special, two-panel back cover. Once the ques- explains the study. Then the questionnaire is left tionnaire was completed, the respondent needed for the respondent to complete, and the researcher only to fold out the extra panel, wrap it around picks it up later. the booklet, and seal the whole thing with the ad- Home delivery and the mail can also be used in hesive strip running along the edge of the panel. combination. Questionnaires are mailed to fami- The foldout panel contained my return address lies, and then research workers visit homes to pick and postage. When I repeated the study a couple SELF-ADMINISTERED QUESTIONNAIRES 287 of years later, I improved on the design. Both the complete and return the questionnaires? Research- front and back covers had foldout panels: one for ers shouldn’t sit back idly as questionnaires are sending the questionnaire out and the other for returned; instead, they should undertake a care- getting it back—thus avoiding the use of envelopes ful recording of the varying rates of return among altogether. respondents. The point here is that anything you can do to An invaluable tool in this activity is a return- make the job of completing and returning the ques- rate graph. The day on which questionnaires were tionnaire easier will improve your study. Imagine mailed is labeled Day 1 on the graph, and every receiving a questionnaire that made no provisions day thereafter the number of returned question- for its return to the researcher. Suppose you had to naires is logged on the graph. It’s usually best to (1) fi nd an envelope, (2) write the address on it, (3) compile two graphs. One shows the number refi gure out how much postage it required, and (4) turned each day—rising, then dropping. The sec- put the stamps on it. How likely is it that you would ond reports the cumulative number or percentage. return the questionnaire? In part, this activity provides the researchers with A few brief comments on postal options are in gratifi cation, as they get to draw a picture of their order. On outgoing mail, your main choices are successful data collection. More important, how- fi rst-class postage and bulk rate. First class is more ever, it is their guide to how the data collection is certain, but bulk rate is far cheaper. (Check your going. If follow-up mailings are planned, the graph local post offi ce for rates and procedures.) On re- provides a clue about when such mailings should turn mail, your choices are postage stamps and be launched. (The dates of subsequent mailings business-reply permits. Here, the cost differential should be noted on the graph.) is more complicated. If you use stamps, you pay for As completed questionnaires are returned, them whether people return their questionnaires each should be opened, scanned, and assigned an or not. With the business-reply permit, you pay identifi cation (ID) number. These numbers should for only those that are used, but you pay an addi- be assigned serially as the questionnaires are re- tional surcharge of about a nickel. This means that turned, even if other ID numbers have already stamps are cheaper if a lot of questionnaires are been assigned. Two examples should illustrate the returned, but business-reply permits are cheaper important advantages of this procedure. if fewer are returned (and you won’t know in ad- Let’s assume you’re studying attitudes toward vance how many will be returned). a political fi gure. In the middle of the data collec- There are many other considerations involved tion, the media break the story that the politician in choosing among the several postal options. is having extramarital affairs. By knowing the date Some researchers, for example, feel that the use of of that public disclosure and the dates when ques- postage stamps communicates more “humanness” tionnaires were received, you’ll be in a position to and sincerity than do bulk rate and business-reply determine the effects of the disclosure. (Recall the permits. Others worry that respondents will re- discussion in Chapter 8 of history in connection move the stamps and use them for some purpose with experiments.) other than returning the questionnaires. Because In a less sensational way, serialized ID num- both bulk rate and business-reply permits require bers can be valuable in estimating nonresponse establishing accounts at the post offi ce, you’ll prob- biases in the survey. Barring more direct tests of ably fi nd stamps much easier in small surveys. bias, you may wish to assume that those who failed to answer the questionnaire will be more Monitoring Returns like respondents who delayed answering than like those who answered right away. An analysis of The mailing of questionnaires sets up a new re- questionnaires received at different points in the search question that may prove valuable to a data collection might then be used for estimates study: How many and which respondents will of sampling bias. For example, if the GPAs re- 288 CHAPTER 9 SURVEY RESEARCH ported by student respondents decrease steadily collection. The initial mailings will be followed by through the data collection, with those replying a rise and subsequent fall of returns; the follow- right away having higher GPAs and those reply- up mailings will spur a resurgence of returns; and ing later having lower GPAs, you might tenta- more follow-ups will do the same. In practice, three tively conclude that those who failed to answer mailings (an original and two follow-ups) seem the at all have lower GPAs yet. Although it would not most effi cient. be advisable to make statistical estimates of bias The timing of follow-up mailings matters as in this fashion, you could take advantage of ap- well. Here the methodological literature offers less proximate estimates based on the patterns you’ve precise guides, but it has been my experience that observed. two or three weeks is a reasonable space between If respondents have been identifi ed for purposes mailings. (This period might be increased by a few of follow-up mailing, then preparations for those days if the mailing time—out and in—is more than mailings should be made as the questionnaires are two or three days.) returned. The case study later in this section dis- If the individuals in the survey sample are not cusses this process in greater detail. identifi ed on the questionnaires, it might not be possible to remail only to nonrespondents. In such Follow-up Mailings a case, send your follow-up mailing to all members of the sample, thanking those who may have Follow-up mailings can be administered in sev- already participated and encouraging those who eral ways. In the simplest, nonrespondents are have not to do so. (The case study reported later sent a letter of additional encouragement to par- describes another method you can use in an anon- ticipate. A better method is to send a new copy ymous mail survey.) of the survey questionnaire with the follow-up letter. If potential respondents have not returned Acceptable Response Rates their questionnaires after two or three weeks, the questionnaires have probably been lost or mis- A question that new survey researchers frequently placed. Receiving a follow-up letter might encour- ask concerns the percentage return rate, or the re- age them to look for the original questionnaire, sponse rate, that a mail survey should achieve. but if they can’t fi nd it easily, the letter may go for The body of inferential statistics used in connec- naught. tion with survey analysis assumes that all mem- The methodological literature strongly sug- bers of the initial sample complete and return gests that follow-up mailings provide an effective their questionnaires. Because this almost never method for increasing return rates in mail surveys. happens, response bias becomes a concern, with In general, the longer a potential respondent de- the researcher testing (and hoping) for the pos- lays replying, the less likely he or she is to do so at sibility that the respondents look essentially like all. Properly timed follow-up mailings, then, pro- a random sample of the initial sample, and thus vide additional stimuli to respond. a somewhat smaller random sample of the total The effects of follow-up mailings will be seen population. in the response-rate curves recorded during data Nevertheless, overall response rate is one guide to the representativeness of the sample respondents. A high response rate means less response rate The number of people participating in chance of signifi cant response bias than does a a survey divided by the number selected in the sample, in low rate. Conversely, a low response rate is a dan- the form of a percentage. This is also called the completion rate or, in self-administered surveys, the return ger signal, because the nonrespondents are likely rate: the percentage of questionnaires sent out that are to differ from the respondents in ways other than returned. just their willingness to participate in your survey. SELF-ADMINISTERED QUESTIONNAIRES 289

Richard Bolstein (1991), for example, found that closed paper money. Michael Davern and his col- those who did not respond to a preelection po- leagues (2003) found that fi nancial incentives also litical poll were less likely to vote that those who increased completion rates in face-to-face inter- did participate. Estimating the turnout rate from view surveys. the survey respondents, then, would have over- Several factors have reduced response rates estimated the number who would show up at the in telephone surveys. Peter Tuckel and Harry polls. O’Neill (2002) and others have examined the im- But what is a high or low response rate? A pact of such factors as Caller ID, answering ma- quick review of the survey literature uncovers chines, telemarketing, and phone lines being tied a wide range of response rates. Each of these up by faxes and Internet access. All these consti- might be accompanied by a statement like “This tute diffi culties modern survey researchers must is regarded as a relatively high response rate for deal with. a survey of this type.” (A U.S. senator made this Don Dillman (1999) provides an excellent re- statement regarding a poll of constituents that view of the various techniques that survey re- achieved a 4 percent return rate.) Even so, it’s pos- searchers have used to increase return rates on sible to state some general guidelines about remail surveys, and he evaluates the impact of each. turn rates. A review of published social research More important, Dillman stresses the necessity of literature suggests that 50 percent is considered paying attention to all aspects of the study—what adequate for analysis and reporting. A response of he calls the “Total Design Method”—rather than 60 percent is good; a response rate of 70 percent one or two special gimmicks. is very good. Bear in mind, however, that these are More recently, Francis Yammarino, Steven Skin- only rough guides; they have no statistical basis, ner, and Terry Childers (1991) have undertaken an and a demonstrated lack of response bias is far in-depth analysis of the response rates achieved in more important than a high response rate. If you many studies with various techniques. Their fi nd- want to pursue this matter further, Delbert Miller ings are too complex to summarize easily, but you (1991:145–55) has reviewed several specifi c surveys might fi nd some guidance there for effective sur- to offer a better sense of the variability of response vey design. rates. As you can imagine, one of the more persis- A Case Study tent discussions among survey researchers concerns ways of increasing response rates. You’ll The steps involved in the administration of a mail recall that this was a chief concern in the earlier survey are many and can best be appreciated in a discussion of options for mailing out and receiv- walk-through of an actual study. Accordingly, this ing questionnaires. Survey researchers have de- section concludes with a detailed description of veloped many ingenious techniques addressing how the student survey we discussed in Chapter 7 this problem. Some have experimented with novel as an illustration of systematic sampling was ad- formats. Others have tried paying respondents to ministered. This study did not represent the theo- participate. The problem with paying, of course, is retical ideal for such studies, but in that regard it that it’s expensive to make meaningfully high pay- serves present purposes all the better. The study ments to hundreds or thousands of respondents, was conducted by the students in my graduate but some imaginative alternatives have been used. seminar in survey research methods. Some researchers have said, “We want to get your As you may recall, 1,100 students were selected two-cents’ worth on some issues, and we’re will- from the university registration tape through a ing to pay”—enclosing two pennies. Another en- stratifi ed, systematic sampling procedure. For each closed a quarter, suggesting that the respondent student selected, the computer produced six self- make some little child happy. Still others have en- adhesive mailing labels. 290 CHAPTER 9 SURVEY RESEARCH

By the time we were ready to distribute the those who might want more information about the questionnaires, it became apparent that our mea- study. (Five students called for information.) ger research funds wouldn’t cover several mailings By printing the introductory letter on the ques- to the entire sample of 1,100 students (question- tionnaire, we avoided the necessity of enclosing a naire printing costs were higher than anticipated). separate letter in the outgoing envelope, thereby As a result, we chose a systematic two-thirds sam- simplifying the task of assembling mailing pieces. ple of the mailing labels, yielding a subsample of The materials for the initial mailing were as- 733 students. sembled as follows. (1) One mailing label for each Earlier, we had decided to keep the survey student was stuck on a postcard. (2) Another la- anonymous in the hope of encouraging candid bel was stuck on an outgoing manila envelope. (3) responses to some sensitive questions. (Later sur- One postcard and one questionnaire were placed veys of the same issues among the same popula- in each envelope—with a glance to ensure that the tion indicated that this anonymity was unneces- name on the postcard and on the envelope were sary.) Thus, the questionnaires would carry no the same in each case. identifi cation of students on them. At the same The distribution of the survey questionnaires time, we hoped to reduce the follow-up mailing had been set up for a bulk rate mailing. Once the costs by mailing only to nonrespondents. questionnaires had been stuffed into envelopes, To achieve both of these aims, a special post- they were grouped by zip code, tied in bundles, card method was devised. Each student was mailed and delivered to the post offi ce. a questionnaire that carried no identifying marks, Shortly after the initial mailing, questionnaires plus a postcard addressed to the research offi ce— and postcards began arriving at the research of- with one of the student’s mailing labels affi xed to fi ce. Questionnaires were opened, scanned, and the reverse side of the card. The introductory letter assigned identifi cation numbers, as described ear- asked the student to complete and return the ques- lier in this chapter. For every postcard received, a tionnaire—assuring anonymity—and to return the search was made for that student’s remaining la- postcard simultaneously. Receiving the postcard bels, and they were destroyed. would tell us—without indicating which question- After two or three weeks, the remaining mailing naire it was—that the student had returned his or labels were used to organize a follow-up mailing. her questionnaire. This procedure would then fa- This time a special, separate letter of appeal was cilitate follow-up mailings. included in the mailing piece. The new letter indi- The 32-page questionnaire was printed in book- cated that many students had returned their ques- let form. The three-panel cover described earlier in tionnaires already, and it was very important for all this chapter permitted the questionnaire to be re- others to do so as well. turned without an additional envelope. The follow-up mailing stimulated a resurgence A letter introducing the study and its purposes of returns, as expected, and the same logging pro- was printed on the front cover of the booklet. It cedures were continued. The returned postcards explained why the study was being conducted (to told us which additional mailing labels to destroy. learn how students feel about a variety of issues), Unfortunately, time and fi nancial pressures made how students had been selected for the study, the it impossible to undertake a third mailing, as had importance of each student’s responding, and the been initially planned, but the two mailings re- mechanics of returning the questionnaire. sulted in an overall return rate of 62 percent. Students were assured that their responses to This illustration should give you a fairly good the survey were anonymous, and the postcard sense of what’s involved in the execution of mailed method was explained. A statement followed about self-administered questionnaires. Let’s turn now the auspices under which the study was being con- to the second principal method of conducting sur- ducted, and a telephone number was provided for veys: in-person interviews. INTERVIEW SURVEYS 291

INTERVIEW SURVEYS thereby obtaining relevant responses. (As we’ll discuss shortly, such clarifi cations must be strictly controlled through formal specifi cations.) The interview is an alternative method of collect- Finally, the interviewer can observe respon- ing survey data. Rather than asking respondents to dents as well as ask questions. For example, the read questionnaires and enter their own answers, interviewer can note the respondent’s race if this is researchers send interviewers to ask the questions considered too delicate a question to ask. Similar orally and record respondents’ answers. Interview- observations can be made regarding the quality of ing is typically done in a face-to-face encounter, the dwelling, the presence of various possessions, but telephone interviewing, discussed in the next the respondent’s ability to speak English, the re- section, follows most of the same guidelines. spondent’s general reactions to the study, and so Most interview surveys require more than forth. In one survey of students, respondents were one interviewer, although you might undertake a given a short, self-administered questionnaire to small-scale interview survey yourself. Portions of complete—concerning sexual attitudes and be- this section will discuss methods for training and havior—during the course of the interview. While supervising a staff of interviewers assisting you a student completed the questionnaire, the inter- with a survey. viewer made detailed notes regarding the dress This section deals specifi cally with survey inter- and grooming of the respondent. viewing. Chapter 10 discusses the less structured, This procedure raises an ethical issue. Some rein-depth interviews often conducted in qualitative searchers have objected that such practices violate fi eld research. the spirit of the agreement by which the respondent has allowed the interview. Although ethical The Role of the Survey Interviewer issues are seldom clear-cut in social research, it’s important to be sensitive to them (see Chapter 3). There are several advantages to having a ques- Survey research is of necessity based on an un- tionnaire administered by an interviewer rather realistic stimulus-response theory of cognition and than a respondent. To begin with, interview sur- behavior. Researchers must assume that a ques- veys typically attain higher response rates than do tionnaire item will mean the same thing to every mail surveys. A properly designed and executed respondent, and every given response must mean interview survey ought to achieve a completion the same when given by different respondents. rate of at least 80 to 85 percent. (Federally funded Although this is an impossible goal, survey ques- surveys often require one of these response rates.) tions are drafted to achieve the ideal as closely as Respondents seem more reluctant to turn down possible. an interviewer standing on their doorstep than to The interviewer must also fi t into this ideal situ- throw away a mailed questionnaire. ation. The interviewer’s presence should not affect The presence of an interviewer also generally a respondent’s perception of a question or the an- decreases the number of “don’t knows” and “no swer given. In other words, the interviewer should answers.” If minimizing such responses is impor- be a neutral medium through which questions and tant to the study, the interviewer can be instructed answers are transmitted. to probe for answers (“If you had to pick one of the As such, different interviewers should obtain ex- answers, which do you think would come closest actly the same responses from a given respondent. to your feelings?”). Interviewers can also serve as a guard against interview A data-collection encounter in which one questionnaire items that are confusing. If the re- person (an interviewer) asks questions of another (a spondent clearly misunderstands the intent of respondent). Interviews may be conducted face-to-face or a question, the interviewer can clarify matters, by telephone. 292 CHAPTER 9 SURVEY RESEARCH

(Recall our earlier discussions of reliability.) This middle-class neatness and cleanliness may not be neutrality has a special importance in area sam- accepted by all sectors of U.S. society, they remain ples. To save time and money, a given interviewer the primary norm and are the most likely to be ac- is typically assigned to complete all the interviews ceptable to the largest number of respondents. in a particular geographic area—a city block or Dress and grooming are typically regarded as a group of nearby blocks. If the interviewer does signs of a person’s attitudes and orientations. At anything to affect the responses obtained, the bias the time this is being written, torn jeans, green hair, thus interjected might be interpreted as a charac- and razor blade earrings may communicate—cor- teristic of that area. rectly or incorrectly—that the interviewer is politi- Let’s suppose that a survey is being done to de- cally radical, sexually permissive, favorable to drug termine attitudes toward low-cost housing in or- use, and so forth. Any of these impressions could der to help in the selection of a site for a new gov- bias responses or affect the willingness of people ernment-sponsored development. An interviewer to be interviewed. assigned to a given neighborhood might—through In demeanor, interviewers should be pleasant word or gesture—communicate his or her own if nothing else. Because they’ll be prying into a distaste for low-cost housing developments. Re- respondent’s personal life and attitudes, they must spondents might therefore tend to give responses communicate a genuine interest in getting to know in general agreement with the interviewer’s own the respondent without appearing to spy. They position. The results of the survey would indicate must be relaxed and friendly without being too ca- that the neighborhood in question strongly resists sual or clinging. Good interviewers also have the construction of the development in its area when ability to determine very quickly the kind of person in fact their apparent resistance simply refl ects the the respondent will feel most comfortable with, the interviewer’s attitudes. kind of person the respondent would most enjoy talking to. Clearly, the interview will be more successful if the interviewer can become the kind of General Guidelines person the respondent is comfortable with. Fur- for Survey Interviewing ther, because respondents are asked to volunteer a portion of their time and to divulge personal in- The manner in which interviews ought to be con- formation, they deserve the most enjoyable experi- ducted will vary somewhat by survey population ence the researcher and interviewer can provide. and, to some degree, by the nature of the survey content. Nevertheless, some general guidelines Familiarity with Questionnaire If an interviewer apply to most interviewing situations. is unfamiliar with the questionnaire, the study suffers and the respondent bears an unfair burden. Appearance and Demeanor As a rule, inter- The interview is likely to take more time than viewers should dress in a fashion similar to that of necessary and be unpleasant. Moreover, the in- the people they’ll be interviewing. A richly dressed terviewer cannot acquire familiarity by skimming interviewer will probably have diffi culty getting through the questionnaire two or three times. He good cooperation and responses from poorer re- or she must study it carefully, question by question, spondents; a poorly dressed interviewer will have and must practice reading it aloud. similar diffi culties with richer respondents. To the Ultimately, the interviewer must be able to read extent that the interviewer’s dress and groom- the questionnaire items to respondents without ering differ from those of the respondents, it should ror, without stumbling over words and phrases. A be in the direction of cleanliness and neatness in good model is the actor reading lines in a play or modest apparel. If cleanliness is not next to godli- movie. The lines must be read as though they con- ness, it appears to be next to neutrality. Although stituted a natural conversation, but that conversa- INTERVIEW SURVEYS 293 tion must follow exactly the language set down in done to limit their numbers. Another might say the questionnaire. that more roads are needed. If the interviewer re- By the same token, the interviewer must be corded these two responses with the same sum- familiar with the specifi cations prepared in con- mary—“congested traffi c”—the researchers would junction with the questionnaire. Inevitably some not be able to take advantage of the important dif- questions will not exactly fi t a given respondent’s ferences in the original responses. situation, and the interviewer must determine how Sometimes, verbal responses are too inarticu- the question should be interpreted in that situa- late or ambiguous to permit interpretation. How- tion. The specifi cations provided to the interviewer ever, the interviewer may be able to understand should give adequate guidance in such cases, but the intent of the response through the respon- the interviewer must know the organization and dent’s gestures or tone. In such a situation, the contents of the specifi cations well enough to refer interviewer should still record the exact verbal to them effi ciently. It would be better for the inter- response but also add marginal comments giving viewer to leave a given question unanswered than both the interpretation and the reasons for arriv- to spend fi ve minutes searching through the speci- ing at it. fi cations for clarifi cation or trying to interpret the More generally, researchers can use any mar- relevant instructions. ginal comments explaining aspects of the response not conveyed in the verbal recording, such as the Following Question Wording Exactly The fi rst respondent’s apparent anger, embarrassment, un- part of this chapter discussed the signifi cance of certainty in answering, and so forth. In each case, question wording for the responses obtained. A however, the exact verbal response should also be slight change in the wording of a given question recorded. may lead a respondent to answer “yes” rather than “no.” It follows that interviewers must be instructed Probing for Responses Sometimes respondents to follow the wording of questions exactly. Other- in an interview will give an inappropriate or incom- wise all the effort that the developers have put into plete answer. In such cases, a probe, or request carefully phrasing the questionnaire items to ob- for an elaboration, can be useful. For example, a tain the information they need and to ensure that closed-ended question may present an attitudi- respondents interpret items precisely as intended nal statement and ask the respondent to strongly will be wasted. agree, agree somewhat, disagree somewhat, or strongly disagree. The respondent, however, may Recording Responses Exactly Whenever the reply: “I think that’s true.” The interviewer should questionnaire contains open-ended questions, follow this reply with “Would you say you strongly those soliciting the respondent’s own answer, the agree or agree somewhat?” If necessary, inter- interviewer must record that answer exactly as viewers can explain that they must check one or given. No attempt should be made to summarize, the other of the categories provided. If the respon- paraphrase, or correct bad grammar. dent adamantly refuses to choose, the interviewer This exactness is especially important because should write in the exact response given by the the interviewer will not know how the responses respondent. are to be coded. Indeed, the researchers themselves may not know the coding until they’ve read a hundred or so responses. For example, the ques- A technique employed in interviewing to solicit tionnaire might ask respondents how they feel probe a more complete answer to a question. It is a nondirec- about the traffi c situation in their community. One tive phrase or question used to encourage a respondent respondent might answer that there are too many to elaborate on an answer. Examples include “Anything cars on the roads and that something should be more?” and “How is that?” 294 CHAPTER 9 SURVEY RESEARCH

Probes are more frequently required in eliciting Even though the interviewers may be involved responses to open-ended questions. For example, only in the data-collection phase of the project, it in response to a question about traffi c conditions, will be useful to them to understand what will be the respondent might simply reply, “Pretty bad.” done with the interviews they conduct and what The interviewer could obtain an elaboration on purpose will be served. Morale and motivation this response through a variety of probes. Some- are usually lower when interviewers don’t know times the best probe is silence; if the interviewer what’s going on. sits quietly with pencil poised, the respondent will The training on how to interview should begin probably fi ll the pause with additional comments. with a discussion of general guidelines and pro- (This technique is used effectively by newspaper cedures, such as those discussed earlier in this reporters.) Appropriate verbal probes might be section. Then the whole group should go through “How is that?” or “In what ways?” Perhaps the most the questionnaire together—question by question. generally useful probe is “Anything else?” Don’t simply ask if anyone has any questions about Often interviewers need to probe for answers the fi rst page of the questionnaire. Read the fi rst that will be suffi ciently informative for analytical question aloud, explain the purpose of the ques- purposes. In every case, however, such probes tion, and then entertain any questions or com- must be completely neutral; they must not in any ments the interviewers may have. Once all their way affect the nature of the subsequent response. questions and comments have been handled, go Whenever you anticipate that a given question on to the next question in the questionnaire. may require probing for appropriate responses, It’s always a good idea to prepare specifi cations you should provide one or more useful probes next to accompany an interview questionnaire. Speci- to the question in the questionnaire. This prac- fi cations are explanatory and clarifying comments tice has two important advantages. First, you’ll about handling diffi cult or confusing situations that have more time to devise the best, most neutral may occur with regard to particular questions in probes. Second, all interviewers will use the same the questionnaire. When drafting the question- probes whenever they’re needed. Thus, even if the naire, try to think of all the problem cases that probe isn’t perfectly neutral, all respondents will might arise—the bizarre (or not so bizarre) circum- be presented with the same stimulus. This is the stances that might make a question diffi cult to an- same logical guideline that we discussed for ques- swer. The survey specifi cations should provide de- tion wording. Although a question should not be tailed guidelines on how to handle such situations. loaded or biased, every respondent must be pre- For example, even as simple a matter as age might sented with the same question, even if it’s biased. present problems. Suppose a respondent says he or she will be 25 next week. The interviewer might Coordination and Control not be sure whether to take the respondent’s current age or the nearest one. The specifi cations for Most interview surveys require the assistance of that question should explain what should be done. several interviewers. In large-scale surveys, in- (Probably, you would specify that the age as of last terviewers are hired and paid for their work. Stu- birthday should be recorded in all cases.) dent researchers might fi nd themselves recruiting If you’ve prepared a set of specifi cations, review friends to help them interview. Whenever more them with the interviewers when you go over the than one interviewer is involved in a survey, their individual questions in the questionnaire. Make efforts must be carefully controlled. This control sure your interviewers fully understand the speci- has two aspects: training interviewers and superfi cations and the reasons for them as well as the vising them after they begin work. questions themselves. The interviewers’ training session should begin This portion of the interviewer training is likely with the description of what the study is all about. to generate many troublesome questions from TELEPHONE SURVEYS 295 your interviewers. They’ll ask, “What should I do have them practice on people you’ve selected in if . . . ?” In such cases, avoid giving a quick, offhand your sample, however. After each interviewer answer. If you have specifi cations, show how the has completed three to fi ve interviews, have him solution to the problem could be determined from or her check back with you. Look over the com- the specifi cations. If you do not have specifi cations, pleted questionnaires for any evidence of misun- show how the preferred handling of the situation derstanding. Again, answer any questions that the fi ts within the general logic of the question and the interviewers may have. Once you’re convinced purpose of the study. Giving unexplained answers that a given interviewer knows what to do, assign to such questions will only confuse the interview- some actual interviews, using the sample you’ve ers and cause them to take their work less seri- selected for the study. ously. If you don’t know the answer to such a ques- It’s essential to continue supervising the work tion when it is asked, admit it and ask for some of interviewers over the course of the study. You time to decide on the best answer. Then think out should check in with them after they conduct no the situation carefully and be sure to give all the in- more than 20 or 30 interviews. You might assign 20 terviewers your answer, explaining your reasons. interviews, have the interviewer bring back those Once you’ve gone through the whole ques- questionnaires when they’re completed, look them tionnaire, conduct one or two demonstration over, and assign another 20 or so. Although this interviews in front of everyone. Preferably, you may seem overly cautious, you must continually should interview someone other than one of the protect yourself against any misunderstandings interviewers. Realize that your interview will be a not evident early in the study. model for those you’re training, so make it good. If you’re the only interviewer in your study, It would be best, moreover, if the demonstration these comments may not seem relevant. However, interview were done as realistically as possible. it would be wise, for example, to prepare specifi ca- Do not pause during the demonstration to point tions for potentially troublesome questions in your out how you’ve handled a complicated situation: questionnaire. Otherwise, you run the risk of mak- Handle it, and then explain later. It is irrelevant if ing ad hoc decisions during the course of the study the person you’re interviewing gives real answers that you’ll later regret or forget. Also, the emphasis or takes on some hypothetical identity for the pur- on practice applies equally to the one-person proj- pose, as long as the answers consistently repre- ect and to the complex funded survey with a large sent the identity being presented. interviewing staff. After the demonstration interviews, pair off your interviewers and have them practice on each other, going through the entire process twice and TELEPHONE SURVEYS reversing roles each time. Interviewing is the best training for interviewing. As your interviewers Positive and Negative Factors practice on each other, wander around, listening in on the practice so you’ll know how well they’re For years telephone surveys had a rather bad repu- doing. Once the practice is completed, the whole tation among professional researchers. Telephone group should discuss their experiences and ask surveys are limited by defi nition to people who any other questions they may have. have telephones. Years ago, this method produced The fi nal stage of the training for interview- a substantial social-class bias by excluding poor ers should involve some “real” interviews. Have people from the surveys. This was vividly demon- them conduct some interviews under the actual strated by the Literary Digest fi asco of 1936. Recall conditions that will pertain to the fi nal survey. You that, even though voters were contacted by mail, may want to assign them people to interview, or the sample was partially selected from telephone you may let them pick people themselves. Do not subscribers, who were hardly typical in a nation 296 CHAPTER 9 SURVEY RESEARCH just recovering from the Great Depression. By are engaged in the project. If all the interviewers 2003, however, 95.5 percent of all housing units are calling from the research offi ce, they can get had telephones, so the earlier form of class bias clarifi cation from the person in charge whenever has substantially diminished (U.S. Bureau of the problems occur, as they inevitably do. Alone in the Census 2006:737, Table 1117). boondocks, an in-person interviewer may have to A related sampling problem involved unlisted wing it between weekly visits with the interview- numbers. A survey sample selected from the pages ing supervisor. of a local telephone directory would totally omit Finally, another important factor involved in the all those people—typically richer—who requested growing use of telephone surveys has to do with that their numbers not be published. This poten- personal safety. Don Dillman describes the situa- tial bias has been erased through a technique that tion this way: has advanced telephone sampling substantially: random-digit dialing. Interviewers must be able to operate comfort- Telephone surveys have many advantages, ably in a climate in which strangers are viewed which underlie the popularity of this method. Prob- with distrust and must successfully counter ably the greatest advantages are money and time, respondents’ objections to being interviewed. in that order. In a face-to-face household inter- Increasingly, interviewers must be willing to view, you may drive several miles to a respondent’s work at night to contact residents in many home, fi nd no one there, return to the research of- households. In some cases, this necessitates fi ce, and drive back the next day—possibly fi nding providing protection for interviewers working in no one there again. It’s cheaper and quicker to let potentially dangerous locations. — (1978:4) your fi ngers make the trips. When interviewing by telephone, you can dress Concerns for safety thus work two ways to ham- any way you please without affecting the answers per face-to-face interviews. Potential respondents respondents give. And sometimes respondents may refuse to be interviewed, fearing the stranger- will be more honest in giving socially disapproved interviewer. And the interviewers themselves may answers if they don’t have to look you in the eye. incur some risks. All this is made even worse by Similarly, it may be possible to probe into more the possibility of the researchers being sued for sensitive areas, though this isn’t necessarily the huge sums if anything goes wrong. case. People are, to some extent, more suspicious There are problems involved in telephone in- when they can’t see the person asking them ques- terviewing, however. As I’ve already mentioned, tions—perhaps a consequence of “surveys” aimed the method is hampered by the proliferation of at selling magazine subscriptions and time-share bogus “surveys” that are actually sales campaigns condominiums. disguised as research. If you have any questions Interviewers can communicate a lot about about any such call you receive, by the way, ask themselves over the phone, however, even though the interviewer directly whether you’ve been se- they can’t be seen. For example, researchers worry lected for a survey only or if a sales “opportunity” about the impact of an interviewer’s name (paris involved. It’s also a good idea, if you have any ticularly if ethnicity is relevant to the study) and de- doubts, to get the interviewer’s name, phone num- bate the ethics of having all interviewers use bland ber, and company. Hang up if the caller refuses to “stage names” such as Smith or Jones. (Female in- provide any of these. terviewers sometimes ask permission to do this, For the researcher, the ease with which people to avoid subsequent harassment from men they can hang up is another shortcoming of telephone interview.) surveys. Once you’ve been let inside someone’s Telephone surveys can allow greater con- home for an interview, the respondent is unlikely trol over data collection if several interviewers to order you out of the house in the middle of the TELEPHONE SURVEYS 297 interview. It’s much easier to terminate a tele- (“Could you tell me how many people live at this phone interview abruptly, saying something like, address?”). “Whoops! Someone’s at the door. I gotta go.” or When the respondent answers the phone, the “OMIGOD! The pigs are eating my Volvo!” (That interviewer says hello, introduces the study, and sort of thing is much harder to fake when the inter- asks the fi rst question displayed on the screen. viewer is sitting in your living room.) When the respondent answers the question, the Another potential problem for telephone in- interviewer types that answer into the computer terviewing is the prevalence of answering ma- terminal—either the verbatim response to an chines. A study conducted by Walker Research open-ended question or the code category for the (1988) found that half of the owners of answering appropriate answer to a closed-ended question. machines acknowledged using their machines to The answer is immediately stored in the computer. “screen” calls at least some of the time. Research The second question appears on the video screen by Peter Tuckel and Barry Feinberg (1991), how- and is asked, and the answer is entered into the ever, showed that answering machines had not yet computer. Thus, the interview continues. had a signifi cant effect on the ability of telephone In addition to the obvious advantages in terms researchers to contact prospective respondents. of data collection, CATI automatically prepares the Nevertheless, the researchers concluded that as data for analysis; in fact, the researcher can begin answering machines continue to proliferate, “the analyzing the data before the interviewing is com- sociodemographic characteristics of owners will plete, thereby gaining an advanced view of how change.” This fact makes it likely that “different be- the analysis will turn out. Sill another innovation havior patterns associated with the utilization of that computer technology makes possible is de- the answering machine” will emerge (1991:216). scribed in the box “Voice Capture.” More-recent research has shown that several factors, including answering machines, have reduced Response Rates in Interview Surveys response rates in telephone surveys (Tuckel and O’Neill 2002). Earlier in this chapter we looked at the issue of response rates in mail surveys, and this is an equally Computer-Assisted important issue for interview surveys. In Chapter 7, Telephone Interviewing (CATI) when we discussed formulas for calculating sampling error to determine the accuracy of survey In Chapter 14, we’ll be looking at some of the estimates, the implicit assumption was that every- ways computers have infl uenced the conduct of one selected in a sample would participate—which social research—particularly data processing and is almost never the case. Lacking perfection, reanalysis. Computers are also changing the nature searchers must maximize participation by those of telephone interviewing. One innovation is com- selected. Although interview surveys tend to pro- puter-assisted telephone interviewing (CATI). This duce higher response rates than do mail surveys, method is increasingly used by academic, govern- interview success has recently declined. ment, and commercial survey researchers. Here’s By analyzing response-rate trends in the Uni- a general example of what using CATI can look versity of Michigan’s Survey of Consumer Atti- like. tudes, Richard Curtin, Stanley Presser, and Eleanor Imagine an interviewer wearing a telephone Singer (2005) have sketched a pattern of general headset, sitting in front of a computer monitor. decline over recent years. Between 1979 and 1996, The central computer has been programmed to the response rate in this telephone survey dropped select a telephone number at random and dials it. from 72 to 60 percent, representing an average an- On the video screen is an introduction (“Hello, my nual decline of three-quarters of a percent. Since name is . . . “) and the fi rst question to be asked 1996, the rate of decline has doubled. The in- 298 CHAPTER 9 SURVEY RESEARCH

ISSUES AND INSIGHTS VOICE CAPTURE

by James E. Dannemiller can hear, in turn, the responses of the young SMS Research, Honolulu men, young women, middle-aged men, and so forth. In one such study we found the younger The development of various CATI techniques and older men tending to watch one TV news has been a boon to survey and marketing re- show, while the middle-aged men watched search, though mostly it has supported the col- something else. Listening to the responses of lection, coding, and analysis of “data as usual.” the middle-aged men, one after another, we The Voice Capture technique developed by heard a common comment: “Well, now that I’m Survey Systems, however, offers quite unusual older . . . “ This kind of aside might have been possibilities, which we are only beginning to lost in the notes hastily typed by interviewers, explore. but such comments stood out dramatically in In the course of a CATI-based telephone in- the oral data. The middle-aged men seemed to terview, the interviewer can trigger the com- be telling us they felt “maturity” required them puter to begin digitally recording the conversa- to watch a particular show, while more years tion with the respondent. Having determined under their belts let them drift back to what they that the respondent has recently changed his or liked in the ﬁ rst place. her favorite TV news show, for example, the in- These kinds of data are especially compel- terviewer can ask, “Why did you change?” and ling to clients, particularly in customer satisfac- begin recording the verbatim response. (Early in tion studies. Rather than summarize what we the interview, the interviewer has asked permis- feel a client’s customers like and don’t like, we sion to record parts of the interview.) can let the respondents speak directly to the cli- Later on, coders can play back the responses ent in their own words. It’s like a focus group and code them—much as they would do with on demand. Going one step further, we have the interviewer’s typescript of the responses. found that letting line employees (bank tellers, This offers an easier and more accurate way of for example) listen to the responses has more accomplishing a conventional task. But that’s a impact than having their supervisors tell them tame use of the new capability. what they are doing right or wrong. It’s also possible to incorporate such oral As exciting as these experiences are, I have data as parts of a cross-tabulation during analy- the strong feeling that we have scarcely begun sis. We may create a table of gender by age by to tap into the possibilities for such unconven- reasons for switching TV news shows. Thus, we tional forms of data.

creased nonresponses reﬂ ected both refusals and Their decline came primarily from refusals, be- those they were unable to contact. cause household interviews produce higher rates By contrast, the General Social Survey, using of contact than do telephone surveys. personal interviews, experienced response rates Most researchers believe that the widespread between 73.5 and 82.4 percent in the years from growth of telemarketing has played a large role in 1975 to 1998. In the 2000 and 2002 surveys, how- the problems experienced by legitimate telephone ever, the GSS completion rate was 70 percent. surveys, and they hope that the state and national NEW TECHNOLOGIES AND SURVEY RESEARCH 299

“do not call” lists may ease that problem. Further, niques, and they do not appear to result in a reduc- we saw that other factors such as answering mation of data quality. chines also contribute to these problems (Tuckel Jeffery Walker (1994) has explored the possibil- and O’Neill 2002). Response rate is likely to remain ity of conducting surveys by fax machine. Ques- an issue of high concern in survey research. tionnaires are faxed to respondents, who are asked to fax their answers back. Of course, such NEW TECHNOLOGIES surveys can only represent that part of the popula- AND SURVEY RESEARCH tion that has fax machines. Walker reports that fax surveys don’t achieve as high a response rate as do face-to-face interviews, but, because of the per- As we’ve already seen in the case of computer-as- ceived urgency, they do produce higher response sisted telephone interviewing, many of the new rates than do mail or telephone surveys. In one test technologies affecting people’s lives also open case, all those who had ignored a mail question- new possibilities for survey research. For example, naire were sent a fax follow-up, and 83 percent of recent innovations in self-administered question- this group responded. naires make use of the computer. Among the tech- I’ve already noted that, as a consumer of social niques that are being tested are these (Nicholls, research, you should be wary of “surveys” whose Baker, and Martin 1996): apparent purpose is to raise money for the sponsor. This practice has already invaded the realm of CAPI (computer-assisted personal interviewing): “fax surveys,” evidenced by a fax entitled, “Should Similar to CATI but used in face-to-face Hand Guns Be Outlawed?” Two fax numbers were interviews rather than over the phone. provided for expressing either a “Yes” or “No” opin- CASI (computer-assisted self-interviewing): A ion. The smaller print noted, “Calls to these num- research worker brings a computer to the bers cost $2.95 per minute, a small price for greater respondent’s home, and the respondent reads democracy. Calls take approx. 1 or 2 minutes.” You questions on the computer screen and enters can imagine where the $2.95 went. his or her own answers. The new technology of survey research in- CSAQ (computerized self-administered cludes the use of the Internet and the World Wide questionnaire): The respondent receives the Web—two of the most far-reaching developments questionnaire via fl oppy disk, bulletin board, of the late twentieth century. Some researchers or other means and runs the software, which feel that the Internet can also be used to conduct asks questions and accepts the respondent’s meaningful survey research, and this technique answers. The respondent then returns the has become especially popular in market research, data fi le. for example. TDE (touchtone data entry): The respondent Mick Couper provides an excellent overview of initiates the process by calling a number at the the issues concerning the present and prospective research organization. This prompts a series of state of online surveys: computerized questions, which the respondent answers by pressing keys on the telephone The rapid development of surveys on the World keypad. Wide Web (WWW) is leading some to argue that VR (voice recognition): Instead of asking the soon Internet (and, in particular, Web) surveys respondent to use the telephone keypad, as in will replace traditional methods of survey data TDE, this system accepts spoken responses. collection. Others are urging caution or even Nicholls and colleagues report that such tech- voicing skepticism about the future role Web niques are more effi cient than conventional tech- surveys will play. Clearly, we stand at the thresh- 300 CHAPTER 9 SURVEY RESEARCH

old of a new era for survey research, but how The researchers compared their results with this will play out is not yet clear. — (2001:464) 52 telephone polls that addressed the same races. Online polling correctly picked 21 of the 22 win- One immediate objection that many social ners, or 95 percent. However, simply picking the researchers make to online surveys concerns winner is not a suffi cient test of effectiveness: How representativeness: Will the people who can be close did the polls come to the actual percentages surveyed online be representative of meaningful received by the various candidates? Taylor and populations, such as all U.S. adults, all voters, and Terhanian report their online polls missed the ac- so on? This is the criticism raised with regard to tual vote by an average of 6.8 percentage points. surveys via fax and, earlier, with regard to tele- The 52 telephone polls missed the same votes by phone surveys. an average of 6.2 percentage points. Camilo Wilson (1999), the founder of Cogix, Warren Mitofsky (1999) is a critic of online poll- points out that some populations are ideally suited ing. In addition to disagreeing with the way Taylor to online surveys: specifi cally, those who visit a and Terhanian calculated the ranges of error just particular website. For example, Wilson indicates reported, he has called for a sounder, theoretical that market research for online companies should basis on which to ground the new technique. be conducted online, and his fi rm has developed One key to online polling is the proper assess- software, ViewsFlash, for precisely that purpose. ment and use of weights for different kinds of Although website surveys could easily collect data respondents—as was discussed in the context of from all who visit a particular site, Wilson suggests quota sampling in Chapter 7. Taylor and Terhanian that survey sampling techniques can provide suf- are aware of the criticisms of quota sampling, but fi cient consumer data without irritating thousands their initial experiences with online polling suggest or millions of potential customers. to them that the technique should be pursued. In- deed, they conclude by saying, “This is an unstop- To learn more about online surveys, you pable train, and it is accelerating. Those who don’t can go to the website for Cogix: http:// get on board run the risk of being left far behind.” www.cogix.com or use a search engine to —(1999:23). fi nd “online surveys.” Many of the cautions urged in relation to online surveys today resemble those urged several decades ago in relation to telephone surveys. But how about general population surveys? For Couper’s observations refl ect these cautions: the past several years, a debate has been brewing within the survey research community. Humphrey Several years ago, I predicted that the rapid Taylor and George Terhanian (1999) prompted part spread of electronic data collection methods of the debate with an article, “Heady Days Are Here such as the Internet would produce a bifurca- Again.” Acknowledging the need for caution, they tion in the survey industry between high-quality urged that online polling be given a fair hearing: surveys based on probability samples and using traditional data collection methods, on the one One test of the credibility of any new data col- hand, and surveys focused more on low cost lection method hinges on its ability to reliably and rapid turnaround than on representative- and accurately forecast voting behavior. For this ness and accuracy on the other. In hindsight, I reason, last fall we attempted to estimate the was wrong, and I underestimated the impact of 1998 election outcomes for governor and US the Web on the survey industry. It has become Senate in 14 states on four separate occasions much more of a fragmentation than a bifurca- using internet surveys. — (1999:20) tion (in terms of Web surveys at least), with NEW TECHNOLOGIES AND SURVEY RESEARCH 301

vendors trying to ﬁ nd or create a niche for their Do test incentives, rewards, and prize drawings particular approach or product. No longer is it to determine the optimal offer for best just “quick and dirty” in one corner and “expen- response. Longer surveys usually require larger sive but high quality” in the other; rather, there is incentives. a wide array of approaches representing varying Do limit studies to 15 minutes or less.* levels of quality and cost. — (2001:466)

In the meantime, researchers are amassing a The web is already seeing extensive use body of experience with this new technique, yield- as a marketplace for surveys and other re- ing lessons for increasing success. For example, search techniques. As only a few illustrative Survey Sampling, Inc., suggests the following dos examples, see the following: and don’ts for conducting online surveys: • The Gallup Organization: http://www.gallup.com/ Do use consistent wording between the invitation • Harris Poll Online: http://www and the survey. Don’t use terms such as .harrisinteractive.com/ “unique ID number” in the invitation, then ask • SMS Research: http://www respondents to type their “password” when .smshawaii.com/ they get to the survey. Changing terminology • The Survey/Marketing Research e-Store: can be confusing. http://www.streamlinesurveys.com/ Do use plain, simple language. Streamline/estore/index.htm Don’t force the respondent to scroll down the • Zogby International: http://www screen for the URL for the study location. .zogby.com/ Do offer to share selected results from the study with everyone who completes the survey. Respondents will often welcome information In another study of ways to improve response as a reward for taking the study, especially rates in online surveys, Stephen Porter and Mi- when they are young adults and teens. chael Whitcomb (2003) found that some of the Do plan the time of day and day of week to techniques effective in mail surveys, such as per- mail, depending on the subject of the study sonalizing the appeal or varying the apparent sta- and type of respondent. Send the invitation tus of the researcher, had little or no impact in the late afternoon, evening, or weekend, when new medium. At the same time, specifying that the respondents are most likely to be reading respondents had been specially selected for the mail at home, especially if the study requests survey and setting a deadline for participation did respondents to check an item in the kitchen increase response rates. The years ahead will see or other area in the home. If a parent-child many experiments aimed at improving the effec- questionnaire is planned, send the invitation tiveness of online surveys. late afternoon when children are home, not During the 2004 election campaign, the role early in the day, when respondents can’t and nature of polls—in-person, telephone, and complete the study because children are at online—drew considerable public attention. Mark school. Blumenthal (2005) has reviewed the public discus- Do be aware of technical limitations. For sion and considered the implications for the future example, WebTV users currently cannot access of polling. surveys using Java. If respondents’ systems need to be Java-enabled or require access

to streaming video, alert panelists at the *Source: http://www.worldopinion.com/the_frame/ beginning of the study, not midway through. frame4.html. Reprinted with permission. 302 CHAPTER 9 SURVEY RESEARCH

COMPARISON OF THE pertains generally to all complicated contingency DIFFERENT SURVEY METHODS questions. With interviews, you can conduct a survey based on a sample of addresses or phone numbers Now that we’ve seen several ways to collect sur- rather than on names. An interviewer can arrive vey data, let’s take a moment to compare them at an assigned address or call the assigned num- directly. ber, introduce the survey, and even—following in- Self-administered questionnaires are generally structions—choose the appropriate person at that cheaper and quicker than face-to-face interview address to respond to the survey. In contrast, self- surveys. These considerations are likely to be im- administered questionnaires addressed to “occu- portant for an unfunded student wishing to under- pant” receive a notoriously low response. take a survey for a term paper or thesis. Moreover, Finally, as we’ve seen, interviewers questioning if you use the self-administered mail format, it respondents face-to-face can make important ob- costs no more to conduct a national survey than servations aside from responses to questions asked a local one of the same sample size. In contrast, a in the interview. In a household interview, they national interview survey (either face-to-face or by may note the characteristics of the neighborhood, telephone) would cost far more than a local one. the dwelling unit, and so forth. They may also note Also, mail surveys typically require a small staff: characteristics of the respondents or the quality of One person can conduct a reasonable mail survey their interaction with the respondents—whether alone, although you shouldn’t underestimate the the respondent had diffi culty communicating, was work involved. Further, respondents are some- hostile, seemed to be lying, and so on. times reluctant to report controversial or deviant The chief advantages of telephone surveys attitudes or behaviors in interviews but are will- over those conducted face-to-face center primar- ing to respond to an anonymous self-administered ily on time and money. Telephone interviews are questionnaire. much cheaper and can be mounted and executed Interview surveys also offer many advantages. quickly. Also, interviewers are safer when inter- For example, they generally produce fewer incom- viewing people in high-crime areas. Moreover, the plete questionnaires. Although respondents may impact of the interviewers on responses is some- skip questions in a self-administered question- what lessened when they can’t be seen by the re- naire, interviewers are trained not to do so. In CATI spondents. As only one indicator of the popular- surveys, the computer offers a further check on ity of telephone interviewing, when Johnny Blair this. Interview surveys, moreover, typically achieve and his colleagues (1995) compiled a bibliography higher completion rates than do self-administered on sample designs for telephone interviews, they questionnaires. listed over 200 items. Although self-administered questionnaires may Online surveys have many of the strengths be more effective for sensitive issues, interview and weaknesses of mail surveys. Once the avail- surveys are defi nitely more effective for compli- able software has been further developed, they are cated ones. Prime examples include the enumera- likely to be substantially cheaper. An important tion of household members and the determination weakness, however, lies in the diffi culty of assur- of whether a given address corresponds to more ing that respondents to an online survey will be than one housing unit. Although the concept of representative of some more general population. housing unit has been refi ned and standardized by Clearly, each survey method has its place in the Bureau of the Census and interviewers can be social research. Ultimately, you must balance the trained to deal with the concept, it’s extremely dif- advantages and disadvantages of the different fi cult to communicate in a self-administered ques- methods in relation to your research needs and tionnaire. This advantage of interview surveys your resources. STRENGTHS AND WEAKNESSES OF SURVEY RESEARCH 303

STRENGTHS AND WEAKNESSES subjects. The survey researcher is bound to this OF SURVEY RESEARCH requirement, however, by having to ask exactly the same questions of all subjects and having to impute the same intent to all respondents giving a Regardless of the specifi c method used, sur- particular response. veys—like other modes of observation in social Survey research also has several weaknesses. research—have special strengths and weaknesses. First, the requirement of standardization often You should keep these in mind when determining seems to result in the fi tting of round pegs into whether a survey is appropriate for your research square holes. Standardized questionnaire items goals. often represent the least common denominator Surveys are particularly useful in describing the in assessing people’s attitudes, orientations, cir- characteristics of a large population. A carefully cumstances, and experiences. By designing ques- selected probability sample in combination with tions that will be at least minimally appropriate to a standardized questionnaire offers the possibil- all respondents, you may miss what is most ap- ity of making refi ned descriptive assertions about propriate to many of them. In this sense, surveys a student body, a city, a nation, or any other large often appear superfi cial in their coverage of com- population. Surveys determine unemployment plex topics. Although sophisticated analyses can rates, voting intentions, and the like with uncanny partly offset this problem, it is inherent in survey accuracy. Although the examination of offi cial research. documents—such as marriage, birth, or death re- Similarly, survey research can seldom deal with cords—can provide equal accuracy for a few top- the context of social life. Although questionnaires ics, no other method of observation can provide can provide information in this area, the survey this general capability. researcher rarely develops a feel for the total life Surveys—especially self-administered ones— situation in which respondents are thinking and make large samples feasible. Surveys of 2,000 acting that, say, the participant observer can (see respondents are not unusual. A large number of Chapter 10). cases is very important for both descriptive and In many ways, surveys are infl exible. Studies in- explanatory analyses, especially wherever several volving direct observation can be modifi ed as fi eld variables are to be analyzed simultaneously. conditions warrant, but surveys typically require In one sense, surveys are fl exible. They allow that an initial study design remain unchanged you to ask many questions on a given topic, giv- throughout. As a fi eld researcher, for example, you ing you considerable fl exibility in your analyses. can become aware of an important new variable Whereas an experimental design may require operating in the phenomenon you’re studying and you to commit yourself in advance to a particu- begin making careful observations of it. The sur- lar operational defi nition of a concept, surveys let vey researcher would probably be unaware of the you develop operational defi nitions from actual new variable’s importance and could do nothing observations. about it in any event. Finally, standardized questionnaires have an Finally, surveys are subject to the artifi ciality important strength in regard to measurement gen- mentioned in connection with experiments. Find- erally. Earlier chapters have discussed the ambigu- ing out that a person gives conservative answers ous nature of most concepts: They have no ulti- in a questionnaire does not necessarily mean the mately real meanings. One person’s religiosity is person is conservative; fi nding out that a person quite different from another’s. Although you must gives prejudiced answers in a questionnaire does be able to defi ne concepts in those ways most rel- not necessarily mean the person is prejudiced. evant to your research goals, you may not fi nd it This shortcoming is especially salient in the realm easy to apply the same defi nitions uniformly to all of action. Surveys cannot measure social action; 304 CHAPTER 9 SURVEY RESEARCH they can only collect self-reports of recalled past SECONDARY ANALYSIS action or of prospective or hypothetical action. The problem of artifi ciality has two aspects. As a mode of observation, survey research involves First, the topic of study may not be amenable to the following steps: (1) questionnaire construc- measurement through questionnaires. Second, the tion, (2) sample selection, and (3) data collection, act of studying that topic—an attitude, for exam- through either interviewing or self-administered ple—may affect it. A survey respondent may have questionnaires. As you’ve gathered, surveys are given no thought to whether the governor should usually major undertakings. It’s not unusual for a be impeached until asked for his or her opinion by large-scale survey to take several months or even an interviewer. He or she may form an opinion on more than a year to progress from conceptualiza- the spot. tion to data in hand. (Smaller-scale surveys can, of Survey research is generally weak on validity course, be done more quickly.) Through a method and strong on reliability. In comparison with fi eld called secondary analysis, however, researchers research, for example, the artifi ciality of the sur- can pursue their particular social research inter- vey format puts a strain on validity. As an illustra- ests—analyzing survey data from, say, a national tion, people’s opinions on issues seldom take the sample of 2,000 respondents—while avoiding the form of strongly agreeing, agreeing, disagreeing, enormous expenditure of time and money such a or strongly disagreeing with a specifi c statement. survey entails. Their survey responses in such cases must be re- Secondary analysis is a form of research in garded as approximate indicators of what the re- which the data collected and processed by one researchers had in mind when they framed the ques- searcher are reanalyzed—often for a different pur- tions. This comment, however, needs to be held in pose—by another. Beginning in the 1960s, survey the context of earlier discussions of the ambiguity researchers became aware of the potential value of validity itself. To say something is a valid or an in archiving survey data for analysis by scholars invalid measure assumes the existence of a “real” who had nothing to do with the survey design and defi nition of what’s being measured, and many data collection. Even when one researcher had scholars now reject that assumption. conducted a survey and analyzed the data, those Reliability is a clearer matter. By presenting same data could be further analyzed by others all subjects with a standardized stimulus, survey who had slightly different interests. Thus, if you research goes a long way toward eliminating un- were interested in the relationship between po- reliability in observations made by the researcher. litical views and attitudes toward gender equality, Moreover, careful wording of the questions can also you could examine that research question through reduce signifi cantly the subject’s own unreliability. the analysis of any data set that happened to con- As with all methods of observation, a full aware- tain questions relating to those two variables. ness of the inherent or probable weaknesses of The initial data archives were very much like survey research can partially resolve them in some book libraries, with a couple of differences. First, cases. Ultimately, though, researchers fi nd the saf- instead of books, the data archives contained est ground when they employ several research data sets: fi rst as punched cards, then as magnetic methods in studying a given topic. tapes. Today they’re typically contained on computer disks, CD-ROMs, or online servers. Second, whereas you’re expected to return books to a con- secondary analysis A form of research in which ventional library, you can keep the data obtained the data collected and processed by one researcher are from a data archive. reanalyzed—often for a different purpose—by another. The best-known current example of second- This is especially appropriate in the case of survey data. ary analysis is the General Social Survey (GSS). Data archives are repositories or libraries for the storage The National Opinion Research Center (NORC) at and distribution of data for secondary analysis. SECONDARY ANALYSIS 305 the University of Chicago conducts this major na- With a somewhat different format, Polling the tional survey, currently every other year, to collect Nations offers access to thousands of surveys con- data on a large number of social science variables. ducted around the world. Instead of providing the These surveys are conducted precisely for the pur- data, Polling the Nations lets subscribers retrieve pose of making data available to scholars at little the distribution of responses to questions in those or no cost and are supported by a combination of surveys, with the questions linked to an index of private and government funding. Recall that the topics. Although the cost of the subscription may be GSS was created by James A. Davis in 1972; it is too steep for the average student, you might check currently directed by Davis, Tom W. Smith, and Pe- to see if your school’s library has subscribed. ter V. Marsden. Their considerable ongoing efforts Whereas secondary analysis typically involves make an unusual contribution to social science re- obtaining a data set and undertaking an extensive search and to education in social science. analysis, I would like you to consider another ap- Numerous other resources are available for proach as well. Often you can do limited analyses identifying and acquiring survey data for second- for just a little investment of time. Let’s say you are ary analysis. The Roper Center for Public Opinion writing a term paper about the impact of religion Research at the University of Connecticut is one in contemporary American life. You want to com- excellent resource. The center also publishes the ment on the role of the Roman Catholic church in journal Public Perspective on public opinion poll- the debate over abortion. Although you might get ing. Polling the Nations is an online repository for away with an offhand, unsubstantiated assertion, thousands of polls conducted in the United States imagine how much more powerful your paper and 70 other nations. A paid subscription allows would be with this addition: users to obtain specifi c data results from studies 1. Go to the General Social Survey Website they specify, rather than obtaining whole studies. (see Figure 9-7): http://webapp.icpsr.umich Outside the United States, the Netherlands In- .edu/GSS/ stitute for Scientifi c Information Services allows 2. Click on “Subject” under “Codebook Indexes,” users to track down European studies that contain then go to “Abortion.” This will show you the variables of interest. You might also try the Central questions asked about attitudes on abortion Archive for Social Science Research at the Univer- (e.g., ABANY asks if a women should be per- sity of Cologne in Germany. mitted a legal abortion for any reason). 3. Click the “Analyze” button. 4. Select “Frequencies or crosstabulation” and See the following sites for more on social click the “Start” button. science data sources: 5. Enter ABANY as the dependent (“Row”) vari- • GSS: http://webapp.icpsr.umich able and RELIG as the independent (“Column”) .edu/GSS/ variable. In the “Selection Filter” fi eld, enter • Roper Center for Public Opinion “YEAR(2000)” to limit the analysis to that year’s Research: http://www.ropercenter survey, then choose “No Weight,” as shown in .uconn.edu/ Figure 9-8. • Polling the Nations: http://www .pollingthenations.com/ 6. Press “Run the Table” to run the requested • Netherlands Institute for analysis. Scientifi c Information Services: The results of your analysis, shown in Figure http://www2.niwi.knaw.nl/en/ 9-9, may surprise you. Whereas Catholics are less maatschappijwetenschappen/toon supportive of abortion (33.3 percent) than Jews • Central Archive for Social Science Research at the University of Cologne: (76.1 percent) and those with no religion (57.3 per- http://www.gesis.org/en/za/ cent), they do not differ substantially from Ameri- can Protestants (35.2 percent). 306 CHAPTER 9 SURVEY RESEARCH

FIGURE 9-7 The General Social Survey Codebook Website

FIGURE 9-9 Impact of Religion on Attitude FIGURE 9-8 Entering the Table Request toward Abortion

Imagine a term paper that says, “Whereas the and, depending on who did the original survey, you Roman Catholic Church has taken a strong, of- may benefi t from the work of topfl ight profession- fi cial position on abortion, many Catholics do als. The ease of secondary analysis has also en- not necessarily agree, as shown in Table . . . “ hanced the possibility of meta-analysis, in which a Moreover, this might be just the beginning of an researcher brings together a body of past research analysis that looks a bit more deeply into the mat- on a particular topic. To gain confi dence in your ter, as described in Chapter 14, on quantitative understanding of the relationship between religion analysis. and abortion, for example, you could go beyond The key advantage of secondary analysis is that the GSS to analyze similar data collected in dozens it’s cheaper and faster than doing original surveys, or even hundreds of other studies. MAIN POINTS 307

There are disadvantages inherent in secondary analysis, however. The key problem involves the WHAT DO YOU THINK? REVISITED recurrent question of validity. When one researcher collects data for one particular purpose, you have Professional survey research has been no assurance that those data will be appropriate damaged in recent years by the actions of for your research interests. Typically, you’ll fi nd telemarketers who pretend they are conduct- that the original researcher asked a question that ing surveys. Potential respondents some- “comes close” to measuring what you’re interested times refuse to participate in a legitimate in, but you’ll wish the question had been asked just survey because they suspect that it’s really a little differently—or that another, related ques- a sales call. tion had also been asked. Your question, then, Here are a few ways to determine the le- is whether the question that was asked provides gitimacy of a survey. a valid measure of the variable you want to analyze. Nevertheless, secondary analysis can be im- 1. Ask who is conducting the survey. The mensely useful. Moreover, it illustrates once again caller probably said something about it the range of possibilities available in fi nding the quickly at the outset, but ask him or her answers to questions about social life. Although to repeat the information so you can write no single method unlocks all puzzles, there is no it down. limit to the ways you can fi nd out about things. 2. Ask for the telephone number of a su- And when you zero in on an issue from several pervisor or manager so you can call the independent directions, you gain that much more people running the survey. expertise. 3. Ask whether the call involves a sales I’ve discussed secondary analysis in this chap- solicitation. ter on survey research because it’s the type of If the caller is reluctant to answer any of analysis most associated with the technique. How- these questions, assume that the call is not ever, the reanalysis of social research data is not a professional survey. You may respond as limited to those collected in surveys. Nigel Fielding you deem appropriate. One possibility: Let (2004), for example, has examined the possibilities them recite their entire sales pitch, and then for the archiving and reanalysis of qualitative data ask, “Could you repeat that?” as well.

ETHICS AND SURVEY RESEARCH

respondents. Even if the information they provide Survey research almost always involves a request is kept confi dential, simply forcing them to think that people provide us with information about about some matters can be upsetting. Imagine themselves that is not readily available. Some- asking people for their attitudes toward suicide times, we ask for information (such as attitudes when one of them has recently experienced the and behaviors) that would be embarrassing to the suicide of a family member or close friend. Or ask- respondents if that information became publicly ing people to report on their attitudes about differ- known. In some cases, such revelations could re- ent racial groups, which may cause them to refl ect sult in the loss of a job or a marriage. Hence, main- on whether they may be racists or at least appear taining the norm of confi dentiality, mentioned ear- as such to the interviewers. The possibilities for lier in the book, is particularly important in survey harming survey respondents are endless. Although research. this fact should not prevent you from doing sur- Another ethical concern relates to the possi- veys, it should increase your considered efforts to bility of psychological injury that can be done to avoid the problem wherever possible. 308 CHAPTER 9 SURVEY RESEARCH

Main Points ❏ The order of items in a questionnaire can infl u- ence the responses given. Introduction ❏ Clear instructions are important for getting ap- ❏ Survey research, a popular social research propriate responses in a questionnaire. method, is the administration of question- ❏ Questionnaires should be pretested before be- naires to a sample of respondents selected ing administered to the study sample. from some population. ❏ Questionnaires can be administered in three Topics Appropriate for Survey Research basic ways: through self-administered ques- ❏ Survey research is especially appropriate for tionnaires, face-to-face interviews, or tele- making descriptive studies of large popula- phone surveys. tions; survey data may be used for explanatory purposes as well. Self-Administered Questionnaires ❏ Questionnaires provide a method of collect- ❏ It’s generally advisable to plan follow-up mailing data by (1) asking people questions or ings in the case of self-administered question- (2) asking them to agree or disagree with naires, sending new questionnaires to those statements representing different points of respondents who fail to respond to the initial view. Questions can be open-ended (whereby appeal. Properly monitoring questionnaire respondents supply their own answers) or returns provides a good guide for determining closed-ended (whereby they select from a list when a follow-up mailing is appropriate. of provided answers). Interview Surveys Guidelines for Asking Questions ❏ The essential characteristic of interviewers ❏ Items in a questionnaire should observe is that they be neutral; their presence in the several guidelines: (1) The items must be clear data-collection process must not have any and precise; (2) the items should ask only effect on the responses given to questionnaire about one thing (i.e., double-barreled ques- items. tions should be avoided); (3) respondents must ❏ Interviewers must be carefully trained to be be competent to answer the item; (4) respon- familiar with the questionnaire, to follow the dents must be willing to answer the item; (5) question wording and question order exactly, questions should be relevant to the respon- and to record responses exactly as they are dent; (6) items should ordinarily be short; (7) given. negative terms should be avoided so as not to ❏ Interviewers can use probes to elicit an confuse respondents; (8) the items should be elaboration on an incomplete or ambiguous worded to avoid biasing responses. response. Probes should be neutral. Ideally, all Questionnaire Construction interviewers should use the same probes. ❏ The format of a questionnaire can infl uence the quality of data collected. Telephone Surveys ❏ Telephone surveys can be cheaper and more ❏ A clear format for contingency questions is effi cient than face-to-face interviews, and they necessary to ensure that the respondents an- can permit greater control over data collec- swer all the questions intended for them. tion. The development of computer-assisted ❏ The matrix question is an effi cient format for telephone interviewing (CATI) is especially presenting several items sharing the same promising. response categories. ONLINE STUDY RESOURCES 309

New Technologies and Survey Research to gain a full sense of social processes in their ❏ New technologies offer additional opportuni- natural settings. In general, survey research is ties for social researchers. They include vari- comparatively weak on validity and strong on ous kinds of computer-assisted data collection reliability. and analysis as well as the chance to conduct surveys by fax or over the Internet. The latter Secondary Analysis two methods, however, must be used with ❏ Secondary analysis provides social research- caution because respondents may not be rep- ers with an important option for “collecting” resentative of the intended population. data cheaply and easily but at a potential cost in validity. Comparison of the Different Survey Methods Ethics and Survey Research ❏ Surveys often ask for private information, ❏ The advantages of a self-administered which researchers must keep confi dential. questionnaire over an interview survey are economy, speed, lack of interviewer bias, and ❏ Because asking questions can cause psycho- the possibility of anonymity and privacy to en- logical discomfort or harm to respondents, the courage candid responses on sensitive issues. researcher should minimize this risk. ❏ The advantages of an interview survey over a self-administered questionnaire are fewer in- Key Terms complete questionnaires and fewer misunderstood questions, generally higher return rates, bias probe and greater fl exibility in terms of sampling and closed-ended questions questionnaire special observations. contingency question respondent interview response rate ❏ The principal advantages of telephone surveys open-ended questions secondary analysis over face-to-face interviews are the savings in cost and time. Telephone interviewers are also safer than in-person interviewers, and they Review Questions may have a smaller effect on the interview 1. What closed-ended questions could you con- itself. struct from each of the following open-ended ❏ Online surveys have many of the strengths and questions? weaknesses of mail surveys. Although they are a. What was your family’s total income last cheaper to conduct, it can be diffi cult to ensure year? that the respondents represent a more general population. b. How do you feel about the space program? c. How important is religion in your life? Strengths and Weaknesses d. What was your main reason for attending of Survey Research college? ❏ Survey research in general offers advantages in terms of economy, the amount of data that e. What do you feel is the biggest problem can be collected, and the chance to sample facing your community? a large population. The standardization of 2. What are the main advantages and dis- the data collected represents another special advantages of conducting surveys over the strength of survey research. Internet? ❏ Survey research has the weaknesses of being 3. A newspaper headline proclaims, “Most somewhat artifi cial, potentially superfi cial, and Americans Oppose Abortion, According to relatively infl exible. It’s diffi cult to use surveys New Survey.” What methodological details do 310 CHAPTER 9 SURVEY RESEARCH

you want to know about the survey to help Bradburn, Norman M., and Seymour Sudman. 1988. you interpret the results? Polls and Surveys: Understanding What They Tell Us. San Francisco: Jossey-Bass. These veteran survey 4. Look at your appearance right now. What researchers answer questions about their craft the aspects of your appearance might create a general public commonly ask. problem if you were interviewing a general Couper, Mick P., Michael W. Traugott, and Mark J. La- cross section of the public? mias. 2001. “Web Survey Design and Administration.” Public Opinion Quarterly 65:230–53. An experimental Online Study Resources study to determine which of three different web survey designs elicited the best results. Dillman, Don A. 1999. Mail and Telephone Surveys: The Tailored Design Method. 2nd ed. New York: Wiley. An excellent review of the methodological literature on mail and telephone surveys. Dillman makes many Go to good suggestions for improving response rates. http://sociology.wadsworth.com/babbie_basics4e (This classic book was originally published in 1978 under the name Mail and Telephone Surveys: The Total and click on ThomsonNow for access to this Design Method.) powerful online study tool. You will get a per- Elder, Glen H., Jr., Eliza K. Pavalko, and Elizabeth C. sonalized study plan based on your responses to Clipp. 1993. Working with Archival Data: Studying a diagnostic pretest. Once you have mastered Lives. Thousand Oaks, CA: Sage. This book discusses the material with the help of interactive learning the possibilities and techniques for using existing data archives in the United States, especially those tools, you can take a posttest to confi rm that you providing longitudinal data. are ready to move on to the next chapter. Feick, Lawrence F. 1989. “Latent Class Analysis of Survey Questions That Include Don’t Know Responses.” Website for Public Opinion Quarterly 53:525–47. “Don’t know” can The Basics of Social Research mean a variety of things, as this analysis indicates. 4th edition Fowler, Floyd J., Jr. 1995. Improving Survey Questions: Design and Evaluation. Thousand Oaks, CA: Sage. A At the book companion website (http://sociology comprehensive discussion of questionnaire construc- .wadsworth.com/babbie_basics4e) you will fi nd tion, including suggestions for pretesting questions. many resources in addition to ThomsonNow to This book discusses the logic of obtaining informa- aid you in studying for your exams. For example, tion through survey questions and gives numerous you will fi nd Tutorial Quizzes with feedback, Inter- guidelines for being effective. It also offers several net Exercises, Flashcards, and Chapter Tutorials, as examples of questions you might use. well as Extended Projects, InfoTrac College Edition Groves, Robert M. 1990. “Theories and Methods of search terms, Social Research in Cyberspace, GSS Telephone Surveys.” Pp. 221–40 in Annual Review of Sociology (vol. 16), edited by W. Richard Scott and Data, Web Links, and primers for using various Judith Blake. Palo Alto, CA: Annual Reviews. An data analysis software such as SPSS and NVivo. attempt to place telephone surveys in the context of sociological and psychological theories and to Additional Readings address the various kinds of errors common to this research method. Babbie, Earl. 1990. Survey Research Methods. Belmont, Holbrook, Allyson L., Melanie C. Green, and Jon A. CA: Wadsworth. A comprehensive overview of Krosnick. 2003. “Telephone Versus Face-to-Face In- survey methods. (You thought I’d say it was lousy?) terviewing of National Probability Samples with Long This textbook, although overlapping the present one Questionnaires: Comparisons of Respondent Satisfi c- somewhat, covers aspects of survey techniques omit- ing and Social Desirability Response Bias.” Public ted here. Opinion Quarterly 67:79–125. A detailed examination ADDITIONAL READINGS 311

of the differences to be expected from these two names affect the overall responses given to survey methods of conducting interview surveys. questionnaires, they also affect such things as the Miller, Delbert. 1991. Handbook of Research Design and correlation between education and the number of Social Measurement. Thousand Oaks, CA: Sage. A “don’t know” answers. powerful reference work. This book, especially Part Swafford, Michael. 1992. “Soviet Survey Research: 6, cites and describes a wide variety of operational The 1970’s vs. the 1990’s.” AAPOR News 19 (3): 3–4. measures used in earlier social research. In several The author contrasts the general repression of cases, the questionnaire formats used are presented. survey research during his fi rst visit in 1973–1974 Although the quality of these illustrations is uneven, with the renewed use of the method in more recent they provide excellent examples of possible varia- times. He notes, for example, that the Soviet govern- tions. ment commissioned a national survey to determine Moore, David W. 2002. “Measuring New Types of Ques- public opinion on the possible reunifi cation of tion-Order Effects: Additive and Subtractive.” Public Germany. Opinion Quarterly 66:80–91. An extensive examina- Tourangeau, Roger, Kenneth A. Rasinski, Norman tion of the various ways in which question wording Bradburn, and Roy D’Andrade. 1989. “Carryover can affect responses. Effects in Attitude Surveys.” Public Opinion Quarterly Sheatsley, Paul F. 1983. “Questionnaire Construction and 53:495–524. The authors asked six target questions in Item Writing.” Pp. 195–230 in Handbook of Survey a telephone survey of 1,100 respondents, varying the Research, edited by Peter H. Rossi, James D. Wright, questions immediately preceding the target ques- and Andy B. Anderson. New York: Academic Press. tions. They found substantial differences. An excellent examination of the topic by an expert in Williams, Robin M., Jr. 1989. “The American Soldier: the fi eld. An Assessment, Several Wars Later.” Public Opinion Smith, Eric R. A. N., and Peverill Squire. 1990. “The Ef- Quarterly 53:155–74. One of the classic studies in the fects of Prestige Names in Question Wording.” Public history of survey research is reviewed by one of its Opinion Quarterly 54:97–116. Not only do prestigious authors. 101 HUMANQUALITATIVE INQUIRY FIELD AND RESEARCH SCIENCE Photo credit Paul Conklin/PhotoEdit

What You’ll Learn in thisThis ChapterChapter

HereWe’ll you’ll examine see that the qualitative way people ﬁ eld learn research about enablestheir world researchers and the mistakes to observe they makesocial alonglife in theits natural way. We’ll habitat: also tobegin go where to see thewhat action makes is andsience watch. different This fromtype of

otherresearch ways can of produce knowing a things.richer understanding of many social phenomena than can 2S be achieved through other observational methods, provided that the researcher 1S N observes in a deliberate, well-planned, and active way. L 312 In this chapter . . . WHAT DO YOU THINK?

Introduction The impact of the Topics Appropriate to Field Research observer is a fun- Special Considerations in Qualitative damental issue in Field Research Image not available due to copyright restrictions social research. If The Various Roles of the Observer you participate in Relations to Subjects the events you are studying, observing Some Qualitative Field Research them directly, up Paradigms close and personal, won’t your presence Naturalism there change things? How can you observe Ethnomethodology something as though you aren’t actually Grounded Theory there observing it? In other words, how close Case Studies and the Extended Case Method is too close? Institutional Ethnography Participatory Action Research See the “What Do You Think? Revisited” box toward the end of the chapter. Conducting Qualitative Field Research Preparing for the Field Qualitative Interviewing Focus Groups ing room, or on an airplane. Whenever we report Recording Observations our observations to others, we are reporting our fi eld research efforts. Strengths and Weaknesses of Qualitative Such research is at once very old and very Field Research new in social science, stretching at least from the Validity nineteenth century studies of preliterate societies, Reliability through fi rsthand examinations of urban com- Ethics in Qualitative Field Research munity life in the “Chicago School” of the 1930s and 1940s, to contemporary observations of web INTRODUCTION chat-room interactions. Many of the techniques discussed in this chapter have been used by social researchers for centuries. Within the social Several chapters ago, I suggested that you’ve been sciences, anthropologists are especially associated doing social research all your life. This idea should with this method and have contributed greatly to become even clearer as we turn to what probably its development as a scientifi c technique. More- seems like the most obvious method of making over, many people who might not, strictly speak- observations: qualitative fi eld research. In a sense, ing, be regarded as social science researchers em- we do fi eld research whenever we observe or par- ploy something similar to fi eld research. Welfare ticipate in social behavior and try to understand it, department case workers are one example; news- whether in a college classroom, in a doctor’s wait- paper reporters are another. 313 314 CHAPTER 10 QUALITATIVE FIELD RESEARCH

To take this last example further, consider that it’s a theory-generating activity as well. As a fi eld interviewing is a technique common to both jour- researcher, you’ll seldom approach your task with nalism and sociology. A journalist uses the data to precisely defi ned hypotheses to be tested. More report a subject’s attitude, belief, or experience— typically, you’ll attempt to make sense out of that’s usually it. Sociologists, on the other hand, an ongoing process that cannot be predicted in treat an interview as data that need to be analyzed advance—making initial observations, develop- in depth; their ultimate goal is to understand social ing tentative general conclusions that suggest life in the context of theory, using established ana- particular types of further observations, mak- lytical techniques. Although sociology and jour- ing those observations and thereby revising your nalism use similar techniques, the two disciplines conclusions, and so forth. In short, the alterna- view and use data differently. tion of induction and deduction discussed in Part Although many of the techniques involved in 1 of this book is perhaps nowhere more evident fi eld research are “natural” activities, they are also and essential than in good fi eld research. For ex- skills to be learned and honed. This chapter dis- pository purposes, however, this chapter focuses cusses these skills in some detail, examining some primarily on some of the theoretical foundations of the major paradigms of fi eld research and de- of fi eld research and on techniques of data collec- scribing some specifi c techniques that make sci- tion. Chapter 13 discusses how to analyze qualita- entifi c fi eld research more useful than the casual tive data. observation we all engage in. Keep in mind that the types of methods re- As we’ll see, there are many paradigms associ- searchers use depend in part on the specifi c re- ated with fi eld research, which comprises a wide search questions they want to answer. For exam- range of studies. This range stems in part from dif- ple, a question such as “How do women construct ferences among paradigms—specifi cally, the va- their everyday lives in order to perform their roles riety of theoretical approaches to basic questions as mothers, partners, and breadwinners?” could be such as “What is data?” “How should we collect addressed by either in-depth interviews or direct data?” and “How should we analyze data?” observations—or both. The assessment of adver- I use the term qualitative fi eld research to dis- tising campaigns might profi t from focus group tinguish this type of observation method from discussions. In most cases, researchers have many methods designed to produce data appropriate fi eld research methods to choose from. for quantitative (statistical) analysis. Thus, surveys provide data from which to calculate the percentage unemployed in a population, mean incomes, TOPICS APPROPRIATE and so forth. Field research more typically yields TO FIELD RESEARCH qualitative data: observations not easily reduced to numbers. Thus, for example, a fi eld researcher may note the “paternalistic demeanor” of leaders One of the key strengths of fi eld research is how at a political rally or the “defensive evasions” of a comprehensive a perspective it can give research- public offi cial at a public hearing without trying to ers. By going directly to the social phenomenon express either the paternalism or the defensive- under study and observing it as completely as ness as numerical quantities or degrees. Although possible, researchers can develop a deeper and fi eld research can be used to collect quantitative fuller understanding of it. As such, this mode of data—for example, noting the number of interac- observation is especially, though not exclusively, tions of various specifi ed types within a fi eld set- appropriate to research topics and social studies ting—typically, fi eld research is qualitative. that appear to defy simple quantifi cation. Field re- Field observation also differs from some other searchers may recognize several nuances of atti- models of observation in that it’s not just a data- tude and behavior that might escape researchers collecting activity. Frequently, perhaps typically, using other methods. TOPICS APPROPRIATE TO FIELD RESEARCH 315

Field research is especially appropriate to the hoods, as opposed to large societies such as study of those attitudes and behaviors best un- nations, which are diffi cult to study derstood within their natural setting, as opposed I. Subcultures and Lifestyles: How large numbers to the somewhat artifi cial settings of experiments of people adjust to life in groups such as a “rul- and surveys. For example, fi eld research provides ing class” or an “urban underclass” a superior method for studying the dynamics of religious conversion at a revival meeting, just as a In all these social settings, fi eld research can re- statistical analysis of membership rolls would be a veal things that would not otherwise be apparent. better way of discovering whether men or women Here’s a concrete example. were more likely to convert. One issue I’m particularly interested in (Bab- Finally, fi eld research is well suited to the study bie 1985) is the nature of responsibility for public of social processes over time. Thus, the fi eld re- matters: Who’s responsible for maintaining the searcher might be in a position to examine the things that we share? Who’s responsible for keep- rumblings and fi nal explosion of a riot as events ing public spaces—parks, malls, buildings, and actually occur rather than afterward in a recon- so on—clean? Who’s responsible for seeing that struction of the events. broken street signs get fi xed? Or, if a strong wind Other good places to apply fi eld research meth- knocks over garbage cans and rolls them around ods include campus demonstrations, courtroom the street, who’s responsible for getting them out proceedings, labor negotiations, public hearings, of the street? or similar events taking place within a relatively On the surface, the answers to these questions limited area and time. Several such observations are pretty clear. We have formal and informal must be combined in a more comprehensive ex- agreements in our society that assign responsi- amination over time and space. bility for these activities. Government custodians In Analyzing Social Settings, John Lofl and and are the ones who keep public places clean. Trans- colleagues (2006:123–132) discuss several ele- portation department employees take care of the ments of social life appropriate to fi eld research. street signs, and perhaps the police deal with the garbage cans rolling around on a windy day. And A. Practices: Various kinds of behavior, such as when these responsibilities are not fulfi lled, we talking or reading a book tend to look for someone to blame. B. Episodes: A variety of events such as divorce, What fascinates me is the extent to which the crime, and illness assignment of responsibility for public things to C. Encounters: Two or more people meeting and specifi c individuals not only relieves others of the interacting responsibility but actually prohibits them from tak- D. Roles and Social Types: The analysis of the posi- ing it on. It’s my notion that it has become unac- tions people occupy and the behavior associ- ceptable for someone like you or me to take per- ated with those positions: occupations, family sonal responsibility for public matters that haven’t roles, ethnic groups been assigned to us. E. Social and Personal Relationships: Behavior ap- Let me illustrate what I mean. If you were walk- propriate to pairs or sets of roles: mother-son ing through a public park and you threw down a relationships, friendships, and the like bunch of trash, you’d discover that your action was F. Groups and Cliques: Small groups, such as unacceptable to those around you. People would friendship cliques, athletic teams, and work glare at you, grumble to each other; perhaps some- groups. one would say something to you about it. What- G. Organizations: Formal organizations, such as ever the form, you’d be subjected to defi nite, nega- hospitals or schools tive sanctions for littering. Now here’s the irony. If H. Settlements and Habitats: Small-scale “societ- you were walking through that same park, came ies” such as villages, ghettos, and neighbor- across a bunch of trash that someone else had 316 CHAPTER 10 QUALITATIVE FIELD RESEARCH dropped, and cleaned it up, it’s likely that your ac- bor who insisted that the mess should be left for tion would also produce negative sanctions from the street cleaners. Everyone who picked up lit- those around you. ter was sneered at, laughed at, and generally put When I fi rst began discussing this pattern with down. One young man was picking up litter scat- students, most felt the notion was absurd. Al- tered around a trash can when a passerby sneered, though littering would bring negative sanctions, “Clumsy!” It became clear to us that there are only cleaning up a public place would obviously bring three acceptable explanations for picking up litter positive ones: People would be pleased with us in a public place: for doing it. Certainly, all my students said they would be pleased if someone cleaned up a public 1. You did it and got caught—somebody forced place. It seemed likely that everyone else would be you to clean up your mess. pleased, too, if we asked them how they would re- 2. You did it and felt guilty. act to someone’s cleaning up litter in a public place 3. You’re stealing litter. or otherwise taking personal responsibility for fi x- ing some social problem. In the normal course of life in the United States, it’s To settle the issue, I suggested that my students simply not acceptable for people to take responsi- start fi xing the public problems they came across bility for public things. in the course of their everyday activities. As they Clearly, we could not have discovered the nature did so, I asked them to note the answers to two and strength of agreements about taking personal questions: responsibility for public things except through fi eld research. Social norms suggest that taking respon- 1. How did they feel while they were fi xing a sibility is a good thing, sometimes referred to as public problem they had not been assigned good citizenship. Asking people what they thought responsibility for? about taking responsibility would have produced a 2. How did others around them react? solid consensus that it was good. Only going out into life, doing it, and watching what happened My students picked up litter, fi xed street signs, gave us an accurate picture. put knocked-over traffi c cones back in place, As an interesting footnote to this story, my stu- cleaned and decorated communal lounges in their dents and I found that whenever people could get dorms, trimmed trees that blocked visibility at past their initial reactions and discover that the intersections, repaired public playground equip- students were simply taking responsibility for fi x- ment, cleaned public restrooms, and took care of a ing things for the sake of having them work, the hundred other public problems that weren’t “their passersby tended to assist. Although there are responsibility.” some very strong agreements making it “unsafe” Most reported feeling very uncomfortable doing to take responsibility for public things, the willing- whatever they did. They felt foolish, goody-goody, ness of one person to rise above those agreements conspicuous, and all the other feelings that usu- seemed to make it safe for others to do so, and ally keep us from performing these activities. In they did. almost every case, the reactions of those around In summary, fi eld research offers the advan- them increased their discomfort. One student was tage of probing social life in its natural habitat. Al- removing a damaged and long-unused newspaper though some things can be studied adequately in box from the bus stop, where it had been a prob- questionnaires or in the laboratory, others cannot. lem for months, when the police arrived, having And direct observation in the fi eld lets research- been summoned by a neighbor. Another student ers observe subtle communications and other decided to clean out a clogged storm drain on his events that might not be anticipated or measured street and found himself being yelled at by a neigh- otherwise. SPECIAL CONSIDERATIONS IN QUALITATIVE FIELD RESEARCH 317

SPECIAL CONSIDERATIONS IN ate for you to talk and act like a university profes- QUALITATIVE FIELD RESEARCH sor or student. This type of research introduces an ethical issue, one on which social researchers themselves Every research method presents specifi c issues are divided. Is it ethical to deceive the people and concerns, and qualitative fi eld research is no you’re studying, in the hope that they will confi de exception. When you use fi eld research methods, in you in ways that they would not confi de in you if you’re confronted with decisions about the role you were an identifi ed researcher? Do the potential you’ll play as an observer and your relations with benefi ts to be gained from the research offset such the people you’re observing. Let’s examine some considerations? Although many professional asso- of the issues involved in these decisions. ciations have addressed this issue, the norms to be followed remain somewhat ambiguous when applied to specifi c situations. The Various Roles of the Observer Related to this ethical consideration is a scien- In fi eld research, observers can play any of several tifi c one. No researcher deceives his or her subjects roles, including participating in what they want solely for the purpose of deception. Rather, it’s to observe (this was the situation of the students done in the belief that the data will be more valid who fi xed public things). In this chapter, I’ve used and reliable—that the subjects will be more natural the term fi eld research rather than the frequently and honest if they do not know the researcher is used term participant observation, because fi eld doing a research project. If the people being stud- researchers need not always participate in what ied know they’re being studied, they might modify they’re studying, though they usually will study it their behavior in a variety of ways. This problem is directly at the scene of the action. As Catherine known as reactivity. Marshall and Gretchen Rossman point out: First, they might expel the researcher. Second, they might modify their speech and behavior to The researcher may plan a role that entails appear more respectable than would otherwise be varying degrees of “participantness”—that is, the the case. Third, the social process itself might be degree of actual participation in daily life. At one radically changed. Students making plans to burn extreme is the full participant, who goes about down the university administration building, for ordinary life in a role or set of roles constructed example, might give up the plan altogether once in the setting. At the other extreme is the com- they learn that one of their group is a social scien- plete observer, who engages not at all in social tist conducting a research project. interaction and may even shun involvement in On the other side of the coin, if you’re a com- the world being studied. And, of course, all pos- plete participant, you may affect what you’re study- sible complementary mixes along the contin- ing. To play the role of participant, you must par- uum are available to the researcher. — (1995:60) ticipate. Yet, your participation may signifi cantly affect the social process you’re studying. Suppose, The complete participant, in this sense, may be for example, that you’re asked for your ideas about a genuine participant in what he or she is studying what the group should do next. No matter what (for example, a participant in a campus demon- you say, you will affect the process in some fash- stration) or may pretend to be a genuine partici- ion. If the group follows your suggestion, your pant. In any event, if you’re acting as the complete participant, you would let people see you only as a participant, not as a researcher. For instance, if reactivity The problem that the subjects of social re- you’re studying a group made up of uneducated search may react to the fact of being studied, thus altering and inarticulate people, it would not be appropri- their behavior from what it would have been normally. 318 CHAPTER 10 QUALITATIVE FIELD RESEARCH infl uence on the process is obvious. If the group decides not to follow your suggestion, the process whereby the suggestion is rejected may affect what happens next. Finally, if you indicate that you just don’t know what should be done next, you may be adding to a general feeling of uncertainty and indecisiveness in the group. Ultimately, anything the participant-observer does or does not do will have some effect on what’s being observed; it’s simply inevitable. More seriously, the research effort may have an important effect on what happens. There is no complete protection against this effect, though sensitivity to the issue may provide a partial protection. (This infl u- ence, called the Hawthorne effect, was discussed more fully in Chapter 8.)

Because of these ethical and scientifi c consid- Earl Babbie erations, the fi eld researcher frequently chooses Field research is a hands-on process, which involves a different role from that of complete participant. going to the scene of the action and checking You could participate fully with the group under it out. study but make it clear that you were also undertaking research. As a member of the volleyball team, for example, you might use your position to Fred Davis (1973) characterizes the extreme launch a study in the sociology of sports, letting roles that observers might play as “the Martian” your teammates know what you’re doing. There and “the Convert.” The latter involves delving are dangers in this role also, however. The people deeper and deeper into the phenomenon under being studied may shift much of their attention study, running the risk of “going native.” We’ll ex- to the research project rather than focus on the amine this risk further in the next section. natural social process, so that the process being To appreciate the “Martian” approach, imagine observed is no longer typical. Or, conversely, you that you were sent to observe some newfound life yourself may come to identify too much with the on Mars. Probably you would feel yourself ines- interests and viewpoints of the participants. You capably separate from the Martians. Some social may begin to “go native” and lose much of your scientists adopt this degree of separation when scientifi c detachment. observing cultures or social classes different from At the other extreme, the complete observer their own. studies a social process without becoming a part Marshall and Rossman (1995:60–61) also note of it in any way. Quite possibly, because of the re- that the researcher can vary the amount of time searcher’s unobtrusiveness, the subjects of study spent in the setting being observed; that is, re- might not realize they’re being studied. Sitting at searchers can be a full-time presence on the scene a bus stop to observe jaywalking at a nearby in- or just show up now and then. Moreover, they can tersection is one example. Although the complete focus their attention on a limited aspect of the so- observer is less likely to affect what’s being stud- cial setting or seek to observe all of it—framing an ied and less likely to “go native” than the complete appropriate role to match their aims. participant, she or he is also less likely to develop a Different situations ultimately require different full appreciation of what’s being studied. Observa- roles for the researcher. Unfortunately, there are tions may be more sketchy and transitory. no clear guidelines for making this choice—you SPECIAL CONSIDERATIONS IN QUALITATIVE FIELD RESEARCH 319 must rely on your understanding of the situation preciate the phenomenon you’ve set out to study, and your own good judgment. In making your de- you need to believe that Jesus is coming Thursday cision, however, you must be guided by both meth- night. In some settings, this can also help you gain odological and ethical considerations. Because rapport with your subjects. these often confl ict, your decision will frequently Adopting an alien point of view is an uncom- be diffi cult, and you may fi nd sometimes that your fortable prospect for most people. It can be hard role limits your study. enough merely to learn about views that seem strange to you; you may sometimes fi nd it hard just Relations to Subjects to tolerate certain views; but to take them on as your own is ten times worse. Robert Bellah (1970, Having introduced the different roles fi eld research- 1974) has offered the term symbolic realism to in- ers might play in connection with their observa- dicate the need for social researchers to treat the tions, we now focus more specifi cally on how re- beliefs they study as worthy of respect rather than searchers may relate to the subjects of their study as objects of ridicule. If you seriously entertain this and to the subjects’ points of view. prospect, you may appreciate why William Shaffi r We’ve already noted the possibility of pretend- and Robert Stebbins (1991:1) concluded that “fi eld- ing to occupy social statuses we don’t really oc- work must certainly rank with the more disagree- cupy. Consider now how you would think and feel able activities that humanity has fashioned for in such a situation. itself.” Suppose you’ve decided to study a religious There is, of course, a danger in adopting the cult that has enrolled many people in your neigh- points of view of the people you’re studying. When borhood. You might study the group by joining you abandon your objectivity in favor of adopting it or pretending to join it. Take a moment to ask such views, you lose the possibility of seeing and yourself what the difference is between “really” understanding the phenomenon within frames of joining and “pretending” to join. The main differ- reference unavailable to your subjects. On the one ence is whether or not you actually take on the hand, accepting the belief that the world will end beliefs, attitudes, and other points of view shared Thursday night allows you to appreciate aspects by the “real” members. If the cult members be- of that belief available only to believers; stepping lieve that Jesus will come next Thursday night to outside that view, however, makes it possible for destroy the world and save the members of the you to consider some reasons why people might cult, do you believe it or do you simply pretend to adopt such a view. You may discover that some did believe it? so as a consequence of personal trauma (such as Traditionally, social scientists have tended to unemployment or divorce) whereas others were emphasize the importance of “objectivity” in such brought into the fold through their participation matters. In this example, that injunction would in particular social networks (for example, their be to avoid getting swept up in the beliefs of the whole bowling team joined the cult). Notice that group. Without denying the advantages associated the cult members might disagree with those “ob- with such objectivity, social scientists today also jective” explanations, and you might not come up recognize the benefi ts gained by immersing them- with them to the extent that you had operated le- selves in the points of view they’re studying, what gitimately within the group’s views. Lofl and and associates (2006:70) refer to as “selec- Anthropologists sometimes use the term emic tive competence” or “insider knowledge, skill, or perspective in reference to taking on the point of understanding.” Ultimately, you will not be able to view of those being studied. In contrast, the etic fully understand the thoughts and actions of the perspective maintains a distance from the native cult members unless you can adopt their points point of view in the interest of achieving more of view as true—at least temporarily. To fully ap- objectivity. 320 CHAPTER 10 QUALITATIVE FIELD RESEARCH

The apparent dilemma here is that both of these be moved by their personal problems and crises. postures offer important advantages but also seem Imagine, for example, that one of the cult mem- mutually exclusive. In fact, you can assume both bers becomes ill and needs a ride to the hospital. postures. Sometimes you can simply shift view- Should you provide transportation? Sure. Suppose points at will. When appropriate, you can fully someone wants to borrow money to buy a stereo. assume the beliefs of the cult; later, you can step Should you loan it? Probably not. Suppose they outside those beliefs (more accurately, you can need the money for food? step inside the viewpoints associated with social There are no black-and-white rules for resolv- science). As you become more adept at this kind ing situations such as these, but you should realize of research, you may come to hold contradictory that you will need to deal with them regardless of viewpoints simultaneously, rather than switch whether or not you reveal that you’re a researcher. back and forth. Such problems do not tend to arise in other types During my study of trance channeling—people of research—surveys and experiments, for exam- who allow spirits to occupy their bodies and speak ple—but they are part and parcel of fi eld research. through them—I found I could participate fully in This discussion of the fi eld researcher’s rela- channeling sessions without becoming alienated tions to subjects fl ies in the face of the conventional from conventional social science. Rather than view of “scientifi c objectivity.” Before concluding “believing” in the reality of channeling, I found it this section, let’s take the issue one step further. possible to suspend beliefs in that realm: neither In the conventional view of science, there are believing it to be genuine (like most of the other implicit differences of power and status separat- participants) nor disbelieving it (like most scien- ing the researcher from the subjects of research. tists). Put differently, I was open to either possibil- When we discussed experimental designs in Chap- ity. Notice how this differs from our normal need to ter 8, for example, who was in charge was obvi- “know” whether such things are legitimate or not. ous: The experimenter organized things and told Social researchers often refer to the concerns the subjects what to do. Often the experimenter just discussed as a matter of refl exivity, in the was the only person who even knew what the re- sense of things acting on themselves. Thus, your search was really about. Something similar might own characteristics can affect what you see and be said about survey research. The person running how you interpret it. The issue is broader than the survey designs the questions, decides who will that, however, and applies to the subjects as well be selected for questioning, and is responsible for as to the researcher. Imagine yourself interview- making sense out of the data collected. ing a homeless person (1) on the street, (2) in a Sociologists often look at these sorts of rela- homeless shelter, or (3) in a social welfare offi ce. tionships as power or status relationships. In ex- The research setting could affect the person’s re- perimental and survey designs, the researcher sponses. In other words, you might get different clearly has more power and a higher status than results because of where you conducted the inter- do the people being studied. The researchers have view. Moreover, you might act differently as a re- a special knowledge that the subjects do not enjoy. searcher in those different settings. If you refl ect on They are not so crude as to say they are superior to this issue, you’ll be able to identify other aspects of their subjects, but there is a sense in which that’s the research encounter that complicate the task of implicitly assumed. (Notice that there is a similar, “simply observing what’s so.” implicit assumption about the writers and readers The problem we’ve just been discussing could of textbooks.) be seen as psychological, occurring mostly in- In fi eld research, such assumptions can be side the researchers’ or subjects’ heads. There is problematic. When the early European anthro- a corresponding problem at a social level, how- pologists set out to study what were originally ever. When you become deeply involved in the called “primitive” societies, there was no ques- lives of the people you’re studying, you’re likely to tion that the anthropologists knew best. Whereas SOME QUALITATIVE FIELD RESEARCH PARADIGMS 321 the natives “believed” in witchcraft, for example, 1930s and 1940s at the University of Chicago’s so- the anthropologists “knew” it wasn’t really true. ciology department, whose faculty and students And whereas the natives said some of their ritu- fanned out across the city to observe and under- als would appease the gods, the anthropologists stand local neighborhoods and communities. The explained that the “real” functions of these rituals researchers of that era and their research approach were the creation of social identity, the establish- are now often referred to as the Chicago School. ment of group solidarity, and so on. One of the earliest and best-known studies that The more social researchers have gone into illustrates this research tradition is William Foote the fi eld to study their fellow humans face-to-face, Whyte’s ethnography of Cornerville, an Italian however, the more they have become conscious of American neighborhood, in his book Street Corner these implicit assumptions about researcher supe- Society (1943). An ethnography is a study that fo- riority, and the more they have considered alterna- cuses on detailed and accurate description rather tives. As we turn now to the various paradigms of than explanation. Like other naturalists, Whyte be- fi eld research, we’ll see some of the ways in which lieved that in order to fully learn about social life that ongoing concern has worked itself out. on the streets, he needed to become more of an insider. He made contact with “Doc,” his key infor- SOME QUALITATIVE mant, who appeared to be one of the street-gang FIELD RESEARCH PARADIGMS leaders. Doc let Whyte enter his world, and Whyte got to participate in the activities of the people of Cornerville. His study offered something that Although I’ve described fi eld research as simply surveys could not: a richly detailed picture of life going where the action is and observing it, there among the Italian immigrants of Cornerville. are actually many different approaches to this An important feature of Whyte’s study is that research method. This section examines several he reported the reality of the people of Cornerville fi eld research paradigms: naturalism, ethnometh- on their terms. The naturalist approach is based odology, grounded theory, case studies and the on telling “their” stories the way they “really are,” extended case method, institutional ethnogra- not the way the ethnographer understands “them.” phy, and participatory action research. Although The narratives collected by Whyte are taken at this survey won’t exhaust the variations on the face value as the social “truth” of the Cornerville method, it should give you a broad appreciation of residents. the possibilities. Forty years later, David Snow and Leon Ander- There are no specifi c methods attached to each son (1987) conducted exploratory fi eld research of these paradigms. You could do ethnomethod- into the lives of homeless people in Austin, Texas. ology or institutional ethnography by analyzing Their main task was to understand how the home- court hearings or conducting group interviews, for less construct and negotiate their identity while example. The important distinctions of this section knowing that the society they live in attaches a are epistemological, that is, having to do with what stigma to homelessness. Snow and Anderson be- data mean, regardless of how they were collected. lieved that, to achieve this goal, the collection of data had to arise naturally. Like Whyte in Street Naturalism Corner Society, they found some key informants whom they followed in their everyday journeys, Naturalism is an old tradition in qualitative research. The earliest fi eld researchers operated on naturalism An approach to fi eld research based on the the positivist assumption that social reality was assumption that an objective social reality exists and can “out there,” ready to be naturally observed and re- be observed and reported accurately. ported by the researcher as it “really is” (Gubrium ethnography A report on social life that focuses on and Holstein 1997). This tradition started in the detailed and accurate description rather than explanation. 322 CHAPTER 10 QUALITATIVE FIELD RESEARCH such as at their day-labor pickup sites or under future of U.S. society, most are more interested in bridges. Snow and Anderson chose to memorize creating alternative lives and cultures for them- the conversations they participated in or the “talks” selves than in blowing up the mainstream society. that homeless people had with each other. At the That’s not to suggest none of the survivalists are a end of the day, the two researchers debriefed and threat, but Mitchell’s examination moves beyond wrote detailed fi eld notes about all the “talks” they the McVeighs, Koreshes, and Weavers to draw a encountered. They also taped in-depth interviews broader picture of the whole phenomenon. with their key informants. In Chapter 9, we saw how the Internet is affect- Snow and Anderson reported “hanging out” ing survey research. Eric Anderson (2005) used with homeless people over the course of 12 months the Internet to launch a qualitative, in-depth infor a total of 405 hours in 24 different settings. Out terviewing study of male cheerleaders: “Twelve of these rich data, they identifi ed three related pat- collegiate male cheerleaders were contacted for terns in homeless people’s conversations. First, the interviews by using the member profi le search on homeless showed an attempt to “distance” them- American Online which provides a search engine selves from other homeless people, from the low- for accessing the stated interests of AOL’s 33 mil- status job they currently had, or from the Salvation lion subscribers. After communicating with these Army they depended on. Second, they “embraced” cheerleaders through instant messaging, I asked their street-life identity—their group membership them for in-depth, taped telephone interviews” or a certain belief about why they are homeless. (2005:340). Anderson then used snowball sam- Third, they told “fi ctive stories” that always con- pling to increase the number of cheerleaders to trasted with their everyday life. For example, they study. would often say that they were making much more Whereas this chapter aims at introducing you to money than they really were, or even that they some of the different approaches available to you were “going to be rich.” in qualitative fi eld research, please realize that this Richard Mitchell (2002) offers another, timely discussion of ethnography merely sketches some illustration of the power of ethnographic report- of the many avenues social researchers have es- ing. Recent U.S. history has raised the specter of tablished. If you’re interested in this general ap- violence from secretive survivalist groups, drama- proach, you might want to explore the idea of virtual tized by the 1992 siege at Ruby Ridge, Idaho, which ethnography, which uses ethnographic techniques left the wife and son of the white supremacist for inquiry into cyberspace. Or, in a different direc- Randy Weaver dead; the 1993 shootout with David tion, autoethnography intentionally assumes a per- Koresh and his Branch Davidians in Waco, Texas; sonal stance, breaking with the general proscrip- and Timothy McVeigh’s 1995 bombing, which left tion against the researcher getting involved at that 168 dead under the rubble of the nine-story Mur- level. You can learn more about these variants on rah Federal Building in Oklahoma City. ethnography by searching the web or your campus Mitchell describes a variety of survivalist indi- library. A later section of this chapter will examine viduals and groups, seeking to understand their institutional ethnography, which links individuals reasoning, their plans, and the threat they may and organizations. pose for the rest of us. Although he fi nds the survivalists disillusioned with and uncertain about the Ethnomethodology

Ethnomethodology, which I introduced as a re- An approach to the study of ethnomethodology search paradigm in Chapter 2, is a unique approach social life that focuses on the discovery of implicit, usually unspoken assumptions and agreements; this method to qualitative fi eld research. It has its roots in the often involves the intentional breaking of agreements as a philosophical tradition of phenomenology, which way of revealing their existence. can explain why ethnomethodologists are skepti- SOME QUALITATIVE FIELD RESEARCH PARADIGMS 323 cal about the way people report their experience of She appeared momentarily stunned. Then reality (Gubrium and Holstein 1997). Alfred Schutz she answered in a hostile way: “What do you (1967, 1970), who introduced phenomenology, ar- mean, ‘What do you mean?’ A fl at tire is a fl at gued that reality was socially constructed rather tire. That is what I meant. Nothing special. What than being “out there” for us to observe. People a crazy question.” describe their world not “as it is” but “as they make Case 6 sense of it.” Thus, phenomenologists would argue The victim waved his hand cheerily. that Whyte’s street-corner men were describing (S) How are you? their gang life as it made sense to them. Their re- (E) How I am in regard of what? My health, ports, however, would not tell us how and why it my fi nances, my school work, my peace of mind, made sense to them. For this reason, researchers my . . . ? cannot rely on their subjects’ stories to depict so- (S) (Red in the face and suddenly out of con- cial realities accurately. trol.) Look I was just trying to be polite. Frankly, I Whereas traditional ethnographers believe in don’t give a damn how you are. immersing themselves in a particular culture and reporting their informants’ stories as if they rep- By setting aside or “bracketing” their expecta- resent reality, phenomenologists see a need to tions from these everyday conversations, the ex- “make sense” out of the informants’ perceptions perimenters made visible the subtleties of mun- of the world. Following in this tradition, some fi eld dane interactions. For example, although “How researchers have tried to devise techniques that are you?” has many possible meanings, none of us reveal how people make sense of their everyday have any trouble knowing what it means in casual world. As we saw in Chapter 2, the sociologist Har- interactions, as the unsuspecting subject revealed old Garfi nkel suggested that researchers “break the in his fi nal comment. rules” so that people’s taken-for-granted expecta- Ethnomethodologists, then, are not simply in- tions would become apparent. This is the tech- terested in subjects’ perceptions of the world. In nique that Garfi nkel called ethnomethodology. these cases, we could imagine that the subjects Garfi nkel became known for engaging his stu- may have thought that the experimenters were dents to perform a series of “breaching experi- rude, stupid, or arrogant. The conversation itself, ments” designed to break away from the ordinary not the informants, becomes the object of ethno- (Heritage 1984). For instance, Garfi nkel (1967) methodological studies. In general, in ethnometh- asked his students to do a “conversation clarifi ca- odology the focus is on the “underlying patterns” of tion experiment.” Students were told to engage in interactions that regulate our everyday lives. an ordinary conversation with an acquaintance or Ethnomethodologists believe that researchers a friend and to ask for clarifi cation about any of who use a naturalistic analysis “[lose] the ability this person’s statements. Through this technique, to analyze the commonsense world and its culture they uncovered elements of conversation that are if [they use] analytical tools and insights that are normally taken for granted. Here are two examples themselves part of the world or culture being stud- of what Garfi nkel’s students reported — (1967:42): ied” (Gubrium and Holstein 1997:43). D. L. Wieder has provided an excellent example of how much a Case 1 naturalistic approach differs from an ethnometh- The subject was telling the experimenter, a odological approach (Gubrium and Holstein 1997). member of the subject’s car pool, about having In his study, Language and Social Reality: The Case had a fl at tire while going to work the previous of Telling the Convict Code (1988), Wieder started to day. approach convicts in a halfway house in a tradi- (S) I had a fl at tire. tional ethnographic style: He was going to become (E) What do you mean, you had a fl at tire? an insider by befriending the inmates and by con- 324 CHAPTER 10 QUALITATIVE FIELD RESEARCH ducting participant observations. He took careful Grounded Theory notes and recorded interactions among inmates Grounded theory originated from the collaboration and between inmates and staff. His fi rst concern of Barney Glaser and Anselm Strauss, sociologists was to describe the life of the convicts of the half- who brought together two main traditions of re- way house the way it “really was” for them. Wie- search: positivism and interactionism. Essentially, der’s observations allowed him to report on a “con- grounded theory is the attempt to derive theories vict code” that he thought was the source of the from an analysis of the patterns, themes, and com- deviant behavior expressed by the inmates toward mon categories discovered in observational data. the staff. This code, which consisted of a series of The fi rst major presentation of this method can be rules such as “Don’t kiss ass,” “Don’t snitch,” and found in Glaser and Strauss’s book, The Discovery “Don’t trust the staff,” was followed by the inmates of Grounded Theory (1967). Grounded theory can who interfered with the staff members’ attempts to be described as an approach that attempts to com- help them make the transition from prison to the bine a naturalist approach with a positivist con- community. cern for a “systematic set of procedures” in doing It became obvious to Wieder that the code was qualitative research. more than an explanation for the convicts’ deviant Anselm Strauss and Juliet Corbin (1998:43–46) behavior; it was a “method of moral persuasion have suggested that grounded theory allows the and justifi cation” (Wieder 1988:175). At this point researcher to be scientifi c and creative at the he changed his naturalistic approach to an ethno- same time, as long as the researcher follows these methodological one. Recall that whereas natural- guidelines: istic fi eld researchers aim to understand social life as the participants understand it, ethnomethodologists are more intent on identifying the meth- • Think Comparatively: The authors suggest that ods through which understanding occurs. In the researchers must compare numerous incidents case of the convict code, Wieder came to see that as a way of avoiding the biases that can arise convicts used the code to make sense of their own from interpretations of initial observations. interactions with other convicts and with the staff. • Obtain Multiple Viewpoints: In part this refers The ethnography of the halfway house thus shifted to the different points of view of participants to an ethnography of the code. For instance, the in the events under study, but Strauss and convicts would say, “You know I won’t snitch,” re- Corbin suggest that different observational ferring to the code as a way to justify their refusal techniques may also provide a variety of to answer Wieder’s question (1988:168). Accord- viewpoints. ing to Wieder, the code “operated as a device for • Periodically Step Back: As data accumulate, stopping or changing the topic of conversation” you’ll begin to frame interpretations about (1988:175). Even the staff would refer to the code what is going on, and it’s important to keep to justify their reluctance to help the convicts. Al- checking your data against those interpreta- though the code was something that constrained tions. As Strauss and Corbin (1998:45) say, behavior, it also functioned as a tool for the control “The data themselves do not lie.” of interactions. • Maintain an Attitude of Skepticism: As you begin to interpret the data, you should regard all those interpretations as provisional, using new grounded theory An inductive approach to the study observations to test those interpretations, not of social life that attempts to generate a theory from the constant comparing of unfolding observations. This differs just confi rm them. greatly from hypothesis testing, in which theory is used to • Follow the Research Procedures: Grounded generate hypotheses to be tested through observations. theory allows for fl exibility in data collection SOME QUALITATIVE FIELD RESEARCH PARADIGMS 325

as theories evolve, but Strauss and Corbin Assessing how data from each of the two types (1998:46) stress that three techniques are es- of universities fi t with the other helped refi ne the- sential: “making comparisons, asking ques- ory building. This refi nement process stands in tions, and sampling.” contrast to a naturalist approach, in which the process of building theory would have stopped with Grounded theory emphasizes research proce- Conrad’s fi rst interpretation. dures. In particular, systematic coding is impor- Conrad concluded that changes in university tant for achieving validity and reliability in the data curricula are based on the following process: Con- analysis. Because of this somewhat positivistic fl ict and interest groups emerge because of interview of data, grounded theorists are quite open to nal and external social structural forces; they push the use of qualitative studies in conjunction with for administrative intervention and recommenda- quantitative ones. Here are two examples of the tion to make changes in the current academic pro- implementation of this approach. gram; these changes are then made by the most powerful decision-making body. Studying Academic Change Clifton F. Conrad’s (1978) study of academic change in universi- Shopping Romania Much has been written ties is an early example of the grounded theory about large-scale changes caused by the shift from approach. Conrad hoped to uncover the major socialism to capitalism in the former USSR and its sources of changes in academic curricula and at Eastern European allies. Patrick C. Jobes and his the same time understand the process of change. colleagues (1997) wanted to learn about the transi- Using the grounded theory idea of theoretical sam- tion on a smaller scale among average Romanians. pling—whereby groups or institutions are selected They focused on the task of shopping. on the basis of their theoretical relevance—Con- Noting that shopping is normally thought of as a rad chose four universities for the purpose of his routine, relatively rational activity, the researchers study. In two, the main vehicle of change was the suggested that it could become a social problem formal curriculum committee; in the other two, the in a radically changing economy. They used the vehicle was an ad hoc group. grounded theory method to examine Romanian Conrad explained, step by step, the advantage shopping as a social problem, looking for the ways of using the grounded theory approach in building in which ordinary people solved the problem. his theory of academic change. He described the Their fi rst task was to learn something about process of systematically coding data in order to how Romanians perceived and understood the create categories that must “emerge” from the data task of shopping. The researchers—participants and then assessing the fi tness of these categories in a social problems class—began by interviewing in relation to one other. Going continuously from 40 shoppers and asking whether they had experi- data to theory and theory to data allowed him to enced problems in connection with shopping and reassess the validity of his initial conclusions about what actions they had taken to cope with those academic change. problems. For instance, it fi rst seemed that academic Once the initial interviews were completed, change was caused mainly by an administrator the researchers reviewed their data, looking for who was pushing for it. By reexamining the data categories of responses—the shoppers’ most com- and looking for more-plausible explanations, Con- mon problems and solutions. One of the most rad found the pressure of interest groups a more common problems was a lack of money. This led convincing source of change. The emergence of to the researchers’ fi rst working hypothesis: The these interest groups actually allowed the admin- “socio-economic position of shoppers would be istrator to become an agent of change. associated with how they perceived problems and 326 CHAPTER 10 QUALITATIVE FIELD RESEARCH sought solutions” (Jobes et al. 1997:133). This and a particular instance of something is the essential other hypotheses helped the researchers focus characteristic of the case study. their attention on more-specifi c variables in subse- The chief purpose of a case study can be de- quent interviewing. scriptive, as when an anthropologist describes As they continued, they also sought to inter- the culture of a preliterate tribe. Or the in-depth view other types of shoppers. When they inter- study of a particular case can yield explanatory in- viewed students, for example, they discovered that sights, as when the community researchers Robert different types of shoppers were concerned with and Helen Lynd (1929, 1937) and W. Lloyd Warner different kinds of goods, which in turn affected the (1949) sought to understand the structure and pro- problems faced and the solutions tried. cess of social stratifi cation in small-town USA. As additional hypotheses were developed in Case study researchers may seek only an idio- response to the continued interviewing, the re- graphic understanding of the particular case under searchers began to develop a more or less stan- examination, or—as we’ve seen with grounded dardized set of questions to ask shoppers. Initially, theory—case studies can form the basis for the de- all the questions were open-ended, but they even- velopment of more general, nomothetic theories. tually developed closed-ended items as well. Michael Burawoy and his colleagues (1991) This study illustrates the key, inductive princi- have suggested a somewhat different relation- ples of grounded theory: data are collected in the ship between case studies and theory. For them, absence of hypotheses. The initial data are used to the extended case method has the purpose of determine the key variables as perceived by those discovering fl aws in, and then modifying, existing being studied, and hypotheses about relationships social theories. This approach differs importantly among the variables are similarly derived from the from some of the others already discussed. data collected. Continuing data collection yields Whereas the grounded theorists seek to enter refi ned understanding and, in turn, sharpens the the fi eld with no preconceptions about what they’ll focus of data collection itself. fi nd, Burawoy suggests just the opposite: to try “to lay out as coherently as possible what we expect to Case Studies and the fi nd in our site before entry” (Burawoy et al. 1991:9). Extended Case Method Burawoy sees the extended case method as a way to rebuild or improve theory instead of approving Social researchers often speak of case studies, or rejecting it. Thus, he looks for all the ways in which focus attention on one or a few instances which observations confl ict with existing theories of some social phenomenon, such as a village, a and what he calls “theoretical gaps and silences” family, or a juvenile gang. As Charles Ragin and (1991:10). This orientation to fi eld research implies Howard Becker (1992) point out, there is little con- that knowing the literature beforehand is actually sensus on what constitutes a “case” and the term a must for Burawoy and his colleagues, whereas is used broadly. The case being studied, for exam- grounded theorists would worry that knowing ple, might be a period of time rather than a particu- what others have concluded might bias their ob- lar group of people. The limitation of attention to servations and theories. To illustrate the extended case method, I’ll use two examples of studies by Burawoy’s students. case study The in-depth examination of a single instance of some social phenomenon, such as a village, a Teacher-Student Negotiations Leslie Hurst family, or a juvenile gang. (1991) set out to study the patterns of interaction extended case method A technique developed by Michael Burawoy in which case study observations are between teachers and students of a junior high used to discover fl aws in and to improve existing social school. She went into the fi eld armed with existing, theories. contradictory theories about the “offi cial” functions SOME QUALITATIVE FIELD RESEARCH PARADIGMS 327 of the school. Some theories suggested that the sense of property” (1991:185), or as Hurst defi nes purpose of schools was to promote social mobil- it, “babysit” the student’s body and soul. ity, whereas others suggested that schools mainly Hurst says she differs from the traditional socio- reproduced the status quo in the form of a strati- logical perspectives as follows: fi ed division of labor. The offi cial roles assigned I do not approach schools with a futuristic to teachers and students could be interpreted in eye. I do not see the school in terms of train- terms of either view. ing, socializing, or slotting people into future Hurst was struck, however, by the contrast hierarchies. To approach schools in this manner between these theories and the types of interac- is to miss the negotiated, chaotic aspects of the tions she observed in the classroom. In her own classroom and educational experience. A futur- experiences as a student, teachers had total rights ist perspective tends to impose an order and over the mind, body, and soul of their pupils. She purpose on the school experience, missing its observed something quite different at a school in day-to-day reality. — (1991:186) a lower-middle-class neighborhood in Berkeley, California—Emerald Junior High School, where she In summary, what emerges from Hurst’s study is volunteered as a tutor. She had access to several an attempt to improve the traditional sociological classrooms, the lunchroom, and the English De- understanding of education by adding the idea that partment’s meetings. She wrote fi eld notes based classroom, school, and family have separate func- on the negotiation interactions between students tions, which in turn can explain the emergence of and teachers. She explained the nature of the “negotiated order” in the classroom. student-teacher negotiations she witnessed, by focusing on the separation of functions among the The Fight against AIDS Katherine Fox (1991) set school, the teacher, and the family. out to study an agency whose goal was to fi ght the In Hurst’s observation, the school fulfi lled the AIDS epidemic by bringing condoms and bleach function of controlling its students’ “bodies”—for (for cleaning needles) to intravenous drug users. example, by regulating their general movements Her study offers a good example of fi nding the limi- and activities within the school. The students’ tations of well-used models of theoretical explana- “minds” were to be shaped by the teacher, whereas tion in the realm of understanding deviance—spe- students’ families were held responsible for their cifi cally, the “treatment model” that predicted that “souls”; that is, families were expected to social- drug users would come to the clinic and ask for ize students regarding personal values, attitudes, treatment. Fox’s interactions with outreach work- sense of property, and sense of decorum. When ers—most of whom were part of the community students don’t come to school with these values of drug addicts or former prostitutes—contradicted in hand, the teacher, according to Hurst, “must that model. fi rst negotiate with the students some compromise To begin, it was necessary to understand the on how the students will conduct themselves and drug users’ subculture and use that knowledge to on what will be considered classroom decorum” devise more-realistic policies and programs. The —(1991:185). target users had to be convinced, for example, that Hurst explained the constant bargaining be- the program workers could be trusted, that they tween teachers and students is an expression of were really interested only in providing bleach and the separation between “the body,” which is the condoms. The target users needed to be sure they school’s concern, and “the soul” as family domain. were not going to be arrested. The teachers, who had limited sanctioning power Fox’s fi eld research didn’t stop with an examina- to control their students’ minds in the classroom, tion of the drug users. She also studied the agency were using forms of negotiations with students so workers, discovering that the outreach program that they could “control . . . the student’s body and meant different things to the research directors and 328 CHAPTER 10 QUALITATIVE FIELD RESEARCH the outreach workers. Some of the volunteers who sion that more traditional types of research often were actually providing the bleach and condoms overlook. were frustrated about the minor changes they felt Smith’s methodology is similar to ethnometh- they could make. Many thought the program was odology in the sense that the inquiry does not fo- just a bandage on the AIDS and drug-abuse prob- cus on the subjects themselves. The institutional lems. Some resented having to take fi eld notes. Di- ethnographer starts with the personal experiences rectors, on the other hand, needed reports and fi eld of individuals but proceeds to uncover the institu- notes so that they could validate their research in tional power relations that structure and govern the eyes of the federal and state agencies that fi - those experiences. In this process, the researcher nanced the project. Fox’s study showed how the can reveal aspects of society that would have been AIDS research project developed the bureaucratic missed by an inquiry that began with the offi cial inertia typical of established organizations: Its goal purposes of institutions. became that of sustaining itself. This approach links the “microlevel” of every- Both of these studies illustrate how the extended day personal experiences with the “macrolevel” of case method can operate. The researcher enters institutions. As M. L. Campbell puts it: the fi eld with full knowledge of existing theories Institutional ethnography, like other forms of but aims to uncover contradictions that require the ethnography, relies on interviewing, obser- modifi cation of those theories. vations and document as data. Institutional One criticism of the case study method is the ethnography departs from other ethnographic limited generalizability of what may be observed approaches by treating those data not as the in a single instance of some phenomenon. This topic or object of interest, but as “entry” into the risk is reduced, however, when more than one social relations of the setting. The idea is to tap case is studied in depth: the comparative case into people’s expertise. — (1998:57) study method. You can fi nd examples of this in the discussion of comparative and historical research Here are two examples of this approach. methods in Chapter 11 of this book. Mothering, Schooling, and Child Development Institutional Ethnography Our fi rst example of institutional ethnography is a study by Alison Griffi th (1995), who collected data Institutional ethnography is an approach origi- with Dorothy Smith on the relationship among nally developed by Dorothy Smith (1978) to bet- mothering, schooling, and children’s development. ter understand women’s everyday experiences by Griffi th started by interviewing mothers from three discovering the power relations that shape those cities of southern Ontario on their everyday work experiences. Today this methodology has been of creating a relationship between their families extended to the ideologies that shape the experi- and the school. This was the starting point for ences of any oppressed subjects. other interviews with parents, teachers, school ad- Smith and other sociologists believe that if re- ministrators, social workers, school psychologists, searchers ask women or other members of subor- and central offi ce administrators. dinated groups about “how things work,” they can In her fi ndings, Griffi th explained how the dis- discover the institutional practices that shape their course about mothering had shifted its focus over realities (M. L. Campbell 1998; D. Smith 1978). The time from mother-child interactions to “child- goal of such inquiry is to uncover forms of oppres- centered” recommendations. She saw a distinct similarity in the discourse used by schools, the media (magazines and television programs), the institutional ethnography A research technique in which the personal experiences of individuals are used to state, and child development professionals. reveal power relationships and other characteristics of the Teachers and child development professionals institutions within which they operate. saw the role of mothers in terms of a necessary SOME QUALITATIVE FIELD RESEARCH PARADIGMS 329 collaboration between mothers and schools for opportunity for them to act effectively in their own the child’s success not only in school but also in interest. The disadvantaged subjects defi ne their life. Because of unequal resources, all mothers do problems, defi ne the remedies desired, and take not participate in this discourse of “good” child de- the lead in designing the research that will help velopment the same way. Griffi th found that work- them realize their aims. ing-class mothers were perceived as weaker than This approach began in Third World research middle-class mothers in the “stimulation” effort of development, but it spread quickly to Europe and schooling. Griffi th argued that this child develop- North America (Gaventa 1991). It comes from a ment discourse, embedded in the school institu- vivid critique of classical social science research. tion, perpetuates the reproduction of class by mak- According to the PAR paradigm, traditional re- ing middle-class ideals for family-school relations search is an “elitist model” (Whyte, Greenwood, the norm for everyone. and Lazes 1991) that reduces the “subjects” of research to “objects” of research. According to many Compulsory Heterosexuality The second illus- advocates of this perspective, the distinction be- tration of institutional ethnography is taken from tween the researcher and the researched should Didi Khayatt’s (1995) study of the institutionaliza- disappear. They argue that the subjects who will tion of compulsory heterosexuality in schools and be affected by research should also be responsible its effects on lesbian students. In 1990 Khayatt be- for its design. gan her research by interviewing 12 Toronto les- Implicit in this approach is the belief that re- bians, 15 to 24 years of age. Beginning with the search functions not only as a means of knowl- young women’s viewpoint, she then expanded her edge production but also as a “tool for the educa- inquiry to other students, teachers, guidance coun- tion and development of consciousness as well as selors, and administrators. mobilization for action” (Gaventa 1991:121–22). Khayatt found that the school’s administrative Advocates of participatory action research equate practices generated a compulsory heterosexuality, access to information with power and argue that which produced a sense of marginality and vulner- this power has been kept in the hands of the domi- ability among lesbian students. For example, the nant class, sex, ethnicity, or nation. Once people school didn’t punish harassment and name-calling see themselves as researchers, they automatically against gay students. The issue of homosexuality regain power over knowledge. was excluded from the curriculum lest it appear to Participatory action research poses a special students as an alternative to heterosexuality. challenge to researchers. On the one hand, a cen- In both of the studies I’ve described, the inquiry tral intention is to empower participants to frame began with the women’s standpoint—mothers and research relevant to their needs, as they defi ne lesbian students. However, instead of emphasizing those needs. At the same time, the researcher the subjects’ viewpoints, both analyses focused on brings special skills and insights that nonresearch- the power relations that shaped these women’s ers lack. So, who should be in charge? Andrew experiences and reality. J. Sense (2006:1) suggests that this decision may have to be made in the moment, varying by par- Participatory Action Research ticular circumstances: “Do I take the ‘passenger’ position on the bus or do I take the ‘driver’ seat Our fi nal fi eld research paradigm takes us further and be a little more provocative to energise the along in our earlier discussion of the status and power relationships linking researchers to the sub- participatory action research An approach to so- jects of their research. Within the participatory cial research in which the people being studied are given action research paradigm (PAR), the researcher’s control over the purpose and procedures of the research; function is to serve as a resource to those being intended as a counter to the implicit view that researchers studied—typically, disadvantaged groups—as an are superior to those they study. 330 CHAPTER 10 QUALITATIVE FIELD RESEARCH session. My view at this moment is to judge it on tition. Once again, management was happy about the day.” the money saved by the teams of workers. Examples of the PAR approach include commu- The Xerox case study is interesting because nity power structure research, corporate research, it shows how the production of knowledge does and “right-to-know” movements (Whyte, Green- not always have to be an elitist enterprise. The wood, and Lazes 1991). Here are three more- “experts” do not necessarily have to be the profes- detailed examples of research that used a PAR sionals. According to Whyte and his colleagues, approach. “at Xerox, participatory action research created and guided a powerful process of organizational The Xerox Corporation A participatory action learning—a process whereby leaders of labor and research project took place at the Xerox corpora- management learned from each other and from tion at the instigation of leaders of both manage- the consultant/facilitator, while he learned from ment and the union. Management’s goal was to them” — (1991:30). lower costs so that the company could thrive in an increasingly competitive market. The union sug- PAR and Welfare Policy Participatory action re- gested a somewhat broader scope: improving the search often involves poor people, as they are typi- quality of working life while lowering manufactur- cally less able than others to infl uence the policies ing costs and increasing productivity. and actions that affect their lives. Bernita Quoss, Company managers began by focusing atten- Margaret Cooney, and Terri Longhurst (2000) re- tion on shop-level problems; they were less con- port a research project involving welfare policy in cerned with labor contracts or problematic mana- Wyoming. University students, many of them wel- gerial policies. At the time, management had a plan fare recipients, undertook research and lobbying to start an “outsourcing” program that would lay efforts aimed at getting Wyoming to accept post- off 180 workers, and the union had begun mobiliz- secondary education as “work” under the state’s ing to oppose the plan. Peter Lazes, a consultant new welfare regulations. hired by Xerox, spent the fi rst month convincing This project began against the backdrop of the management and the union to create a “cost study 1996 Personal Responsibility and Work Opportu- team” (CST) that included workers in the wire har- nity Act (PRWORA), which ness department. eliminated education waivers that had been Eight full-time workers were assigned to the available under the previous welfare law, the CST for six months. Their task was to study the 1988 Family Support Act (FSA). These waivers possibilities of making changes that would save had permitted eligible participants in the cash the company $3.2 million and keep the 180 jobs. assistance AFDC program to attend college as The team had access to all fi nancial information an alternative to work training requirements. and was authorized to call on anyone within the Empirical studies of welfare participants who company. This strategy allowed workers to make received these waivers have provided evidence suggestions outside the realm usually available to that education, in general, is the most effective them. According to Whyte and his colleagues, “re- way to stay out of poverty and achieve self- shaping the box enabled the CST to call upon man- suffi ciency. — (QUOSS, COONEY, AND LONGHURST agement to explain and justify all staff services” 2000:47) (1991:27). Because of the changes suggested by the CST and implemented by management, the The students began by establishing an organi- company saved the targeted $3.2 million. zation, Empower, and making presentations on Management was so pleased by this result that campus to enlist broad student and faculty support. it expanded the wire harness CST project to three They compiled existing research relevant to the is- other departments that were threatened by compe- sue and established relationships with members of SOME QUALITATIVE FIELD RESEARCH PARADIGMS 331 the state legislature. By the time the 1997 legisla- in 1924 in a town of around a thousand inhabit- tive session opened, they were actively engaged in ants, the hotel fell victim to population growth the process of modifying state welfare laws to take and urban development. Lofl and says his role as account of the shift in federal policy. researcher began on September 18, 2000, as the The students prepared and distributed fact demolition of the building began. sheets and other research reports that would be relevant to the legislators’ deliberations. They at- Before that, I was only and simply an involved tended committee meetings and lobbied legisla- citizen. Along with many other people, I was tors on a one-to-one basis. When erroneous or attempting to preserve the Terminal Building in misleading data were introduced into the discus- some manner. This also explains why there are sions, the student-researchers were on hand to so few photographs in this book taken by me point out the errors and offer corrections. before that date, but many after that date. I had Ultimately, they were successful. Welfare re- then begun seriously to document what was go- cipients in Wyoming were allowed to pursue post- ing on with a camera and fi eld notes. secondary education as an effective route out of Therefore, questions of “informed consent” poverty. (now so often raised regarding research) were Some researchers speak of emancipatory not pertinent before September 18. After that research, which Ardha Danieli and Carol Wood- day, it was my practice to indicate to everyone hams (2005:284) defi ne as “fi rst and foremost a I encountered that I was “writing a book” about process of producing knowledge which will be of the building. — (LOFLAND 2003:20) benefi t to oppressed people; a political outcome.” Both qualitative and quantitative methods can be Recall the discussion of informed consent in used to pursue this goal, but it goes well beyond Chapter 3, a method of protecting research sub- simply learning what’s so, even as seen from the jects. In this case, as Lofl and notes elsewhere, ex- subjects’ point of view. The authors focus on the plicit consent was not necessarily needed here be- study of disability, and they note similarities in the cause the behavior being studied was public. Still development of emancipatory research and early his instincts as a social researcher were to ensure feminist research. See the box “Pencils and Pho- that he treat subjects appropriately. tos in the Hands of Research Subjects” for more One of Lofl and’s purposes was to study this on this topic. failed attempt to secure “historic preservation” status for a building, thus providing useful infor- Blocking a Demolition In another example of mation to activists in the future. This indicates that researchers being directly involved in what they there can be many different forms of participatory study, John Lofl and (2003) detailed the demolition action research. of a historic building in Davis, California, and com- At the same time, this is a valuable case for a munity attempts to block the demolition. One thing study of research methods, because Lofl and, as the that makes his book especially unusual is its reli- author of research methods textbooks, is particu- ance on photographs and facsimile news articles larly sensitive to the methodological aspects of the and government documents as raw data for the study. analysis (and for the reader): what Lofl and refers The depth and intensity of my involvement to as “documentary sociology.” is a two-edged sword. On the one edge, my As Lofl and explains, he was involved in the is- involvement provided me with a view closer sue fi rst as an active participant, joining with other community members in the attempt to block demolition of the Hotel Aggie (also known as the emancipatory research Research conducted for the “Terminal Building” and “Terminal Hotel”). Built purpose of benefi ting disadvantaged groups. 332 CHAPTER 10 QUALITATIVE FIELD RESEARCH

ISSUES AND INSIGHTS PENCILS AND PHOTOS IN THE HANDS OF RESEARCH SUBJECTS

How would you go about studying the life condi- shine business. Their clients were mainly in tions of Peruvian Indians living in the Amazon the city, not in the barrio where they lived. rainforest? With a minimal telecommunications As their shoe-shine boxes were too heavy infrastructure and a slow ferry-based postal for them to carry, these boys rented a nail on service in the vast region, a mail or telephone a wall (usually in a shop), where they could survey wouldn’t be the best approach. It might hang their boxes for the night. To them, that occur to you to conduct in-depth interviews in nail on the wall represented “exploitation.” which you would work from an outline of top- The “nail on the wall” photograph spurred ics to be covered. Arvind Singhal and Elizabeth widespread discussions in the Peruvian Rattine-Flaherty (2006) opted for a very differ- barrio about other forms of institutionalized ent approach, which put the subjects of study exploitation, including ways to overcome more in control of the research and allowed for them. — (2006:314) important but unexpected discoveries. They de- Singhal and Rattine-Flaherty’s research in- rived their inspiration from the work of the revolved gauging the quality of life in the Peruvian nowned Brazilian educator, Paulo Freire, who Amazon and assessing the impact of programs once set out to measure exploitation among launched by a Peruvian nongovernmental or- street children. Instead of interviewing them, ganization (NGO), Minga Peru. To view society he gave them cameras and asked them to bring through the eyes of children, the researchers set back photographs of exploitation. As Singhal up drawing sessions with colored pencils. In the and Rattine-Flaherty report: spirit of reciprocity, one of the authors sketched One child took a photo of a nail on a wall. It pictures of snowmen and jack-o’-lanterns that made no sense to adults, but other children were a part of her childhood in the Midwest. In were in strong agreement. The ensuing addition to depicting life in their villages and discussions showed that many young boys their close relationship with the natural envi- of that neighborhood worked in the shoe- ronment, the children’s sketches often featured

than that of some other people. I was one type reader less dependent on me than is the case of “insider.” This means I could gather data of with some other methods of representing what certain sorts that were not available to the less happened. — (LOFLAND 2003:20) involved. On the other edge, my partisanship clearly As you can see, the seemingly simple process poses the threat of bias. I have always been of observing social action as it occurs has subtle aware of this, and I have tried my best to correct though important variations. As we saw in Chapter for it. But, in the end, I cannot be the ﬁ nal judge. 2, all our thoughts occur within, and are shaped by, Each reader will have to form her or his own as- paradigms, whether we’re conscious of it or not. sessment. I can hope, however, that the “digital Qualitative ﬁ eld researchers have been unusually documentary” evidence I mention above helps deliberate in framing a variety of paradigms to en- the study tell itself, so to speak. It makes the rich the observation of social life. CONDUCTING QUALITATIVE FIELD RESEARCH 333

examples of social change being brought about with his arms and feet spread in an open by development programs of the NGO. position. Especially noticeable is the physical distance of about 10 feet that separates These include sketches of chicken coops, the woman and the man. When Adela was ﬁ sh farms, and agro-forestry projects. These asked why she took the picture and why enterprises, all launched by Minga Peru, were the man and woman sitting so far began in the Peruvian Amazon only in apart, she noted: “The woman is sitting at the past few years. For children to sketch one side of the house and he is on the other these “new” initiatives in their pictures on and this was not anything unusual.” Upon their own, without prompts, is noteworthy. probing, we learned that Amazonian men —(2006:322) determine how close the couple sits. If they The photographs taken by the adult women are sitting closer, and if the man has his arm were equally revealing. Several drew attention around his partner, it is his decision to do to the patriarchal social structure. As the au- so. This authority also applies to initiation thors report, of sex: The man determines if and when sex will happen. — (2006:323–24) Several photographs depicted the subservient position of the Amazonian women This research not only illustrates some unusual relative to men, a situation that Minga Peru data collection techniques, but it also represents seeks to address. For instance, Adela’s the spirit of participatory action research, dis- picture shows a middle-aged Amazonian cussed in this chapter. woman and her husband sitting on their porch and having a conversation. The Source: Arvind Singhal and Elizabeth Rattine- Flaherty, “Pencils and Photos as Tools of Communi- woman, sporting a forlorn expression, sits cative Research and Praxis: Analyzing Minga Peru’s with her legs crossed while her husband Quest for Social Justice in the Amazon,” International stares directly into the camera, squatting Communication Gazette, 2006, 68(4): 313-330.

CONDUCTING QUALITATIVE research, beginning with how researchers prepare FIELD RESEARCH for work in the ﬁ eld.

Preparing for the Field So far in this chapter we’ve considered the kinds of topics appropriate to qualitative fi eld research, spe- Suppose for the moment that you’ve decided to cial considerations in doing this kind of research, undertake fi eld research on a campus political or- and a sampling of paradigms that direct different ganization. Let’s assume further that you’re not types of research efforts. Along the way we’ve a member of that group, that you do not know a seen some examples that illustrate fi eld research great deal about it, and that you will identify your- in action. To round out the picture, we turn now to self to the participants as a researcher. To cover specifi c ideas and techniques for conducting fi eld more of the activities common to research, we’ll 334 CHAPTER 10 QUALITATIVE FIELD RESEARCH also assume that you have decided not to take a do it will depend, in part, on the role you intend to grounded theory or similar approach. This section play. Especially if you decide to take on the role of will use this example and others to discuss some complete participant, you must fi nd a way to de- of the ways you might prepare yourself before un- velop an identity with the people to be studied. If dertaking direct observations. you wish to study dishwashers in a restaurant, the As is true of most research methods (grounded most direct method would be to get a job as a dish- theory being the most obvious exception), you washer. In the case of the student political group, would be well advised to begin with a search of you might simply join the group. the relevant literature, fi lling in your knowledge Many of the social processes appropriate to of the subject and learning what others have said fi eld research are open enough to make your con- about it (library research is discussed at length in tact with the people to be studied rather simple Appendix A). and straightforward. If you wish to observe a mass In the next phase of your research, you might demonstration, just be there. If you wish to ob- wish to discuss the student political group with serve patterns in jaywalking, hang around busy others who have already studied it or with any- streets. one else likely to be familiar with it. In particular, Whenever you wish to make more formal con- you might fi nd it useful to discuss the group with tact with the people, identifying yourself as a re- one or more informants (discussed in Chapter searcher, you must establish a rapport with them. 7). Perhaps you have a friend who is a member, You might contact a participant with whom you or you can meet someone who is. This aspect of feel comfortable and gain that person’s assistance. your preparation is likely to be more effective if In studying a formal group, you might approach your relationship with the informant extends be- the groups’ leaders, or you may fi nd that one of yond your research role. In dealing with members your informants can introduce you. of the group as informants, you should take care Whereas you’ll probably have many options in that your initial discussions do not compromise or making your initial contact with the group, real- limit later aspects of your research. Keep in mind ize that your choice can infl uence your subsequent that the impression you make on the informant, observations. Suppose, for example, that you’re the role you establish for yourself, may carry over studying a university and begin with high-level ad- into your later effort. For example, creating the ini- ministrators. This choice is likely to have a couple tial impression that you may be an undercover FBI of important consequences. First, your initial im- agent is unlikely to facilitate later observations of pressions of the university will be shaped to some the group. extent by the administrators’ views, which will dif- You should also be wary about the information fer signifi cantly from those of students or faculty. you get from informants. Although they may have This initial impression may infl uence the way you more direct, personal knowledge of the subject un- observe and interpret events subsequently—espe- der study than you do, what they “know” is prob- cially if you’re unaware of the infl uence. ably a mixture of fact and point of view. Members Second, if the administrators approve of your of the political group in our example would be un- research project and encourage students and fac- likely to provide completely unbiased information ulty to cooperate with you, the latter groups will (so would members of opposing political groups). probably look on you as somehow aligned with the Before making your fi rst contact with the student administration, which can affect what they say to group, then, you should already be quite familiar you. For example, faculty members might be reluc- with it, and you should understand its general phil- tant to tell you about plans to organize through the osophical context. teamsters’ union. There are many ways to establish your initial In making a direct, formal contact with the peo- contact with the people you plan to study. How you ple you want to study, you’ll be required to give CONDUCTING QUALITATIVE FIELD RESEARCH 335 them some explanation of the purpose of your port structure of family members already in the study. Here again, you face an ethical dilemma. United States, as you might imagine, but Menjívar Telling them the complete purpose of your re- found that her interviewees were often reluctant search might eliminate their cooperation alto- to call on relatives for help. On the one hand, they gether or signifi cantly affect their behavior. On the might jeopardize those family members who were other hand, giving only what you believe would be here illegally and living in poverty. At the same an acceptable explanation might involve outright time, asking for help would put the immigrants deception. Your decisions in this and other matters in debt to those helping them out. Menjívar also will probably be largely determined by the purpose discovered that Salvadoran gender norms put of your study, the nature of what you’re studying, women immigrants in an especially diffi cult situ- the observations you wish to use, and similar fac- ation, because they were largely prohibited from tors, but ethical considerations must be taken into seeking the help of men they weren’t related to, account as well. lest they seem to obligate themselves sexually. Previous fi eld research offers no fi xed rule— These are the kinds of discoveries that can emerge methodological or ethical—to follow in this regard. from open-ended, in-depth interviewing. Your appearance as a researcher, regardless of We’ve already discussed interviewing in Chap- your stated purpose, may result in a warm wel- ter 9, and much of what was said there applies come from people who are fl attered that a scientist to qualitative fi eld interviewing. The interview- fi nds them important enough to study. Or, it may ing you’ll do in connection with fi eld observation, result in your being totally ostracized or worse. It however, differs enough to demand a separate probably wouldn’t be a good idea, for example, to treatment. In surveys, questionnaires are rigidly burst into a meeting of an organized crime syndi- structured; however, less structured interviews are cate and announce that you’re writing a term pa- more appropriate to fi eld research. As Herbert and per on organized crime. Riene Rubin describe the distinction, “Qualitative interviewing design is fl exible, iterative, and contin- Qualitative Interviewing uous, rather than prepared in advance and locked in stone” (1995:43). They elaborate in this way: In part, fi eld research is a matter of going where Design in qualitative interviewing is iterative. the action is and simply watching and listening. That means that each time you repeat the basic As the baseball legend Yogi Berra said, “You can process of gathering information, analyzing it, see a lot just by observing”—provided that you’re winnowing it, and testing it, you come closer to paying attention. At the same time, as I’ve already a clear and convincing model of the phenom- indicated, fi eld research can involve more active enon you are studying. . . . inquiry. Sometimes it’s appropriate to ask people The continuous nature of qualitative inter- questions and record their answers. Your on-the- viewing means that the questioning is rede- spot observations of a full-blown riot will lack signed throughout the project. — (1995:46, 47) something if you don’t know why people are rioting. Ask somebody. Unlike a survey, a qualitative interview is an When Cecilia Menjívar (2000) wanted to learn interaction between an interviewer and a respon- about the experiences of Salvadoran immigrants in dent in which the interviewer has a general plan of San Francisco, she felt in-depth interviews would inquiry including the topics to be covered, but not be a useful technique, along with personal observations. Before she was done, she had discovered qualitative interview Contrasted with survey a much more complex system of social processes interviewing, the qualitative interview is based on a set and structures than one would have imagined. It of topics to be discussed in depth rather than the use of was important for new immigrants to have a sup- standardized questions. 336 CHAPTER 10 QUALITATIVE FIELD RESEARCH a set of questions that must be asked with particu- collect a great deal of information about students’ lar words and in a particular order. At the same attitudes toward the infamous ruling, they may time, the qualitative interviewer, like the survey in- be rioting for some other reason. Perhaps most terviewer, must be fully familiar with the questions are simply joining in for the excitement. Properly to be asked. This allows the interview to proceed done, fi eld research interviewing enables you to smoothly and naturally. fi nd out. A qualitative interview is essentially a conver- Although you may set out with a reasonably sation in which the interviewer establishes a gen- clear idea of what you want to ask in your inter- eral direction for the conversation and pursues views, one of the special strengths of fi eld research specifi c topics raised by the respondent. Ideally, the is its fl exibility. In particular, the answers evoked respondent does most of the talking. If you’re talk- by your initial questions should shape your sub- ing more than 5 percent of the time, that’s probably sequent ones. It doesn’t work merely to ask pre- too much. established questions and record the answers. Steinar Kvale (1996:3–5) offers two metaphors Instead, you need to ask a question, listen care- for interviewing: the interviewer as a “miner” or fully to the answer, interpret its meaning for your as a “traveler.” The fi rst model assumes that the general inquiry, and then frame another question subject possesses specifi c information and that either to dig into the earlier answer or to redirect the interviewer’s job is to dig it out. By contrast, in the person’s attention to an area more relevant to the second model, the interviewer your inquiry. In short, you need to be able to listen, think, and talk almost at the same time. wanders through the landscape and enters into The discussion of probes in Chapter 9 provides conversations with the people encountered. The a useful guide to getting answers in more depth traveler explores the many domains of the coun- without biasing later answers. More generally, fi eld try, as unknown territory or with maps, roaming interviewers need to be good listeners. This means freely around the territory. . . . The interviewer being more interested than interesting; learning to wanders along with the local inhabitants, asks say things like “How is that?” “In what ways?” “How questions that lead the subjects to tell their own do you mean that?” “What would be an example of stories of their lived world. that?” and learning to look and listen expectantly Asking questions and noting answers is a natu- and letting the person you’re interviewing fi ll in the ral human process, and it seems simple enough to silence. add it to your bag of tricks as a fi eld researcher. Be a At the same time, you can’t afford to be a totally little cautious, however. Recall that wording ques- passive receiver. You’ll go into your interviews tions is a tricky business. All too often, the way we with some general (or specifi c) questions you want ask questions subtly biases the answers we get. answered and some topics you want addressed. At Sometimes we put our respondent under pressure times you’ll need the skill of subtly directing the to look good. Sometimes we put the question in a fl ow of conversation. particular context that omits altogether the most The martial arts offer a useful analogy in this relevant answers. regard. The aikido master never resists an oppo- Suppose, for example, that you want to fi nd nent’s blow but instead accepts it, joins with it, and out why a group of students is rioting and pillag- then subtly redirects it in a more appropriate direc- ing on campus. You might be tempted to focus tion. So, instead of trying to halt your respondent’s your questioning on how students feel about the line of discussion, learn to take what he or she has dean’s recent ruling that requires students always just said and branch that comment back in the di- to carry The Basics of Social Research with them on rection appropriate to your purposes. Most people campus. (Makes sense to me.) Although you may love to talk to anyone who’s really interested. Stop- CONDUCTING QUALITATIVE FIELD RESEARCH 337 ping their line of conversation tells them that you velop the ability to “control” conversations in this are not interested; asking them to elaborate in a fashion. particular direction tells them that you are. Herbert and Riene Rubin offer several ways Consider this hypothetical example in which to control a “guided conversation,” such as the you’re interested in why college students chose following: their majors. If you can limit the number of main topics, YOU: What are you majoring in? it is easier to maintain a conversational fl ow RESP: Engineering. from one topic to another. Transitions should YOU: I see. How did you come to choose be smooth and logical. “We have been talking engineering? about mothers, now let’s talk about fathers,” RESP: I have an uncle who was voted the best sounds abrupt. A smoother transition might engineer in Arizona in 1981. be, “You mentioned your mother did not care YOU: Gee, that’s great. how you performed in school—was your father RESP: Yeah. He was the engineer in charge more involved?” The more abrupt the transi- of developing the new civic center in tion, the more it sounds like the interviewer has Tucson. It was written up in most of the an agenda that he or she wants to get through, engineering journals. rather than wanting to hear what the inter- YOU: I see. Did you talk to him about your viewee has to say.— (1995:123) becoming an engineer? Because fi eld research interviewing is so much RESP: Yeah. He said that he got into engineering like normal conversation, researchers must keep by accident. He needed a job when he reminding themselves that they are not having a graduated from high school, so he went normal conversation. In normal conversations, to work as a laborer on a construction each of us wants to come across as an interest- job. He spent eight years working his ing, worthwhile person. If you watch yourself the way up from the bottom, until he decided next time you chat with someone you don’t know to go to college and come back nearer too well, you’ll probably fi nd that much of your at- the top. tention is spent on thinking up interesting things YOU: So is your main interest civil engineering, to say—contributions to the conversation that will like your uncle, or are you more make a good impression. Often, we don’t really interested in some other branch of hear each other, because we’re too busy thinking engineering? of what we’ll say next. As an interviewer, the de- RESP: Actually, I’m leaning more toward sire to appear interesting is counterproductive. The electrical engineering—computers, in interviewer needs to make the other person seem particular. I started messing around with interesting and can do so by being interested—and microcomputers when I was in high listening more than talking. (Do this in ordinary school, and my long-term plan is . . . conversations, and people will actually regard you Notice how the interview fi rst begins to wan- as a great conversationalist.) der off into a story about the respondent’s uncle. Lofl and and colleagues (2006:69–70) suggest The fi rst attempt to focus things back on the stu- that investigators adopt the role of the “socially dent’s own choice of major (“Did you talk to your acceptable incompetent” when interviewing. That uncle . . . ?”) fails. The second attempt (“So is your is, you offer yourself as someone who does not main interest . . . ?”) succeeds. Now the student understand the situation you fi nd yourself in and is providing the kind of information you’re look- must be helped to grasp even the most basic and ing for. It’s important for fi eld researchers to de- obvious aspects of that situation: “A naturalistic 338 CHAPTER 10 QUALITATIVE FIELD RESEARCH investigator, almost by defi nition, is one who does Focus Groups not understand. She or he is ‘ignorant’ and needs Although our discussions of fi eld research so far to be ‘taught.’ This role of watcher and asker of have focused on studying people in the process of questions is the quintessential student role” (Lo- living their lives, researchers sometimes bring peo- fl and et al. 2006:69). ple into the laboratory for qualitative interviewing Interviewing needs to be an integral part of the and observation. The focus group method, which entire fi eld research process. Later, I’ll stress the is also called group interviewing, is essentially a need to review your observational notes every qualitative method. It’s based on structured, semi- night—making sense out of what you’ve observed, structured, or unstructured interviews. It allows getting a clearer feel for the situation you’re study- the researcher/interviewer to question several in- ing, and fi nding out what you should pay more dividuals systematically and simultaneously. Focus attention to in further observations. In the same group techniques are typically used in market re- fashion, you’ll need to review your notes on in- search, but not exclusively. terviews, recording especially effective questions Imagine that you’re thinking about introducing and detecting all those questions you should have a new product. Let’s suppose that you’ve invented asked but didn’t. Start asking such questions the a new computer that not only does word process- next time you interview. If you have recorded the ing, spreadsheets, data analysis, and the like, but interviews, replay them as a useful preparation for also contains a fax machine, AM/FM/TV tuner, future interviews. CD player, dual-cassette unit, microwave oven, Steinar Kvale (1996:88) details seven stages in denture cleaner, and coffeemaker. To highlight the complete interviewing process: its computing and coffee-making features, you’re 1. Thematizing: clarifying the purpose of the inter- thinking of calling it the “Compulator.” You fi gure views and the concepts to be explored the new computer will sell for about $28,000, and 2. Designing: laying out the process through you want to know whether people are likely to buy which you’ll accomplish your purpose, includ- it. Your prospects might be well served by focus ing a consideration of the ethical dimension groups. 3. Interviewing: doing the actual interviews In a focus group, typically 12 to 15 people are 4. Transcribing: creating a written text of the brought together in a room to engage in a guided interviews discussion of some topic—in this case, the ac- 5. Analyzing: determining the meaning of gath- ceptability and salability of the Compulator. The ered materials in relation to the purpose of the subjects are selected on the basis of relevance to study the topic under study. Given the likely cost of the 6. Verifying: checking the reliability and validity of Compulator, your focus group participants would the materials probably be limited to upper-income groups, for 7. Reporting: telling others what you’ve learned example. Other, similar considerations might fi g- ure into the selection. As with all other aspects of fi eld research, in- Participants in focus groups are not likely to terviewing improves with practice. Fortunately, it’s be chosen through rigorous, probability sampling something you can practice any time you want. methods. This means that the participants do not Practice on your friends. statistically represent any meaningful population. However, the purpose of the study is to explore focus group A group of subjects interviewed together, rather than to describe or explain in any defi ni- prompting a discussion. The technique is frequently used tive sense. Nevertheless, typically more than one by market researchers, who ask a group of consumers to evaluate a product or discuss a type of commodity, for focus group is convened in a given study because example. of the serious danger that a single group of 7 to 12 CONDUCTING QUALITATIVE FIELD RESEARCH 339 people will be too atypical to offer any generaliz- In a focus group interview, more than in any other able insights. type of interview, the interviewer has to be a skilled William Gamson (1992) used focus groups moderator. Controlling the dynamic within the to examine how U.S. citizens frame their views group is a major challenge. Letting one interviewee of political issues. Having picked four issues— dominate the focus group interview reduces the affi rmative action, nuclear power, troubled in- likelihood that the other subjects will participate. dustries, and the Arab-Israeli confl ict—Gamson This can generate the problem of group confor- undertook a content analysis of press coverage mity or what Janis called “groupthink,” which is to get an idea of the media context within which the tendency for people in a group to conform with we think and talk about politics. Then the focus the opinions and decisions of the most outspoken groups were convened for a fi rsthand observation members of the group. Interviewers need to be of the process of people discussing issues with their aware of this phenomenon and try to get everyone friends. to participate fully on all the issues brought up in Richard Krueger points to fi ve advantages of fo- the interview. In addition, interviewers must resist cus groups: bringing their own views into play by overdirecting the interview and the interviewees. 1. The technique is a socially oriented research Although focus group research differs from method capturing real-life data in a social other forms of qualitative fi eld research, it further environment. illustrates the possibilities for doing social research 2. It has fl exibility. face-to-face with those we wish to understand. In 3. It has high face validity. addition, David Morgan (1993) suggests that fo- 4. It has speedy results. cus groups are an excellent device for generating 5. It is low in cost. — (1988:47) questionnaire items for a subsequent survey. In their typical form—centered on a particular topic and taking a limited amount of time—focus In addition to these advantages, group dynam- groups would not be regarded as an in-depth re- ics frequently bring out aspects of the topic that search technique. However, Carolina Överlien, would not have been anticipated by the researcher Karin Aronsson, and Margareta Hydén (2005) have and would not have emerged from interviews with used the technique successfully for extended dis- individuals. In a side conversation, for example, a cussions of sexuality among Swedish teenagers in couple of the participants might start joking about a youth detention home. the results of leaving out one letter from a prod- Like other social research techniques, focus uct’s name. This realization might save the manu- groups are adapting to new communication mo- facturer great embarrassment later on. dalities. George Silverman (2005), for example, Krueger also notes some disadvantages of the offers a discussion of telephone and online focus focus group method, however: groups.

1. Focus groups afford the researcher less control than individual interviews. To see more on market research, check 2. Data are difﬁ cult to analyze. out the Survey/Marketing eStore listing* 3. Moderators require special skills. at http://www.streamlinesurveys.com/ 4. Difference between groups can be Streamline/estore/focus.htm. troublesome. 5. Groups are difﬁ cult to assemble.

6. The discussion must be conducted in a condu- *Each time the Internet icon appears, you’ll be given help- cive environment. — (1988:44–45) ful leads for searching the World Wide Web. 340 CHAPTER 10 QUALITATIVE FIELD RESEARCH

ISSUES AND INSIGHTS INTERVIEW TRANSCRIPT ANNOTATED WITH RESEARCHER MEMOS

Thursday August 26, 12:00–1:00 script is lousy, and you have to make a silk purse out of sow’s hair. You know, you R: What is challenging for women directors on have a lot of extra strikes against you than a daily experience, on a daily life? the average guy who has similar problems, J: Surviving. because you are a woman and they look R: OK. Could you develop a little bit on at it, and women are more visible than that? [I need to work on my interview men . . . in unique positions. [It seems that schedule so that my interviewee answers Joy is talking about the particularities of with more elaboration without having to the fi lm industry. There are not that many probe.] opportunities and in order to keep working, J: Yeah, I mean it’s all about trying to get, she needs to build a certain reputation. It you know, in, trying to get the job, and try, is only by continuing to direct that she can you know, to do a great job so that you maintain or improve her reputation. She are invited back to the next thing. And thinks that it is even harder for women but particularly since they are so many, you does not explain it.] know, diffi culties in women directing. It R: Hum . . . what about on the set did you makes it twice as hard to gain into this experience, did it feel . . . did people make position where you do an incredible job, it clear that you were a woman, and you because . . . you can’t just do an average felt treated differently? [I am trying to get job, you have to [347] do this job that just her to speak about more specifi c and more knocks your socks off all the time, and personal experiences without leading her sometimes you don’t get the opportunity answer] to do that, because either you don’t have J: Yeah, oh yeah, I mean . . . a lot of women a good producer or you have so many have commiserated about, you know when pressures that you can’t see straight or your

Recording Observations to identify these different kinds of notes for what they are. For example, you might note that Person The greatest advantage of the ﬁ eld research X spoke out in opposition to a proposal made by a method is the presence of an observing, thinking group leader (an observation), that you think this researcher at the scene of the action. Not even represents an attempt by Person X to take over tape recorders and cameras can capture all the leadership of the group (an interpretation), and relevant aspects of social processes. Consequently, that you think you heard the leader comment to in both direct observation and interviewing, mak- that effect in response to the opposition (a tenta- ing full and accurate notes of what goes on is vital. tive observation). If possible, take notes on your observations while Of course, you cannot hope to observe every- you observe. When that’s not feasible, write down thing; nor can you record everything you do ob- your notes as soon as possible afterward. serve. Just as your observations will represent a In your notes, include both your empirical ob- sample of all possible observations, your notes will servations and your interpretations of them. In represent a sample of your observations. The idea, other words, record what you “know” has hap- of course, is to record the most pertinent ones. pened and what you “think” has happened. Be sure CONDUCTING QUALITATIVE FIELD RESEARCH 341

you have to walk on the set for the fi rst time, dividual. She is in a double bind situation where they’re all used to working like a well-oiled whether she complains or not, she will not be machine and they say, “Oh, here is the woman, treated equal to men. Time seems to be one something different” and sometimes they can quantifi able measurement of how well she does be horrible, they can resist your directing and her job and, as observed in other professions, they can, they can sabotage you, by taking a the fact that she is a woman is perceived as a long time to light, or to move sets, or to do handicap. Review literature on women in high something . . . and during that time you’re management position. I need to keep asking wasting time, and that goes on a report, and about the dynamics between my interviewees the report goes to the front [368] offi ce, and, and the crewmembers on the set. The cinema- you know, and so on and so on and so on and tographer has the highest status on the set un- so forth. And people upstairs don’t know what der the director. Explore other interviews about the circumstances are, and they are not about reasons for confl ict between them.] to fi re a cinematographer that is on their show [Methods (note to myself for the next inter- for ever and ever . . . nor do they want to know views): try to avoid phone interviews unless spe- that this guy is a real bastard, and making your cifi c request from the interviewee. It is diffi cult to life a horror. They don’t want to know that, so assess how the interviewee feels with the ques- therefore, they go off, because she’s a woman tions. Need body language because I become let’s not hire any more women, since he has more nervous about the interview process.] problems with women. You know, so, there is that aspect. Note: A number in brackets represents a word that was inaudible from the interview. It is the number [I need to review the literature on institutional that appeared on the transcribing machine, with each interview starting at count 0. The numbers help discrimination. It seems that the challenges that the researcher locate a passage quickly when he or Joy is facing are not a matter of a particular in- she reviews the interview.

The accompanying box, “Interview Transcript An- might develop a symbolic shorthand in advance notated with Researcher Memos,” provides an ex- to speed up recording. For studying audience par- ample given by Sandrine Zerbib from an in-depth ticipation at a mass meeting, you might want to interview with a woman ﬁ lm director. construct a numbered grid representing the differ- Some of the most important observations can ent sections of the meeting room; then you could be anticipated before you begin the study; others record the location of participants easily, quickly, will become apparent as your observations prog- and accurately. ress. Sometimes you can make note taking easier None of this advance preparation should limit by preparing standardized recording forms in ad- your recording of unanticipated events and aspects vance. In a study of jaywalking, for example, you of the situation. Quite the contrary, the speedy might anticipate the characteristics of pedestrians handling of anticipated observations can give you that are most likely to be useful for analysis—age, more freedom to observe the unanticipated. gender, social class, ethnicity, and so forth—and You’re already familiar with the process of tak- prepare a form in which observations of these ing notes, just as you already have at least informal variables can be recorded easily. Alternatively, you experience with ﬁ eld research in general. Like good 342 CHAPTER 10 QUALITATIVE FIELD RESEARCH

fi eld research, however, good note taking requires servation session? The general guideline is yes. careful and deliberate attention and involves spe- Generally, in fi eld research you can’t be really sure cifi c skills. Some guidelines follow. (You can learn of what’s important and what’s unimportant until more from Lofl and et al. 2006:110–17.) you’ve had a chance to review and analyze a great First, don’t trust your memory any more than volume of information, so you should record even you have to; it’s untrustworthy. To illustrate this things that don’t seem important at the outset. point, try this experiment. Recall the last three or They may turn out to be signifi cant after all. Also, four movies you saw that you really liked. Now, the act of recording the details of something “un- name fi ve of the actors or actresses. Who had the important” may jog your memory about something longest hair? Who was the most likely to start con- that is important. versations? Who was the most likely to make sug- Realize that your fi nal report on the project gestions that others followed? Now, if you didn’t will not refl ect most of your fi eld notes. Put more have any trouble answering any of those questions, harshly, most of your notes will be “wasted.” But how sure are you of your answers? Would you be take heart: Even the richest gold ore yields only willing to bet a hundred dollars that a panel of im- about 30 grams of gold per metric ton, meaning partial judges would observe what you recall? that 99.997 percent of the ore is wasted. Yet, that Even if you pride yourself on having a photo- 30 grams of gold can be hammered out to cover an graphic memory, it’s a good idea to take notes ei- area 18 feet square—the equivalent of about 685 ther during the observation or as soon afterward book pages. So take a ton of notes, and plan to as possible. If you take notes during observation, select and use only the gold. do it unobtrusively, because people will tend to be- Like other aspects of fi eld research (and all re- have differently if they see you taking down every- search for that matter), profi ciency comes with thing they say or do. practice. The nice thing about fi eld research is that Second, it’s usually a good idea to take notes you can begin practicing now and can continue in stages. In the fi rst stage, you may need to take practicing in almost any situation. You don’t have sketchy notes (words and phrases) in order to to be engaged in an organized research project keep abreast of what’s happening. Then go off by to practice observation and recording. You might yourself and rewrite your notes in more detail. If start by volunteering to take the minutes at com- you do this soon after the events you’ve observed, mittee meetings, for example. Or just pick a sunny the sketchy notes should allow you to recall most day on campus, fi nd a shady spot, and try observ- of the details. The longer you delay, the less likely ing and recording some specifi c characteristics of you’ll be able to recall things accurately and fully. the people who pass by. You can do the same thing I know this method sounds logical, but it takes at a shopping mall or a busy street corner. Remem- self-discipline to put it into practice. Careful obser- ber that observing and recording are professional vation and note taking can be tiring, especially if skills, and, like all worthwhile skills, they improve it involves excitement or tension and if it extends with practice. over a long period. If you’ve just spent eight hours observing and making notes on how people have STRENGTHS AND WEAKNESSES OF been coping with a disastrous fl ood, your fi rst QUALITATIVE FIELD RESEARCH desire afterward will likely be to get some sleep, change into dry clothes, or get a drink. You may need to take some inspiration from newspaper re- Like all research methods, qualitative fi eld research porters who undergo the same sorts of hardships has distinctive strengths and weaknesses. As I’ve then write to meet their deadlines. already indicated, fi eld research is especially effec- Third, you’ll inevitably wonder how much you tive for studying subtle nuances in attitudes and should record. Is it really worth the effort to write behaviors and for examining social processes over out all the details you can recall right after the ob- time. As such, the chief strength of this method lies STRENGTHS AND WEAKNESSES OF QUALITATIVE FIELD RESEARCH 343 in the depth of understanding it permits. Whereas “Being there” is a powerful technique for gain- other research methods may be challenged as ing insights into the nature of human affairs in all “superfi cial,” fi eld research seldom receives this their rich complexity. Listen, for example, to what criticism. one nurse reports about the impediments to pa- Flexibility is another advantage of fi eld re- tients’ coping with cancer: search. As discussed earlier, you can modify your fi eld research design at any time. Moreover, you’re Common fears that may impede the coping always prepared to engage in fi eld research, when- process for the person with cancer can include ever the occasion should arise, whereas you could the following: not as easily initiate a survey or an experiment. —Fear of death—for the patient, and the im- Field research can be relatively inexpensive as plications his or her death will have for signifi - well. Other social research methods may require cant others. costly equipment or an expensive research staff, —Fear of incapacitation—because cancer but fi eld research typically can be undertaken by can be a chronic disease with acute episodes one researcher with a notebook and a pencil. This that may result in periodic stressful periods, the is not to say that fi eld research is never expensive. variability of the person’s ability to cope and A particular project may require a large number of constantly adjust may require a dependency trained observers, for instance. Expensive record- upon others for activities of daily living and may ing equipment may be needed. Or you may wish to consequently become a burden. undertake participant observation of interactions —Fear of alienation—from signifi cant others in pricey Paris nightclubs. and health care givers, thereby creating help- Field research has several weaknesses as well. lessness and hopelessness. First, being qualitative rather than quantitative, it —Fear of contagion—that cancer is transmis- is not an appropriate means for arriving at statisti- sible and/or inherited. cal descriptions of a large population. Observing —Fear of losing one’s dignity—losing control casual political discussions in Laundromats, for of all bodily functions and being totally vulner- example, would not yield trustworthy estimates of able. — (GARANT 1980:2167) the future voting behavior of the total electorate. Observations and conceptualizations such as Nevertheless, the study could provide important these are valuable in their own right. In addition, insights into how political attitudes are formed. they can provide the basis for further research— To assess fi eld research further, we should fo- both qualitative and quantitative. cus on the issues of validity and reliability. Recall Now listen to what Joseph Howell has to say that validity concerns whether measurements ac- about “toughness” as a fundamental ingredient of tually measure what they’re supposed to rather life on Clay Street, a white, working-class neigh- than something else. Reliability, on the other hand, borhood in Washington, D.C.: is a matter of dependability: If you made the same measurement again and again, would you get the Most of the people on Clay Street saw them- same result? Let’s see how fi eld research stacks up selves as fi ghters in both the fi gurative and literal in these respects. sense. They considered themselves strong, independent people who would not let themselves Validity be pushed around. For Bobbi, being a fi ghter meant battling the welfare department and Field research seems to provide measures with cussing out social workers and doctors upon greater validity than do survey and experimental occasion. It meant spiking Barry’s beer with measurements, which are often criticized as su- sleeping pills and bashing him over the head perfi cial and not really valid. Let’s review a couple with a broom. For Barry it meant telling off his of fi eld research examples to see why this is so. boss and refusing to hang the door, an act that 344 CHAPTER 10 QUALITATIVE FIELD RESEARCH

led to his being fi red. It meant going through the ritual of a duel with Al. It meant pushing Bubba WHAT DO YOU THINK? REVISITED around and at times getting rough with Bobbi. June and Sam had less to fi ght about, though The impact of the observer affects most if pressed they both hinted that they, too, would forms of social research. We’ve seen that fi ght. Being a fi ghter led Ted into near confl ict experimenters can infl uence the way peo- with Peg’s brothers, Les into confl ict with Lon- ple behave in experiments, and survey re- nie, Arlene into confl ict with Phyllis at the bowl- searchers can affect how people respond to ing alley, etc. — (1973:292) questionnaires. The problem is also present when participant observers set out to study Even without having heard the episodes Howell human behavior in its natural setting. As refers to in this passage, we get the distinct im- we’ve seen, researchers sometimes conceal pression that Clay Street is a tough place to live in. their research identity as a way of reducing That “toughness” comes through far more power- their impact, but we’ve also seen that any- fully through these fi eld observations than it would thing they do in a social setting will have in a set of statistics on the median number of fi st- some impact. fi ghts occurring during a specifi ed period. Ultimately, the solution to this problem is These examples point to the superior validity of awareness of it, because this allows you to fi eld research, as compared with surveys and ex- have some control over the impact you have. periments. The kinds of comprehensive measure- This approach is coupled with replication ments available to the fi eld researcher tap a depth by other researchers. Different researchers of meaning in concepts such as common fears of bring various impacts to different research cancer patients and “toughness” (or such as liberal situations. If they nonetheless discover the and conservative) that is generally unavailable to same patterns of behavior, our confi dence surveys and experiments. Instead of specifying that we’ve learned something about social concepts, fi eld researchers commonly give de- life, not just something about our role in it, tailed illustrations. increases.

Reliability

Field research has, however, a potential problem with reliability. Suppose you were to characterize though I might feel that I’ve been observing a fi re- your best friend’s political orientations according breathing radical. to everything you know about him or her. Your as- As I’ve suggested earlier, researchers who use sessment of your friend’s politics would appear to qualitative techniques are conscious of this issue have considerable validity; certainly it’s unlikely to and take pains to address it. Individual researchers be superfi cial. We couldn’t be sure, however, that often sort out their own biases and points of view, another observer would characterize your friend’s and the communal nature of science means that politics the same way you did, even with the same their colleagues will help them in that regard. Nev- amount of observation. ertheless, it’s prudent to be wary of purely descrip- Similarly, in-depth, fi eld research measure- tive measurements in fi eld research—your own, ments are often quite personal. How I judge your or someone else’s. If a researcher reports that the friend’s political orientation depends very much on members of a club are fairly conservative, such a my own, just as your judgment depends on your judgment is unavoidably linked to the researcher’s political orientation. Conceivably, then, you could own politics. You can be more trusting of compara- describe your friend as middle-of-the-road, al- tive evaluations: identifying who is more conserva- MAIN POINTS 345 tive than who, for example. Even if you and I had • Is it ethical to talk to people when they do not different political orientations, we would probably know you will be recording their words? agree pretty much in ranking the relative conser- • Is it ethical to get information for your own vatism of the members of a group. purposes from people you hate? As we’ve seen, fi eld research is a potentially • Is it ethical to see a severe need for help and powerful tool for social scientists, one that provides not respond to it directly? a useful balance against the strengths and weak- • Is it ethical to be in a setting or situation but nesses of experiments and surveys. The remaining not commit yourself wholeheartedly to it? chapters of Part 3 present additional modes of ob- • Is it ethical to develop a calculated stance servation available to social researchers. toward other humans, that is, to be strategic in your relations? ETHICS IN QUALITATIVE • Is it ethical to take sides or to avoid taking FIELD RESEARCH sides in a factionalized situation? • Is it ethical to “pay” people with trade-offs for access to their lives and minds? As I’ve noted repeatedly, all forms of social re- • Is it ethical to “use” people as allies or infor- search raise ethical issues. By bringing research- mants in order to gain entree to other people ers into direct and often intimate contact with their or to elusive understandings? subjects, fi eld research raises ethical concerns in a particularly dramatic way. Here are some of Planning and conducting fi eld research in a re- the issues mentioned by Lofl and and colleagues sponsible way requires attending to these and —(2006:78–79): other ethical concerns.

Main Points Special Considerations in Qualitative Field Research Introduction ❏ Among the special considerations involved in ❏ Field research involves the direct observation fi eld research are the various possible roles of of social phenomena in their natural settings. the observer and the researcher’s relations to Typically, fi eld research is qualitative rather subjects. As a fi eld researcher, you must decide than quantitative. whether to observe as an outsider or as a ❏ In fi eld research, observation, data process- participant, whether or not to identify yourself ing, and analysis are interwoven, cyclical as a researcher, and how to negotiate your processes. relationships with subjects.

Topics Appropriate to Field Research Some Qualitative Field ❏ Field research is especially appropriate to top- Research Paradigms ics and processes that are not easily quantiﬁ - ❏ Field research can be guided by any one of able, that are best studied in natural settings, several paradigms, such as naturalism, ethno- or that change over time. Among these topics methodology, grounded theory, case studies are practices, episodes, encounters, roles, rela- and the extended case method, institutional tionships, groups, organizations, settlements, ethnography, and participatory action social worlds, and lifestyles or subcultures. research. 346 CHAPTER 10 QUALITATIVE FIELD RESEARCH

Conducting Qualitative Field Research Review Questions ❏ Preparing for the fi eld involves doing background research, determining how to make 1. Think of some group or activity you partici- contact with subjects, and resolving issues of pate in or are very familiar with. In two or what your relationship to your subjects will be. three paragraphs, describe how an outsider might effectively go about studying that group ❏ Field researchers often conduct in-depth inter- or activity. What should he or she read, what views that are much less structured than those contacts should be made, and so on? conducted in survey research. Qualitative interviewing is more of a guided conversation 2. Choose any two of the paradigms discussed than a search for specifi c information. Effective in this chapter. How might your hypothetical interviewing involves active listening and the study from item 1 be conducted if you fol- ability to direct conversations unobtrusively. lowed each? Compare and contrast the way these paradigms might work in the context of ❏ To create a focus group, researchers bring sub- your study. jects together and observe their interactions as they explore a specifi c topic. 3. To explore the strengths and weaknesses of experiments, surveys, and fi eld research, ❏ Whenever possible, fi eld observations should choose a general research area (e.g., preju- be recorded as they are made; otherwise, dice, political orientation, education) and they should be recorded as soon afterward as write brief descriptions of studies in that possible. area that could be conducted using each of these three methods. In each case, why is the Strengths and Weaknesses chosen method the most appropriate for the of Qualitative Field Research study you describe? ❏ Among the advantages of fi eld research are the depth of understanding it can provide, its fl ex- 4. Return to the example you devised in re- ibility, and (usually) its lack of costs. sponse to item 1 above. What fi ve ethical issues can you imagine having to confront if ❏ Compared with surveys and experiments, fi eld you were to undertake your study? research measurements generally have more validity but less reliability. Also, fi eld research 5. Using the web, fi nd a research report using is generally not appropriate for arriving at sta- the grounded theory method. Briefl y, what are tistical descriptions of large populations. the study design and main fi ndings?

Ethics in Qualitative Field Research Online Study Resources ❏ Conducting ﬁ eld research responsibly involves confronting several ethical issues that arise from the researcher’s direct contact with subjects. Go to

Key Terms http://sociology.wadsworth.com/babbie_basics4e and click on ThomsonNow for access to this case study grounded theory powerful online study tool. You will get a per- emancipatory research institutional ethnography sonalized study plan based on your responses to ethnography naturalism a diagnostic pretest. Once you have mastered ethnomethodology participatory action research the material with the help of interactive learning extended case method qualitative interview focus group reactivity tools, you can take a posttest to conﬁ rm that you are ready to move on to the next chapter. ADDITIONAL READINGS 347

Website for report on their experiences with a variety of comput- The Basics of Social Research er programs used in the analysis of qualitative data. 4th edition Kvale, Steinar. 1996. InterViews: An Introduction to Qualitative Research Interviewing. Thousand Oaks, CA: At the book companion website (http://sociology Sage. An in-depth presentation of in-depth interview- .wadsworth.com/babbie_basics4e) you will fi nd ing. Besides presenting techniques, Kvale places interviewing in the context of postmodernism and many resources in addition to ThomsonNow to other philosophical systems. aid you in studying for your exams. For example, Lofl and, John, David Snow, Leon Anderson, and Lyn H. you will fi nd Tutorial Quizzes with feedback, Inter- Lofl and. 2006. Analyzing Social Settings: A Guide to net Exercises, Flashcards, and Chapter Tutorials, as Qualitative Observation and Analysis. 4th ed. Belmont, well as Extended Projects, InfoTrac College Edition CA: Wadsworth. An unexcelled presentation of fi eld search terms, Social Research in Cyberspace, GSS research methods from beginning to end. This emi- Data, Web Links, and primers for using various nently readable book manages successfully to draw data analysis software such as SPSS and NVivo. the links between the logic of scientifi c inquiry and the nitty-gritty practicalities of observing, communicating, recording, fi ling, reporting, and everything Additional Readings else involved in fi eld research. In addition, the book contains a wealth of references to fi eld research il- Adler, Patricia A., and Peter Adler. 2003. “The Promise lustrations. and Pitfalls of Going into the Field.” Contexts 2 (2): Morgan, David L., ed. 1993. Successful Focus Groups: Ad- 41–47. An excellent report on some of the complexi- vancing the State of the Art. Thousand Oaks, CA: Sage. ties of fi eld research, with tips on distinguishing good This collection of articles on the uses of focus groups from not-so-good ethnography. points to many aspects not normally considered. Denzin, Norman K., and Yvonna S. Lincoln, eds. 1994. Shaffi r, William B., and Robert A. Stebbins, eds. 1991. Handbook of Qualitative Research. Thousand Oaks, Experiencing Fieldwork: An Inside View of Qualitative CA: Sage. This handbook is an extensive collec- Research. Thousand Oaks, CA: Sage. Several fi eld tion of articles covering issues regarding the wide research practitioners discuss the nature of the craft fi eld of qualitative research. This book also exists and recall experiences in the fi eld. Here’s an oppor- in three volumes: Vol. 1. The Landscape of Qualita- tunity to gain a “feel” for the method as well as learn tive Research: Theories and Issues; Vol. 2. Strategies of some techniques. Qualitative Inquiry; and Vol. 3. Interpreting Qualitative Shostak, Arthur, ed. 1977. Our Sociological Eye: Personal Materials. Essays on Society and Culture. Port Washington, NY: Gans, Herbert J. 1999. “Participant Observation in the Era Alfred. An orgy of social scientifi c introspection, this of ‘Ethnography.’” Journal of Contemporary Ethnogra- delightful collection of fi rst-person research accounts phy 28 (5): 540–48. A thoughtful discussion of recent offers concrete, inside views of the thinking process developments and problems in participant observa- in sociological research, especially fi eld research. tion. Silverman, David. 1999. Doing Qualitative Research: A Gubrium, Jaber F., and James A. Holstein. 1997. The New Practical Handbook. Thousand Oaks, CA: Sage. This Language of Qualitative Method. New York: Oxford book focuses on the process of collecting and inter- University Press. This book provides the necessary preting qualitative data. foundations for understanding some of the main ap- Strauss, Anselm, and Juliet Corbin. 1998. Basics of Quali- proaches or traditions in qualitative fi eld research. tative Research: Procedures for Developing Grounded Johnson, Jeffrey C. 1990. Selecting Ethnographic In- Theory. Thousand Oaks, CA: Sage. This is a very formants. Thousand Oaks, CA: Sage. The author important book to read before data collection and discusses the various strategies that apply to the task during data analysis if you choose to take a grounded of sampling in fi eld research. theory approach. Kelle, Udo, ed. 1995. Computer-Aided Qualitative Data Uwe, Flick. 1998. An Introduction to Qualitative Research. Analysis: Theory, Methods, and Practice. Thousand Thousand Oaks, CA: Sage. This book provides a good Oaks, CA: Sage. An international group of scholars entrance to the large fi eld of qualitative research. 111 HUMANUNOBTRUSIVE INQUIRY RESEARCH AND SCIENCE Photo credit Dion Ogust/The Image Works

What You’ll Learn in thisThis ChapterChapter

ThisWe’ll chapter examine will presentthe way overviews people learn of threeabout unobtrusive their world researchand the mistakes methods: they makecontent along analysis, the way. the analysisWe’ll also of begin existing to seestatistics, what makes and comparative sience different and historicalfrom

otheranalysis. ways Each of knowing of these methodsthings. allows researchers to study social life from afar, 2S without inﬂ uencing it in the process. 1S N L 348 In this chapter . . . WHAT DO YOU THINK?

Introduction This chapter pres- Content Analysis ents several re- Topics Appropriate to Content Analysis search techniques Sampling in Content Analysis Image not available due to copyright restrictions that by deﬁ nition Coding in Content Analysis have no impact on Illustrations of Content Analysis what is being stud- Strengths and Weaknesses of Content Analysis ied. If the impact of Analyzing Existing Statistics the observer is such Durkheim’s Study of Suicide a problem in social research, why don’t so- The Consequences of Globalization cial scientists limit themselves to unobtru- Units of Analysis sive techniques? Problems of Validity See the “What Do You Think? Revisited” box Problems of Reliability toward the end of the chapter. Sources of Existing Statistics

Comparative and Historical Research Examples of Comparative and Historical Research fecting it. As you’ll see, unobtrusive measures can Sources of Comparative and Historical Data be qualitative or quantitative. Analytical Techniques This chapter examines three types of unobtrusive research methods: content analysis, analysis Ethics and Unobtrusive Measures of existing statistics, and comparative and historical research. In content analysis, researchers ex- INTRODUCTION amine a class of social artifacts that usually are written documents such as newspaper editorials. The Durkheim study is an example of the analysis With the exception of the complete observer in of existing statistics. As you’ll see, there are great fi eld research, each of the modes of observation masses of data all around you, awaiting your use discussed so far requires the researcher to intrude in understanding social life. Finally, comparative to some degree on whatever he or she is studying. and historical research, a form of research with This is most obvious in the case of experiments, a venerable history in the social sciences, is cur- followed closely by survey research. Even the fi eld rently enjoying a resurgence of popularity. Like researcher, as we’ve seen, can change things in fi eld research, comparative and historical research the process of studying them. is usually a qualitative method, one in which the At least one previous example in this book, main resources for observation and analysis are however, was totally exempt from that danger. Durkheim’s analysis of suicide did nothing to affect unobtrusive research Methods of studying social suicides one way or the other (see Chapter 5). His behavior without affecting it. This includes content study is an example of unobtrusive research, or analysis, analysis of existing statistics, and comparative and methods of studying social behavior without af- historical research. 349 350 CHAPTER 11 UNOBTRUSIVE RESEARCH historical records. The method’s name includes important social scientifi c question—all you have the word comparative because social scientists—in to do is think of the question. contrast to historians who may simply describe a Although problems of validity and reliability particular set of events—seek to discover common crop up in unobtrusive measures, a little ingenuity patterns that recur in different times and places. can either handle them or put them in perspective. To set the stage for our examination of these I encourage you to look at Webb’s book. It’s enjoy- three research methods, I want to draw your at- able reading, and it can be a source of stimulation tention to an excellent book that should sharpen and insight for social inquiry through data that al- your senses about the potential for unobtrusive ready exist. For now, let’s turn our attention to the measures in general. It is, among other things, fi rst of three unobtrusive methods often employed the book from which I take the term unobtrusive by social scientists: content analysis. measures. In 1966 Eugene Webb and three colleagues CONTENT ANALYSIS published an ingenious little book on social research that has become a classic: Unobtrusive Re- search (revised in 2000 as Unobtrusive Measures). As I mentioned in the chapter introduction, con- As you might have guessed, it focuses on the idea tent analysis is the study of recorded human of unobtrusive or nonreactive research. Webb and communications. Among the forms suitable for his colleagues have played freely with the task study are books, magazines, web pages, poems, of learning about human behavior by observing newspapers, songs, paintings, speeches, letters, what people inadvertently leave behind them. Do e-mail messages, bulletin board postings on the you want to know which exhibits are the most Internet, laws, and constitutions, as well as any popular at a museum? You could conduct a poll, components or collections thereof. Shulamit Re- but people might tell you what they thought you inharz points out that feminist researchers have wanted to hear or what might make them look in- used content analysis to study “children’s books, tellectual and serious. You could stand by different fairy tales, billboards, feminist nonfi ction and fi c- exhibits and count the viewers that came by, but tion books, children’s art work, fashion, fat-letter people might come over to see what you were do- postcards, Girl Scout Handbooks, works of fi ne art, ing. Webb and his colleagues suggest instead that newspaper rhetoric, clinical records, research pub- you check the wear and tear on the fl oor in front of lications, introductory sociology textbooks, and ci- various exhibits. Those that have the most-worn tations, to mention only a few” (1992:146–47). In tiles are probably the most popular. Want to know another example, when William Mirola set out to which exhibits are popular with little kids? Look for discover the role of religion in the movements to mucus on the glass cases. To get a sense of the establish the eight-hour working day in America, most popular radio stations, you could arrange his data were taken “from Chicago’s labor, reli- with an auto mechanic to check the radio dial set- gious, and secular presses, from pamphlets, and tings for cars brought in for repair. from speeches given by eight-hour proponents The possibilities are limitless. Like a detective from three representative factions within the investigating a crime, the social researcher looks movement” (2003:273). for clues. If you stop to notice, you’ll fi nd that clues of social behavior are all around you. In a sense, Topics Appropriate to Content Analysis everything you see represents the answer to some Content analysis is particularly well suited to the study of communications and to answering the content analysis The study of recorded human com- classic question of communications research: munications, such as books, websites, paintings, and laws. “Who says what, to whom, why, how, and with CONTENT ANALYSIS 351 what effect?” Are popular French novels more discuss some of the ways you could do that. Ulti- concerned with love than are American ones? Was mately, you’d need a plan that would allow you to popular British music of the 1960s more politically watch TV, classify sponsors, and rate the degree of cynical than popular German music during that violence on particular shows. period? Do political candidates who primarily ad- Next, you’d have to decide what to watch. dress “bread and butter” issues get elected more Probably you’d decide (1) what stations to watch, often than those who address issues of high prin- (2) for what days or period, and (3) at what hours. ciple? Each of these questions addresses a social Then, you’d stock up on beer and potato chips and scientifi c research topic: The fi rst might address start watching, classifying, and recording. Once national character, the second political orienta- you’d completed your observations, you’d be able tions, and the third the political process. Although to analyze the data you collected and determine we could study such topics through observation of whether men’s product manufacturers sponsored individual people, content analysis provides an- more blood and gore than did other sponsors. other approach. Gabriel Rossman (2002) had a somewhat dif- An early example of content analysis is the ferent question regarding the mass media. Public work of Ida B. Wells. In 1891 Wells, whose parents concern over the concentration of media in fewer had been slaves, wanted to test the widely held and fewer corporate hands has grown, so Ross- assumption that black men were being lynched man decided to ask the following question. If a in the South primarily for raping white women. newspaper is owned by the same conglomerate As a research method, she examined newspaper that owns a movie production company, can you articles on the 728 lynchings reported during the trust that newspaper’s movie reviews of its parent previous ten years. In only a third of the cases were company’s productions? the lynching victims even accused of rape, much You can’t, according to Rossman’s fi ndings. Be- less proven guilty. Primarily, they were charged cause many newspapers rate movies somewhat with being insolent, not staying in “their place” quantitatively (as in three stars out of four), he (cited in Reinharz 1992:146). could perform a simple quantitative analysis. For More recently, the best-selling Megatrends 2000 each movie review, he asked two main questions: (Naisbitt and Aburdene 1990) used content analy- (1) Was the movie produced by the same com- sis to determine major trends in modern U.S. life. pany that owned the newspaper? and (2) What The authors regularly monitored thousands of lo- rating did the fi lm receive? He found that, indeed, cal newspapers a month to discover local and re- movies produced by the parent company received gional trends, for publication in a series of quar- higher ratings than did other movies. Further, the terly reports. Their book examines some of the ratings given to movies by newspapers with the trends they observed in the nation at large. same parent company were higher than the rat- Some topics are more appropriately addressed ings those movies received from other newspa- by content analysis than by any other method pers. This discrepancy, moreover, was strongest in of inquiry. Suppose that you’re interested in vio- the case of big-budget movies in which the parent lence on TV. Maybe you suspect that the manufac- company had invested heavily. See the box “Un- turers of men’s products are more likely to sponsor raveling Bias in the Media and in Tobacco Ads” violent TV shows than are other kinds of spon- for more on the practical applications of content sors. Content analysis would be the best way of analysis. fi nding out. As a mode of observation, content analysis Briefl y, here’s what you would do. First, you’d requires a thoughtful handling of the “what” that develop operational defi nitions of the two key vari- is being communicated. The analysis of data col- ables in your inquiry: men’s products and violence. lected in this mode, as in others, addresses the The section on coding, later in this chapter, will “why” and “with what effect.” 352 CHAPTER 11 UNOBTRUSIVE RESEARCH

unit of analysis. But we’ll have to ask individual IN THE REAL WORLD members of families how much money they make. UNRAVELING BIAS IN THE Thus, individuals will be the units of observation, MEDIA AND IN TOBACCO ADS even though the individual family remains the unit of analysis. Similarly, we may wish to com- Content analysis has many practical applica- pare crime rates of different cities in terms of their tions. Consider, for example, the periodic al- size, geographic region, racial composition, and legations that the mass media have particu- other differences. Even though the characteristics lar biases: Conservatives often claim that the of these cities are partly a function of the behav- media are liberal, and liberals often claim iors and characteristics of their individual resi- the opposite. Notice how the tools of content dents, the cities would ultimately be the units of analysis permit you to select media materi- analysis. als for analysis, defi ne indicators of liberal or The complexity of this issue is often more ap- conservative slant, and then replace simple parent in content analysis than in other research allegations with research fi ndings. methods, especially when the units of observation Or consider charges that tobacco ads are differ from the units of analysis. A few examples aimed at youth and minority groups. Is it re- should clarify this distinction. ally true? What elements in ads could be re- Let’s suppose we want to fi nd out whether garded as indicators of such targeting? How criminal law or civil law makes the most distinc- would you select a sample of advertisements tions between men and women. In this instance, for study? Again, you could conduct research individual laws would be both the units of obser- to address the issue on factual grounds. vation and the units of analysis. We might select a sample of a state’s criminal and civil laws and then categorize each law by whether it makes a distinction between men and women. In this fashion, we Sampling in Content Analysis could determine whether criminal or civil law distinguishes by gender the most. In the study of communications, as in the study Somewhat differently, we might wish to deter- of people, you often can’t observe directly all that mine whether states that enact laws distinguish- you would like to explore. In your study of TV vio- ing between different racial groups are more likely lence and sponsorship, I advise against attempting than other states to enact laws distinguishing be- to watch everything that’s broadcast. It wouldn’t tween men and women. Although the examina- be possible, and your brain would probably short- tion of this question would also involve coding circuit before you got close to discovering that for individual acts of legislation, the unit of analysis in yourself. Usually, then, it’s appropriate to sample. this case is the individual state, not the law. Let’s begin by revisiting the idea of units of analy- Or, changing topics radically, let’s suppose sis. We’ll then review some of the sampling tech- we’re interested in representationalism in paint- niques that might be applied to them in content ing. If we wish to compare the relative popularity analysis. of representational and nonrepresentational paintings, the individual paintings will be our units of Units of Analysis As discussed in Chapter 4, analysis. If, on the other hand, we wish to discover determining appropriate units of analysis—the whether representationalism in painting is more individual units that we make descriptive and ex- characteristic of wealthy or impoverished painters, planatory statements about—can be a complicated of educated or uneducated painters, of capitalist task. For example, if we wish to compute the av- or socialist painters, the individual painters will be erage family income, the individual family is the our units of analysis. CONTENT ANALYSIS 353

Books may often subsample—select samples of subcat- egories—for each individual unit of analysis. Thus, if writers are the units of analysis, we might (1) select a sample of writers from the total population of writers, (2) select a sample of books written by each writer selected, and (3) select portions of each selected book for observation and coding. Finally, let’s look at a trickier example: the study of TV violence and sponsors. What’s the unit of analysis for the research question “Are the manufacturers of men’s products more likely to sponsor violent shows than are other sponsors?” Is it the Pages TV show? The sponsor? The instance of violence? In the simplest study design, it would be none of these. Though you might structure your inquiry in various ways, the most straightforward design would be based on the commercial as the unit of analysis. You would use two kinds of observational units: the commercial and the program (the show that gets squeezed in between commercials). You’d want to observe both units. You would classify commercials by whether they advertised men’s Paragraphs Lines products and the programs by their violence. The program classifi cations would be transferred to the commercials occurring near them. Figure 11-2 provides an example of the kind of record you FIGURE 11-1 A Few Possible Units of Analysis might keep. for Content Analysis Notice that in the research design illustrated in Figure 11-2, all the commercials occurring in the same program break are grouped and get the It’s essential that this issue be clear, because same scores. Also, the number of violent instances sample selection depends largely on what the unit recorded as following one commercial is the same of analysis is. If individual writers are the units of as the number preceding the next break. This sim- analysis, the sample design should select all or a ple design allows us to classify each commercial sample of the writers appropriate to the research by its sponsorship and the degree of violence as- question. If books are the units of analysis, we sociated with it. Thus, for example, the fi rst Grunt should select a sample of books, regardless of their Aftershave commercial is coded as being a men’s authors. Bruce Berg (1989:112–13) points out that product and as having 10 instances of violence as- even if you plan to analyze some body of textual sociated with it. The Buttercup Bra commercial is materials, the units of analysis might be words, coded as not being a men’s product and as having themes, characters, paragraphs, items (such as a no violent instances associated with it. book or letter), concepts, semantics, or combina- In the illustration, we have four men’s prod- tions of these. Figure 11-1 illustrates some of those uct commercials with an average of 7.5 violent possibilities. instances each. The four commercials classifi ed I’m not suggesting that sampling should be as defi nitely not men’s products have an average based solely on the units of analysis. Indeed, we of 1.75, and the two that might or might not be 354 CHAPTER 11 UNOBTRUSIVE RESEARCH

Men’s Number of Instances Product? of Violence Commercial Sponsor Break Before the After the Yes No ? Commercial Commercial Break Break

FIGURE 11-2 Example of Recording Sheet for TV Violence

considered men’s products have an average of 1 period of the study—number of days? And during violent instance each. If this pattern of differences which hours of each day will you observe? Then, persisted across a much larger number of observa- how many commercials do you want to observe tions, we’d probably conclude that manufacturers and code for analysis? Watch television for a while of men’s products are more likely to sponsor TV and fi nd out how many commercials occur each violence than are other sponsors. hour; then you can fi gure out how many hours of The point of this illustration is to demonstrate observation you’ll need. how units of analysis fi gure into data collection Now you’re ready to design the sample selec- and analysis. You need to be clear about your unit tion. As a practical matter, you wouldn’t have to of analysis before planning your sampling strategy, sample among the different stations if you had as- but in this case you can’t sample commercials. Un- sistants—each of you could watch a different chan- less you have access to the stations’ broadcasting nel during the same time period. But let’s suppose logs, you won’t know when the commercials are you’re working alone. Your fi nal sampling frame, going to occur. Moreover, you need to observe the from which a sample will be selected and watched, programming as well as the commercials. As a re- might look something like this: sult, you must set up a sampling design that will include everything you need to observe. Jan. 7, Channel 2, 7–9 P.M. In designing the sample, you’ll need to establish Jan. 7, Channel 4, 7–9 P.M. the universe to be sampled from. In this case, what Jan. 7, Channel 9, 7–9 P.M. TV stations will you observe? What will be the Jan. 7, Channel 2, 9–11 P.M. CONTENT ANALYSIS 355

Jan. 7, Channel 4, 9–11 P.M. U.S. newspapers, for example, we might fi rst group Jan. 7, Channel 9, 9–11 P.M. all newspapers by the region of the country or size Jan. 8, Channel 2, 7–9 P.M. of the community in which they are published, Jan. 8, Channel 4, 7–9 P.M. frequency of publication, or average circulation. Jan. 8, Channel 9, 7–9 P.M. We might then select a stratifi ed random or sys- Jan. 8, Channel 2, 9–11 P.M. tematic sample of newspapers for analysis. Hav- Jan. 8, Channel 4, 9–11 P.M. ing done so, we might select a sample of editorials Jan. 8, Channel 9, 9–11 P.M. from each selected newspaper, perhaps stratifi ed Jan. 9, Channel 2, 7–9 P.M. chronologically. Jan. 9, Channel 4, 7–9 P.M. Cluster sampling is equally appropriate to con- etc. tent analysis. Indeed, if individual editorials were to be the unit of analysis in the previous example, Notice that I’ve made several decisions for you then the selection of newspapers at the fi rst stage in the illustration. First, I’ve assumed that chan- of sampling would be a cluster sample. In an analy- nels 2, 4, and 9 are the ones appropriate to your sis of political speeches, we might begin by select- study. I’ve assumed that you found the 7 to 11 P.M. ing a sample of politicians; each politician would prime-time hours to be the most relevant and that represent a cluster of political speeches. The TV two-hour periods would do the job. I picked Janu- commercial study described previously is another ary 7 out of a hat for a starting date. In practice, of example of cluster sampling. course, all these decisions should be based on your Again, sampling need not end when we reach careful consideration of what would be appropri- the unit of analysis. If novels are the unit of analy- ate to your particular study. sis in a study, we might select a sample of novel- Once you’ve become clear about your units ists, subsamples of novels written by each selected of analysis and the observations appropriate to author, and a sample of paragraphs within each those units and have created a sampling frame novel. We would then analyze the content of the like the one I’ve illustrated, sampling is simple paragraphs for the purpose of describing the nov- and straightforward. The alternative procedures els themselves. available to you are the same ones described in Let’s turn now to the coding or classifi cation of Chapter 7: random, systematic, stratifi ed, and so on. the material being observed. Part 4 discusses the manipulation of such classifi cations to draw de- Sampling Techniques As we’ve seen, in the con- scriptive and explanatory conclusions. tent analysis of written prose, sampling may occur at any or all of several levels, including the contexts Coding in Content Analysis relevant to the works. Other forms of communication may also be sampled at any of the conceptual Content analysis is essentially a coding opera- levels appropriate to them. tion. Coding is the process of transforming raw In content analysis, we could employ any of the data into a standardized form. In content analy- conventional sampling techniques discussed in sis, communications—oral, written, or other—are Chapter 7. We might select a random or systematic coded or classifi ed according to some conceptual sample of French and U.S. novelists, of laws passed framework. Thus, for example, newspaper editori- in the state of Mississippi, or of Shakespearean so- als can be coded as liberal or conservative. Radio liloquies. We might select (with a random start) broadcasts can be coded as propagandistic or not, every 23rd paragraph in Tolstoy’s War and Peace. Or, we might number all the songs recorded by the Beatles and select a random sample of 25. coding The process whereby raw data are transformed Stratifi ed sampling is also appropriate to con- into standardized form suitable for machine processing tent analysis. To analyze the editorial policies of and analysis. 356 CHAPTER 11 UNOBTRUSIVE RESEARCH novels as romantic or not, paintings as representa- advantage of ease and reliability in coding and tional or not, and political speeches as containing of letting the reader of the research report know character assassinations or not. Recall that terms precisely how eroticism was measured. It would such as these can produce many interpretations, have a disadvantage, on the other hand, in terms so the researcher must create clear and specifi c of validity. Surely the phrase erotic novel conveys defi nitions. a richer and deeper meaning than the number of Coding in content analysis involves the logic of times the word love is used. conceptualization and operationalization as dis- Alternatively, you could code the latent con- cussed in Chapter 5. As in other research methods, tent of the communication: its underlying mean- you must refi ne your conceptual framework and ing. In the present example, you might read an develop specifi c methods for observing in relation entire novel or a sample of paragraphs or pages to that framework. and make an overall assessment of how erotic the novel was. Although your total assessment might Manifest and Latent Content In the earlier very well be infl uenced by the appearance of words discussions of fi eld research, we found that the such as love and kiss, it would not depend fully on researcher faces a fundamental choice between their frequency. depth and specifi city of understanding. Often, Clearly, this second method seems better de- this represents a choice between validity and resigned for tapping the underlying meaning of com- liability, respectively. Typically, fi eld researchers munications, but its advantage comes at a cost to opt for depth, preferring to base their judgments reliability and specifi city. Especially if more than on a broad range of observations and informa- one person is coding the novel, somewhat differ- tion, even at the risk that another observer might ent defi nitions or standards may be employed. A reach a different judgment of the same situation. passage that one coder regards as erotic may not Survey research—through the use of standardized seem erotic to another. Even if you do all the cod- questionnaires—represents the other extreme: to- ing yourself, there’s no guarantee that your defi ni- tal specifi city, even though the specifi c measures tions and standards will remain constant through- of variables may not be fully satisfactory as valid out the enterprise. Moreover, the reader of your refl ections of those variables. The content analyst research report will likely be uncertain about the has some choice in this matter, however. defi nitions you’ve employed. Coding the manifest content—the visible, sur- Wherever possible, the best solution to this di- face content—of a communication is analogous to lemma is to use both methods. For example, Carol using a standardized questionnaire. To determine, Auster was interested in changes in the socializa- for example, how erotic certain novels are, you tion of young women in Girl Scouts. To explore might simply count the number of times the word this, she undertook a content analysis of the Girl love appears in each novel or the average number Scout manuals as revised over time. In particu- of appearances per page. Or, you might use a list lar, Auster was interested in the view that women of such words as love, kiss, hug, and caress, each should be limited to homemaking. Her analysis of of which might serve as an indicator of the erotic the manifest content suggested a change: “I found nature of the novel. This method would have the that while 23% of the badges in 1913 centered on home life, this was true of only 13% of the badges in 1963 and 7% of the badges in 1980” — (1985:361). manifest content In connection with content analy- An analysis of the latent content also pointed sis, the concrete terms contained in a communication, as distinguished from latent content. to an emancipation of Girl Scouts, similar to that occurring in U.S. society at large. The change of latent content In connection with content analysis, the underlying meaning of communications as distin- uniform was one indicator: “The shift from skirts guished from their manifest content. to pants may refl ect an acknowledgement of the CONTENT ANALYSIS 357 more physically active role of women as well as over, should be mutually exclusive and exhaus- the variety of physical images available to modern tive. A newspaper editorial, for example, should women” (Auster 1985:362). Supporting evidence not be described as both liberal and conservative, was found in the appearance of badges such as though you should probably allow for some to be “Science Sleuth,” “Aerospace,” and “Ms. Fix-It.” middle-of-the-road. It may be suffi cient for your purposes to code novels as erotic or nonerotic, but Conceptualization and the Creation of Code you may also want to consider that some could be Categories For all research methods, conceptu- anti-erotic. Paintings might be classifi ed as repre- alization and operationalization typically involve sentational or not, if that satisfi ed your research the interaction of theoretical concerns and empiri- purpose, or you might wish to classify them as im- cal observations. If, for example, you believe some pressionistic, abstract, allegorical, and so forth. newspaper editorials to be liberal and others to be Realize further that different levels of mea- conservative, ask yourself why you think so. Read surement may be used in content analysis. You some editorials, asking yourself which ones are may, for example, use the nominal categories of liberal and which ones are conservative. Was the liberal and conservative for characterizing news- political orientation of a particular editorial most paper editorials, or you may wish to use a more clearly indicated by its manifest content or by its refi ned ordinal ranking, ranging from extremely tone? Was your decision based on the use of cer- liberal to extremely conservative. Bear in mind, tain terms (for example, leftist, fascist, and so on) or however, that the level of measurement implicit in on the support or opposition given to a particular your coding methods—nominal, ordinal, interval, issue or political personality? or ratio—does not necessarily refl ect the nature of Both inductive and deductive methods should your variables. If the word love appeared 100 times be used in this activity. If you’re testing theoretical in Novel A and 50 times in Novel B, you would propositions, your theories should suggest empiri- be justifi ed in saying that the word love appeared cal indicators of concepts. If you begin with spe- twice as often in Novel A, but not that Novel A was cifi c empirical observations, you should attempt twice as erotic as Novel B. Similarly, agreeing with to derive general principles relating to them and twice as many anti-Semitic statements in a ques- then apply those principles to the other empirical tionnaire does not necessarily make one twice as observations. anti-Semitic. Bruce Berg places code development in the context of grounded theory and likens it to solving Counting and Record Keeping If you plan to a puzzle: evaluate your content analysis data quantitatively, your coding operation must be amenable to data Coding and other fundamental procedures as- processing. This means, fi rst, that the end product sociated with grounded theory development are of your coding must be numerical. If you’re count- certainly hard work and must be taken seri- ing the frequency of certain words, phrases, or ously, but just as many people enjoy fi nishing a other manifest content, the coding is necessarily complicated jigsaw puzzle, many researchers numerical. But even if you’re coding latent content fi nd great satisfaction in coding and analysis. on the basis of overall judgments, it will be nec- As researchers . . . begin to see the puzzle essary to represent your coding decision numeri- pieces come together to form a more complete cally: 1 very liberal, 2 moderately liberal, 3 picture, the process can be downright thrilling. moderately conservative, and so on. — (1989:111) Second, your record keeping must clearly dis- Throughout this activity, remember that the op- tinguish between your units of analysis and your erational defi nition of any variable is composed of units of observation, especially if these two are dif- the attributes included in it. Such attributes, more- ferent. The initial coding, of course, must relate to 358 CHAPTER 11 UNOBTRUSIVE RESEARCH

SUBJECTIVE EVALUATION 1. Very liberal Newspaper Number of 2. Moderately liberal Number of Number of Number of ID editorials 3. Middle-of-road “isolationist” “pro–United “anti–United evaluated 4. Moderately conservative editorials Nations” Nations” 5. Very conservative editorials editorials

FIGURE 11-3 Sample Tally Sheet (Partial)

the units of observation. If novelists are the units of entation, even if it must be coded “no apparent analysis, for example, and you wish to character- orientation.” ize them through a content analysis of their nov- Let’s suppose we want to describe and explain els, your primary records will represent novels as the editorial policies of different newspapers. Fig- the units of observation. You may then combine ure 11-3 presents part of a tally sheet that might re- your scoring of individual novels to characterize sult from the coding of newspaper editorials. Note each novelist, the unit of analysis. that newspapers are the units of analysis. Each Third, when you’re counting, you usually need newspaper has been assigned an identifi cation to record the base from which the counting is number to facilitate mechanized processing. The done. It would probably be useless to know the second column has a space for the number of edi- number of realistic paintings produced by a given torials coded for each newspaper. This will be an painter without knowing the total number he or important piece of information, because we want she had painted; the painter would be regarded to be able to say, for example, “Of all the editori- as realistic if a high percentage of paintings were als, 22 percent were pro–United Nations,” not just of that genre. Similarly, it would tell us little that “There were eight pro–United Nations editorials.” the word love appeared 87 times in a novel if we One column in Figure 11-3 is for assigning a didn’t know about how many words there were in subjective overall assessment of the newspapers’ the entire novel. The issue of observational base is editorial policies. (Such assignments might later most easily resolved if every observation is coded be compared with the several objective measures.) in terms of one of the attributes making up a vari- Other columns provide space for recording num- able. Rather than simply counting the number of bers of editorials refl ecting specifi c editorial posi- liberal editorials in a given collection, for example, tions. In a real content analysis, there would be you would code each editorial by its political ori- spaces for recording other editorial positions plus CONTENT ANALYSIS 359 noneditorial information about each newspaper, There are, of course, dangers in this form of such as the region in which it is published, its cir- analysis, as in all others. The chief risk is that you’ll culation, and so forth. misclassify observations so as to support your emerging hypothesis. You may erroneously con- Qualitative Data Analysis Not all content analy- clude that a nonleader didn’t graduate from col- sis results in counting. Sometimes a qualitative lege or you may decide that the job of factory fore- assessment of the materials is most appropriate, man is “close enough” to being white-collar. as in Carol Auster’s examination of changes in Girl Berg (1989:124) offers techniques for avoiding Scout uniforms and handbook language. these errors: Bruce Berg (1989:123–25) discusses “nega- 1. If there are suffi cient cases, select some at tive case testing” as a technique for qualitative random from each category in order to avoid hypothesis testing. First, in the grounded theory merely picking those that best support the tradition, you begin with an examination of the hypothesis. data, which may yield a general hypothesis. Let’s 2. Give at least three examples in support of say that you’re examining the leadership of a new every assertion you make about the data. community association by reviewing the minutes 3. Have your analytic interpretations carefully of meetings to see who made motions that were reviewed by others uninvolved in the research subsequently passed. Your initial examination of project to see whether they agree. the data suggests that the wealthier members are 4. Report whatever inconsistencies you do dis- the most likely to assume this leadership role. cover—any cases that simply do not fi t your The second stage in the analysis is to search hypotheses. Realize that few social patterns your data to fi nd all the cases that would contra- are 100 percent consistent, so you may have dict the initial hypothesis. In this instance, you discovered something important even if it would look for poorer members who made suc- doesn’t apply to absolutely all of social life. cessful motions and wealthy members who never However, you should be honest with your did. Third, you must review each of the disconfi rm- readers in that regard. ing cases and either (1) give up the hypothesis or (2) see how it needs to be fi ne-tuned. Let’s say that in your analysis of disconfi rming Illustrations of Content Analysis cases, you notice that each of the unwealthy lead- Several studies have indicated that women are ers has a graduate degree, whereas each of the stereotyped on television. R. Stephen Craig (1992) wealthy nonleaders has very little formal educa- took this line of inquiry one step further to examine tion. You may revise your hypothesis to consider the portrayal of both men and women during dif- both education and wealth as routes to leadership ferent periods of television programming. in the association. Perhaps you’ll discover some To study gender stereotyping in television com- threshold for leadership (a white-collar job, a level mercials, Craig selected a sample of 2,209 network of income, and a college degree) beyond which commercials during several periods between Janu- those with the most money, education, or both are ary 6 and 14, 1990. the most active leaders. This process is an example of what Barney The weekday day part (in this sample, Monday– Glaser and Anselm Strauss (1967) called analytic Friday, 2–4 P.M.) consisted exclusively of soap induction. It’s inductive in that it primarily begins operas and was chosen for its high percentage with observations, and it’s analytic because it goes of women viewers. The weekend day part (two beyond description to fi nd patterns and relation- consecutive Saturday and Sunday afternoons ships among variables. during sports telecasts) was selected for its high 360 CHAPTER 11 UNOBTRUSIVE RESEARCH

percentage of men viewers. Evening “prime were proportionately more likely to be portrayed time” (Monday–Friday, 9–11 P.M.) was chosen as as interviewer/demonstrators, parent/spouses, a basis for comparison with past studies and the or sex object/models in every day part. . . . other day parts. — (1992:199) Women were proportionately more likely to appear as sex object/models during the weekend Each of the commercials was coded in several than during the day. — (1992:204) ways. “Characters” were coded as

All male adults The research also showed that different prod- All female adults ucts were advertised during different time periods. All adults, mixed gender As you might have imagined, almost all the week- Male adults with children or teens (no women) day daytime commercials dealt with body, food, or Female adults with children or teens (no men) home products. These products accounted for only Mixture of ages and genders one in three on the weekends. Instead, weekend commercials stressed automotive products (29 In addition, Craig’s coders noted which charac- percent), business products or services (27 per- ter was on the screen longest during the commer- cent), or alcohol (10 percent). There were virtu- cial—the “primary visual character”—as well as ally no alcohol ads during evenings and weekday the roles played by the characters (such as spouse, daytime. celebrity, parent), the type of product advertised As you might suspect, women were most likely (such as body product, alcohol), the setting (such to be portrayed in home settings, men most likely as kitchen, school, business), and the voice-over to be shown away from home. Other ﬁ ndings dealt narrator. with the different roles played by men and women. Table 11-1 indicates the differences in the times when men and women appeared in commercials. The women who appeared in weekend ads Women were more common during the daytime were almost never portrayed without men and (with its soap operas), men predominated during seldom as the commercial’s primary character. the weekend commercials (with its sports pro- They were generally seen in roles subservient to gramming), and men and women were equally men (e.g., hotel receptionist, secretary, or stew- represented during evening prime time. ardess), or as sex objects or models in which Craig found other differences in the ways men their only function seemed to be to lend an and women were portrayed. aspect of eroticism to the ad. — (CRAIG 1992:208)

Further analysis indicated that male primary Although some of Craig’s ﬁ ndings may seem characters were proportionately more likely unsurprising, remember that “common knowl- than females to be portrayed as celebrities and edge” does not always correspond with reality. professionals in every day part, while women It’s always worthwhile to check out widely held assumptions. And even when we think we know TABLE 11-1 Percentages of Adult Primary about a given situation, it’s often useful to know Visual Characters by Sex Appearing in speciﬁ c details such as those provided by a content Commercials in Three Day Parts analysis like this one. In another example of content analysis, Charis Daytime Evening Weekend Kubrin (2005) also drew on popular culture but Adult male 40 52 80 undertook a primarily qualitative analysis. Kubrin Adult female 60 48 20 focused on the themes put forth in rap music, particularly gangsta rap, and the relationship of those Source: R. Stephen Craig, “The Effect of Television Day Part on themes to neighborhood culture and “the street Gender Portrayals in Television Commercials: A Content Analysis,” Sex Roles 26, nos. 5/6 (1992): 204. code.” CONTENT ANALYSIS 361

In response to societal and neighborhood majority. Many of the violent (and patriarchical, conditions, black youth in disadvantaged com- materialistic, sexist, etc.) ways of thinking that munities have created a substitute social order are glorifi ed in gangsta rap are a refl ection of the governed by their own code—a street code—and prevailing values created and sustained in the rituals of authenticity. . . . This social order larger society. — (2005:454) refl ects the subcultural locus of interests that She traces the implications of this for understand- emerges from pervasive race and class inequal- ing street life as well as for the likely success of ity and the social isolation of poor black com- various crime-control strategies. munities. — (2005:439)

She began her study by identifying all the plati- num rap albums released between 1992 and 2000: Strengths and Weaknesses 130 albums containing 1,922 songs. She then drew of Content Analysis a simple random sample of one-third of the songs Probably the greatest advantage of content analy- (632) and set about the task of listening to each. sis is its economy in terms of both time and money. She did this twice with each song. A single college student could undertake a content First, I listened to a song in its entirety while analysis, whereas undertaking a survey, for ex- reading the printed lyrics to determine what the ample, might not be feasible. There is no require- song was about. Second, I listened to the song ment for a large research staff; no special equip- again and coded each line to determine whether ment is required. As long as you have access to the the street code elements described earlier were material to be coded, you can undertake content present: (1) respect, (2) willingness to fi ght or analysis. use violence, (3) material wealth, (4) violent Content analysis also allows the correction of retaliation, (5) objectifi cation of women, and errors. If you discover you’ve botched up a survey (6) nihilism. — (2005:443) or an experiment, you may be forced to repeat the whole research project with all its attendant costs Kubrin was particularly interested in the theme in time and money. If you botch up your fi eld re- of nihilism, the rejection of traditional moral prin- search, it may be impossible to redo the project; ciples and a fundamental skepticism about the the event under study may no longer exist. In con- meaning of life. She wanted to know how that tent analysis, it’s usually easier to repeat a portion theme was portrayed in gangsta rap music and of the study than it is in other research methods. how it fi t into the street code. You might be required, moreover, to recode only a Though she began with a sample 632 songs, she portion of your data rather than all of it. found that no new themes appeared to be show- A third advantage of content analysis is that it ing up after about 350 songs had been analyzed. permits you to study processes occurring over a To be safe, she coded another 50 songs and found long time. You might focus on the imagery of Afri- no new themes, completing her coding process at can Americans conveyed in U.S. novels of 1850 to that point. 1860, for example, or you might examine changing Kubrin notes that rap music is typically regarded imagery from 1850 to the present. as antisocial and resistant to organized society, but Finally, content analysis has the advantage of her in-depth analysis of lyrics suggested something all unobtrusive measures, namely, that the content different. analyst seldom has any effect on the subject be- Rap music does not exist in a cultural vacuum; ing studied. Because the novels have already been rather it expresses the cultural crossing, mixing, written, the paintings already painted, and the and engagement of black youth culture with speeches already presented, content analyses can the values, attitudes and concerns of the white have no effect on them. 362 CHAPTER 11 UNOBTRUSIVE RESEARCH

Content analysis has disadvantages as well. would be interesting and revealing in connection For one thing, it’s limited to the examination of with the data your own research would generate. recorded communications. Such communications Existing statistics, then, can often provide a histori- may be oral, written, or graphical, but they must be cal or conceptual context within which to locate recorded in some fashion to permit analysis. your original research. As we’ve seen, content analysis has both ad- Existing statistics can also provide the main data vantages and disadvantages in terms of validity for a social scientifi c inquiry. An excellent example and reliability. Problems of validity are likely un- is the classic study mentioned at the beginning less you happen to be studying communication of this chapter, Emile Durkheim’s Suicide ([1897] processes per se. 1951). Let’s take a closer look at Durkheim’s work On the other side of the ledger, the concrete- before considering some of the special problems ness of materials studied in content analysis this method presents. strengthens the likelihood of reliability. You can always code and recode and even recode again if Durkheim’s Study of Suicide you want, making certain that the coding is consistent. In fi eld research, by contrast, there’s probably Why do people kill themselves? Undoubtedly, ev- nothing you can do after the fact to ensure greater ery suicide case has a unique history and explana- reliability in observation and categorization. tion, yet all such cases could no doubt be grouped Let’s move from content analysis now and turn according to certain common causes: fi nancial to a related research method: the analysis of ex- failure, trouble in love, disgrace, and other kinds of isting data. Although numbers rather than com- personal problems. The French sociologist Emile munications are the substance analyzed in this Durkheim had a slightly different question in mind case, I think you’ll see the similarity to content when he addressed the matter of suicide, however. analysis. He wanted to discover the environmental conditions that encouraged or discouraged it, especially ANALYZING EXISTING STATISTICS social conditions. The more Durkheim examined the available records, the more patterns of differences became Frequently you can or must undertake social sci- apparent to him. All of these patterns interested entifi c inquiry through the use of offi cial or quasi- him. One of the fi rst things to attract his attention offi cial statistics. This differs from secondary anal- was the relative stability of suicide rates. Looking ysis, in which you obtain a copy of someone else’s at several countries, he found suicide rates to be data and undertake your own statistical analysis. about the same year after year. He also discovered In this section, we’re going to look at ways of us- that a disproportionate number of suicides oc- ing the data analyses that others have already curred in summer, leading him to hypothesize that performed. temperature might have something to do with sui- This method is particularly signifi cant because cide. If this were the case, suicide rates should be existing statistics should always be considered at higher in the southern European countries than in least a supplemental source of data. If you were the temperate ones. However, Durkheim discov- planning a survey of political attitudes, for ex- ered that the highest rates were found in countries ample, you would do well to examine and present in the central latitudes, so temperature couldn’t be your fi ndings within a context of voting patterns, the answer. rates of voter turnout, or similar statistics relevant He explored the role of age (35 was the most to your research interest. Or, if you were doing common suicide age), gender (men outnumbered evaluation research on an experimental morale- women around four to one), and numerous other building program on an assembly line, probably factors. Eventually, a general pattern emerged statistics on absenteeism, sick leave, and so on from different sources. ANALYZING EXISTING STATISTICS 363

In terms of the stability of suicide rates over TABLE 11-2 Suicide Rates in Various German time, for instance, Durkheim found the pattern Provinces, Arranged in Terms of Religious Affi liation was not totally stable. There were spurts in the rates during times of political turmoil, which oc- Religious Character Suicides per curred in several European countries around 1848. of Province Million Inhabitants This observation led him to hypothesize that sui- Bavarian Provinces (1867–1875)* cide might have something to do with “breaches in social equilibrium.” Put differently, social stability Less than 50% Catholic and integration seemed to be a protection against Rhenish Palatinate 167 suicide. Central Franconia 207 This general hypothesis was substantiated and Upper Franconia 204 specifi ed through Durkheim’s analysis of a differ- Average 192 ent set of data. The countries of Europe had radi- 50% to 90% Catholic cally different suicide rates. The rate in Saxony, Lower Franconia 157 for example, was about ten times that of Italy, and Swabia 118 the relative ranking of various countries persisted Average 135 over time. As Durkheim considered other differ- More than 90% Catholic ences among the various countries, he eventually Upper Palatinate 64 noticed a striking pattern: Predominantly Protes- Upper Bavaria 114 tant countries had consistently higher suicide rates Lower Bavaria 19 than did Catholic ones. The predominantly Protes- Average 75 tant countries had 190 suicides per million popula- Prussian Provinces (1883–1890) tion; mixed Protestant-Catholic countries, 96; and predominantly Catholic countries, 58 (Durkheim More than 90% Protestant [1897] 1951:152). Saxony 309.4 Although suicide rates thus seemed to be re- Schleswig 312.9 lated to religion, Durkheim reasoned that some Pomerania 171.5 other factor, such as level of economic and cul- Average 264.6 tural development, might explain the observed dif- 68% to 89% Protestant ferences among countries. If religion had a genu- Hanover 212.3 ine effect on suicide, then the religious difference Hesse 200.3 would have to be found within given countries as Brandenburg and Berlin 296.3 well. To test this idea, Durkheim fi rst noted that the East Prussia 171.3 German state of Bavaria had both the most Catho- Average 220.0 lics and the lowest suicide rates in that country, 40% to 50% Protestant whereas heavily Protestant Prussia had a much West Prussia 123.9 higher suicide rate. Not content to stop there, how- Silesia 260.2 ever, Durkheim examined the provinces compos- Westphalia 107.5 ing each of those states. Average 163.6 Table 11-2 shows what he found. As you can 28% to 32% Protestant see, in both Bavaria and Prussia, provinces with Posen 96.4 the highest proportion of Protestants also had the Rhineland 100.3 highest suicide rates. Increasingly, Durkheim be- Hohenzollern 90.1 came confi dent that religion played a signifi cant Average 95.6 role in the matter of suicide. *Note: The population below 15 years has been omitted. Returning eventually to a more general theoret- Source: Adapted from Emile Durkheim, Suicide (Glencoe, IL: Free ical level, Durkheim combined the religious fi nd- Press, [1897] 1951), 153. 364 CHAPTER 11 UNOBTRUSIVE RESEARCH ings with the earlier observation about increased fects the developing countries that host the pro- suicide rates during times of political turmoil. Put cess. To that end, he used data available from most simply, Durkheim suggested that many sui- the World Bank’s “World Development Indica- cides are a product of anomie, “normlessness,” or tors.” Noting past variations in the way global- a general sense of social instability and disintegra- ization was measured, Kentor used the amount tion. During times of political strife, people may feel of foreign investment in a country’s economy as that the old ways of society are collapsing. They a percentage of that country’s whole economy. become demoralized and depressed, and suicide He reasoned that dependence on foreign invest- is one answer to the severe discomfort. Seen from ments was more important that the amount of the the other direction, social integration and solidar- investment. ity—refl ected in personal feelings of being part of a coherent, enduring social whole—offer protection against depression and suicide. That was You can learn more about the World where the religious difference fi t in. Catholicism, Bank’s data* at http://www.worldbank as a far more structured and integrated religious .org/data/. system, gave people a greater sense of coherence and stability than did the more loosely structured Protestantism. In his analysis of 88 countries with a per capita From these theories, Durkheim created the gross domestic product (the total goods and ser- concept of anomic suicide. More important, as you vices produced in a country) of less that $10,000, know, he added the concept of anomie to the lexi- Kentor found that dependence on foreign invest- con of the social sciences. ment tended to increase income inequality among This account of Durkheim’s classic study is the citizens of a country. The greater the degree greatly simplifi ed, of course. Anyone studying of dependence, the greater the income inequal- social research would profi t from studying the ity. Kentor reasoned that globalization produced original. For our purposes, Durkheim’s approach well-paid elites who, by working with the foreign provides a good illustration of the possibilities for corporations, maintained a status well above that research contained in the masses of data regularly of the average citizen. But because the profi ts de- gathered and reported by government agencies. rived from the foreign investments tended to be returned to the investors’ countries rather than enriching the poor countries, the great majority of The Consequences of Globalization the population in the latter reaped little or no eco- The notion of “globalization” has become increas- nomic benefi t. ingly controversial in the United States and around Income inequality, in turn, was found to in- the world, with reactions ranging from scholarly crease birth rates and, hence, population growth, debates to violent confrontations in the streets. One in a process too complex to summarize here. Pop- point of view sees the spread of U.S.-style capital- ulation growth, of course, brings a wide range of ism to developing countries as economic salvation problems to countries already too poor to provide for those countries. A very different point of view for the basic needs of their people. sees globalization as essentially neocolonial ex- This research example, along with our brief ploitation, in which multinational conglomerates look at Durkheim’s studies, should broaden your exploit the resources and people of poor countries. understanding of the kinds of social phenomena And, of course, there are numerous variations on these contradictory views.

Jeffrey Kentor (2001) wanted to bring data *Each time the Internet icon appears, you’ll be given help- to bear on the question of how globalization af- ful leads for searching the World Wide Web. ANALYZING EXISTING STATISTICS 365 that we can study through data already collected counterexplanation, such as the one I just made and compiled by others. up, less likely. Second, by extensively retesting his conclusions in a variety of ways, Durkheim further Units of Analysis strengthened the likelihood that they were correct. Suicide rates were higher in Protestant countries The unit of analysis involved in the analysis of exist- than in Catholic ones; higher in Protestant regions ing statistics is often not the individual. Durkheim, of Catholic countries than in Catholic regions of for example, was required to work with political- Protestant countries; and so forth. The replication geographical units: countries, regions, states, and of fi ndings added to the weight of evidence sup- cities. You would likely face the same requirement porting his conclusions. if you were to undertake a study of crime rates, accident rates, disease, and so forth. By their nature, Problems of Validity most existing statistics are aggregated: They describe groups. Whenever we base our research on an analysis of The aggregate nature of existing statistics can data that already exist, we’re obviously limited to present a problem, though not an insurmountable what exists. Often, the existing data don’t cover one. As we saw, for example, Durkheim wanted to exactly what we’re interested in, and our measure- determine whether Protestants or Catholics were ments may not be altogether valid. more likely to commit suicide. The diffi culty was Two characteristics of science are used to han- that none of the records available to him indicated dle the problem of validity in the analysis of ex- the religion of those people who committed sui- isting statistics: logical reasoning and replication. cide. Ultimately, then, it was not possible for him to Durkheim’s strategy provides an example of logi- say whether Protestants committed suicide more cal reasoning. Although he could not determine often than did Catholics, although he inferred as the religion of people who committed suicide, he much. Because Protestant countries, regions, and reasoned that most of the suicides in a predomi- states had higher suicide rates than did Catholic nantly Protestant region would be Protestants. countries, regions, and states, he drew the obvious As you have seen, replication can resolve conclusion. problems of validity in many social research ap- There’s danger in drawing this kind of conclu- proaches. Recall the earlier discussion of the in- sion, however. It’s always possible that patterns of terchangeability of indicators (Chapter 5). Crying behavior at a group level do not refl ect correspond- in sad movies isn’t necessarily a valid measure of ing patterns on an individual level. Such errors re- compassion; neither is putting little birds back in fl ect the ecological fallacy, which was discussed in their nests nor giving money to charity. None of Chapter 4. In the case of Durkheim’s study, it was these things, taken alone, would prove that one altogether possible, for example, that it was Catho- group (women, say) were more compassionate lics who committed suicide in the predominantly than another (men). But if women appeared more Protestant areas. Perhaps Catholics in predomi- compassionate than men by all these measures, nantly Protestant areas were so badly persecuted that would create a weight of evidence in sup- that they fell into despair and suicide. In that case port of the conclusion. In the analysis of existing Protestant countries could have high suicide rates statistics, a little ingenuity and reasoning can usu- without any Protestants committing suicide. ally turn up several independent tests of a given Durkheim avoided the danger of the ecologi- hypothesis. If all the tests seem to confi rm the hy- cal fallacy in two ways. First, his general con- pothesis, then the weight of evidence supports the clusions were based as much on rigorous, theo- validity of the measure. Durkheim’s analysis of re- retical deductions as on the empirical facts. The gions and other subsamples of countries is a form of correspondence between theory and fact made a replication. 366 CHAPTER 11 UNOBTRUSIVE RESEARCH

Problems of Reliability abusive to police offi cers, and so forth. When we consider unreported crimes, sometimes estimated The analysis of existing statistics depends heavily to be ten times the number of crimes known to on the quality of the statistics themselves: Do they police, the reliability of crime statistics gets even accurately report what they claim to report? This shakier. can be a substantial problem, because the weighty These comments concern crime statistics at tables of government statistics, for example, are a local level. Often it’s useful to analyze national sometimes grossly inaccurate. crime statistics, such as those reported in the FBI’s Consider research into crime. Because a great annual Uniform Crime Reports. Additional prob- deal of this research depends on offi cial crime sta- lems are introduced at the national level. Different tistics, this body of data has come under critical local jurisdictions defi ne crimes differently. Also, evaluation. The results have not been encourag- participation in the FBI program is voluntary, so ing. As an illustration, suppose you were interested the data are incomplete. in tracing long-term trends in marijuana use in the Finally, the process of record keeping affects United States. Offi cial statistics on the numbers of the records that are kept and reported. When- people arrested for selling or possessing it would ever a law-enforcement unit improves its record- seem to be a reasonable measure of use, right? Not keeping system—computerizes it, for example— necessarily. the apparent crime rates increase dramatically. To begin, you face a hefty problem of validity. This can happen even if the number of crimes Before the passage of the Marihuana Tax Act in committed, reported, and investigated does not 1937, “grass” was legal in the United States, so ar- increase. rest records would not give you a valid measure Researchers’ fi rst protection against the prob- of use. But even if you limited your inquiry to the lems of reliability in the analysis of existing statis- post-1937 era, you would still have problems of re- tics is awareness—knowing that the problem may liability, stemming from the nature of law enforce- exist. Investigating the nature of the data collec- ment and crime record keeping. tion and tabulation may enable you to assess the Law enforcement, for example, faces various nature and degree of unreliability so you can judge pressures. A public outcry against marijuana, led its potential impact on your research interest. If perhaps by a vocal citizens’ group, often results in you also use logical reasoning and replication, you a police crackdown on drug traffi cking—especially can usually cope with the problem. if it occurs during an election or budget year. A The box “Is America #1?” provides an example sensational story in the press can have a similar of what you might discover by carefully examining effect. In addition, the volume of other business the use of existing statistics. facing police affects marijuana arrests. In tracing the pattern of drug arrests in Chi- Sources of Existing Statistics cago between 1942 and 1970, Lois DeFleur (1975) demonstrated that the offi cial records present a It would take a whole book just to list the sources far less accurate history of drug use than of police of data available for analysis. In this section, I’ll practices and political pressure on police. On a dif- mention a few sources and point you in the di- ferent level of analysis, Donald Black (1970) and rection of fi nding others relevant to your research others analyzed the factors infl uencing whether interests. an offender is actually arrested by police or let off The single most valuable book you can buy is with a warning. Ultimately, offi cial crime statistics the annual Statistical Abstract of the United States, are infl uenced by whether specifi c offenders are published by the United States Department of well or poorly dressed, whether they are polite or Commerce. Unquestionably the best source of ANALYZING EXISTING STATISTICS 367

ISSUES AND INSIGHTS IS AMERICA #1?

On September 19, 1999, ABC-TV broadcast a Iceland, Ireland, New Zealand, Norway and special show, hosted by John Stossel, to exam- Sweden. ine where the United States stood in the ranking of the world’s societies. As the show unfolded, it Stossel went on to contrast Hong Kong (a capi- became clear that the USA was doing OK—argu- talist success story) with the alternative to a free ably #1—and that the key to our success was due economy: “stagnation, and often poverty. Con- primarily to our laissez-faire capitalist system. sider China, now mired in Third World poverty. To make the latter point more strongly, Stossel They were once the leader of the world.” Again, pointed to other countries that also owed their FAIR suggested a different assessment: success to laissez-faire capitalism. Actually, China’s economy is anything but According to Stossel, Hong Kong stood out “stagnant.” As the Treasury Department’s among the world’s nations as the leader of free- Lawrence Summers said in a speech last market economics. As evidence of Hong Kong’s year, “China has been the fastest growing success, Stossel reported that it had “the only economy in history since [economic] reform government in the world that makes a surplus, a began in 1980.” While China has adopted big surplus.” What do you think about that con- some aspects of market economics, a large clusion? Is it convincing to you? proportion of its business ﬁ rms are still Here’s what the media watchdog, Fairness owned by the government. and Accuracy in Reporting (FAIR), had to say about Stossel’s assertion: In the media and elsewhere, you’ll often ﬁ nd as- As anyone who pays attention to Wash- sertions of fact that appear to be based on statis- ington politics knows, the U.S. government tical analyses. However, it’s usually a good idea has been running a federal budget surplus to check the facts. for more than a year; it amounted to $70 billion last year. Other countries with budget Source: Fairness and Accuracy in Reporting, “Action Alert: ABC News Gives up on Accuracy?” Sep tem- surpluses last year included the United King- ber 28, 1999, http://www.fair.org/activism/ dom, Canada, Australia, Denmark, Finland, stossel-america.html.

data about the United States, it includes statistics offers the same book in soft cover for less cost. The on the individual states and (less extensively) cit- commercial version, entitled The American Alma- ies as well as on the nation as a whole. Where else nac, should not be confused with other almanacs can you learn the number of work stoppages in the that are less reliable and less useful for social sci- country year by year, the residential property taxes entiﬁ c research. Better yet, you can buy the Statisti- of major cities, the number of water pollution dis- cal Abstract on a CD-ROM, making the search for charges reported around the country, the number and transfer of data quite easy. Best of all, you can of business proprietorships in the nation, and hun- download the Statistical Abstract from the web for dreds of other such handy bits of information? To free (your tax dollars at work for you); see the In- make things even better, Hoover’s Business Press ternet box for the web address. 368 CHAPTER 11 UNOBTRUSIVE RESEARCH

These data point to a persistent difference be- Here are a few of the many data sources tween the incomes of men and women, even you can fi nd on the World Wide Web: when both groups have achieved the same levels • Bureau of the Census: http://www of education. Other variables could explain the .census.gov/ • Bureau of Labor Statistics: differences, however; we’ll return to this issue in http://stats.bls.gov/ Chapter 14. • Bureau of Transportation Statistics: Federal agencies—the Departments of Labor, http://www.bts.gov/ Agriculture, Transportation, and so forth—publish • Central Intelligence Agency: numerous data series. To fi nd out what’s available, http://www.cia.gov/ go to your library, fi nd the government documents • Department of Education: section, and spend a few hours browsing through http://www.ed.gov/index.jhtml the shelves. You’ll come away with a clear sense • Federal Bureau of Investigation: of the wealth of data available to your insight and http://www.fbi.gov/ ingenuity. You can also visit these departments’ • State and Local Governments: websites or the U.S. Government Printing Offi ce http://www.statelocalgov.net/ • Statistical Abstract: http://www site and look around. .census.gov/compendia/statab/ World statistics are available through the United • U.S. Government Printing Offi ce: Nations. Its Demographic Yearbook presents an- http://www.gpoaccess.gov/index.html nual vital statistics (births, deaths, and other data • The World Bank: http://www relevant to population) for the individual nations .worldbank.org/ of the world. Other publications report a variety of other kinds of data. Again, a trip to your library, along with a web search, is the best introduction Suppose you were interested in the issue of in- to what’s available. come discrimination by gender. You could exam- The amount of data provided by nongovern- ine this rather easily through the Statistical Abstract ment agencies is as staggering as the amount your data. Here, for example, is a look at gender, educa- taxes buy. Chambers of commerce often publish tion, and income (adapted from U.S. Bureau of the data reports on business, as do private consumer Census 2006: Table 686, p. 467): groups. Ralph Nader has information on automobile safety, and Common Cause covers politics and Average Earnings of Year-Round, government. And, as mentioned earlier, George Full-Time Workers, 2003 Gallup publishes reference volumes on public Ratio of opinion as tapped by Gallup polls since 1935. Women/ Organizations such as the Population Reference Men Men Women Earnings Bureau publish a variety of demographic data, U.S. and international, that a secondary analyst could All workers 53,039 37,197 0.70 use. Their World Population Data Sheet and Popula- Less than 9th grade 23,972 20,979 0.88 tion Bulletin are heavily used by social scientists. 9th–12th grades 29,100 21,426 0.74 Social indicator data can be found in the journal HS graduates 38,331 27,956 0.73 SINET: A Quarterly Review of Social Reports and Re- Some college 46,332 31,655 0.68 search on Social Indicators, Social Trends, and the Associate degree 48,683 36,528 0.75 Quality of Life. Bachelors or more 81,007 53,215 0.66 The sources I’ve listed are only a tiny fraction of the thousands that are available. With so much You can access this table online at http://www.census.gov/prod/ 2005pubs/06statab/income.pdf. data already collected, the lack of funds to support COMPARATIVE AND HISTORICAL RESEARCH 369 expensive data collection is no reason for not do- After describing some major instances of coming good and useful social research. parative and historical research, past and present, The availability of existing statistics makes it this section discusses some of the key elements of possible to create some fairly sophisticated mea- this method. sures. The accompanying box, “Suffering around the World,” describes an analysis published by the Examples of Comparative Population Crisis Committee based on the kinds of and Historical Research data available in government practice. Let’s move now from an inherently quantitative August Comte, who coined the term sociologie, method to one that is typically qualitative: com- saw that new discipline as the fi nal stage in a his- parative and historical research. torical development of ideas. With his broadest brush, he painted an evolutionary picture that took COMPARATIVE AND humans from a reliance on religion to metaphysics HISTORICAL RESEARCH to science. With a fi ner brush, he portrayed science as evolving from the development of biology and the other natural sciences to the development of Comparative and historical research is the ex- psychology and, fi nally, to the development of sci- amination of societies (or other social units) over entifi c sociology. time and in comparison with one another. It differs A great many later social scientists have also substantially from the methods discussed so far, turned their attention to broad historical processes. though it overlaps somewhat with fi eld research, Several have examined the historical progression content analysis, and the analysis of existing sta- of social forms from the simple to the complex, tistics. This type of research involves the use of his- from rural-agrarian to urban-industrial societies. torical methods by sociologists, political scientists, The U.S. anthropologist Lewis Morgan, for exam- and other social scientists. ple, saw a progression from “savagery” to “barba- The discussion of longitudinal research designs rism” to “civilization” (1870). Robert Redfi eld, an- in Chapter 4 notwithstanding, our examination other anthropologist, more recently wrote of a shift of research methods so far has focused primar- from “folk society” to “urban society” (1941). Emile ily on studies anchored in one point in time and Durkheim saw social evolution largely as a process in one locale, whether a small group or a nation. of ever-greater division of labor ([1893] 1964). In a Although accurately portraying the main thrust more specifi c analysis, Karl Marx examined eco- of contemporary social scientifi c research, this nomic systems progressing historically from primi- focus conceals the fact that social scientists are tive to feudal to capitalistic forms ([1867] 1967). All also interested in tracing the development of so- history, he wrote in this context, was a history of cial forms over time and comparing those develop- class struggle—the “haves” struggling to maintain mental processes across cultures. James Mahoney their advantages and the “have-nots” struggling and Dietrich Rueschemeyer (2003:4) suggest that for a better lot in life. Looking beyond capitalism, current comparative and historical researchers Marx saw the development of socialism and fi nally “focus on a wide range of topics, but they are communism. united by a commitment to providing historically Not all historical studies in the social sciences grounded explanations of large-scale and substan- have had this evolutionary fl avor, however. Some tively important outcomes.” Thus, you fi nd comparative and historical studies dealing with social comparative and historical research The exami- class, capitalism, religion, revolution, and similar nation of societies (or other social units) over time and in topics. comparison with one another. 370 CHAPTER 11 UNOBTRUSIVE RESEARCH

INSIGHTS AND ISSUES SUFFERING AROUND THE WORLD

In 1992 the Population Crisis Committee, a non- Extreme Human Suffering proﬁ t organization committed to combating the 93—Mozambique population explosion, undertook analyzing the 92—Somalia relative degree of suffering in nations around 89—Afghanistan, Haiti, Sudan the world. Every country with a population of 88—Zaire one million or more was evaluated in terms of 87—Laos the following ten indicators—with a score of 10 86—Guinea, Angola on any indicator representing the highest level 85—Ethiopia, Uganda of adversity: 84—Cambodia, Sierra Leone 82—Chad, Guinea-Bissau Life expectancy 81—Ghana, Burma Daily per capita calorie supply 79—Malawi Percentage of the population with access to 77—Cameroon, Mauritania clean drinking water 76—Rwanda, Vietnam, Liberia Proportion of infant immunization 75—Burundi, Kenya, Madagascar, Yemen Rate of secondary school enrollment High Human Suffering Gross national product Inﬂ ation 74—Ivory Coast Number of telephones per 1,000 people 73—Bhutan, Burkina Faso, Central African Republic Political freedom 71—Tanzania, Togo Civil rights 70—Lesotho, Mali, Niger, Nigeria 69—Guatemala, Nepal 68—Bangladesh, Bolivia, Zambia Here’s how the world’s nations ranked in terms 67—Pakistan of these indicators. Remember, high scores are 66—Nicaragua, Papua-New Guinea, Senegal, signs of overall suffering. Swaziland, Zimbabwe

social scientiﬁ c readings of the historical record, of view, which he called “ideational” and “sensate” in fact, point to grand cycles rather than to linear (1937–1940). Sorokin’s sensate point of view de- progressions. No scholar better represents this ﬁ nes reality in terms of sense experiences. The ide- view than Pitirim A. Sorokin. A participant in the ational, by contrast, places a greater emphasis on Russian Revolution of 1917, Sorokin served as sec- spiritual and religious factors. Sorokin’s reading of retary to Prime Minister Kerensky. Both Kerensky the historical record further indicated that the pas- and Sorokin fell from favor, however, and Sorokin sage between the ideational and sensate occurred began his second career—as a U.S. sociologist. through a third point of view, which he called the Whereas Comte read history as a progression “idealistic.” This third view combined elements of from religion to science, Sorokin suggested that the sensate and ideational in an integrated, ratio- societies alternate cyclically between two points nal view of the world. COMPARATIVE AND HISTORICAL RESEARCH 371

65—Iraq 38—Cuba, Panama 64—Gambia, Congo, El Salvador, 37—Chile, Uruguay, North Korea Indonesia, Syria 34—Costa Rica, South Korea, United 63—Comores, India, Paraguay, Peru Arab Emirates 62—Benin, Honduras 33—Poland 61—Lebanon, China, Guyana, South Africa 32—Bulgaria, Hungary, Qatar 59—Egypt, Morocco 31—Soviet Union (former) 58—Ecuador, Sri Lanka 29—Bahrain, Hong Kong, Trinidad and Tobago 57—Botswana 28—Kuwait, Singapore 56—Iran 25—Czechoslovakia, Portugal, Taiwan 55—Suriname 54—Algeria, Thailand Minimal Human Suffering 53—Dominican Republic, Mexico, 21—Israel Tunisia, Turkey 19—Greece 51—Libya, Colombia, Venezuela 16—United Kingdom 50—Brazil, Oman, Philippines 12—Italy Moderate Human Suffering 11—Barbados, Ireland, Spain, Sweden 8—Finland, New Zealand 49—Solomon Islands 7—France, Iceland, Japan, Luxembourg 47—Albania 6—Austria, Germany 45—Vanuatu 5—United States 44—Jamaica, Romania, Saudi Arabia, Seychelles, Yugoslavia (former) 4—Australia, Norway 43—Mongolia 3—Canada, Switzerland 41—Jordan 2—Belgium, Netherlands 40—Malaysia, Mauritius 1—Denmark 39—Argentina

These examples indicate some of the topics showed that a function of European churches was comparative and historical researchers have ex- to justify and support the capitalist status quo—re- amined. To get a better sense of what this sort of ligion was a tool of the powerful in maintaining research entails, let’s look at a few examples in their dominance over the powerless. “Religion is somewhat more detail. the sigh of the oppressed creature,” Marx wrote in a famous passage, “the sentiment of a heartless Weber and the Role of Ideas In his analysis world, and the soul of soulless conditions. It is the of economic history, Marx put forward a view of opium of the people” (Bottomore and Rubel [1843] economic determinism. That is, he postulated that 1956:27). economic factors determined the nature of all other Max Weber, a German sociologist, disagreed. aspects of society. For example, Marx’s analysis Without denying that economic factors could and 372 CHAPTER 11 UNOBTRUSIVE RESEARCH did affect other aspects of society, Weber argued vestment of capital—strengthening his conclusion that economic determinism did not explain every- about the role of Protestantism in that regard. thing. Indeed, Weber said, economic forms could come from noneconomic ideas. In his research in Japanese Religion and Capitalism Weber’s the- the sociology of religion, Weber examined the ex- sis regarding Protestantism and capitalism has tent to which religious institutions were the source become a classic in the social sciences. Not sur- of social behavior rather than mere refl ections of prisingly, other scholars have attempted to test it economic conditions. His most noted statement of in other historical situations. No analysis has been this side of the issue is found in The Protestant Ethic more interesting, however, than Robert Bellah’s and the Spirit of Capitalism ([1905] 1958). Here’s a examination of the growth of capitalism in Japan brief overview of Weber’s thesis. during the late nineteenth and early twentieth cen- John Calvin (1509–1564), a French theologian, turies, entitled Tokugawa Religion (1957). played an important role in the Protestant refor- As both an undergraduate and a graduate stu- mation of Christianity. Calvin taught the doctrine dent, Bellah had developed interests in Weber and of predestination: that the ultimate salvation or in Japanese society. Given these two interests, it damnation of every individual had already been was perhaps inevitable that he would, in 1951, decided by God. Calvin also suggested that God fi rst conceive his Ph.D. thesis topic as “nothing less communicated his decisions to people by making than an ‘Essay on the Economic Ethic of Japan’ to them either successful or unsuccessful during their be a companion to Weber’s studies of China, In- earthly existence. God gave each person an earthly dia, and Judaism: The Economic Ethic of the World “calling”—an occupation or profession—and mani- Religions” (recalled in Bellah 1967:168). Originally, fested their success or failure through that medium. Bellah sketched his research design as follows: Ironically, this point of view led Calvin’s followers to seek proof of their coming salvation by working Problems would have to be specifi c and lim- hard, saving their money, and generally striving for ited—no general history would be attempted— economic success. since time span is several centuries. Field work In Weber’s analysis, Calvinism provided an im- in Japan on the actual economic ethic practiced portant stimulus for the development of capital- by persons in various situations, with, if pos- ism. Rather than “wasting” their money on worldly sible, controlled matched samples from the U.S. comforts, the Calvinists reinvested it in their eco- (questionnaires, interviews, etc.). — (1967:168) nomic enterprises, thus providing the capital nec- Bellah’s original plan, then, called for surveys of essary for the development of capitalism. In arriv- contemporary Japanese and Americans. However, ing at this interpretation of the origins of capitalism, he did not receive the fi nancial support necessary Weber researched the offi cial doctrines of the early for the study as originally envisioned. So instead Protestant churches, studied the preachings of Cal- he immersed himself in the historical records of vin and other church leaders, and examined other Japanese religion, seeking the roots of the rise of relevant historical documents. capitalism in Japan. In three other studies, Weber conducted de- Over the course of several years’ research, Bel- tailed historical analyses of Judaism ([1934] 1952) lah uncovered numerous leads. In a 1952 term and the religions of China ([1934] 1951) and India paper on the subject, Bellah felt he had found the ([1934] 1958). Among other things, Weber wanted answer in the samurai code of Bushido and in the to know why capitalism had not developed in the Confucianism practiced by the samurai class: ancient societies of China, India, and Israel. In none of the three religions did he fi nd any teaching that Here I think we fi nd a real development of this would have supported the accumulation and rein- worldly asceticism, at least equaling anything COMPARATIVE AND HISTORICAL RESEARCH 373

found in Europe. Further, in this class the idea people, the answer to this puzzle is a matter of duty in occupation involved achievement of faith in the miraculous destiny of Christian- without traditionalistic limits, but to the limits of ity. Without debunking Christian faith, Stark one’s capacities, whether in the role of bureau- looks for a scientifi c explanation, undertaking crat, doctor, teacher, scholar, or other role open an analysis of existing historical records that to the Samurai. — (QUOTED IN BELLAH 1967:171) sketch out the population growth of Christian- ity during its early centuries. He notes, among The samurai, however, made up only a portion other things, that the early growth rate of of Japanese society. So Bellah kept looking at the Christianity, rather than being unaccountably religions among Japanese generally. His under- rapid, was very similar to the contemporary standing of the Japanese language was not yet growth of Mormonism. He then goes on to very good, but he wanted to read religious texts in examine elements in early Christian prac- the original. Under these constraints and experi- tice that gave it growth advantages over the encing increased time pressure, Bellah decided to predominant paganism of the Roman Empire. concentrate his attention on a single group: Shin- For example, the early Christian churches were gaku, a religious movement among merchants in friendlier to women than paganism was, and the eighteenth and nineteenth centuries. He found much of the early growth occurred among that Shingaku had two infl uences on the develop- women—who often converted their husbands ment of capitalism. It offered an attitude toward later on. And in an era of deadly plagues, the work similar to the Calvinist notion of a “calling,” early Christians were more willing to care for and it had the effect of making business a more stricken friends and family members, which acceptable calling for Japanese. Previously, com- not only enhanced the survival of Christians merce had had a very low standing in Japan. but also made it a more attractive conversion In other aspects of his analysis, Bellah examined prospect. At every turn in the analysis, Stark the religious and political roles of the Emperor and makes rough calculations of the demographic the economic impact of periodically appearing em- impact of cultural factors. This study illustrates peror cults. Ultimately, Bellah’s research pointed to how social research methods can shed light on the variety of religious and philosophical factors nonscientifi c realms such as faith and religion. that laid the groundwork for capitalism in Japan. • International Policing: Mathieu Defl em (2002) It seems unlikely that he would have achieved set out to learn how contemporary systems of anything approaching that depth of understanding international cooperation among police agen- if he had been able to pursue his original plan to cies came about. While all of us have heard interview matched samples of U.S. and Japanese movie and TV references to Interpol, Defl em citizens. went back to the middle of the nineteenth I’ve presented these two studies in some depth century and traced developments through to demonstrate the way comparative and historical World War II. In part, his analysis examines the researchers dig down into the variables relevant strains between the bureaucratic integration to their analyses. Here are a few briefer examples of police agencies in their home governments to illustrate some of the topics interesting to com- and the need for independence from those parative and historical scholars today. governments. • The Rise of Christianity: Rodney Stark (1997) • Understanding America: Charles Perrow (2002) lays out his research question in the book’s wanted to understand the roots of the uniquely subtitle: How the Obscure, Marginal Jesus Move- American form of capitalism. Compared with ment Became the Dominant Religious Force in European nations, the United States has the Western World in a Few Centuries. For many shown less interest in providing for the needs 374 CHAPTER 11 UNOBTRUSIVE RESEARCH

of average citizens and has granted greater power to gigantic corporations. Perrow feels IN THE REAL WORLD the die was pretty much cast by the end of RESEARCH METHODS the nineteenth century, resting primarily on IN YOUR DAILY LIFE Supreme Court decisions in favor of corporations and on the experiences of the textile and The various techniques introduced in this railroad industries. chapter relate to a variety of skills that can • American Democracy: Theda Skocpol (2003) be useful to nonresearch activities. As a turns her attention to something that fas- method of social research, content analysis cinated Alexis de Tocqueville in his 1840 involves the rigorous analysis—either quali- Democracy in America: the grassroots commit- tative or quantitative—of recorded commu- ment to democracy in all aspects of American nications. More generally construed, how- community life. It almost seemed as though ever, content analysis is an everyday task for democratic decision making was genetic in professionals ranging from literary critics to the new world, but what happened? Skocpol’s lawyers. Looking beneath the surface of hu- analysis of contemporary U.S. culture sug- man communications to discover meanings gests a “diminished democracy” that cannot be and/or patterns is something you’ll be doing easily explained by the ideologies of either the all your life in many different contexts. right or the left. Similarly, you’ll frequently fi nd yourself examining compiled statistics—in relation These examples of comparative and historical to investments, household or offi ce budgets, research should give you some sense of the poten- batting averages, and many other activities tial power of the method. The box “Research Meth- that get tabulated. And you’ll be called on to ods in Your Daily Life” underscores this discussion. review and understand the history of events Let’s turn now to an examination of the sources in your personal life or profession. Or if you and techniques used in this approach. fi nd yourself debating whether a particular government misadventure is simply a repeat Sources of Comparative of earlier misadventures, you may fi nd your- and Historical Data self refl ecting on the skills of comparative and historical analysis. As we saw in the case of existing statistics, there is no end of data available for analysis in historical research. To begin, historians may already have reported on whatever it is you want to examine, they examined letters written by the immigrants and their analyses can give you an initial ground- to their families in Poland. (They obtained the let- ing in the subject, a jumping-off point for more in- ters through newspaper advertisements.) Other re- depth research. searchers have analyzed old diaries. Such personal Most likely you’ll ultimately want to go beyond documents merely scratch the surface, however. In others’ conclusions and examine some “raw data” discussing procedures for studying the history of to draw your own conclusions. These data vary, family life, Ellen Rothman points to the following of course, according to the topic under study. In sources: Bellah’s study of Tokugawa religion, raw data included the sermons of Shingaku teachers. When In addition to personal sources, there are public W. I. Thomas and Florian Znaniecki (1918) studied records which are also revealing of family his- the adjustment process for Polish peasants coming tory. Newspapers are especially rich in evidence to the United States in the early twentieth century, on the educational, legal, and recreational COMPARATIVE AND HISTORICAL RESEARCH 375

aspects of family life in the past as seen from slaves. Later court cases and laws clarifi ed the a local point of view. Magazines refl ect more matter—holding African Americans to be some- general patterns of family life; students often thing less than human. fi nd them interesting to explore for data on per- The sources of data for historical analysis are ceptions and expectations of mainstream family too extensive to cover even in outline here, though values. Magazines offer several different kinds the few examples we’ve looked at should suggest of sources at once: visual materials (illustrations some ideas. Whatever resources you use, how- and advertisements), commentary (editorial and ever, a couple of cautions are in order. advice columns), and fi ction. Popular periodicals As we saw in the case of existing statistics, you are particularly rich in the last two. Advice on can’t trust the accuracy of records—offi cial or un- many questions of concern to families—from the offi cial, primary or secondary. Your protection lies proper way to discipline children to the econom- in replication: In the case of historical research, ics of wallpaper—fi lls magazine columns from that means corroboration. If several sources point the early nineteenth century to the present. to the same set of “facts,” your confi dence in them Stories that suggest common experiences or might reasonably increase. perceptions of family life appear with the same At the same time, you need always be wary of continuity. — (1981:53) bias in your data sources. If all your data on the development of a political movement are taken Organizations generally document themselves, from the movement itself, you’re unlikely to gain so if you’re studying the development of some or- a well-rounded view of it. The diaries of well-to- ganization—as Bellah studied Shingaku, for exam- do gentry of the Middle Ages may not give you ple—you should examine its offi cial documents: an accurate view of life in general during those charters, policy statements, speeches by leaders, times. Where possible, obtain data from a variety and so on. Once when I was studying the rise of a of sources representing different points of view. contemporary Japanese religious group—Sokagak- Here’s what Bellah said regarding his analysis of kai—I discovered not only weekly newspapers and Shingaku: magazines published by the group but also a published collection of all the speeches given by the One could argue that there would be a bias in original leaders. With these sources, I could trace what was selected for notice by Western schol- changes in recruitment patterns over time. At the ars. However, the fact that there was material outset, followers were enjoined to enroll all the from Western scholars with varied interests world. Later, the emphasis shifted specifi cally to from a number of countries and over a period of Japan. Once a sizable Japanese membership had nearly a century reduced the probability of bias. been established, an emphasis on enrolling all the — (1967:179) world returned (Babbie 1966). Often, offi cial government documents provide The issues raised by Bellah are important ones. the data needed for analysis. To better appreciate As Ron Aminzade and Barbara Laslett indicate the history of race relations in the United States, A. in the box “Reading and Evaluating Documents,” Leon Higginbotham, Jr. (1978) examined 200 years there is an art to knowing how to regard such doc- of laws and court cases involving race. The fi rst uments and what to make of them. African American appointed as a federal judge, Incidentally, the critical review that Aminzade Higginbotham found that the law, rather than and Laslett urge for the reading of historical docu- protecting African Americans, embodied bigotry ments is useful in many areas of your life besides and oppression. In the earliest court cases, there the pursuit of comparative and historical research. was considerable ambiguity over whether African Consider applying some of their questions to presi- Americans were indentured servants or, in fact, dential press conferences, advertising, or (gasp) 376 CHAPTER 11 UNOBTRUSIVE RESEARCH

Text not available due to copyright restrictions

college textbooks. None of these offers a direct are no easily listed steps to follow in the analysis view of reality; all have human authors and human of historical data. Nevertheless, a few comments subjects. are in order. Max Weber used the German term verste- Analytical Techniques hen—“understanding”—in reference to an essential quality of social research. He meant that the The analysis of comparative and historical data is researcher must be able to take on, mentally, the another large subject that I can’t cover exhaustively circumstances, views, and feelings of those being here. Moreover, because comparative and histori- studied, so that the researcher can interpret their cal research is usually a qualitative method, there actions appropriately. Certainly this concept ap- COMPARATIVE AND HISTORICAL RESEARCH 377

REAU CY BU CR A

Potential Problems

DDivisionivision Inefficiency ooff llaborabor and rigidity

HHierarchyierarchy Resistance to change ofof authorityauthority

Perpetuation of race, RulesRules andand regulationsregulations class, and gender inequalities

EEmploymentmployment bbasedased oonn ttechnicalechnical qqualificationualification

IImpersonalitympersonality

plies to comparative and historical research. The bureaucracy. Having observed numerous actual researcher’s imaginative understanding is what bureaucracies, Weber ([1925] 1946) detailed those breathes life and meaning into the evidence being qualities essential to bureaucracies in general: ju- analyzed. risdictional areas, hierarchically structured author- The comparative and historical researcher ity, written fi les, and so on. Weber did not merely must fi nd patterns among the voluminous details list those characteristics common to all the actual describing the subject matter of study. Often, this bureaucracies he observed. Rather, to create a the- takes the form of what Weber called ideal types: oretical model of the “perfect” (ideal type) bureau- conceptual models composed of the essential cracy, he needed to understand fully the essen- characteristics of social phenomena. Thus, for ex- tials of bureaucratic operation. Figure 11-4 offers ample, Weber himself did considerable research on a more recent, graphic portrayal of some positive 378 CHAPTER 11 UNOBTRUSIVE RESEARCH and negative aspects of bureaucracy as a general assumption that history as documented necessar- social phenomenon. ily coincides with what actually happened. Often, comparative and historical research is informed by a particular theoretical paradigm. Thus, Marxist scholars may undertake historical ETHICS AND UNOBTRUSIVE MEASURES analyses of particular situations—such as the history of Latinos and Latinas in the United States—to determine whether they can be understood in terms The use of unobtrusive measures avoids many of of the Marxist version of confl ict theory. Some- the ethical issues we’ve discussed in connection times comparative and historical researchers at- with other data-collection techniques. However, tempt to replicate prior studies in new situations— if you refl ect on the general principles we’ve dis- for example, Bellah’s study of Tokugawa religion cussed, I think you’ll see potential risks that rein the context of Weber’s studies of religion and searchers need to avoid in such measures. economics. The general principle of confi dentiality may be Although comparative and historical research relevant in some projects, for example. Let’s sup- is often regarded as a qualitative technique, quan- pose you want to examine an immigrant subculture titative methods can work in certain cases. For through a content analysis of letters written back example, historical analysts sometimes use time- to the old country, as in the study of Polish peas- series data to monitor changing conditions over time, such as data on population, crime rates, unemployment, and infant mortality rates. The analysis of such data sometimes requires sophistica- WHAT DO YOU THINK? REVISITED tion, however. For example, Larry Isaac and Larry Griffi n (1989) discuss the uses of a variation on regression in determining the meaningful breaking Unobtrusive research techniques allow re- points in historical processes as well as for specify- searchers to avoid having an effect on what ing the periods within which certain relationships is being studied. Given that, why don’t so- occur among variables. Criticizing the tendency to cial scientists limit themselves to these regard history as a steadily unfolding process, the techniques? authors focus their attention on the statistical rela- As we’ve seen, each of the unobtrusive tionship between unionization and the frequency techniques presented in this chapter has of strikes, demonstrating that the relationship has shortcomings of its own. The most general shifted importantly over time. is that we may not be able to fi nd existing Isaac and Griffi n raise several important issues statistics, recorded communications, or his- regarding the relationship among theory, research torical records that provide valid and reliable methods, and the “historical facts” they address. data relevant to the topic we wish to study. Their analysis, once again, warns against the naive Other techniques, such as experiments, surveys, and fi eld research, allow us to generate original data to fi ll such voids. The ideal approach is to use multiple The American Sociological Association’s techniques, including unobtrusive ones. The section on Comparative and Historical So- ciology has an excellent website presenting use of multiple approaches to research can examples of this research approach as well substantiate our fi ndings when they agree, as resources for original research. Go to and they can broaden our understanding of http://www2.asanet.org/sectionchs/. the subject matter when they do not agree. KEY TERMS 379 ants mentioned earlier in the chapter (Thomas and your fi ndings honestly, with the purpose of discov- Znaniecki 1918). To begin, you should obtain those ering what’s so rather than attempting to support a letters legally and ethically (no getting a govern- favored hypothesis or personal agenda. Although ment agency to intercept the letters for you), and agreeing with such a principle may be easy, ap- you need to protect the privacy of the letter writers plying it when you actually conduct research may and recipients. prove somewhat more diffi cult. Your ethical sen- As with all other research techniques, you are sibilities will be challenged more by the vast gray obliged to collect data, analyze them, and report areas than by the black-and-white ones.

Main Points Analyzing Existing Statistics ❏ A variety of government and nongovernment Introduction agencies provide aggregate statistical data for ❏ Unobtrusive measures are ways of study- studying aspects of social life. ing social behavior without affecting it in the ❏ Problems of validity in the analysis of existing process. statistics can often be handled through logical reasoning and replication. Content Analysis ❏ Because existing statistics often generate prob- ❏ Content analysis is a social research method lems of reliability, researchers must use them appropriate for studying human communica- with caution. tions. Researchers can use it to study not only communication processes but other aspects of Comparative and Historical Research social behavior as well. ❏ Social scientists use comparative and historical ❏ Common units of analysis in content analy- methods to discover patterns in the histories of sis include units of communication—words, different cultures. paragraphs, books, and so forth. Standard ❏ Although often regarded as a qualitative probability sampling techniques are sometimes method, comparative and historical research appropriate in content analysis. can make use of quantitative techniques. ❏ Content analysis involves coding—transforming raw data into categories based on some Ethics and Unobtrusive Measures conceptual scheme. Coding may attend to both ❏ Sometimes even unobtrusive measures can manifest and latent content. The determina- raise the possibility of violating subjects’ tion of latent content requires judgments on privacy. the part of the researcher. ❏ The general principles of honest observation, ❏ Both quantitative and qualitative techniques analysis, and reporting apply to all research are appropriate for interpreting content analy- techniques. sis data.

❏ The advantages of content analysis include Key Terms economy, safety, and the ability to study

processes occurring over a long time. Its coding latent content disadvantages are that it is limited to recorded comparative and historical manifest content communications and can raise issues of reli- research unobtrusive research ability and validity. content analysis 380 CHAPTER 11 UNOBTRUSIVE RESEARCH

Review Questions Website for The Basics of Social Research 1. Is the Republican or the Democratic party the 4th edition more supportive of free speech? In two or three paragraphs, outline a content analysis At the book companion website (http://sociology design to answer this question. Be sure to .wadsworth.com/babbie_basics4e) you will ﬁ nd specify units of analysis, sampling methods, many resources in addition to ThomsonNow to and relevant measurements. aid you in studying for your exams. For example, you will ﬁ nd Tutorial Quizzes with feedback, 2. Social scientists often contrast the sense of Internet Exercises, Flashcards, and Chapter Tutori- “community” in villages, small towns, and als, as well as Extended Projects, InfoTrac College neighborhoods with life in large, urban soci- Edition search terms, Social Research in Cyber- eties. What, in your opinion, are the essential space, GSS Data, Web Links, and primers for using qualities of an ideal type of community? various data analysis software such as SPSS and 3. How might you compare lifestyles in different NVivo. societies around the world, using pictures on the World Wide Web? What cultural features could you look for? How would you identify differences and similarities? Additional Readings

4. How “old” is the college or university you’re Baker, Vern, and Charles Lambert. 1990. “The National attending? When you decide on an age, Collegiate Athletic Association and the Governance specify what is that old. Is it people, buildings, of Higher Education.” Sociological Quarterly 31 (3): or something else? Cite the sources you might 403–21. A historical analysis of the factors producing use in arriving at your conclusion. Discuss and shaping the NCAA. any ambiguities that might exist in determin- Berg, Bruce L. 1989. Qualitative Research Methods for the Social Sciences. Boston: Allyn and Bacon. Con- ing the age of your college or university. tains excellent materials on unobtrusive measures, including a chapter on content analysis. While focusing on qualitative research, Berg shows the Online Study Resources logical links between qualitative and quantitative approaches. Evans, William. 1996. “Computer-Supported Content Analysis: Trends, Tools, and Techniques.” Social Sci- ence Computer Review 14 (3): 269–79. Here’s a review of current computer software for content analysis, Go to such as CETA, DICTION, INTEXT, MCCA, MECA, TEXTPACK, VBPro, and WORDLINK. http://sociology.wadsworth.com/babbie_basics4e Gould, Roger V., ed. 2000. The Rational Choice Contro- and click on ThomsonNow for access to this versy in Historical Sociology. Chicago: University of Chicago Press. A lively debate over the state of theory powerful online study tool. You will get a per- in historical research and about rational choice in sonalized study plan based on your responses to sociology in general. a diagnostic pretest. Once you have mastered Øyen, Else, ed. 1990. Comparative Methodology: Theory the material with the help of interactive learning and Practice in International Social Research. Thou- tools, you can take a posttest to conﬁ rm that you sand Oaks, CA: Sage. Here are a variety of viewpoints are ready to move on to the next chapter. on different aspects of comparative research. Appro- ADDITIONAL READINGS 381

priately, the contributors come from many different Webb, Eugene T., Donald T. Campbell, Richard D. countries. Schwartz, and Lee Sechrest. 2000. Unobtrusive Mea- U.S. Bureau of the Census. 2006. Statistical Abstract of the sures. Rev. ed. Thousand Oaks, CA: Sage. A com- United States, 2006, National Data Book and Guide to pendium of unobtrusive measures. Includes physical Sources. Washington, DC: U.S. Government Print- traces, a variety of archival sources, and observa- ing Ofﬁ ce. This is absolutely the best book bargain tions. Good discussion of the ethics involved and the available (present company excluded). Although limitations of such measures. the hundreds of pages of tables of statistics do not Weber, Robert Philip. 1990. Basic Content Analysis. Thou- provide exciting bedtime reading—the plot is a little sand Oaks, CA: Sage. Here’s an excellent beginner’s thin—it is an absolutely essential resource volume book for the design and execution of content analy- for every social scientist. This document is now also sis. Both general issues and speciﬁ c techniques are available on CD-ROM. presented. 121 HUMANEVALUATION INQUIRY RESEARCH AND SCIENCE Wojnarowicz/The Image Works Photo credit Wojnarowicz/The

What You’ll Learn in thisThis ChapterChapter

NowWe’ll you’re examine going the to seeway one people of the learn most about rapidly their growing world anduses the of socialmistakes research: they makethe evaluation along the of way. social We’ll interventions. also begin toYou’ll see whatcome makes away from sience this different chapter from able to

otherjudge wayswhether of knowing social programs things. have succeeded or failed. 2S 1S N L 382 In this chapter . . . WHAT DO YOU THINK?

Introduction

Topics Appropriate to Evaluation Research

Formulating the Problem: Issues of Measurement Image not available due to copyright restrictions Specifying Outcomes Measuring Experimental Contexts Specifying Interventions Why is there so much continuing debate Specifying the Population over issues that straightforward social re- New versus Existing Measures search would likely resolve? For example, Operationalizing Success/Failure people still debate whether the threat of the Types of Evaluation Research Designs death penalty successfully deters murder- Experimental Designs ers. Can’t that outcome be tested once and Quasi-Experimental Designs for all? Can’t we tell yes from no, black from Qualitative Evaluations white, up from down? Logistical Problems See the “What Do You Think? Revisited” box Use of Research Results toward the end of the chapter. Social Indicators Research The Death Penalty and Deterrence Computer Simulation Tanzanian government in response to two problems facing that country: (1) a population growth Ethics and Evaluation Research rate over twice that of the rest of the world and (2) an AIDS epidemic particularly heavy along the international truck route, where more than a fourth INTRODUCTION of the truck drivers and over half the commercial sex workers were found to be HIV positive in 1991. You may not be familiar with Twende na Wakati The prevalence of contraceptive use was 11 per- (“Let’s Go with the Times”), but it’s the most popu- cent (Rogers et al. 1996:5–6). lar radio show in Tanzania. It’s a soap opera. The The purpose of the soap opera was to bring main character, Mkwaju, is a truck driver with about a change in knowledge, attitudes, and prac- some pretty traditional ideas about gender roles tices (KAP) relating to contraception and fam- and sex. By contrast, Fundi Mitindo, a tailor, and ily planning. Rather than instituting a conven- his wife, Mama Waridi, have more modern ideas tional educational campaign, PCI felt it would be regarding the roles of men and women, particu- more effective to illustrate the message through larly in relation to the issues of overpopulation and entertainment. family planning. Between 1993 and 1995, 108 episodes of Twende Twende na Wakati was the creation of Popula- na Wakati were aired, aiming at the 67 percent of tion Communications International (PCI) and other Tanzanians who listen to the radio. Eighty-four per- organizations working in conjunction with the cent of the radio listeners reported listening to the 383 384 CHAPTER 12 EVALUATION RESEARCH

PCI soap opera, making it the most popular show in the country. Ninety percent of the show’s listen- IN THE REAL WORLD ers recognized Mkwaju, the sexist truck driver, and SOAP OPERA RESEARCH SUCCESS only three percent regarded him as a positive role model. Over two-thirds identifi ed Mama Waridi, a The research evaluating the soap operas businesswoman, and her tailor husband as posi- produced in Tanzania serves many practi- tive role models. cal functions. To begin, it tells the producers Surveys conducted to measure the impact of whether they have been successful in deliv- the show indicated that it had affected knowledge, ering each of their messages. These data can attitudes, and behavior. For example, 49 percent of help them fi ne-tune their presentations and the married women who listened to the show said make it easier to promote similar programs. they now practiced family planning, compared Soap operas promoting small families, safe with only 19 percent of the nonlisteners. There sex, and the liberation of women have been were other impacts: produced in several other countries in Africa as well as Asia and Latin America, and the Some 72 percent of the listeners in 1994 said list is still growing. that they adopted an HIV/AIDS prevention behavior because of listening to “Twende na Wakati,” and this percentage increased to 82 percent in our 1995 survey. Seventy-seven search points to a more general trend in the social percent of these individuals adopted monogamy, sciences. As a researcher, you’ll likely be asked to 16 percent began using condoms, and 6 per- conduct evaluations of your own. cent stopped sharing razors and/or needles. In part, the growth of evaluation research re- — (ROGERS ET AL. 1996:21) fl ects social scientists’ increasing desire to make a difference in the world. At the same time, we can’t We can judge the effectiveness of the soap opera discount the infl uence of (1) an increase in federal because of a particular form of social science. Eval- requirements that program evaluations must ac- uation research refers to a research purpose rather company the implementation of new programs than a specifi c research method. This purpose is and (2) the availability of research funds to fulfi ll to evaluate the impact of social interventions: new those requirements. In any case, it seems clear that teaching methods, innovations in parole, and a social scientists will be bringing their skills into the host of others. (See the box “Soap Opera Research real world more than ever before. Success” to see some of the practical functions of This chapter looks at some of the key elements evaluation research in our example.) Many meth- in this form of social research. After considering ods—surveys, experiments, and so on—can be the kinds of topics commonly subjected to evalu- used in evaluation research. ation, we’ll move through some of its main opera- Evaluation research is probably as old as social tional aspects: measurement, study design, and research itself. Whenever people have instituted execution. As you’ll see, formulating questions is a social reform for a specifi c purpose, they have as important as answering them. Because it occurs paid attention to its actual consequences, even if within real life, evaluation research has special they have not always done so in a conscious, de- problems, some of which we’ll examine. Besides liberate, or sophisticated fashion. In recent years, logistical problems, special ethical issues arise however, the fi eld of evaluation research has be- from evaluation research generally and in its spe- come an increasingly popular and active research cifi c, technical procedures. As you review reports specialty, as refl ected in textbooks, courses, and of program evaluations, you should be especially projects. Moreover, the growth of evaluation re- sensitive to these problems. TOPICS APPROPRIATE TO EVALUATION RESEARCH 385

Evaluation is a form of applied research—that The intent of evaluation research takes many is, it’s intended to have some real-world effect. It forms. Needs assessment studies aim to determine will be useful, therefore, to consider whether and the existence and extent of problems, typically how it’s actually applied. As you’ll see, the obvious among a segment of the population, such as the implications of an evaluation research project do elderly. Cost-benefi t studies determine whether the not necessarily affect real life. They may become results of a program can be justifi ed by its expense the focus of ideological, rather than scientifi c, de- (both fi nancial and other). Monitoring studies pro- bates. They may simply be denied out of hand, for vide a steady fl ow of information about something political or other reasons. Perhaps most typically, of interest, such as crime rates or the outbreak of they may simply be ignored and forgotten, left to an epidemic. Sometimes the monitoring involves collect dust in bookcases across the land. incremental interventions. Read the following de- Toward the end of this chapter, we’ll look at scription of “adaptive management” by the Nature a particular resource for large-scale evaluation— Conservancy, a public-interest group seeking to social indicators research. This type of research protect natural areas: is also a rapidly growing specialty. Essentially, it First, partners assess assumptions and set involves the creation of aggregated indicators of management goals for the conservation area. the “health” of society, similar to the economic Based on this assessment, the team takes ac- indicators that give diagnoses and prognoses of tion, then monitors the environment to see how economies. it responds. After measuring results, partners refi ne their assumptions, goals and monitoring TOPICS APPROPRIATE regimen to refl ect what they’ve learned from TO EVALUATION RESEARCH past experiences. With refi nements in place, the entire process begins again. — (2005:3)

Evaluation research is appropriate whenever some Much of evaluation research is referred to as social intervention occurs or is planned. A social program evaluation or outcome assessment: intervention is an action taken within a social con- the determination of whether a social interven- text for the purpose of producing some intended tion is producing the intended result. Here’s an result. In its simplest sense, evaluation research example. is a process of determining whether a social inter- Some years ago, a project evaluating the na- vention has produced the intended result. tion’s driver education programs, conducted by The topics appropriate to evaluation research the National Highway and Transportation Safety are limitless. When the federal government abol- Administration (NHTSA), stirred up a controversy. ished the selective service system, military re- Philip Hilts reported on the study’s ﬁ ndings: searchers began paying special attention to the For years the auto insurance industry has given impact on enlistment. As individual states have large insurance discounts for children who take liberalized their marijuana laws, researchers have drivers’ education courses, because statistics sought to learn the consequences, both for mari- show that they have fewer accidents. juana use and for other forms of social behavior. Do no-fault divorce reforms increase the number of divorces, and do related social problems de- evaluation research Research undertaken for the purpose of determining the impact of some social crease? Has no-fault automobile insurance really intervention, such as a program aimed at solving a social brought down insurance policy premiums? Agen- problem. cies providing foreign aid conduct evaluations program evaluation/outcome assessment The to determine whether the desired effects were determination of whether a social intervention is produc- produced. ing the intended result. 386 CHAPTER 12 EVALUATION RESEARCH

The preliminary results of a new major study, pact within Hungarian society. One conclusion was however, indicate that drivers’ education does that “the second economy, which earlier probably not prevent or reduce the incidence of traffi c ac- tended to diminish income inequalities or at least cidents at all. — (HILTS 1981:4) improved the standard of living of the poorest part of the population, in the 1980s increasingly con- Based on an analysis of 17,500 young people tributed to the growth of inequalities” (Andorka in DeKalb County, Georgia (including Atlanta), the 1990:111). preliminary fi ndings indicate that students who As you can see, the questions appropriate to took drivers’ education had just as many accidents evaluation research are of great practical signifi - and traffi c violations as those who didn’t. The cance: Jobs, programs, and investments as well as study also seemed to reveal some subtle aspects beliefs and values are at stake. Let’s now examine of driver training. how these questions are answered—how evalua- First, it suggested that the apparent impact of tions are conducted. driver education was largely a matter of self-selection. The kind of students who took drivers’ education were less likely to have accidents and FORMULATING THE PROBLEM: traffi c violations—with or without driver training. ISSUES OF MEASUREMENT Students with high grades, for example, were more likely to sign up for driver training, and they were also less likely to have accidents. Several years ago, I headed an institutional re- More startling, however, was the suggestion search offi ce that conducted research directly rel- that driver training courses may have actually in- evant to the operation of the university. Often, we creased traffi c accidents! The existence of drivers’ were asked to evaluate new programs in the cur- education may encourage some students to get riculum. The following description shows the prob- their licenses earlier than if there were no such lem that arose in that context, and it points to one courses. In a study of ten Connecticut towns that of the key barriers to good evaluation research. discontinued driver training, about three-fourths Faculty members would appear at my offi ce to of those who probably would have been licensed say they’d been told by the university administra- through their classes delayed getting licenses until tion to arrange for an evaluation of the new pro- they were 18 or older (Hilts 1981:4). gram they had permission to try. This points to a As you might imagine, these results were not common problem: Often the people whose pro- well received by those most closely associated grams are being evaluated aren’t thrilled at the with driver training. This matter was complicated, prospect. For them, an independent evaluation moreover, by the fact that the NHTSA study was threatens the survival of the programs and per- also evaluating a new, more intensive training pro- haps even their jobs. gram—and the preliminary results showed that the The main problem I want to introduce, how- new program was effective. ever, has to do with the purpose of the interven- Here’s a very different example of evaluation tion to be evaluated. The question “What is the research. Rudolf Andorka, a Hungarian sociolo- intended result of the new program?” often pro- gist, has been particularly interested in his coun- duced a vague response such as “Students will get try’s shift to a market economy. Even before the an in-depth and genuine understanding of math- dramatic events in Eastern Europe in 1989, An- ematics, instead of simply memorizing methods of dorka and his colleagues had been monitoring the calculations.” Fabulous! And how could we mea- nation’s “second economy”—jobs pursued outside sure that “in-depth and genuine understanding”? the socialist economy. Their surveys followed the Often, I was told that the program aimed at pro- rise and fall of such jobs and examined their im- ducing something that could not be measured by FORMULATING THE PROBLEM: ISSUES OF MEASUREMENT 387 conventional aptitude and achievement tests. No The cultural biases inherent in how middle-class problem there; that’s to be expected when we’re white researchers interpret the experiences of innovating and being unconventional. What would low-income minorities may lead to erroneous be an unconventional measure of the intended re- assumptions and faulty propositions concerning sult? Sometimes this discussion came down to an causal relationships, to invalid social theory, and assertion that the effects of the program would be consequently to invalid program theory. De- “unmeasurable.” scriptive theories derived from faulty premises, There’s the common rub in evaluation re- which have been legitimized in the literature as search: measuring the “unmeasurable.” Evaluation existing knowledge, may have negative conse- research is a matter of fi nding out whether some- quences for program participants. — (1992:38) thing is there or not there, whether something In setting up an evaluation, then, research- happened or didn’t happen. To conduct evaluation ers must pay careful attention to issues of mea- research, we must be able to operationalize, ob- surement. Let’s take a closer look at the types of serve, and recognize the presence or absence of measurements that evaluation researchers must what is under study. deal with. Often, outcomes can be derived from published program documents. Thus, when Edward Howard and Darlene Norman (1981) evaluated Specifying Outcomes the performance of the Vigo County Public Library As I’ve already suggested, a key variable for eval- in Indiana, they began with the statement of pur- uation researchers to measure is the outcome, pose previously adopted by the library’s Board of or what is called the response variable. If a social Trustees: program is intended to accomplish something, To acquire by purchase or gift, and by recording we must be able to measure that something. If we and production, relevant and potentially useful want to reduce prejudice, we need to be able to information that is produced by, about, or for the measure prejudice. If we want to increase marital citizens of the community; harmony, we need to be able to measure that. To organize this information for effi cient Achieve agreements on defi nitions in advance delivery and convenient access, furnish the is essential: equipment necessary for its use, and provide The most diffi cult situation arises when there assistance in its utilization; and is disagreement as to standards. For example, To effect maximum use of this information many parties may disagree as to what defi nes toward making the community a better place serious drug abuse—is it defi ned best as 15% in which to live through aiding the search for or more of students using drugs weekly, 5% or understanding by its citizens. — (1981:306) more using hard drugs such as cocaine or PCP As the researchers said, “Everything that VCPL monthly, students beginning to use drugs as does can be tested against the Statement of Pur- young as seventh grade, or some combination pose.” They then set about creating operational of the dimensions of rate of use, nature of use, measures for each of the purposes. and age of user? . . . Applied researchers should, Although “offi cial” purposes of interventions to the degree possible, attempt to achieve are often the key to designing an evaluation, these consensus from research consumers in advance may not always suffi ce. Anna-Marie Madison of the study (e.g., through advisory groups) or (1992), for example, warns that programs designed at least ensure that their studies are able to to help disadvantaged minorities do not always reproduce data relevant to the standards posited fl ect what the proposed recipients of the aid may by all potentially interested parties. — (HEDRICK, need and desire: BICKMAN, AND ROG 1993:27) 388 CHAPTER 12 EVALUATION RESEARCH

In some cases you may fi nd that the defi nitions important to measure those aspects of the context of a problem and a suffi cient solution are defi ned of an experiment that researchers think might af- by law or agency regulations; if so, you must be fect the experiment. Though external to the experi- aware of such specifi cations and accommodate ment itself, some variables may affect it. them. Moreover, whatever the agreed-on defi ni- Suppose, for example, that you were conduct- tions, you must also achieve agreement on how ing an evaluation of a program aimed at training the measurements will be made. Because there unskilled people for employment. The primary out- are different possible methods for estimating the come measure would be their success at gaining percentage of students “using drugs weekly,” for employment after completing the program. You example, you would have to be sure that all the would, of course, observe and calculate the sub- parties involved understood and accepted the jects’ employment rate, but you should also de- method(s) you had chosen. termine what has happened to the employment/ In the case of the Tanzanian soap opera, there unemployment rates of society at large during were several outcome measures. In part, the pur- the evaluation. A general slump in the job market pose of the program was to improve knowledge should be taken into account in assessing what about both family planning and AIDS. Thus, for might otherwise seem a pretty low employment example, one show debunked the belief that the rate for subjects. Or, if all the experimental sub- AIDS virus was spread by mosquitoes and could jects get jobs following the program, you should be avoided by the use of insect repellant. Studies of consider any general increase in available jobs. listeners showed a reduction in that belief (Rogers Combining complementary measures with proper et al. 1996:21). control-group designs should allow you to pinpoint PCI also wanted to change Tanzanian attitudes the effects of the program you’re evaluating. toward family size, gender roles, HIV/AIDS, and other related topics; the research indicated that the Specifying Interventions show had affected these as well. Finally, the program aimed at affecting behavior. We’ve already Besides making measurements relevant to the seen that radio listeners reported changing their outcomes of a program, researchers must mea- behavior with regard to AIDS prevention. They sure the program intervention—the experimental reported a greater use of family planning as well. stimulus. In part, this measurement will be handled However, because there’s always the possibility of by the assignment of subjects to experimental and a gap between what people say they do and what control groups, if that’s the research design. As- they actually do, the researchers sought indepen- signing a person to the experimental group is the dent data to confi rm their conclusions. same as scoring that person “yes” on the stimulus, Tanzania’s national AIDS-control program had and assignment to the control group represents a been offering condoms free of charge to citizens. In score of “no.” In practice, however, it’s seldom that the areas covered by the soap opera, the number simple. of condoms given out increased sixfold between Let’s stick with the job-training example. Some 1992 and 1994. This far exceeded the increase of people will participate in the program; others will 1.4 times in the control area, where broadcasters not. But imagine for a moment what job-training did not carry the soap opera. programs are probably like. Some subjects will participate fully; others will miss a lot of sessions Measuring Experimental Contexts or fool around when they are present. So we may need measures of the extent or quality of partici- Measuring the dependent variables directly in- pation in the program. If the program is effective, volved in the experimental program is only a be- you should fi nd that those who participated fully ginning. As Henry Riecken and Robert Boruch have higher employment rates than do those who (1974:120–21) point out, it’s often appropriate and participated less. FORMULATING THE PROBLEM: ISSUES OF MEASUREMENT 389

Other factors may further confound the admin- New versus Existing Measures istration of the experimental stimulus. Suppose In providing for the measurement of these different we’re evaluating a new form of psychotherapy kinds of variables, the researcher must continually that’s designed to cure sexual impotence. Several choose whether to create new measures or to use therapists administer it to subjects composing an ones already devised by others. If a study addresses experimental group. We plan to compare the re- something that’s never been measured before, the covery rate of the experimental group with that of choice is easy. If it addresses something that oth- a control group, which receives some other ther- ers have tried to measure, the researcher will need apy or none at all. It may be useful to include the to evaluate the relative worth of various existing names of the therapists treating specifi c subjects measurement devices in terms of her or his specifi c in the experimental group, because some may be research situations and purpose. Recall that this is more effective than others. If this turns out to be the a general issue in social research that applies well case, we must fi nd out why the treatment worked beyond evaluation research. Let’s examine briefl y better for some therapists than for others. What we the advantages and disadvantages of creating new learn will further develop our understanding of the measures versus using existing ones. therapy itself. Creating measurements specifi cally for a study can offer greater relevance and validity. If the psy- Specifying the Population chotherapy we’re evaluating aims at a specifi c aspect of recovery, we can create measures that In evaluating an intervention, it’s important to pinpoint that aspect. We might not be able to fi nd defi ne the population of subjects for whom the any standardized psychological measures that hit program is appropriate. Ideally, all or a sample of that aspect right on the head. However, creating appropriate subjects will then be assigned to ex- our own measure will cost us the advantages to perimental and control groups as warranted by the be gained from using preexisting measures. Cre- study design. Defi ning the population, however, ating good measures takes time and energy, both can itself involve specifying measurements. If of which could be saved by adopting an existing we’re evaluating a new form of psychotherapy, it’s technique. Of greater scientifi c signifi cance, mea- probably appropriate for people with mental prob- sures that have been used frequently by other re- lems. But how will “mental problems” be defi ned searchers carry a body of possible comparisons and measured? The job-training program men- that might be important to our evaluation. If the tioned previously is probably intended for people experimental therapy raises scores by an average who are having trouble fi nding work, but what of ten points on a standardized test, we’ll be in a counts as “having trouble”? position to compare that therapy with others that Beyond defi ning the relevant population, then, have been evaluated using the same measure. Fi- the researcher should make fairly precise mea- nally, measures with a long history of use usually surements on the variables considered in the defi - have known degrees of validity and reliability, but nition. For example, even though the randomiza- newly created measures will require pretesting or tion of subjects in the psychotherapy study would will be used with considerable uncertainty. ensure an equal distribution of those with mild and severe mental problems into the experimental and control groups, we’d need to keep track of the rela- Operationalizing Success/Failure tive severity of different subjects’ problems in case the therapy turns out to be effective for only those Potentially, one of the most taxing aspects of evalu- with mild disorders. Similarly, we should measure ation research is determining whether the program such demographic variables as gender, age, race, under review succeeded or failed. The purpose of and so forth in case the therapy works only for the foreign language program mentioned earlier women, the elderly, or some other group. may be to help students better learn the language, 390 CHAPTER 12 EVALUATION RESEARCH but how much better is enough? The purpose of In summary, researchers must take measure- the conjugal visit program at a prison may be to ment quite seriously in evaluation research, care- raise morale, but how high does morale need to be fully determining all the variables to be measured raised to justify the program? and getting appropriate measures for each. As I’ve As you may anticipate, clear-cut answers to implied, however, such decisions are typically not questions like these almost never materialize. This purely scientifi c ones. Evaluation researchers often problem has surely led to what is generally called must work out their measurement strategy with cost-benefi t analysis. How much does the program the people responsible for the program being eval- cost in relation to what it returns in benefi ts? If the uated. It usually doesn’t make sense to determine benefi ts outweigh the cost, keep the program go- whether a program achieves Outcome X when its ing. If the reverse, junk it. That’s simple enough, purpose is to achieve Outcome Y. (Realize, how- and it seems to work in straightforward economic ever, that evaluation designs sometimes have the situations: If it cost you $20 to produce something purpose of testing for unintended consequences.) and you can sell it for only $18, there’s no way you There is a political aspect to these choices, also. can make up the difference in volume. Because evaluation research often affects other Unfortunately, the situations usually faced by people’s professional interests—their pet program evaluation researchers are seldom amenable to may be halted, or they may be fi red or lose profes- straightforward economic accounting. The for- sional standing—the results of evaluation research eign language program may cost the school dis- are often contested. trict $100 per student, and it may raise students’ Let’s turn now to some of the evaluation de- performances on tests by an average of 15 points. signs that researchers commonly employ. Because the test scores can’t be converted into dollars, though, no obvious ground for weighing the costs and benefi ts exists. TYPES OF EVALUATION Sometimes, as a practical matter, the criteria of RESEARCH DESIGNS success and failure can be handled through competition among programs. If a different foreign As I noted at the start of this chapter, evaluation language program costs only $50 per student and research is not itself a method, but rather one ap- produces an increase of 20 points in test scores, plication of social research methods. As such, it it would undoubtedly be considered more suc- can involve any of several research designs. Here cessful than the fi rst program—assuming that test we’ll consider three main types of research design scores were seen as an appropriate measure of that are appropriate for evaluations: experimental the purposes of both programs and the less ex- designs, quasi-experimental designs, and qualita- pensive program had no negative, unintended tive evaluations. consequences. Ultimately, the criteria of success and failure are often a matter of agreement. The people respon- Experimental Designs sible for the program may commit themselves in advance to a particular outcome that will be re- Many of the experimental designs introduced in garded as an indication of success. If that’s the Chapter 8 can be used in evaluation research. By case, all you need to do is make absolutely certain way of illustration, let’s see how the classical ex- that the research design will measure the speci- perimental model might be applied to our evalu- fi ed outcome. I mention this obvious requirement ation of the new psychotherapeutic treatment for simply because researchers sometimes fail to meet sexual impotence. it, and there’s little or nothing more embarrassing In designing our evaluation, we should begin by than that. identifying a population of patients appropriate to TYPES OF EVALUATION RESEARCH DESIGNS 391 the therapy. This identifi cation might be made by sion be? Would they see the movies in private or in researchers experimenting with the new therapy. groups? Should therapists be present? Perhaps we Let’s say we’re dealing with a clinic that already should observe the patients while the movies are has 100 patients being treated for sexual impo- being shown and include our observations among tence. We might take that group and the clinic’s the measurements of the experimental stimulus. defi nition of sexual impotence as a starting point, Do some patients watch the movies eagerly but and we should maintain any existing assessments others look away from the screen? We’d have to of the severity of the problem for each specifi c ask these kinds of questions and create specifi c patient. measurements to address them. For purposes of the evaluation research, how- Having thus designed the study, all we have to ever, we would need to develop a more specifi c do is “roll ‘em.” The study is set in motion, the ob- measure of impotence. Maybe it would involve servations are made and recorded, and the mass whether patients have sexual intercourse at all of data is accumulated for analysis. Once the study (within a specifi ed time), how often they have in- has run its course, we can determine whether the tercourse, or whether and how often they reach new therapy had its intended—or perhaps some orgasm. Alternatively, the outcome measure might unintended—consequences. We can tell whether be based on the assessments of independent ther- the movies were most effective for patients with apists not involved in the therapy who interview mild problems or severe ones, whether they the patients later. In any event, we would need to worked for young subjects but not older ones, and agree on the measures to be used. so forth. In the simplest design, we would assign the This simple illustration should show you how 100 patients randomly to experimental and control the standard experimental designs presented in groups; the former would receive the new therapy, Chapter 8 can be used in evaluation research. and the latter would be taken out of therapy al- Many, perhaps most, of the evaluations reported in together during the experiment. Because ethi- the research literature don’t look exactly like this cal practice would probably prevent withdrawing illustration, however. Because it’s nested in real therapy altogether from the control group, how- life, evaluation research often calls for quasi-ex- ever, it’s more likely that the control group would perimental designs. Let’s see what this means. continue to receive their conventional therapy. Having assigned subjects to the experimental and control groups, we would need to agree on Quasi-Experimental Designs the length of the experiment. Perhaps the design- Quasi experiments are distinguished from “true” ers of the new therapy feel it ought to be effective experiments primarily by the lack of random as- within two months, and an agreement could be signment of subjects to an experimental and a reached. The duration of the study doesn’t need to control group. In evaluation research, it’s often be rigid, however. One purpose of the experiment impossible to achieve such an assignment of sub- and evaluation might be to determine how long it jects. Rather than forgo evaluation altogether, re- actually takes for the new therapy to be effective. searchers sometimes create and execute research Conceivably, then, an agreement could be struck designs that give some evaluation of the program to measure recovery rates weekly, say, and let the in question. This section describes some of these ultimate length of the experiment rest on a con- designs. tinual review of the results. Let’s suppose the new therapy involves showing pornographic movies to patients. We’d need quasi experiments Nonrigorous inquiries somewhat to specify that stimulus. How often would patients resembling controlled experiments but lacking key ele- see the movies, and how long would each ses- ments such as pre- and posttesting and/or control groups. 392 CHAPTER 12 EVALUATION RESEARCH

Time-Series Designs To illustrate the time- 12 series design—studies that involve measure- 11 ments taken over time—I’ll begin by asking you 10 Controversial 9 to assess the meaning of some hypothetical data. Discussion Suppose I come to you with what I say is an ef- 8 fective technique for getting students to participate 7 in classroom sessions of a course I’m teaching. To 6 prove my assertion, I tell you that on Monday only 5 four students asked questions or made a comment 4 3 in class; on Wednesday I devoted the class time to

or asking questions in class 2 an open discussion of a controversial issue raging 1 on campus; and on Friday, when we returned to the Number of students making comments 0 subject matter of the course, eight students asked Monday Wednesday Friday questions or made comments. In other words, I FIGURE 12-1 Two Observations of Class Partici- contend, the discussion of a controversial issue on pation: Before and After an Open Discussion Wednesday has doubled classroom participation. This simple set of data is presented graphically in Figure 12-1. to the discussion. The long-term data suggest that Have I persuaded you that the open discussion the trend would have occurred even without the on Wednesday has had the consequence I claim discussion on Wednesday. The fi rst pattern, then, for it? Probably you’d object that my data do not contradicts my assertion that the special discus- prove the case. Two observations (Monday and sion increased class participation. Friday) aren’t really enough to prove anything. Ide- The second pattern contradicts my assertion by ally I should have had two classes, with students indicating that class participation has been bounc- assigned randomly to each, held an open discus- ing up and down in a regular pattern throughout sion in only one, and then compared the two on the semester. Sometimes it increases from one Friday. But I don’t have two classes with random class to the next, and sometimes it decreases; the assignment of students. Instead, I’ve been keep- open discussion on that Wednesday simply came ing a record of class participation throughout the at a time when the level of participation was about semester for the one class. This record allows you to increase. More to the point, we note that class to conduct a time-series evaluation. participation decreased again in the class follow- Figure 12-2 presents three possible patterns of ing the alleged postdiscussion increase. class participation over time, both before and after Only the third pattern in Figure 12-2 supports the open discussion on Wednesday. Which of these my contention that the open discussion mattered. patterns would give you some confi dence that the As depicted there, the level of discussion before discussion had the impact I contend it had? that Wednesday had been a steady four students If the time-series results looked like the fi rst pat- per class. Not only did the level of participation tern in Figure 12-2, you’d probably conclude that double following the day of discussion, but it con- the process of greater class participation had be- tinued to increase afterward. Although these data gun on the Wednesday before the discussion and do not protect us against the possible infl uence of had continued, unaffected, after the day devoted some extraneous factor (I might also have mentioned that participation would fi gure into students’ grades), they do exclude the possibility that time-series design A research design that involves measurements made over some period, such as the study the increase results from a process of maturation of traffi c accident rates before and after lowering the (indicated in the fi rst pattern) or from regular fl uc- speed limit. tuations (indicated in the second). Pattern 1 16

12 Controversial 10 Discussion 8

or asking questions in class 2

Number of students making comments 0 Wed Fri Mon Wed Fri Mon Wed Fri Mon WedFri Mon Wed Fri

Pattern 2 16

12 Controversial 10 Discussion 8

or asking questions in class 2

Number of students making comments 0 Wed Fri Mon Wed Fri Mon Wed Fri Mon WedFri Mon Wed Fri

Pattern 3 16

12 Controversial 10 Discussion 8

or asking questions in class 2

Number of students making comments 0 Wed Fri Mon Wed Fri Mon Wed Fri Mon WedFri Mon Wed Fri FIGURE 12-2 Three Patterns of Class Participation in a Longer Historical Period 394 CHAPTER 12 EVALUATION RESEARCH

Nonequivalent Control Groups The time-series to be effective and believed that the onset of design just described involves only an “experimen- smoking was relatively uncommon among his tal” group; it doesn’t provide the value to be gained students. The communities served by the two from having a control group. Sometimes, when schools were very similar. The rate of paren- researchers can’t create experimental and control tal smoking reported by the students was just groups by random assignment from a common above 40 percent in both schools. — (MCALISTER pool, they can ﬁ nd an existing “control” group that ET AL. 1980:720) appears similar to the experimental group. Such a In the initial set of observations, the experimen- group is called a nonequivalent control group. tal and control groups reported virtually the same If an innovative foreign language program is be- (low) frequency of smoking. Over the 21 months of ing tried in one class in a large high school, for the study, smoking increased in both groups, but example, you may be able to ﬁ nd another foreign it increased less in the experimental group than in language class in the same school that has a very the control group, suggesting that the program had similar student population: one that has about the had an impact on students’ behavior. same composition in terms of grade in school, gender, ethnicity, IQ, and so forth. The second class, then, could provide a point of comparison. At the Multiple Time-Series Designs Sometimes the end of the semester, both classes could be given evaluation of processes occurring outside of “pure” the same foreign language test, and you could experimental controls can be made easier by the compare performances. use of more than one time-series analysis. Mul- Here’s how two junior high schools were se- tiple time-series designs are an improved ver- lected for purposes of evaluating a program aimed sion of the nonequivalent control group design just at discouraging tobacco, alcohol, and drug use: described. Carol Weiss presents a useful example:

The pairing of the two schools and their assign- An interesting example of multiple time series ment to “experimental” and “control” conditions was the evaluation of the Connecticut crack- was not random. The local Lung Association down on highway speeding. Evaluators col- had identifi ed the school where we delivered lected reports of traffi c fatalities for several peri- the program as one in which administrators ods before and after the new program went into were seeking a solution to admitted problems of effect. They found that fatalities went down after smoking, alcohol, and drug abuse. The “control” the crackdown, but since the series had had an school was chosen as a convenient and nearby unstable up-and-down pattern for many years, demographic match where administrators were it was not certain that the drop was due to the willing to allow our surveying and breath-testing program. They then compared the statistics with procedures. The principal of that school consid- time-series data from four neighboring states ered the existing program of health education where there had been no changes in traffi c enforcement. Those states registered no equivalent drop in fatalities. The comparison lent credence nonequivalent control group A control group that to the conclusion that the crackdown had had is similar to the experimental group but is not created by some effect. — (1972:69) the random assignment of subjects. This sort of control group does differ signifi cantly from the experimental Although this study design is not as good as group in terms of the dependent variable or variables one in which subjects are assigned randomly, it’s related to it. nonetheless an improvement over assessing the multiple time-series designs The use of more than one set of data that were collected over time, as in experimental group’s performance without any accident rates over time in several states or cities, so that comparison. That’s what makes these designs comparison can be made. quasi experiments instead of just fooling around. TYPES OF EVALUATION RESEARCH DESIGNS 395

The key in assessing this aspect of evaluation The outcome variables considered by the eval- studies is comparability, as the following example uation all had to do with the extent to which mem- illustrates. bers of the peer groups initiated group activities A growing concern in the poor countries of the designed to improve their situation. Six types of world, rural development, has captured the atten- initiative were studied. “Active initiative,” for ex- tion and support of many rich countries. Through ample, was defi ned as “active effort to infl uence national foreign assistance programs and through persons or events affecting group members ver- international agencies such as the World Bank, the sus passive response or withdrawal” (Tandon and developed countries are in the process of sharing Brown 1981:180). The data for evaluation came their technological knowledge and skills with the from the journals that the peer group leaders had developing countries. Such programs have had been keeping since their initial technological train- mixed results, however. Often, modern techniques ing. The researchers read through the journals and do not produce the intended results when applied counted the number of initiatives taken by mem- in traditional societies. bers of the peer groups. Two researchers coded the Rajesh Tandon and L. Dave Brown (1981) un- journals independently and compared their work dertook an experiment in which instruction in vil- to test the reliability of the coding process. lage organization would accompany technological Figure 12-3 compares the number of active training. They felt it was important for poor farmers initiatives by members of the two experimental to learn how to organize and exert collective infl u- groups with those coming from the control groups. ence within their villages—getting needed action Similar results were found for the other outcome from government offi cials, for example. Only then measures. would their new technological skills bear fruit. Notice two things about the graph. First, there’s Both intervention and evaluation were attached a dramatic difference in the number of initiatives by to an ongoing program in which 25 villages had the two experimental groups as compared with the been selected for technological training. Two poor eleven controls. This seems to confi rm the effec- farmers from each village had been trained in new tiveness of the special training program. Second, agricultural technologies. Then they had been sent notice that the number of initiatives also increased home to share their new knowledge with their fel- among the control groups. The researchers explain low villagers and to organize other farmers into this latter pattern as a result of contagion. Because “peer groups” who would assist in spreading that all the villages were near each other, the lessons knowledge. Two years later, the authors randomly learned by peer group members in the experimen- selected two of the 25 villages (subsequently called tal groups were communicated in part to members Group A and Group B) for special training and 11 of the control villages. others as controls. A careful comparison of demo- This example illustrates the strengths of mul- graphic characteristics showed the experimental tiple time-series designs in situations where true and control groups to be strikingly similar to each experiments are inappropriate to the program be- other, suggesting they were suffi ciently compara- ing evaluated. ble for the study. The peer groups from the two experimental vil- Qualitative Evaluations lages were brought together for special training in organization building. The participants were given I’ve laid out the steps involved in tightly struc- some information about organizing and making tured, mostly quantitative evaluation research, demands on the government; they were also given but evaluations can also be less structured and opportunities to act out dramas similar to the more qualitative. For example, Pauline Bart and situations they faced at home. The training took Patricia O’Brien (1985) wanted to evaluate differ- three days. ent ways to stop rape, so they undertook in-depth 396 CHAPTER 12 EVALUATION RESEARCH

Text not available due to copyright restrictions

interviews with both rape victims and women who nant drug users that continuing to use drugs would had successfully fended off rape attempts. As a likely result in underweight babies whose skulls general rule, they found that resistance (such as would be an average of 10 percent smaller than yelling, kicking, running away) was more likely to normal. In his in-depth interviews with program be successful than to make the situation worse, as participants, however, he discovered that the pro- women sometimes fear it will. gram omitted one important piece of information: Sometimes even structured, quantitative evalu- that undersized babies were a bad thing. Many of ations can yield unexpected, qualitative results. the young women Steel interviewed thought that Paul Steel is a social researcher specializing in the smaller babies would mean easier deliveries. evaluation of programs aimed at pregnant drug In another program, a local district attorney had users. One program he evaluated involved coun- instituted what would generally be regarded as a seling by public health nurses, who warned preg- progressive, enlightened program. If a pregnant TYPES OF EVALUATION RESEARCH DESIGNS 397 drug user were arrested, she could avoid prosecu- searcher described his introductory conversation, tion if she would (1) agree to stop using drugs and “We would like you to listen to some radio pro- (2) successfully complete a drug rehabilitation programs designed to encourage people to have small gram. Again, in-depth interviews suggested that families, and we’d like you to tell us whether we’ve the program did not always operate on the ground been successful.” Not surprisingly, the small-fam- the way it did in principle. Specifi cally, Steel discov- ily theme came through clearly to the focus group ered that whenever a young woman was arrested subjects. for drug use, the other inmates would advise her to These examples, along with earlier comments get pregnant as soon as she was released on bail. in previous sections, have hinted at the possibility That way, she could avoid prosecution (personal of problems in the actual execution of evaluation communication, November 22, 1993). research projects. Of course, all forms of research The most effective evaluation research is one can run into problems, but evaluation research has that combines qualitative and quantitative com- a special propensity for it. ponents. Making statistical comparisons is useful, but so is gaining an in-depth understanding of the processes producing the observed results—or pre- Logistical Problems venting the expected results from appearing. In a military context, logistics refers to moving The evaluation of the Tanzanian soap opera, supplies around—making sure people have food, presented earlier in this chapter, employed sev- guns, and tent pegs when they need them. Here, I eral research techniques. I’ve already mentioned use it to refer to getting subjects to do what they’re the listener surveys and data obtained from clinics. supposed to do, getting research instruments dis- In addition, the researchers conducted numerous tributed and returned, and other seemingly unchal- focus groups to probe more deeply into the im- lenging tasks. These tasks pose more challenges pact the shows had on listeners. Further, content than you might guess. analyses were done on the soap opera episodes themselves and on the many letters received from listeners. Both quantitative and qualitative analy- Motivating Sailors When Kent Crawford, Ed- ses were undertaken (Swalehe et al. 1995). mund Thomas, and Jeffrey Fink (1980) set out to The soap opera research also gave me a fi rst- fi nd a way to motivate “low performers” in the U.S. hand opportunity to see how different cultures af- Navy, they found out just how many problems can fect the conduct of research. Not long ago I con- occur. The purpose of the research was to test a sulted on the evaluation of soap operas being three-pronged program for motivating sailors planned in Ethiopia. In contrast to the Western who were chronically poor performers and often concern for confi dentiality in social research, rein trouble aboard ship. First, a workshop was to spondents selected for interviews in rural Ethio- be held for supervisory personnel, training them pian villages often took a special pride at being in effective leadership of low performers. Second, selected and wanted their answers broadly known a few supervisors would be selected and trained in the community. as special counselors and role models—people Sometimes, local researchers’ desires to please the low performers could turn to for advice or just the client got in the way of the evaluation. For ex- as sounding boards. Finally, the low performers ample, some pilot episodes were tested in focus themselves would participate in workshops aimed groups to determine whether listeners would rec- at training them to be more motivated and effec- ognize any of the social messages being commu- tive in their work and in their lives. The project was nicated. The results were more encouraging than to be conducted aboard a particular ship, with a could have been expected. When I asked how the control group selected from sailors on four other focus group subjects had been selected, the re- ships. 398 CHAPTER 12 EVALUATION RESEARCH

To begin, the researchers reported that the su- left the Navy, and another was thrown out for disci- pervisory personnel were not exactly thrilled with plinary reasons. The experiment concluded, then, the program. with 8 subjects. Although the evaluation pointed to positive results, the very small number of subjects Not surprisingly, there was considerable resis- warranted caution in any generalizations from the tance on the part of some supervisors toward experiment. dealing with these issues. In fact, their reluc- The special, logistical problems of evaluation re- tance to assume ownership of the problem was search grow out of the fact that it occurs within the refl ected by “blaming” any of several factors that context of real life. Although evaluation research can contribute to their personnel problem. The is modeled after the experiment—which suggests recruiting system, recruit training, parents, and that the researchers have control over what hap- society at large were named as infl uencing low pens—it takes place within frequently uncontrol- performance—factors that were well beyond the lable daily life. Of course, the participant-observer control of the supervisors. — (CRAWFORD, THOMAS, in fi eld research doesn’t have control over what’s AND FINK 1980:488) observed either, but that method doesn’t strive for control. Given the objectives of evaluation re- Eventually, the reluctant supervisors came around search, lack of control can create real dilemmas and “this initial reluctance gave way to guarded for the researcher. optimism and later to enthusiasm” — (1980:489). The low performers themselves presented even Administrative Control As suggested in the pre- more of a problem, however. The research design vious example, the logistical details of an evaluation called for pre- and posttesting of attitudes and per- project often fall to program administrators. Let’s sonalities, so that changes brought about by the suppose you’re evaluating the effects of a conjugal program could be measured and evaluated. visit program on the morale of married prisoners. The program allows inmates periodic visits from Unfortunately, all of the LPs (Low Perform- their spouses during which they can have sexual ers) were strongly opposed to taking these relations. On the fourth day of the program, a male so-called personality tests and it was therefore prisoner dresses up in his wife’s clothes and es- concluded that the data collected under these capes. Although you might be tempted to assume circumstances would be of questionable validity. that his morale was greatly improved by escaping, Ethical concerns also dictated that we not force that turn of events would complicate your study “testing” on the LPs. — (CRAWFORD, THOMAS, AND design in many ways. Perhaps the warden will ter- FINK 1980:490) minate the program altogether, and where’s your evaluation then? Or, if the warden is brave, he or As a consequence, the researchers had to rely on she may review the fi les of all those prisoners you interviews with the low performers and on the selected randomly for the experimental group and judgments of supervisors for their measures of at- veto the “bad risks.” There goes the comparability titude change. The subjects continued to present of your experimental and control groups. As an al- problems, however. ternative, stricter security measures may be intro- Initially, the ship’s command ordered 15 low duced to prevent further escapes, and the security performers to participate in the experiment. Of the measures may have a dampening effect on morale. 15, however, one went into the hospital, another So the experimental stimulus has changed in the was assigned duties that prevented participation, middle of your research project. Some of the data and a third went “over the hill” (absent without will refl ect the original stimulus; other data will re- leave). Thus, the experiment began with 12 sub- fl ect the modifi cation. Although you’ll probably be jects. But before it was completed, three more sub- able to sort it all out, your carefully designed study jects completed their enlistment requirements and has become a logistical snake pit. TYPES OF EVALUATION RESEARCH DESIGNS 399

Or suppose you’ve been engaged to evalu- do or who couldn’t be trusted to do anything im- ate the effect of race-relations lectures on preju- portant. You might learn this bit of information a dice in the Army. You’ve carefully studied the week or so before the deadline for submitting your soldiers available to you for study, and you’ve ran- fi nal report on the impact of the race-relations domly assigned some to attend the lectures and lectures. others to stay away. The rosters have been cir- These are some of the logistical problems con- culated weeks in advance, and at the appointed fronting evaluation researchers. You need to be fa- day and hour, the lectures begin. Everything miliar with the problems to understand why some seems to be going smoothly until you begin pro- research procedures may not measure up to the cessing the fi les: The names don’t match. Checking design of the classical experiment. As you read re- around, you discover that military fi eld exercises, ports of evaluation research, however, you’ll fi nd KP duty, and a variety of emergencies required that—my earlier comments notwithstanding—it is some of the experimental subjects to be elsewhere possible to carry out controlled social research in at the time of the lectures. That’s bad enough, conjunction with real-life experiments. but then you learn that helpful commanding of- The accompanying box, “Testing Soap Operas fi cers sent others to fi ll in for the missing sol- in Tanzania,” describes some of the logistical diers. And whom do you suppose they picked to problems involved in the research discussed at the fi ll in? Soldiers who didn’t have anything else to outset of this chapter.

ISSUES AND INSIGHTS TESTING SOAP OPERAS IN TANZANIA

by William N. Ryerson repeated this nerve-wracking trip each year af- Executive Vice-President terward in order to measure the change in that Population Communications International village. Another interviewer, Mr. Tende, was invited Twende na Wakati (“Let’s Go With the Times”) to participate in a village feast that the villagers has been broadcast on Radio Tanzania since held to welcome him and to indicate their enthu- mid-1993 with support from the United Nations siasm about having been selected for the study. Population Fund. The program was designed to They served him barbequed rats. Though they encourage family planning use and AIDS pre- weren’t part of his normal diet, he ate them any- vention measures. way to be polite and to ensure that the research There were many different elements to the re- interviews could be carried out in that village. search. One was a nationwide, random-sample Still another interviewer, Mrs. Masanja, was survey given prior to the ﬁrst airing of the soap working in a village in the Pwani region along opera in June 1993 and then annually after that. the coast of the Indian Ocean when cholera Many interviewers faced particularly interest- broke out in that village. She wisely chose to ing challenges. For example, one interviewer, abandon the interviews there, which reduced the Fridolan Banzi, had never been in or on water 1993 sample size by one ward. in his life and couldn’t swim. He arranged for a The unsung heroes of this research, the Tan- small boat to take him through the rough waters zanian interviewers, deserve a great deal of of Lake Victoria so he could carry out his inter- credit for carrying out this important work under views at a village that had no access by road. He difﬁcult circumstances. 400 CHAPTER 12 EVALUATION RESEARCH

ISSUES AND INSIGHTS THE IMPACT OF THREE-STRIKES LAWS

SACRAMENTO (AP)—The author of Califor- in crime rates during the 1990s to the new policy nia’s ﬁ ve-year-old “three strikes” law says of getting tough with career criminals. While ac- it’s prevented more than a million crimes knowledging that “three strikes” may not be the and has saved $21.7 billion. only cause of the drop in crime, Jones added, “If Secretary of State Bill Jones offered his you can have a 51 percent reduction in the ho- interpretation of the “three strikes” results to micide rate in ﬁ ve years, I would guarantee you a Doris Tate Crimes Victim Bureau confer- three strikes is a big part of the reason.” ence on Friday in Sacramento. — “STATE In spite of the politicians’ guarantees, other SAVED 21.7 BILLION,” BAYINSIDER, MARCH 1, 1999. observers have looked for additional evidence to support the impact of three-strikes laws. Some The 1990s saw the passage of three-strikes laws critics of these laws, for example, have noted at the federal level and in numerous states. The that crime rates have been dropping dramati- intention was to reduce crime rates by locking cally across the country, not only in California up “career criminals.” Under the 1994 California but in states that have no three-strikes laws and law, for example, having a past felony convic- in those where the courts have not enforced the tion would double your punishment when you three-strikes laws that exist. In fact, crime rates were convicted of your second felony, and the have dropped in those California counties that third felony conviction would bring a manda- have tended to ignore that state’s law. Moreover, tory sentence of 25 years to life. Over the years, the drop in California crime rates began before only California has enforced such laws with any the three-strikes law went into effect. vigor. In 1994 Peter Greenwood and his colleagues at Those who supported the passage of three- the Rand Corporation estimated that implemen- strikes legislation, such as Bill Jones, quoted tation of the law would cost California’s criminal above, have been quick to link the dramatic drop justice system approximately $5.5 billion more

Use of Research Results diligent, multifaceted evaluation, the commission reported that pornography didn’t appear to have One more facts-of-life aspect of evaluation re- any of the negative social consequences often at- search concerns how evaluations are used. Be- tributed to it. Exposure to pornographic materials, cause the purpose of evaluation research is to for example, didn’t increase the likelihood of sex determine the success or failure of social interven- crimes. You might have expected liberalized leg- tions, you might think it reasonable that a program islation to follow from the research. Instead, the would automatically be continued or terminated president said the commission was wrong. based on the results of the research. Less dramatic examples of the failure to fol- Reality isn’t that simple and reasonable, how- low the implications of evaluation research could ever. Other factors intrude on the assessment of be listed endlessly. Undoubtedly every evaluation evaluation research results, sometimes blatantly researcher can point to studies he or she con- and sometimes subtly. As president, Richard Nixon ducted—studies providing clear research results appointed a blue-ribbon national commission to and obvious policy implications—that were ig- study the consequences of pornography. After a TYPES OF EVALUATION RESEARCH DESIGNS 401

per year, especially in prison costs as “career lution to crime, ignoring other, more effective, criminals” were sentenced to longer terms. Al- solutions. Often, imprisonment causes prob- though the Rand group did not deny that the lems more serious than those it was intended three-strikes legislation would have some im- to remedy. pact on crime—those serving long terms in pris- As with many other social interventions, on can’t commit crimes on the streets—a follow- however, much of the support for three-strikes up study (Greenwood, Rydell, and Model 1996) laws in California and elsewhere have mostly suggested it was an ineffi cient way of attacking to do with public emotions about crime and the crime. The researchers estimated that a million political implications of such emotions. Thus, dollars spent on “three strikes” would prevent evaluation research on these laws may eventu- 60 crimes, whereas the same amount spent on ally bring about changes, but it’s impact is like- programs encouraging high school students to ly to be much slower than you might logically stay in school and graduate would prevent 258 expect. crimes. Criminologists have long recognized that Sources: “State Saved $21.7 Billion with Five-Year- most crimes are committed by young men. Fo- Old ‘Three Strikes’ Law,” BayInsider, March 1, 1999, http://www.bayinsider.com/news/1999/03/01/ cusing attention on older “career criminals” has three_strikes.html; Peter W. Greenwood et al., little or no effect on the youthful offenders. In Three Strikes and You’re Out: Estimated Benefi ts and fact, three-strikes sentences disproportionately Costs of California’s New Mandatory-Sentencing Law (Santa Monica, CA: Rand Corporation, 1994); Peter fall on those approaching the end of their crimi- W. Greenwood, C. Peter Rydell, and Karyn Model, nal careers by virtue of growing older. Diverting Children from a Life of Crime: Measuring In a more general critique, John Irwin and Costs and Benefi ts (Santa Monica, CA: Rand Corpora- tion, 1996); John Irwin and James Austin, It’s About James Austin (1997) suggest that people in the Time: America’s Imprisonment Binge (Belmont, CA: United States tend to overuse prisons as a so- Wadsworth 1997).

nored, as the accompanying box, “The Impact of when he said the earth revolved around the sun. ‘Three-Strikes’ Laws,” illustrates. Anybody could tell that the earth was standing still. There are three important reasons why the im- The third barrier to the use of evaluation results is plications of evaluation research results are not vested interests. If I’ve devised a new rehabilitation always put into practice. First, the implications program that I’m convinced will keep ex-convicts may not always be presented in a way that non- from returning to prison, and if people have taken researchers can understand. Second, evaluation to calling it the “Babbie Plan,” how do you think results sometimes contradict deeply held beliefs. I’m going to feel when your evaluation suggests That was certainly true in the case of the pornogra- that the program doesn’t work? I might apologize phy commission. If everybody knows that pornog- for misleading people, fold up my tent, and go into raphy is bad, that it causes all manner of sexual another line of work. More likely, I’d call your re- deviance, then it’s likely that research results to the search worthless and begin intense lobbying with contrary will have little immediate impact. By the the appropriate authorities to have my program same token, people thought Copernicus was crazy continue. 402 CHAPTER 12 EVALUATION RESEARCH

In the earlier example of the evaluation of driv- The researchers summarize the ways in which ers’ education, Philip Hilts reported some of the re- new laws were intended to make a difference: actions to the researchers’ preliminary results: The most changes are: (1) redefi ning rape and Ray Burneson, traffi c safety specialist with the replacing the single crime of rape with a series National Safety Council, criticized the study, of graded offenses defi ned by the presence or saying that it was a product of a group (NHTSA) absence of aggravating conditions; (2) chang- run by people who believe “that you can’t do ing the consent standard by eliminating the anything to train drivers. You can only improve requirement that the victim physically resist her medical facilities and build stronger cars for attacker; (3) eliminating the requirement that the when the accidents happen. . . . This knocks the victim’s testimony be corroborated; and (4) plac- whole philosophy of education.” —(1981:4) ing restrictions on the introduction of evidence By its nature, evaluation research takes place in of the victim’s prior sexual conduct. — (1990:2) the midst of real life, affecting it and being affected It was generally expected that such legislation by it. Here’s another example, well known to social would encourage women to report being raped researchers. and would increase convictions when the cases were brought to court. To examine the latter ex- Rape Reform Legislation For years, many social pectation, the researchers focused on the period scientists and other observers have noted certain from 1970 to 1985 in Cook County, Illinois: “Our problems with the prosecution of rape cases. All too data fi le includes 4,628 rape cases, 405 deviate often, it is felt, the victim ends up suffering almost sexual assault cases, 745 aggravated criminal sex- as much on the witness stand as in the rape itself. ual assault cases, and 37 criminal sexual assault Frequently, defense lawyers portray her as having cases” (1990:4). Table 12-1 shows some of what encouraged the sex act and being of shady moral they discovered. character; other personal attacks are intended Spohn and Horney summarized these fi ndings to defl ect responsibility from the accused rapist. as follows: Criticisms such as these have resulted in a variety of state laws aimed at remedying the problems. The only signifi cant effects revealed by our Cassie Spohn and Julie Horney (1990) were inter- analyses were increases in the average maxi- ested in tracking the impact of such legislation. mum prison sentences; there was an increase

TABLE 12-1 Analysis of Rape Cases before and after Legislation

Rape Before After (N 2,252) (N 2,369)

Outcome of case Convicted of original charge 45.8% 45.4% Convicted of another charge 20.6 19.4 Not convicted 33.6 35.1 Median prison sentence in months For those convicted of original charge 96.0 144.0 For those convicted of another charge 36.0 36.0 TYPES OF EVALUATION RESEARCH DESIGNS 403

of almost 48 months for rape and of almost 36 allowed within the “Indianapolis Prosecution Ex- months for sex offenses. Because plots of the periment” (IPE). data indicated an increase in the average sen- Wife-battering cases can follow a variety of pat- tence before the reform took effect, we modeled terns, as Ford and Regoli summarize: the series with the intervention moved back one After a violent attack on a woman, someone year earlier than the actual reform date. The size may or may not call the police to the scene. If of the effect was even larger and still signifi cant, the police are at the scene, they are expected indicating that the effect should not be attributed to investigate for evidence to support probable to the legal reform. — (1990:10) cause for a warrantless arrest. If it exists, they Notice in the table that there was virtually no may arrest at their discretion. Upon making change in the percentages of cases ending in con- such an on-scene arrest, offi cers fi ll out a prob- viction for rape or some other charge (such as as- able cause affi davit and slate the suspect into sault). Hence, the change in laws had no effect on court for an initial hearing. When the police are the likelihood of conviction. As the researchers not called, or if they are called but do not arrest, note, the one change that is evident—an increase a victim may initiate charges on her own by go- in the length of sentences—cannot be attributed to ing to the prosecutor’s offi ce and swearing out the reform legislation itself. a probable cause affi davit with her allegation In addition to the analysis of existing statistics, against the man. Following a judge’s approval, Spohn and Horney interviewed judges and law- the alleged batterer may either be summoned to yers to determine what they felt about the impact court or be arrested on a warrant and taken to of the laws. Their responses were somewhat more court for his initial hearing. — (1992:184) encouraging. What if a wife brings charges against her hus- Judges, prosecutors and defense attorneys in band and then reconsiders later on? Many courts Chicago stressed that rape cases are taken have a policy of prohibiting such actions, in the more seriously and rape victims treated more belief that they are serving the interests of the vic- humanely as a result of the legal changes. These tim by forcing the case to be pursued to comple- educative effects clearly are important and tion. In the IPE, however, some victims are offered should please advocates of rape reform legisla- the possibility of dropping the charges if they so tion. — (1990:17) choose later in the process. In addition, the court offers several other options. Because wife batter- Thus, the study found other effects besides the ing is largely a function of sexism, stress, and an qualitative results the researchers looked for. This inability to deal with anger, some of the innovative study demonstrates the importance of following possibilities in the IPE involve educational classes up on social interventions to determine whether, with anger-control counseling. in what ways, and to what degree they accomplish If the defendant admits his guilt and is will- their intended results. ing to participate in an anger-control counseling program, the judge may postpone the trial for Preventing Wife Battering In a somewhat simi- that purpose and can later dismiss the charges if lar study, researchers in Indianapolis focused their the defendant successfully completes the program. attention on the problem of wife battering, with a Alternatively, the defendant may be tried and, if special concern for whether prosecuting the bat- found guilty, be granted probation provided he par- terers can lead to subsequent violence. David Ford ticipates in the anger-control program. Finally, the and Mary Jean Regoli (1992) set about studying the defendant can be tried and, if found guilty, given a consequences of various options for prosecution conventional punishment such as imprisonment. 404 CHAPTER 12 EVALUATION RESEARCH

Which of these possibilities most effectively pre- Moreover, the researchers caution that the re- vents subsequent wife battering? That’s the ques- sults of their research point to general patterns, tion Ford and Regoli addressed. Here are some of and that individual battered wives must choose their fi ndings. courses of action appropriate to their particular First, men who are brought to court for a hear- situations and should not act blindly on the basis ing are less likely to continue beating their wives, of the overall patterns. The research is probably no matter what the outcome of the hearing. Simply more useful in what it says about ways of structur- being brought into the criminal justice system has ing the criminal justice system (giving victims the an impact. right to drop charges, for example) than in guiding Second, women who have the right to drop the actions of individual victims. charges later on are less likely to be abused subse- Finally, the IPE offers an example of a com- quently than those who do not have that right. In mon problem in evaluation research. Often, actual particular, the combined policies of arresting de- practices differ from what might be expected in fendants by warrant and allowing victims to drop principle. For example, the researchers considered charges provide victims with greater security from the impact of different alternatives for bringing subsequent violence than do any of the other pros- suspects into court: Specifi cally, the court can issue ecution policies. either a summons ordering the husband to appear However, giving victims the right to drop in court or a warrant to have the husband arrested. charges has a somewhat strange impact. Women The researchers were concerned that having the who exercise that right are more likely to be abused husband arrested might actually add to his anger later than are those who insist on the prosecution over the situation. They were somewhat puzzled, proceeding to completion. The researchers inter- therefore, to fi nd no difference in the anger of hus- pret this as showing that future violence can be bands summoned or arrested. decreased when victims have a sense of control The solution of the puzzle lay in the discrep- supported by a clear and consistent alliance with ancy between principle and practice: criminal justice agencies. Although a warrant arrest should in principle be A decisive system response to any violation at least as punishing as on-scene arrest, in prac- of conditions for pretrial release, including of tice it may differ little from a summons. A man course new violence, should serve notice that usually knows about a warrant for his arrest the victim-system alliance is strong. It tells and often elects to turn himself in at his conve- the defendant that the victim is serious in her nience, or he is contacted by the warrant service resolve to end the violence and that the system agency and invited to turn himself in. Thus, he is unwavering in its support of her interest in may not experience the obvious punishment of, securing protection. — (FORD AND REGOLI 1992:204) say, being arrested, handcuffed, and taken away The effectiveness of anger-control counsel- from a workplace. — (FORD 1989:9–10) ing cannot be assessed simply. Policies aimed at getting defendants into anger-control counseling In summary, many factors besides the scien- seem to be relatively ineffective in preventing new tifi c quality of evaluation research affect how its violence. The researchers note, however, that the results are used. And, as we saw earlier, factors policy effects should not be confused with actual outside the evaluator’s control can affect the qual- counseling outcomes. Some defendants sched- ity of the study itself. But this “messiness” is bal- uled for treatment never received it. Considerably anced by the potential contributions that evalua- more information on implementing counseling is tion research can make toward the betterment of needed for a proper evaluation. human life. TYPES OF EVALUATION RESEARCH DESIGNS 405

The Sabido Methodology One of the clearest observe rural villages, talking with residents and illustrations of the uses of evaluation research sometimes recording the sounds of village life. results comes from the omnibus methodology This formative research provided the writers developed by Miguel Sabido for the use of “En- with ideas about issues to be raised and how to tertainment-Education (E-E)” to promote social raise them. For example, the research indicated programs. The example of Twende na Wakati at that in some regions, abduction was still a com- the outset of this chapter illustrated the methods mon method of mate selection: A man would kid- initially developed by Miguel Sabido in the 1970s nap a young woman, sexually assaulting her and when he was Vice President for Research in the holding her prisoner until she would consent to Mexican broadcasting company Televisa. Sabido’s be his wife. The formative research also revealed fi rst projects used television novellas to promote a widespread belief that condoms were infected literacy and family planning. They were so suc- with HIV, thus meaning that condom use risked cessful that those methods have been used to pro- AIDS rather than preventing it. mote a variety of social issues in the subsequent The initial research also provided a baseline decades. for subsequent evaluations. By knowing public In part, the Sabido methodology concerns the opinion toward family planning prior to the radio nature of the radio or television dramas, particu- programs, researchers could determine how much larly the kinds of characters portrayed. Some char- it had changed afterward. Preprogramming mea- acters represent traditional points of view, some sures of the use of family planning centers could represent the modern views that the programming be compared with levels of use afterward. Many of is designed to promote, and a third type of “tran- these evaluations efforts ran concurrently with the sitional” character begins with traditional views radio programming. For example, regular focus and eventually shifts to the modern views. Typi- groups were used to monitor public reactions to cally, when a transitional character signs up for each of the serial installments, examining whether literacy classes, thousands of audience members people were reacting as intended. do the same shortly thereafter. When the transi- The Sabido methodology provides an excellent tional character begins using condoms for family illustration of how research methods can be used planning or safe sex, family planning clinics are to construct and evaluate social action programs mobbed the next day by men wanting condoms. aimed at resolving social problems. To learn more The Sabido methodology extends beyond char- about the Sabido Methodology, see Barker and Sa- acter defi nitions and plot structures. An E-E proj- bido (2005). ect begins with thorough research into the society As you can see, evaluation research can pro- where the change is being planned. A project in vide a unique and powerful tool for effecting so- Ethiopia by the Population Media Center, for ex- cial change. However, it can also be useful on a ample, aimed to lower the birth rate, encour- personal level, in everyday situations. See the box age safe sex practices, and enhance the status of “Making Changes in Your Life with Evaluation women. The production of radio serial dramas was Research.” preceded by extensive research into the existing situations regarding the project’s aims. What was To download Barker and Sabido (2005),* the birth rate? How did it differ in different regions go to http://www.unfpa.org/publications/ of the country and among different ethnic groups? detail.cfm?ID271&fi lterListType. What were the existing attitudes toward family planning? In part these questions were answered through national surveys. At the same time, quali- *Each time the Internet icon appears, you’ll be given help- tative researchers went into the countryside to ful leads for searching the World Wide Web. 406 CHAPTER 12 EVALUATION RESEARCH

we could look at infant mortality: the number of IN THE REAL WORLD infants who die during their ﬁ rst year of life among MAKING CHANGES IN YOUR LIFE every 1,000 births. Depending on the particular as- WITH EVALUATION RESEARCH pect of health conditions we were interested in, we could devise any number of other measures: physi- Evaluation research typically begins with cians per capita, hospital beds per capita, days of a problem, followed by a possible solution hospitalization per capita, and so forth. Notice that to the problem, and completed with a test intersocietal comparisons are facilitated by calcu- of that solution. You may be able to think lating per capita rates (dividing by the size of the of analogous situations in your personal or population). professional life. As a student, you may have Before we go further, recall from Chapter 11 ideas about how to improve your study skills the problems involved in using existing statistics. or athletic abilities. What you learn about In a word, they’re often unreliable, reﬂ ecting their evaluation research could enable you to test modes of collection, storage, and calculation. With those ideas more rigorously. Or, working this in mind, we’ll look at some of the ways we can in a business, you might want to test your use social indicators for evaluation research on a ideas about increasing sales or manufactur- large scale. ing quality. The possibilities for evaluation research will crop up in all aspects of your life. The Death Penalty and Deterrence

Does the death penalty deter capital crimes such as murder? This question is hotly debated every time SOCIAL INDICATORS RESEARCH a state considers eliminating or reinstating capital punishment and every time someone is executed. Those supporting capital punishment often argue I want to continue our discussion with a type of that the threat of execution will keep potential research that combines evaluation research with murderers from killing people. Opponents of capi- the analysis of existing data. A rapidly growing tal punishment often argue that it has no effect in fi eld in social research involves the development that regard. Social indicators can help shed some and monitoring of social indicators, aggregated light on the question. statistics that refl ect the social condition of a so- If capital punishment actually deters people ciety or social subgroup. Researchers use social from committing murder, then we should expect indicators to monitor aspects of social life in much to fi nd murder rates lower in those states that have the way that economists use indexes such as gross the death penalty than in those that do not. The national product (GNP) per capita as an indicator relevant comparisons in this instance are not only of a nation’s economic development. possible, but they’ve been published. Table 12-2 Suppose we wanted to compare the relative presents data compiled by William Bailey (1975) health conditions in different societies. One strat- that directly contradict the view that the death egy would be to compare their death rates (number penalty deters murderers. In both 1967 and 1968, of deaths per 1,000 population). More specifi cally, those states with capital punishment had dramatically higher murder rates than did those without capital punishment. Some people criticized the in- Measurements that refl ect the social indicators terpretation of Bailey’s data, saying that most states quality or nature of social life, such as crime rates, infant mortality rates, number of physicians per 100,000 popula- had not used the death penalty in recent years, tion, and so forth. Social indicators are often monitored to even when they had it on the books. That could determine the nature of social change in a society. explain why it hadn’t seemed to work as a deter- SOCIAL INDICATORS RESEARCH 407

Text not available due to copyright restrictions

rent. Further analysis, however, contradicts this ample, the 38 death-penalty states had a combined explanation. When Bailey compared those states murder rate of 5.86 per 100,000, compared with a that hadn’t used the death penalty with those that combined murder rate of 3.84 among the 12 states had, he found no real difference in murder rates. that lack the death penalty (U.S. Bureau of the Cen- sus 2001:22, 183). Notice from the preceding discussion that re- For more on the death penalty, see the searchers can use social indicators data for com- Death Penalty Information Center at http:// www.deathpenaltyinfo.org/deter.html. parison across groups either at one time or across some period. Often, doing both sheds the most light on the subject. At present, work on the use of social indicators Another counterexplanation is possible, how- is proceeding on two fronts. On the one hand, re- ever. It could be the case that the interpretation searchers are developing ever more-refi ned indi- given Bailey’s data was backward. Maybe the ex- cators—fi nding which indicators of a general vari- istence of the death penalty as an option was a able are the most useful in monitoring social life. consequence of high murder rates: Those states At the same time, research is being devoted to dis- with high rates instituted it; those with low rates covering the relationships among variables within didn’t institute it or repealed it if they had it on the whole societies. books. It could be the case, then, that instituting the death penalty would bring murder rates down, and repealing it would increase murders and still To fi nd out more about social indicators, produce—in a broad aggregate—the data pre- search for “social indicators” on the web sented in Table 12-2. Not so, however. Analyses or check out the Sociometrics Corporation: over time do not show an increase in murder rates http://www.socio.com/. when a state repeals the death penalty nor a decrease in murders when one is instituted. A more recent examination by Bailey and Ruth Peterson Computer Simulation (1994) confi rmed the earlier fi ndings and also indicated that law enforcement offi cials doubted the One of the most exciting prospects for social indi- deterrent effect. Further, the pattern observed by cators research lies in the area of computer simu- Bailey in 1967 and 1968 has persisted over time, lation. As researchers begin compiling mathemati- even when we take into account the substantial cal equations describing the relationships that link increase in the overall murder rate. In 1999, for ex- social variables to one another (for example, the 408 CHAPTER 12 EVALUATION RESEARCH relationship between growth in population and the This potentiality points to the special value of number of automobiles), those equations can be evaluation research in general. Throughout human stored and linked in a computer. With a suffi cient history, we’ve been tinkering with our social ar- number of adequately accurate equations on tap, rangements, seeking better results. Evaluation re- researchers one day will be able to test the im- search provides a means for us to learn right away plications of specifi c social changes by computer whether a particular tinkering really makes things rather than in real life. better. Social indicators allow us to make that de- Suppose a state contemplated doubling the size termination on a broad scale; coupling them with of its tourism industry, for example. We could en- computer simulation opens up the possibility of ter that proposal into a computer simulation model knowing how much we would like a particular in- and receive in seconds or minutes a description of tervention without having to experience its risks. all the direct and indirect consequences of the increase in tourism. We could know what new public ETHICS AND EVALUATION RESEARCH facilities would be required, which public agencies such as police and fi re departments would have to be increased and by how much, what the labor Because it’s embedded in the day-to-day events of force would look like, what kind of training would real life, evaluation research entails special ethical be required to provide it, how much new income problems. Evaluating the impact of busing school and tax revenue would be produced, and so forth, children to achieve educational integration, for ex- through all the intended and unintended conse- ample, will throw the researchers directly into the quences of the action. Depending on the results, political, ideological, and ethical issues of busing the public planners might say, “Suppose we in- itself. It’s not possible to evaluate a sex education creased the industry only by half,” and have a new program in elementary schools without becom- printout of consequences immediately. ing involved in the heated issues surrounding sex An excellent illustration of computer simula- education itself, and the researcher will fi nd it dif- tion linking social and physical variables is to fi cult to remain impartial. The evaluation study be found in the research of Donella and Dennis design will require that some children receive sex Meadows and their colleagues at Dartmouth and education—in fact, you may very well be the one the Massachusetts Institute of Technology (Mead- who decides which children do. (From a scientifi c ows et al. 1972; Meadows, Meadows, and Randers standpoint, you should be in charge of selection.) 1992). They’ve taken as input data some known This means that when parents become outraged and estimated reserves of various nonreplaceable that their child is being taught about sex, you’ll be natural resources (for example, oil, coal, and iron), directly responsible. past patterns of population and economic growth, Now let’s look on the bright side. Maybe the and the relationships between growth and use of experimental program is of great value to those resources. Using a complex computer simulation participating in it. Let’s say that the new industrial model, they’ve been able to project, among other safety program being evaluated reduces injuries things, the probable number of years various re- dramatically. What about the control group mem- sources will last in the face of alternative usage bers who were deprived of the program by the re- patterns in the future. Going beyond the initially search design? The evaluators’ actions could be an gloomy projections, such models also make it pos- important part of the reason that a control group sible to chart out less gloomy futures, specifying subject suffered an injury. the actions required to achieve them. Clearly, the By its very nature, then, evaluation research is value of computer simulation is not limited to eval- interwoven with real-world issues. We only evalu- uation research, though it can serve an important ate programs when it matters whether they make function in that regard. a difference or not, and that means the results of ETHICS AND EVALUATION RESEARCH 409 the evaluation matter to people. This brings up When professional complaints were fi nally lodged another potential problem. Almost always, some with the U.S. Centers for Disease Control in 1965, people will want the results to turn out a certain there was no reply (Jones 1981). way, and other people may want a different result. My purpose in these comments has not been Often, as in the case of pharmaceutical testing, for to cast a shadow on evaluation research. Rather, I example, those paying for the research may want want to bring home the real-life consequences of a particular result. Further, the researchers them- the evaluation researcher’s actions. Ultimately, as selves may have personal motives toward a given we saw in Chapter 3, all social research has ethical end. Evaluation researchers, therefore, often fi nd components. themselves under internal or external pressure to produce a particular fi nding. Of course, researchers must not be swayed by WHAT DO YOU THINK? REVISITED either personal desires or sponsors’ demands in the design, execution, and analysis of results. This is true in all kinds of social research; however, un- The purpose of evaluation research is to ethical actions in evaluation research can produce determine whether social interventions or particularly severe consequences. For example, programs have had their desired effects. No the results of evaluation research may determine matter how much research is done, how- whether people are subjected to medical or social ever, debates tend to persist. remedies. Imagine a medical researcher slanting As we’ve seen in this chapter, the political drug testing results to suggest that a new drug is and ideological viewpoints that inform posi- more effective than it is or covering up the nega- tions on certain issues often are deeply in- tive side effects of the drug—with the consequence grained and withstand contrary information. of more patients being given the drug. Moreover, evaluation research occurs in the Or imagine an evaluation of a prison rehabilita- “real world,” where such assessments often tion program being slanted to make the program affect people’s self-interests. Research that seem more effective than it is. Limited resources reports a program is ineffective threatens might be diverted to support the ineffective program the jobs of those employed by the program, and possibly even harm the prisoners subjected to not to mention the reputations of those who it. That’s not the worst example, however. created it. In 1932 researchers in Tuskegee, Alabama, Evaluation research typically examines began a program of providing free treatment for studies with multiple variables. This means syphilis to poor, African American men suffering that researchers can easily argue about from the disease. Over the years that followed, which variable caused an apparent effect. several hundred men participated in the program. They may even argue that some other, pre- What they didn’t know was that they were not ac- viously unthought of variable is responsible tually receiving any treatment at all; the physicians (recall the discussion of internal and external conducting the study merely wanted to observe the invalidity in Chapter 8). Further, many of the natural progress of the disease. Even after penicil- variables evaluated tend to be somewhat am- lin was found to be an effective cure, the research- biguous: What is “happiness,” “motivation,” ers still withheld the treatment. Although there is and so forth? In short, there is usually ample unanimous agreement today as to the unethical “wiggle room” for people who disagree with nature of the study, this was not the case at the the fi ndings of evaluation research. As such, time. Even when the study began being reported we can’t settle once and for all whether the in research publications, the researchers refused death penalty prevents murders. to acknowledge they had done anything wrong. 410 CHAPTER 12 EVALUATION RESEARCH

Main Points ❏ The implications of evaluation research won’t necessarily be put into practice, especially if Introduction they confl ict with offi cial points of view. ❏ Evaluation research is a form of applied research that studies the effects of social Social Indicators Research interventions. ❏ Social indicators can provide an understanding of broad social processes. Topics Appropriate ❏ Computer-simulation models hold the promise to Evaluation Research of allowing researchers to study the possible ❏ Evaluation research is appropriate whenever results of social interventions without having some social intervention occurs or is planned, to incur those results in real life. so the potential for topics is limitless. ❏ Much of evaluation research is referred to as Ethics and Evaluation Research program evaluation or outcome assessment: ❏ Evaluation research entails special ethical the determination of whether a social interven- problems because it’s embedded in the day-to- tion is producing the intended result. day events of real life. ❏ Evaluation research may bring added pres- Formulating the Problem: sure to produce specifi c results, as desired by Issues of Measurement interested parties. ❏ A careful formulation of the problem, including ❏ Unethical actions in an evaluation study can relevant measurements and criteria of success have consequences severer than such actions or failure, is essential in evaluation research. in other types of research. In particular, evaluators must carefully specify outcomes, measure experimental contexts, specify the intervention being studied and the Key Terms population targeted by the intervention, decide evaluation research quasi experiments whether to use existing measures or devise multiple time-series designs social indicators new ones, and assess the potential cost-effec- nonequivalent control group time-series design tiveness of an intervention. program evaluation/ outcome assessment Types of Evaluation Research Designs ❏ Evaluation researchers typically use experi- Review Questions mental or quasi-experimental designs. Ex- amples of quasi-experimental designs include 1. Review the evaluation of the Navy low-per- time-series studies and the use of nonequiva- former program discussed in this chapter. lent control groups. How would you redesign the program and ❏ Evaluators can also use qualitative methods of the evaluation to avoid the problems that ap- data collection. Both quantitative and qualita- peared in the actual study? tive data analyses can be appropriate in evalu- 2. Take a minute to think of the many ways your ation research, sometimes in the same study. society has changed during your own life- ❏ The special, logistical problems of evaluation time. How would you specify those changes research grow out of the fact that it occurs as social indicators that could be used in within the context of real life. monitoring the quality of life in your society? ADDITIONAL READINGS 411

3. Identify at least three deliberate social inter- you will fi nd Tutorial Quizzes with feedback, Inter- ventions, such as lowering the voting age to net Exercises, Flashcards, and Chapter Tutorials, as 18. For each, how would you (1) specify the well as Extended Projects, InfoTrac College Edition perceived problem and (2) describe the kind search terms, Social Research in Cyberspace, GSS of research that would evaluate whether the Data, Web Links, and primers for using various intervention was successful in solving the data analysis software such as SPSS and NVivo. perceived problem? 4. Think of something at your college that you feel could be improved. Now think of some- Additional Readings thing that could be done to solve the problem Berg, Richard, and Peter H. Rossi. 1998. Thinking about you’ve identifi ed. Pursue this line of thought Program Evaluation. Thousand Oaks, CA: Sage. Great until you’ve developed a clear operational book if you’re looking for gaining good foundations defi nition of how you would know when the in evaluation research while enjoying a wide range problem had been solved through some in- of examples. tervention. What future measurement would Bickman, Leonard, and Debra J. Rog, eds. 1998. Hand- represent success? book of Applied Social Research Methods. Thousand Oaks, CA: Sage. The two editors of this book have provided examples that illustrate all stages in evalu- Online Study Resources ation, from planning through the collection and analysis of data. In addition, they cover ethical issues in the particular context of an evaluation research. Chen, Huey-Tsyh. 1990. Theory-Driven Evaluations. Thou- sand Oaks, CA: Sage. Chen argues that evaluation Go to research must be fi rmly based in theory if it is to be meaningful and useful. http://sociology.wadsworth.com/babbie_basics4e Cunningham, J. Barton. 1993. Action Research and Orga- nizational Development. Westport, CT: Praeger. This ThomsonNow and click on for access to this book urges researchers to bridge the gap between powerful online study tool. You will get a per- theory and action, becoming engaged participants in sonalized study plan based on your responses to the evolution of organizational life and using social a diagnostic pretest. Once you have mastered research to monitor problems and solutions. the material with the help of interactive learning Hedrick, Terry E., Leonard Bickman, and Debra J. Rog. tools, you can take a posttest to confi rm that you 1993. Applied Research Design: A Practical Guide. are ready to move on to the next chapter. Thousand Oaks, CA: Sage. This introduction to evaluation research is, as its subtitle claims, a practical guide, dealing straight-out with the compromises that must usually be made in research design and Website for The Basics of Social Research execution. Rossi, Peter H., and Howard E. Freeman. 1996. Evalu- 4th edition ation: A Systematic Approach. Thousand Oaks, CA: Sage. This thorough examination of evaluation At the book companion website (http://sociology research is an excellent resource. In addition to dis- .wadsworth.com/babbie_basics4e) you will fi nd cussing the key concepts of evaluation research, the many resources in addition to ThomsonNow to authors provide numerous examples that might help aid you in studying for your exams. For example, you guide your own designs. PaulDavid Conklin/PhotoEdit/PictureQuest Falconer/Folio, Inc. Part Three Part Four OSERVATION MODES OF Part Three OF DATA ANALYSIS Part Four 12 11 10 9 8 15 14 13 Research Evaluation Research Unobtrusive Research Qualitative Field Survey Research Experiments Research Writing Social Reading and Data Analysis Quantitative Analysis Qualitative Data n this part of the book, we’ll discuss the anal- I ysis of qualitative and quantitative data, and we’ll examine the steps that separate observation from the fi nal reporting of fi ndings. In Chapter 1, I made a fundamental distinction between qualitative and quantitative data. In the subsequent discussions, we’ve seen that many of the fundamental concerns in social research apply equally to both types of data. The analysis of qualitative and quantitative data, however, are quite different and will be discussed separately. I hope the chapters that make up this part of the book will give you some tools and insights to produce sophisticated data analyses, whether qualitative or quantitative. Chapter 13 examines ways to analyze qualitative data. We’ll look at some of the basic steps involved, and then we’ll take an in-depth look at some of the computer software now available to qualitative analysts. The logic of quantitative data analysis is presented in Chapter 14. We’ll begin by examining the methods of analyzing and presenting the data related to a single variable. Then we’ll turn to the relationship between two variables and learn how to construct and read simple percentage tables. The chapter ends with a preview of multivariate analysis. Finally, Chapter 15 addresses social research as literature: how to read it and how to write it. The materials of this chapter are essentially bookends for the research process: A review of the literature early in the project involves the skills of reading social research, and writing it comes into play in the communication of your results to others in the form of your research report.

413 131 HUMANQUALITATIVE INQUIRY DATA AND ANALYSIS SCIENCE

Image not available due to copyright restrictions Photo credit

What You’ll Learn in thisThis ChapterChapter

HereWe’ll you’ll examine see that the qualitative way people data learn analysis about is their the nonnumericalworld and the assessmentmistakes they of makeobservations along the made way. through We’ll also participant begin to observation, see what makes content sience analysis, different in-depth from

otherinterviews, ways andof knowing other qualitative things. research techniques. Although qualitative 2S analysis is an art as much as a science, it has its own logic and techniques, some 1S N of which are enhanced by special computer programs. L 414 In this chapter . . . WHAT DO YOU THINK?

Introduction Why do research- Linking Theory and Analysis ers sometimes use Discovering Patterns qualitative analyses Grounded Theory Method Image not available due to copyright restrictions when they might Semiotics have used statis- Conversation Analysis tics? Isn’t a statisti- Qualitative Data Processing cal, or quantitative, Coding analysis a more “sci- Memoing entiﬁ c” way to study poverty, discrimination, Concept Mapping and so forth?

Computer Programs for Qualitative Data See the “What Do You Think? Revisited” box QDA Programs toward the end of the chapter. Leviticus as Seen through NUD*IST Using NVivo to Understand Women Film Directors, by Sandrine Zerbib observations: qualitative analysis—methods for The Qualitative Analysis examining social research data without convert- of Quantitative Data ing them to a numerical format. This approach predates quantitative analysis. It remains useful in Ethics and Qualitative Data Analysis data analysis and is even enjoying a resurgence of interest among social scientists. INTRODUCTION Although statistical analyses may intimidate some students, the steps involved can sometimes be learned by rote. That is, with practice, the rote Chapter 14 will deal with the quantitative analysis exercise of quantitative skills can produce an ever- of social research data, sometimes called statistical more sophisticated understanding of the logic that analysis. Recent decades of social science research lies behind those techniques. have tended to focus on quantitative data-analysis It is much more difﬁ cult to teach qualitative techniques. This focus, however, sometimes con- analysis as a series of rote procedures. In this case, ceals another approach to making sense of social understanding must precede practice. In this chapter, we begin with the links between research and theory in qualitative analysis. Then we examine The nonnumerical examination qualitative analysis some procedures that have proven useful in pur- and interpretation of observations, for the purpose of discovering underlying meanings and patterns of relation- suing the theoretical aims. After considering some ships. This approach is most typical of ﬁ eld research and simple manual techniques, we’ll take some com- historical research. puter programs out for a spin.

415 416 CHAPTER 13 QUALITATIVE DATA ANALYSIS

LINKING THEORY AND ANALYSIS ested in analyzing child abuse in a certain neighborhood. Here are some questions you might ask yourself, in order to make sense out of your data: As suggested in Chapter 10 and elsewhere in this book, qualitative research methods involve a con- 1. Frequencies: How often does child abuse occur tinuing interplay between data collection and the- among families in the neighborhood under ory. In quantitative research, it’s sometimes easy study? (Realize that there may be a difference to get caught up in the logistics of data collection between the frequency and what people are and in the statistical analysis of data, thereby los- willing to tell you.) ing sight of theory for a time. This occurs less in 2. Magnitudes: What are the levels of abuse? qualitative research, where data collection, analy- How brutal are they? sis, and theory intertwine more intimately. 3. Structures: What are the different types of In the discussions to follow, we’ll use the im- abuse: physical, mental, sexual? Are they re- age of theory offered by Anselm Strauss and Ju- lated in any particular manner? liet Corbin (1994:278) as consisting of “plausible 4. Processes: Is there any order among the ele- relationships proposed among concepts and sets ments of structure? Do abusers begin with of concepts.” They stress “plausible” to indicate mental abuse and move on to physical and that theories represent our best understanding of sexual abuse, or does the order of elements how life operates. The more our research confi rms vary? a particular set of relationships among particular 5. Causes: What are the causes of child abuse? concepts, however, the more confi dent we be- Is it more common in particular social classes come that our understanding corresponds to so- or among different religious or ethnic groups? cial reality. Does it occur more often during good times Although qualitative research is sometimes or bad? undertaken for purely descriptive purposes—such 6. Consequences: How does child abuse affect as the anthropologist’s ethnography detailing the the victims, in both the short and the long ways of life in a previously unknown tribe—the rest term? What changes does it cause in the of this chapter focuses primarily on the search for abusers? explanatory patterns. As we’ll see, sometimes the For the most part, in examining your data you’ll patterns occur over time, and sometimes they take look for patterns appearing across several observa- the form of causal relations among variables. Let’s tions that typically represent different cases under look at some of the ways qualitative researchers study. This approach is called cross-case anal- uncover such patterns. ysis. A. Michael Huberman and Matthew Miles (1994) offer two strategies for cross-case analy- Discovering Patterns sis: variable-oriented and case-oriented analysis. Variable-oriented analysis is similar to a model John Lofl and and colleagues (2006:149–65) suggest we’ve already discussed from time to time in this six different ways of looking for patterns in a par- book. If we were trying to predict the decision to ticular research topic. Let’s suppose you’re inter- attend college, Huberman and Miles suggest, we might consider variables such as “gender, socioeconomic status, parental expectations, school cross-case analysis An analysis that involves an examination of more than one case, either a variable- performance, peer support, and decision to attend oriented or case-oriented analysis. college” (1994:435). Thus, we would determine variable-oriented analysis An analysis that de- whether men or women were more likely to at- scribes and/or explains a particular variable. tend college. The focus of our analysis would be LINKING THEORY AND ANALYSIS 417 on interrelations among variables, and the people sive and pursue issues in greater depth. This full, observed would be primarily the carriers of those idiographic examination, however, tells us nothing variables. about people in general. It offers no theory about Variable-oriented analysis may remind you of why people choose to attend college. the discussion in Chapter 1 that introduced the Even so, in addition to understanding one per- idea of nomothetic explanation. The aim here is son in great depth, the researcher sees the critical to achieve a partial, overall explanation using a elements of the subject’s experiences as instances relatively few number of variables. The political of more general social concepts or variables. For pollster who attempts to explain voting intentions example, Nynke’s mother’s social work training on the basis of two or three key variables is us- can also be seen as “mother’s education.” Her ing this approach. There is no pretense that the friend’s decision can be seen as “peer infl uence.” researcher can predict every individual’s behavior More specifi cally, these could be seen as indepen- nor even explain any one person’s motivations in dent variables having an impact on the dependent full. Sometimes, though, it’s useful to have even variable of attending college. a partial explanation of overall orientations and Of course, one case does not a theory make— actions. hence, Huberman and Miles’s reference to cross- You may also recall Chapter 1’s introduction of case analysis, in which the researcher turns to other idiographic explanation, wherein we attempt to subjects, looking into the full details of their lives understand a particular case fully. In the voting ex- as well but paying special note to the variables that ample, we would attempt to learn everything we seemed important in the fi rst case. How much and could about all the factors that came into play in what kind of education did other subjects’ mothers determining one person’s decision on how to vote. have? Is there any evidence of close friends attend- This orientation lies at the base of what Huberman ing college? and Miles call a case-oriented analysis. Some subsequent cases will closely parallel the fi rst one in the apparent impact of particular vari- In a case-oriented analysis, we would look more ables. Other cases will bear no resemblance to the closely into a particular case, say, Case 005, who fi rst. These latter cases may require the identifi ca- is female, middle-class, has parents with high tion of other important variables, which may invite expectations, and so on. These are, however, the researcher to explore why some cases seem to “thin” measures. To do a genuine case analysis, refl ect one pattern whereas others refl ect another. we need to look at a full history of Case 005; Nynke van der Molen, whose mother trained as a social worker but is bitter over the fact Grounded Theory Method that she never worked outside the home, and The cross-case method just described should whose father wants Nynke to work in the family sound somewhat familiar. In the discussion of fl orist shop. Chronology is also important: two grounded theory in Chapter 10, we saw how quali- years ago, Nynke’s closest friend decided to go tative researchers sometimes attempt to establish to college, just before Nynke began work in a theories on a purely inductive basis. This approach stable and just before Nynke’s mother showed begins with observations rather than hypotheses her a scrapbook from social work school. Nynke and seeks to discover patterns and develop theo- then decided to enroll in veterinary studies. ries from the ground up, with no preconceptions, — (1994:436)

This abbreviated commentary should give some case-oriented analysis An analysis that aims to under- idea of the detail involved in this type of analysis. stand a particular case or several cases by looking closely Of course, an entire analysis would be more exten- at the details of each. 418 CHAPTER 13 QUALITATIVE DATA ANALYSIS although some research may build and elaborate the different incidents. This process is similar on earlier grounded theories. to conceptualization as described in Chapter Recall that grounded theory was fi rst devel- 5—specifying the nature and dimensions of the oped by the sociologists Barney Glaser and An- many concepts arising from the data. selm Strauss (1967), in an attempt to come to grips 2. ”Integrating categories and their properties.” with their clinical research in medical sociology. Here the researcher begins to note relation- Since then, it has evolved as a method, with the ships among concepts. In the assessment of cofounders taking it in slightly different directions. social loss, for example, Glaser and Strauss The following discussion will deal with the basic found that nurses took special notice of a concepts and procedures of the Grounded The- patient’s age, education, and family respon- ory Method (GTM). sibilities. For these relationships to emerge, however, it was necessary for the researchers to have noticed all these concepts. You can hear Barney Glaser discuss 3. ”Delimiting the theory.” Eventually, as the pat- grounded theory on the web* at http:// www.groundedtheory.com/vidseries1.html. terns of relationships among concepts become clearer, the researcher can ignore some of the concepts initially noted but evidently irrelevant to the inquiry. In addition to the number of In addition to the fundamental, inductive te- categories being reduced, the theory itself may net of building theory from data, GTM employs become simpler. In the examination of social the constant comparative method. As Glaser loss, for example, Glaser and Strauss found and Strauss originally described this method, it in- that the assessment processes could be gener- volved four stages — (1967:105–13): alized beyond nurses and dying patients: They 1. ”Comparing incidents applicable to each cat- seemed to apply to the ways all staff dealt with egory.” As Glaser and Strauss researched the all patients (dying or not). reactions of nurses to the possible death of pa- 4. ”Writing theory.” Finally, the researcher must tients in their care, the researchers found that put his or her fi ndings into words to be shared the nurses were assessing the “social loss” at- with others. As you may have already experi- tendant on a patient’s death. Once this concept enced for yourself, the act of communicating arose in the analysis of one case, they looked your understanding of a topic actually modi- for evidence of the same phenomenon in other fi es and even improves your own grasp of it. cases. When they found the concept arising In GTM, the writing stage is regarded as a part in the cases of several nurses, they compared of the research process. A later section of this chapter (on memoing) elaborates on this point.

Grounded Theory Method (GTM) An inductive This brief overview should give you an idea of approach to research introduced by Barney Glaser and how grounded theory proceeds. The many tech- Anselm Strauss in which theories are generated solely from an examination of data rather than being derived niques associated with GTM can be found both deductively. in print and on the web. One of the key publica- constant comparative method A component of tions is Anselm Strauss and Juliet Corbin’s Basics the Grounded Theory Method in which observations are of Qualitative Research (1998), which elaborates on compared with one another and with the evolving induc- and extends many of the concepts and techniques tive theory. found in the original Glaser/Strauss volume. GTM is only one analytical approach to quali-

*Each time the Internet icon appears, you’ll be given help- tative data. In the remainder of this section, we’ll ful leads for searching the World Wide Web. look at some other specialized techniques. LINKING THEORY AND ANALYSIS 419

You might want to explore Gaelle T. SIGN MEANING Morin’s “Grounded Theory Methodology” 1. Poinsettia a. Good luck on the web at http://gtm.vlsm.org/ 2. Horseshoe b. First prize gtm-12.en.html. 3. Blue ribbon c. Christmas 4. "Say cheese" d. Acting 5. "Break a leg" e. Smile for a picture

Semiotics FIGURE 13-1 Matching Signs and Their Meanings Semiotics is commonly defi ned as the “science of signs” and has to do with symbols and meanings. It’s often associated with content analysis, which some “emoticons” like : ) —another example of was discussed in Chapter 11, though it can be ap- semiotics.) plied in a variety of research contexts. While there is no doubt a story behind each of Peter Manning and Betsy Cullum-Swan the linkages in Figure 13-1, the meanings you and (1994:466) offer some sense of the applicability of I “know” today are socially constructed. Semiotic semiotics, as follows: “Although semiotics is based analysis involves a search for the meanings inten- on language, language is but one of the many sign tionally or unintentionally attached to signs. systems of varying degrees of unity, applicability, Consider the sign shown in Figure 13-2, from a and complexity. Morse code, etiquette, mathemat- hotel lobby in Portland, Oregon. What’s being com- ics, music, and even highway signs are examples municated by the rather ambiguous sign? The fi rst of semiotic systems.” sentence seems to be saying that the hotel is up to There is no meaning inherent in any sign, how- date with the current move away from tobacco in ever. Meanings reside in minds. So, a particular the United States. Guests who want a smoke-free sign means something to a particular person. How- environment need look no farther: This is a healthy ever, the agreements we have about the meanings place to stay. At the same time, says the second associated with particular signs make semiotics sentence, the hotel would not like to be seen as in- a social science. As Manning and Cullum-Swan hospitable to smokers. There’s room for everyone point out, under this roof. No one need feel excluded. This sign is more easily understood within a marketing For example, a lily is an expression linked con- paradigm than one of logic. ventionally to death, Easter, and resurrection The “signs” examined in semiotics, of course, as a content. Smoke is linked to cigarettes and are not limited to this kind of sign. Most are quite to cancer, and Marilyn Monroe to sex. Each of different, in fact. Signs are any things that are as- these connections is social and arbitrary, so that signed special meanings. They can include such many kinds of links exist between expression things as logos, animals, people, and consumer and content. — (1994:466) products. Sometimes the symbolism is subtle. You can fi nd a classic analysis in Erving Goffman’s Gen- To explore this contention, see if you can link der Advertisements (1979). Goffman focused on ad- the signs with their meanings in Figure 13-1. I’m vertising pictures found in magazines and news- confi dent enough that you know all the “correct” papers. The overt purpose of the ads, of course, associations that there’s no need for me to give the was to sell specifi c products. But what else was answers. (OK, you should have said 1c, 2a, 3b, 4e, 5d.) The point is this: What do any of these signs have to do with their “meanings”? Draft an e-mail semiotics The study of signs and the meanings associ- message to a Martian social scientist explaining ated with them. This is commonly associated with content the logic at work here. (You might want to include analysis. 420 CHAPTER 13 QUALITATIVE DATA ANALYSIS Earl Babbie FIGURE 13-2 Mixed Signals?

communicated? What in particular did the ads say taller than the men. In these cases, the men were about men and women? typically of a lower social status—the chef beside Analyzing pictures containing both men and the society matron, for example. This conﬁ rmed women, Goffman was struck by the fact that men Goffman’s main point that size and height indi- were almost always bigger and taller than the cated social status. women accompanying them. (In many cases, in The same conclusion could be drawn from pic- fact, the picture managed to convey the distinct tures with men of different heights. Those of higher impression that the women were merely accom- status were taller, whether it was the gentleman panying the men.) Although the most obvious ex- speaking to a waiter or the boss guiding the work planation is that men are, on average, heavier and of his younger assistants. Where actual height was taller than women, Goffman suggested the pattern unclear, Goffman noted the placement of heads in had a different meaning: that size and placement the picture. The assistants were crouching down implied status. Those larger and taller presumably while the boss leaned over them. The servant’s had higher social standing—more power and au- head was bowed so it was lower than that of the thority (1979:28). Goffman suggested that the ads master. communicated that men were more important The latent message conveyed by the ads, then, than women. was that the higher a person’s head appeared in the In the spirit of Freud’s comment that “some- ad, the more important that person was. And in the times a cigar is just a cigar” (he was a smoker), great majority of ads containing men and women, how would you decide whether the ads simply the former were clearly portrayed as more impor- reﬂ ected the biological differences in the average tant. The subliminal message in the ads, whether sizes of men and women or whether they sent a intended or not, was that men are more powerful message about social status? In part, Goffman’s and enjoy a higher status than do women. conclusion was based on an analysis of the excep- Goffman examined several differences besides tional cases: those in which the women appeared physical size in the portrayal of men and women. QUALITATIVE DATA PROCESSING 421

As another example, men were typically presented meaning of “Same to you!” varies if preceded by “I in active roles, women in passive ones. The (male) don’t like your looks” or by “Have a nice day.” doctor examined the child while the (female) Third, CA aims to understand the structure and nurse or mother looked on, often admiringly. A meaning of conversation through excruciatingly man guided a woman’s tennis stroke (all the while accurate transcripts of conversations. Not only keeping his head higher than hers). A man gripped are the exact words recorded, but all the uhs, ers, the reins of his galloping horse, while a woman bad grammar, and pauses are also noted. Pauses, rode behind him with her arms wrapped around in fact, are measured to the nearest tenth of a his waist. A woman held the football, while a man second. kicked it. A man took a photo, which contained The practical uses of this type of analysis are only women. many. Ann Marie Kinnell and Douglas Maynard Goffman suggested that such pictorial patterns (1996), for example, analyzed conversations be- subtly perpetuated a host of gender stereotypes. tween staff and clients at an HIV testing clinic Even as people spoke publicly about gender equal- to examine how information about safe sex was ity, these advertising photos established a quiet communicated. Among other things, they found backdrop of men and women in their “proper that the staff tended to provide standard informa- roles.” tion rather than try to speak directly to a client’s specifi c circumstances. Moreover, they seemed re- Conversation Analysis luctant to give direct advise about safe sex, settling for information alone. Ethnomethodology, as you’ll recall, aims to un- These discussions should give you some sense cover the implicit assumptions and structures in of the variety of qualitative analysis methods avail- social life. Conversation analysis (CA) seeks to able to researchers. Now let’s look at some of the pursue that aim through an extremely close scru- data-processing and data-analysis techniques tiny of the way we converse with one another. In commonly used in qualitative research. the examination of ethnomethodology in Chapter 10, you saw some examples of conversation anal- QUALITATIVE DATA PROCESSING ysis. Here we’ll look a little more deeply into that technique. David Silverman (1993:125f), reviewing the Let me begin this section with a warning. The ac- work of other CA theorists and researchers, speaks tivity we are about to examine is as much art as of three fundamental assumptions. First, conversa- science. At the very least, there are no cut-and- tion is a socially structured activity. Like other so- dried steps that guarantee success. cial structures, it includes established rules of be- It’s a lot like learning how to paint with water- havior. For example, we’re expected to take turns, colors or compose a symphony. You can certainly with only one person speaking at a time. In tele- gain education in such activities; you can even take phone conversations, the person answering the university courses in both. Each has its own con- call is expected to speak fi rst (as in “Hello”). You ventions and techniques as well as tips you may can verify the existence of this rule, incidentally, by fi nd useful as you set out to create art or music. picking up the phone without speaking. You may However, instruction can carry you only so far. The recall that this is the sort of thing ethnomethodolo- fi nal product must come from you. Much the same gists tend to do. can be said of qualitative data processing. Second, Silverman points out that conversations must be understood contextually. The same conversation analysis (CA) A meticulous analysis utterance will have totally different meanings in of the details of conversation, based on a complete tran- different contexts. For example, notice how the script that includes pauses, hems, and also haws. 422 CHAPTER 13 QUALITATIVE DATA ANALYSIS

This section presents some ideas on coding tion, we’ll assume that you’ll be doing your coding qualitative data, writing memos, and mapping manually. The next-to-last section of the chapter concepts graphically. Although far from a “how-to” will illustrate the use of computer programs for manual, these ideas give a useful starting point for qualitative data analysis. fi nding order in qualitative data. Coding Units As you may recall from the earlier Coding discussion of content analysis, for statistical analysis it’s important to identify a standardized unit Whether you engage in participant observation, of analysis prior to coding. If you were comparing in-depth interviewing, collecting biographical nar- American and French novels, for example, you ratives, doing content analysis, or some other form might evaluate and code sentences, paragraphs, of qualitative research, you will eventually possess chapters, or whole books. It would be important, a growing mass of data—most typically in the form however, to code the same units for each novel of textual materials. What do you do next? analyzed. This uniformity is necessary in a quanti- The key process in the analysis of qualitative tative analysis, as it allows us to report something social research data is coding—classifying or cat- like “Twenty-three percent of the paragraphs con- egorizing individual pieces of data—coupled with tained metaphors.” This is only possible if we’ve some kind of retrieval system. Together, these pro- coded the same unit—paragraphs—in each of the cedures allow you to retrieve materials you may novels. later be interested in. Coding data for a qualitative analysis, however, Let’s say you’re chronicling the growth of a so- is quite different. The concept is the organizing cial movement. You recall writing up some notes principle for qualitative coding. Here the units of about the details of the movement’s earliest begin- text appropriate for coding will vary within a given nings. Now you need that information. If all your document. Thus, in a study of organizations, “Size” notes have been catalogued by topic, retrieving might require only a few words per coding unit, those you need should be straightforward. As a whereas “Mission” might take a few pages. Or, a simple format for coding and retrieval, you might lengthy description of a heated stockholders meet- have created a set of fi le folders labeled with vari- ing might be coded as “Internal Dissent.” ous topics, such as “History.” Data retrieval in this Realize also that a given code category may case means pulling out the “History” folder and ri- be applied to textual materials of quite different fl ing through the notes contained therein until you lengths. For example, some references to the or- fi nd what you need. ganization’s mission may be brief, others lengthy. As you’ll see later in this chapter, several so- Whereas standardization is a key principle in phisticated computer programs allow for a faster, quantitative analysis, this is not the case in quali- more certain, and more precise retrieval process. tative analysis. Rather than looking through a “History” fi le, you can go directly to notes dealing with the “Earliest Coding as a Physical Act Before continuing with History” or the “Founding” of the movement. the logic of coding, let’s take a moment to see what Coding has another, even more important pur- it actually looks like. Lofl and and colleagues offer pose. As discussed earlier, the aim of data analysis this description of manual fi ling: is the discovery of patterns among the data, patterns that point to a theoretical understanding of Prior to the widespread availability of personal social life. The coding and relating of concepts is computers beginning in the late 1980s, coding key to this process and requires a more refi ned frequently took the specifi c physical form of system than a set of manila folders. In this sec- fi ling. The researcher established an expand- QUALITATIVE DATA PROCESSING 423

ing set of fi le folders with code names on the grouped under more abstract concepts termed tabs and physically placed either the item of “categories.” — (1998:102) data itself or a note that referenced its location Open coding is the logical starting point for in another fi le folder. Before photocopying was GTM qualitative coding. Beginning with some easily available and cheap, some fi eldworkers body of text (part of an interview, for example), you typed their fi eldnotes with carbon paper, wrote read and reread a passage, seeking to identify the codes in the margins of the copies of the notes, key concepts contained within it. Any particular and cut them up with scissors. They then placed piece of data may be given several codes, refl ect- the resulting slips of paper in corresponding fi le ing as many concepts. For example, notice all the folders. — (2006:203) concepts contained in this comment by a student As these researchers point out, personal com- interviewee: puters have greatly simplifi ed this task. However, I thought the professor should have given me at the image of slips of paper that contain text and least partial credit for the homework I turned in. are put in folders representing code categories is useful for understanding the process of coding. In Some obvious codes are “Professor,” “Homework,” the next section, when I suggest that we code a and “Grading.” The result of open coding is the textual passage with a certain code, imagine that identifi cation of numerous concepts relevant to the we have the passage typed on a slip of paper and subject under study. The open coding of more and that we place it in a fi le folder bearing the name more text will lengthen the list of codes. of the code. Whenever we assign two codes to a Besides open coding, two other types of coding passage, imagine placing duplicate copies of the take place in this method. Axial coding aims to passage in two different folders representing the identify the core concepts in the study. Although two codes. axial coding uses the results of open coding, more concepts can be identifi ed through continued open Creating Codes So, what should your code cat- coding after the axial coding has begun. Axial cod- egories be? Glaser and Strauss (1967:101f ) allow ing involves a regrouping of the data, in which for the possibility of coding data for the purpose the researcher uses the open code categories and of testing hypotheses that have been generated by looks for more-analytical concepts. For example, prior theory. In that case, then, the theory would the passage just given also carries the concept of suggest the codes, in the form of variables. “perceptions of fairness,” which might appear fre- In this section, however, we’re going to focus quently in the student interviews, thereby suggest- on the more common process of open coding. ing that it’s an important element in understanding Strauss and Corbin defi ne it as follows: students’ concerns. Another axial code refl ected in the student comment might be “power relation- To uncover, name, and develop concepts, we ships,” because the professor is seen to exercise must open up the text and expose the thoughts, power over the student. ideas, and meanings contained therein. Without the fi rst analytic step, the rest of the analysis and the communication that follows could not occur. Broadly speaking, during open coding, open coding The initial classifi cation and labeling of concepts in qualitative data analysis. In open coding, the data are broken down into discrete parts, closely codes are suggested by the researchers’ examination and examined, and compared for similarities and questioning of the data. differences. Events, happenings, objects, and ac- axial coding A reanalysis of the results of open coding tions/interactions that are found to be concep- in Grounded Theory Method, aimed at identifying the tually similar in nature or related in meaning are important, general concepts. 424 CHAPTER 13 QUALITATIVE DATA ANALYSIS

The last kind of coding, selective coding, analysis of Leviticus, we want to consider these seeks to identify the central code in the study: the two passages. one that all the other codes related to. Both of the Because homosexuality is such a key concept, axial codes just mentioned might be restructured let’s look more closely into what it means within as aspects of a more general concept: “professor- the data under study. We fi rst notice the way ho- student relationships.” Of course, in a real data mosexuality is identifi ed: a man lying with a man analysis, decisions such as the ones we’ve been “as with a woman.” Although we can imagine a discussing would arise from masses of textual lawyer seeking admission to heaven saying, “But data, not from a single quotation. The basic no- here’s my point; if we didn’t actually lie down . . . “ tion of the Grounded Theory Method is that pat- it seems safe to assume the passage refers to hav- terns of relationships can be teased out of an ex- ing sex, though it is not clear what specifi c acts tensive, in-depth examination of a large body of might or might not be included. observations. Notice, however, that the injunctions appear to Here’s a concrete example to illustrate how concern male homosexuality only; lesbianism is you might engage in this form of analysis. Sup- not mentioned. In our analysis, then, each of these pose you’re interested in the religious bases for passages might also be coded “Male Homosexual- homophobia. You’ve interviewed some people op- ity.” This illustrates two more aspects of coding: (1) posed to homosexuality who cite a religious ba- Each unit can have more than one code and (2) sis for their feelings. Specifi cally, they refer you to hierarchical codes (one included within another) these passages in the Book of Leviticus (Revised can be used. Now each passage has two codes as- Standard Version): signed to it. An even more general code might be intro- 18:22 You shall not lie with a male as with a duced at this point: “Prohibited Behavior.” This is woman; it is an abomination. important for two reasons. First, homosexuality 20:13 If a man lies with a male as with a woman, is not inherently wrong, from an analytical stand- both of them have committed an abomination; point. The purpose of the study is to examine the they shall be put to death, their blood is upon way it’s made wrong by the religious texts in ques- them. tion. Second, our study of Leviticus may turn up Although the point of view expressed here other behaviors that are prohibited. seems unambiguous, you might decide to examine There are at least two more critical concepts in it in more depth. Perhaps a qualitative analysis of the passages: “Abomination” and “Put to Death.” Leviticus can yield a fuller understanding of where Notice that whereas these are clearly related to these injunctions against homosexuality fi t into the “Prohibited Behavior,” they are hardly the same. larger context of Judeo-Christian morality. Parking without putting money in the meter is pro- Let’s start our analysis by examining the two hibited, but few would call it an abomination and passages just quoted. We might begin by coding fewer still would demand the death penalty for that each passage with the label “Homosexuality.” This transgression. Let’s assign these two new codes to is clearly a key concept in our analysis. Whenever our fi rst two passages. we focus on the issue of homosexuality in our At this point, we want to branch out from the two key passages and examine the rest of Leviti- cus. We therefore examine and code each of the In Grounded Theory Method, this selective coding remaining chapters and verses. In our subsequent analysis builds on the results of open coding and axial coding to identify the central concept that organizes the other analyses, we’ll use the codes we have already and concepts that have been identifi ed in a body of textual add new ones as appropriate. When we do add materials. new codes, it will be important to review the pas- QUALITATIVE DATA PROCESSING 425 sages already coded to see whether the new codes 11:42 Whatever goes on its belly, and whatever apply to any of them. goes on all fours, or whatever has many feet, Here are the passages we decide to code all the swarming things that swarm upon “Abomination.” (I’ve boldfaced the abominations.) the earth, you shall not eat; for they are an abomination. 7:18 If any of the fl esh of the sacrifi ce of his 11:43 You shall not make yourselves abominable peace offering is eaten on the third day, with any swarming thing that swarms; and he who offers it shall not be accepted, nei- you shall not defi le yourselves with them, lest ther shall it be credited to him; it shall be an you become unclean. abomination, and he who eats of it shall bear 18:22 You shall not lie with a male as with a his iniquity. woman; it is an abomination. 7:21 And if any one touches an unclean thing, 19:6 It shall be eaten the same day you offer it, or whether the uncleanness of man or an unclean on the morrow; and anything left over until the beast or any unclean abomination, and then third day shall be burned with fi re. eats of the fl esh of the sacrifi ce of the 19:7 If it is eaten at all on the third day, it is LORD’s peace offerings, that person shall be an abomination; it will not be accepted, cut off from his people. 19:8 and every one who eats it shall bear his 11:10 But anything in the seas or the rivers iniquity, because he has profaned a holy thing that has not fi ns and scales, of the swarm- of the LORD; and that person shall be cut off ing creatures in the waters and of the living from his people. creatures that are in the waters, is an abomi- 20:13 If a man lies with a male as with a nation to you. woman, both of them have committed an 11:11 They shall remain an abomination to you; abomination; they shall be put to death, their of their fl esh you shall not eat, and their blood is upon them. carcasses you shall have in abomination. 20:25 You shall therefore make a distinction 11:12 Everything in the waters that has not between the clean beast and the unclean, and fi ns and scales is an abomination to you. between the unclean bird and the clean; you 11:13 And these you shall have in abomination shall not make yourselves abominable among the birds, they shall not be eaten, by beast or by bird or by anything with they are an abomination: the eagle, the which the ground teems, which I have set vulture, the osprey, apart for you to hold unclean. 11:14 the kite, the falcon according to its kind, 11:15 every raven according to its kind, Male homosexuality, then, isn’t the only abomi- 11:16 the ostrich, the nighthawk, the sea gull, nation identifi ed in Leviticus. As you compare the hawk according to its kind, these passages, looking for similarities and dif- 11:17 the owl, the cormorant, the ibis, ferences, it will become apparent that most of the 11:18 the water hen, the pelican, the carrion abominations have to do with dietary rules—spe- vulture, cifi cally those potential foods deemed “unclean.” 11:19 the stork, the heron according to its kind, Other abominations fl ow from the mishandling of the hoopoe, and the bat. ritual sacrifi ces. “Dietary Rules” and “Ritual Sacri- 11:20 All winged insects that go upon all fi ces” thus represent additional codes to be used in fours are an abomination to you. our analysis. 11:41 Every swarming thing that swarms upon Earlier, I mentioned the death penalty as an- the earth is an abomination; it shall not be other concept to be explored in our analysis. When eaten. we take this avenue, we discover that many be- 426 CHAPTER 13 QUALITATIVE DATA ANALYSIS haviors besides male homosexuality warrant the death penalty. Among them are these: IN THE REAL WORLD SEXUAL ORIENTATION 20:2 Giving your children to Molech (human CONTROVERSY sacrifi ce) 20:9 Cursing your father or mother 20:10 Adultery with your neighbor’s wife A simple qualitative analysis such as the 20:11 Adultery with your father’s wife Leviticus example sheds new light on a key 20:12 Adultery with your daughter-in-law civil rights issues in the United States today. 20:14 Taking a wife and her mother also People are harassed, discriminated against, 20:15 Men having sex with animals (the animals and even killed because of their sexual ori- are to be killed, also) entation. When GSS respondents are asked 20:16 Women having sex with animals for their opinions about homosexuality, “Al- 20:27 Being a medium or wizard ways wrong” is the most frequent response 24:16 Blaspheming the name of the Lord selected, followed by “Never wrong,” with 24:17 Killing a man small minorities choosing more moderate, mixed views. As you can see, the death penalty is broadly Anti-gay sentiments and actions are of- applied in Levicitus: everything from swearing to ten justifi ed on religious grounds, specifi - murder, including male homosexuality somewhere cally the passages in Leviticus cited in this in between. chapter. But the longer list of abominations An extended analysis of prohibited behavior, in Leviticus was used in the TV series West short of abomination and death, also turns up a Wing to debunk the homophobic preaching lengthy list. Among them are slander, vengeance, of a radio talk-jock. grudges, cursing the deaf, and putting stumbling blocks in front of blind people. In chapter 19, verse 19, Leviticus quotes God as ordering, “You shall not let your cattle breed with a different kind; you shall not sow your fi eld with two kinds of seed; “Sexual Orientation Controversy” for more on this nor shall there come upon you a garment of cloth sort of coding activity. made of two kinds of stuff.” Shortly thereafter, he I hope this brief glimpse into a possible analysis adds, “You shall not eat any fl esh with the blood will give you some idea of the process by which in it. You shall not practice augury or witchcraft. codes are generated and applied. You should also You shall not round off the hair on your temples have begun to see how such coding would allow or mar the edges of your beard.” Tattoos were pro- you to understand better the messages being put hibited, though Leviticus is silent on body piercing. forward in a text and to retrieve data appropriately References to all of these practices would be coded as you need them. “Prohibited Acts” and perhaps given additional codes as well (recall “Dietary Rules”). See the box Memoing

In the Grounded Theory Method, the coding process involves more than simply categorizing chunks of Writing memos that become part of the memoing text. As you code data, you should also be using data for analysis in qualitative research such as grounded theory. Memos can describe and defi ne concepts, deal the technique of memoing—writing memos or with methodological issues, or offer initial theoretical notes to yourself and others involved in the proj- formulations. ect. Some of what you write during analysis may QUALITATIVE DATA PROCESSING 427 end up in your fi nal report; much of it will at least not follow this pattern. It might be characterized as stimulate what you write. a process of creating chaos and then fi nding order In GTM, these memos have a special signifi - within it. cance. Strauss and Corbin (1998:217) distinguish To explore this process further, refer to the three kinds of memos: code notes, theoretical works cited in this discussion and at the end of notes, and operational notes. the chapter. You’ll also fi nd a good deal of infor- Code notes identify the code labels and their mation on the web. Ultimately, the best education meanings. This is particularly important because, in this process comes from practice. Even if you as in all social science research, most of the terms don’t have a research project under way, you can we use with technical meanings also have mean- practice now on class notes. Or start a journal and ings in everyday language. It’s essential, therefore, code it. to write down a clear account of what you mean by the codes used in your analysis. In the Leviticus Concept Mapping analysis, for example, you would want a code note regarding the meaning of “Abomination” and how It should be clear by now that qualitative data ana- you’ve used that code in your analysis of text. lysts spend a lot of time committing thoughts to Theoretical notes cover a variety of topics: refl ec- paper (or to a computer fi le) and fi guring out how tions of the dimensions and deeper meanings of they relate to one another. Often, we can think out concepts, relationships among concepts, theoreti- relationships among concepts even more clearly by cal propositions, and so on. All of us occasionally putting the concepts in a graphical format, a pro- ruminate over the nature of something, try to think cess called concept mapping. Some researchers it out, to make sense out of it. In qualitative data fi nd it useful to put all their major concepts on a analysis, it’s vital to write down these thoughts, single sheet of paper, whereas others spread their even those you’ll later discard as useless. They will thoughts across several sheets of paper, black- vary greatly in length, but you should limit each to boards, magnetic boards, computer pages, or other a single main thought so that you can sort and or- media. Figure 13-3 shows how we might think out ganize them all later. In the Leviticus analysis, one some of the concepts of Goffman’s examination of theoretical note might discuss the way that most of gender and advertising. (This image was created the injunctions implicitly address the behavior of through the use of Inspiration, a concept-mapping men, with women being mostly incidental. computer program.) Operational notes deal primarily with method- Incidentally, many of the topics discussed in this ological issues. Some will draw attention to data- section have useful applications in quantitative as collection circumstances that may be relevant to well as qualitative analyses. Certainly, concept understanding the data later on. Others will con- mapping is appropriate in both types of analysis. sist of notes directing future data collection. The several types of memos would also be useful These memos are written throughout the data- in both. And the discussion of coding readily ap- collection and analysis process. Thoughts de- plies to the coding of open-ended questionnaire manding memos will come to you as you reread responses for the purpose of quantifi cation and notes or transcripts, code chunks of text, or discuss statistical analysis. (We’ll look at coding again in the project with others. It’s a good idea to get in the the next chapter, on quantifying data.) habit of writing out your memos as soon as possible after the thoughts come to you. Notice that whereas we often think of writing concept mapping The graphical display of concepts as a linear process, starting at the beginning and and their interrelations, useful in the formulation of moving through to the conclusion, memoing does theory. 428 CHAPTER 13 QUALITATIVE DATA ANALYSIS

White-out can fully appreciate the glory of com- Active/ puters as a note-taking device. “Easier editing” Physical passive location roles and “easier duplication” simply do not capture the scope of the advance. Moving beyond the basic recording and storage Social Gender of data, simple word-processing programs can be status used for some data analysis. The “fi nd” or “search” command will take you to passages containing key words. Or, going one step further, you can type code words alongside passages in your notes so Servant / Power Authority master that you can search for those keywords later. Database and spreadsheet programs can also be used for processing and analyzing qualitative Social data. Figure 13-4 offers a simple illustration of how worth some of the verses from Leviticus might be manip- FIGURE 13-3 An Example of Concept Mapping ulated within a spreadsheet. The three columns to the left represent three of the concepts we’ve discussed. An “x” means that the passage to the right contains that concept. As shown, the passages are Having noted the overlap of qualitative and sorted in such a way as to gather all those dealing quantitative techniques, it seems fi tting now to with punishment by death. Another simple “sort” address an instrument that is primarily associated command would gather all those dealing with sex, with quantitative research but that is proving quite with homosexuality, or with any of the other con- valuable for qualitative analysts as well—the per- cepts coded. sonal computer. This brief illustration should give you some idea of the possibilities for using readily available programs as tools in qualitative data analysis. Happily, COMPUTER PROGRAMS there are now a large number of programs created FOR QUALITATIVE DATA specifi cally for that purpose.

The advent of computers, both mainframe and per- QDA Programs sonal, has been a boon to quantitative research, The simple spreadsheet in Figure 13-4 should give allowing the rapid calculation of extremely com- you a basic idea of how computers might be used plex statistics. The importance of the computer for the analysis of qualitative social research data. for qualitative research has been somewhat more However, there are now a long list of sophisticated slowly appreciated. Some qualitative researchers computer programs available for this purpose. were quick to adapt the basic capacities of com- Where the analyst’s problem used to be merely puters to nonnumerical tasks, but it took a bit lon- ﬁ nding such a program, the problem now lies in ger for programmers to address the speciﬁ c needs choosing one of so many. Here are a few commonly of qualitative research. Today, however, many used qualitatative-data analysis (QDA) programs powerful programs are available. with online sites where you can learn more about Let’s start this section with a brief overview them and, in many cases, download demo copies. of some of the ways you can use basic computer tools in qualitative research. Perhaps only those • Alceste: http://www.image.cict.fr/english/ who can recall hours spent with carbon paper and index_alceste.htm COMPUTER PROGRAMS FOR QUALITATIVE DATA 429

sex homosex death Verse Passage

XX X20:13 If a man lies with a male as with a woman, both of them have committed an abomination; they shall be put to death, their blood is upon them. XX20:12 If a man lies with his daughter-in-law, both of them shall be put to death; they have committed incest, their blood is upon them. XX20:15 If a man lies with a beast, he shall be put to death; and you shall kill the beast. X 20:09 For every one who curses his father or his mother shall be put to death; he has cursed his father or his mother, his blood is upon him. X 20:02 Any man of the people of Israel, or of the strangers that sojourn in Israel, who gives any of his children to Molech shall be put to death. XX 18:22 You shall not lie with a male as with a woman; it is an abomination.

FIGURE 13-4 Using a Spreadsheet for Qualitative Analysis

• AnSWR: http://www.cdc.gov/hiv/software/ your needs. Sociologists at the University of Surrey, answr.htm England, have prepared an overview of these and • Atlas.ti: http://www.atlasti.com/index.php other programs with descriptions and contact infor- • Ethno 2: http://www.indiana.edu/ mation. You can fi nd this resource at http://www %7Esocpsy/ESA/ .soc.surrey.ac.uk/sru/SRU1.html. Another excel- • Ethnograph: http://www.qualisresearch.com/ lent resource is “Choosing a CAQDAS Package” • HyperQual: http://home.satx. by Ann Lewins and Christina Silver (2006), which rr.com/hyperqual/ can be found at http://caqdas.soc.surrey.ac.uk/. • HyperResearch: http://www.researchware This will familiarize you with some of the key fea- .com/ tures in such computer programs and help you • HyperTranscribe: http://www.researchware choose which one is best suited to your purposes. .com/ Let’s turn now to a couple of illustrations of • MAXqda: http://www.maxqda.com/ QDA programs at work. Although the available • NUD*IST, NVivo 7: http://www.qsr.com programs differ somewhat from one another, I .au/products/productoverview/NVivo_7.htm think these illustrations will give you a good sense • QDA Miner: http://www.provalisresearch of the general use of computers to analyze qualita- .com/QDAMiner/QDAMinerDesc.html tive data. We’ll briefl y examine Leviticus, and then • Qualrus: http://www.ideaworks.com/ we’ll examine a project that used a different pro- Qualrus/index.html gram and focused on understanding the experi- • SPAD: http://eng.spad.eu/ ences of women fi lm directors. • TAMS: http://sourceforge. net/projects/tamsys Leviticus as Seen through NUD*IST • T-LAB: http://www.tlab.it/en/presentazione .asp We’ll fi rst consider one of the programs just men- • Weft: http://www.pressure.to/qda/ tioned, NUD*IST (Nonnumeric Unstructured Data, Index Searching, and Theorizing). This popular There are also some powerful online resources program for teaching qualitative social research to assist you in choosing the program best suited to offers a fair representation of QDA programs. 430 CHAPTER 13 QUALITATIVE DATA ANALYSIS

FIGURE 13-5 How Text Materials Are Displayed in NUD*IST

Although the text materials to be coded can To see the document, select its name in the be typed directly into NUD*IST, usually materi- “Document Explorer” window and click “Browse.” als already in existence—such as ﬁ eld notes or, The text window can be resized and moved around in this case, the verses of Leviticus—are imported the screen to suit your taste. into the program. Menu-based commands do this Note the set of buttons in the upper right corner easily, though the text must be in a plaintext for- of the illustration. These allow you to select por- mat (that is, without word-processing or other tions of the text for purposes of editing, coding, formatting). and other operations. Figure 13-5 shows how the text is displayed Now let’s create a concept code: “homosex.” within NUD*IST. For the illustrations in this sec- This will stand for references to male homosexual- tion, I have used the Macintosh version of the pro- ity. Figure 13-6 shows what the creation of a con- gram, but the Windows version is similar. cept code looks like. COMPUTER PROGRAMS FOR QUALITATIVE DATA 431

FIGURE 13-6 Creating the Code “homosex”

As we create codes for our concepts, we can click the “Browse” button, which presents you with use them to code the text materials. Figure 13-7 a list of the current codes. In this example, I se- illustrates how this is done. In the document lected “homosex” and entered the code ID (100). browser, you can see that verse 20:13 has been As text materials are coded, the program can selected (indicated by the box outline around this then be used for purposes of analysis. As a simple verse). Having done that, we click the button la- example, we might want to pull together all the beled “Add Coding” (not shown in this illustration). passages coded “homosex.” This would allow us This prompts the computer to ask us to identify the to see them all at once, looking for similarities and appropriate code. The easiest way to respond is to differences. 432 CHAPTER 13 QUALITATIVE DATA ANALYSIS

FIGURE 13-7 Coding a Text Passage

Figure 13-8 shows how NUD*IST would bring for qualitative data analysis. Now let’s probe more together the passages referring to male homo- deeply into the possibilities of computerized quali- sexuality. To do this, all you do is select the code tative data analysis. Sandrine Zerbib is a French name in the “Node Explorer” window and click the sociologist interested in understanding the spe- “Make Report” button. cial diffi culties faced by women breaking into the This simple example illustrates the possibili- male-dominated world of fi lm direction. To address ties opened up by a program designed specifi cally this issue, she interviewed 30 women directors in COMPUTER PROGRAMS FOR QUALITATIVE DATA 433

FIGURE 13-8 Reporting on “homosex”

depth. Having compiled hours of recorded inter- Using NVivo to Understand Women views, she turned to a popular program, NVivo (a Film Directors, by Sandrine Zerbib successor to NUD*IST), as a vehicle for analysis. In For those of you who feel uncomfortable using the next section, she directly describes her experi- new programs or computer programs in general, ences with the ongoing process of qualitative data NVivo should work well. It is visually clear and analysis. intuitive, and it requires mostly dragging (moving text or objects using a mouse). 434 CHAPTER 13 QUALITATIVE DATA ANALYSIS

To learn more about the tools in this software paragraph, opened the “career experience” node, package, let’s look at a project fi le I created using and simply dragged the passage to the child node NVivo. Figure 13-9 shows the opened browser win- “early artist” located in the coder within the “career dow containing my interview with Berta, one of the experience” category. 30 fi lm directors I interviewed. The “Coding Stripes” Another helpful tool is the “attribute” function view allows you to visualize “nodes” (i.e., codes) of NVivo. By clicking on the colorful cube icon, you associated with the text. Parts of the same passage can open the attribute browser any time for any were coded with more than one node, which ex- of your interviews or other texts. This allows you plains the overlapping of “stripes” or brackets. The to begin a content analysis by creating attributes “Coder” window is opened on the right. You can (or variables) such as age, sex, date of interview, click on the symbol to visualize “child” nodes (or number of children, and so on. There is no rule subnodes) of a particular “tree” node (main node). as to how many attributes you should create, but As you can see in this fi gure, when you click on the you need to weigh the time spent on choosing at- “fi rst job in the industry” node located in the coder tributes against the potential usefulness of those window, the passages associated with that node attributes. Because values previously created are are automatically highlighted. automatically presented as possible choices, it’s In Figure 13-10, a new interview has been im- easy to keep your names and defi nitions of attri- ported as a “Rich Text Format” fi le into the proj- butes consistent as you move from text to text. For ect. You can either import a fi le with a particular instance, in Figure 13-11, I had typed “Los Angeles” formatting of headings and font styles or make under the category “Live city” (defi ned as “Where these formatting changes while using NVivo. Us- does the interviewee live?”) while coding my inter- ing styles and other text-formatting tools can help view with Ulma. The value “Los Angeles” was later you refi ne your coding system. For instance, it can automatically available when I coded my interview be very useful to create styles that format the in- with Berta. You can see that using “LA” in one interviewer’s and the interviewee’s narratives differ- stance and “Los Angeles” in another might have ently; this way, you can see at a glance which type caused problems. The “attribute” function of NVivo of narrative you’re reading. Of course, taking the can also help you generate an interview profi le or time to format makes more sense in some cases a quantitative analysis. than in others. For instance, I found it extremely To organize your attributes, you can create ei- useful to use formatting in projects that comprise a ther “free” nodes or “tree” nodes. Free nodes are large number of interviews. independent nodes, which means that you can’t As you explore possibilities, you’ll fi nd certain organize them in any type of hierarchy or struc- formatting choices more helpful than others. You’ll ture. They are generally those nodes that cannot want to consider such things as how much a change be related to others. Tree nodes, as their name sug- in font or typeface expands or contracts your text, gests, can be organized into a hierarchy. You can and how easy or hard a given format is to read. create “child” nodes and “sibling” nodes in relation For example, you might increase the size of or use to them. Tree nodes are the most helpful for analy- boldface for the interviewer’s text rather than the sis purposes. To create a tree node, click on “trees” interviewee’s text, because the interviewer speaks next to the green trees symbol and then click on the far less than the interviewee and you do not want right button of your mouse. A window like the one your text to be too long. In Figure 13-10 I chose to in Figure 13-12 will appear. Next, click on “create” italicize my part of the interview. and choose either “child node” or “sibling node.” In Another important feature of NVivo is that it al- Figure 13-12, I had created a node called “abuse” lows you to attach passages to nodes easily. You and needed to create a child node called “school can select passages based on content in units as abuse,” because my interviewee was telling me small as a single word, then drag them where you about the abuse she had experienced at school. I want them. In Figure 13-10, I highlighted part of a created a child node and then typed “school abuse” FIGURE 13-9 Viewing the Interview with Berta

FIGURE 13-10 Example of Formatted Text and Attaching Passages to Nodes FIGURE 13-11 Using the “Attribute” Function

FIGURE 13-12 Creating a Tree Node COMPUTER PROGRAMS FOR QUALITATIVE DATA 437

FIGURE 13-13 Adding a Description

over the default “tree node” just created under the they will not be able to combine separate fi les at “abuse” node. If Ulma reported abuse experienced the end. If this is an issue, N5 (another successor at home, I could have created another child node to NUD*IST) is an alternative program that allows called “domestic abuse” under “abuse” or simply a fi les to be merged. sibling node to “school abuse.” Again, I can select Coding can be tedious and time consuming. any passage from the text and drag it to any num- However, the analysis it allows may be price- ber of nodes. less. You can use NVivo for generating reports for When creating new nodes, be sure to attach all or specifi c nodes or texts. For instance, in Fig- a description to each. It’s easy to forget what you ure 13-14 I inquired about all interviewee narra- originally meant if you don’t write it down. You can tives that were coded under the “gender discrimi- defi ne each node with the “properties” command. nation” node. As you can see, all passages are With this feature, you can also keep track of the extracted under the “gender discrimination” node time you created a particular node and who cre- browser. Each interview name is specifi ed, as well ated it in case you’re working with other coders. In as the size of each passage. From this window, you Figure 13-13, I added the description “Mental and could turn on the coding stripe view and see how physical abuse infl icted by school authority” to the each passage is associated with nodes other than “school abuse” node. You can modify the proper- “gender discrimination.” You could also do further ties of the nodes you create and the documents you coding from this window by simply dragging se- import at any time. Be aware, however, that NVivo lected text to a node. Finally, you could generate a is not conceived to merge project fi les. If you plan report by attribute; for example, you could get all to have several people coding the same data, each passages that have to do with “gender discrimina- will have to work on the same NVivo project fi le at tion” for women who live in Los Angeles or New different times instead of simultaneously; that is, York and compare them. 438 CHAPTER 13 QUALITATIVE DATA ANALYSIS

FIGURE 13-14 Extracting Materials by Node

THE QUALITATIVE ANALYSIS dimensional graph. Even though the graph is OF QUANTITATIVE DATA based on statistical data, it conveys its meaning quite clearly. Although summarizing it in the form of equations may be useful for certain purposes, it Although it is important and appropriate to dis- would add nothing to the clarity of the picture it- tinguish between qualitative and quantitative re- self. Thus, the qualitative assessment of the graph search and to discuss them separately, they are clariﬁ es the quantitative data in a way that no by no means incompatible or in competition. You other representation could. Here’s a case where a need to operate in both modes to explore your full picture is truly worth a thousand words. potential as a social researcher. Chapter 14 explores some ways in which quantitative analyses can strengthen qualitative stud- ETHICS AND QUALITATIVE ies. Before we move on, however, let’s look at an DATA ANALYSIS example of how quantitative data demand qualitative assessment. At least two ethical issues raise special concern in Figure 13-15 presents FBI data on homicides the analysis and reporting of qualitative research. committed in the United States. These data are First, because such analysis calls so directly on often presented in a tabular form, but notice how subjective judgments, there is an obvious risk of clearly the patterns of crime appear in this three- seeing what you are looking for or want to ﬁ nd. MAIN POINTS 439

WHAT DO YOU THINK? REVISITED

800 Quantiﬁ cation requires a simpliﬁ cation of 700 data through a loss of detail. Sometimes 600 those details are critical to understanding 500 the “whole picture.” You’ve experienced this

400 if you’ve ever found yourself being catego- Number 300 rized by someone else. Let’s say you express 0 some political opinion. Someone then asks 200 20 100 40 what your major is, and you reply, “Sociol- 60 ogy.” Then that same person says, “Well, of 0 20 course!”—implying that they now “know” 40 80 60 Offender age 80 a long list of things about you—some true, Victim age 100 100 some false—that will now shape their “un- FIGURE 13-15 Number of One-on-One derstanding” of the political opinion you ex- Homicides by Age of Victim and Age of pressed. You may have experienced being Offender, Raw Data similarly categorized in terms of your reli- Source: Michael D. Maltz, “Visualizing Homicide: A Research gion, race, place of birth, or gender. A similar Note,” Journal of Quantitative Criminology 15, no. 4 (1998): 401. loss can occur in the quantifi cation of data, where a limited number of categories takes The risk increases in the case of participatory ac- the place of varied details. Qualitative analy- tion research or other projects involving an ele- sis, while coding and categorizing, aims at ment of social justice. Researcher bias is hardly an staying closer to the original details. inevitable outcome, however. Experienced qualitative analysts avoid this pitfall in at least two ways: by cultivating a deliberate awareness of their own values and preferences, and by adhering to estab- sults of your analyses, you will often need to make lished techniques for data collection and analysis. concerted efforts to conceal identities. Individuals, And as an additional protection, the peer-review organizations, and communities are often given process in scientifi c research encourages col- pseudonyms toward this end. Sometimes, you leagues to point out any failings in this regard. may need to suppress details that would let outsid- Second, qualitative research makes protecting ers fi gure out who you are talking about. Thus, it subjects’ privacy particularly important. The quali- may be appropriate to speak about interviewing “a tative researcher will often analyze and report data church leader” rather than “the head deacon.” You collected from identifi able individuals. Throughout may also need to suppress or alter age, race, or the book, I’ve indicated the importance of not re- gender references if that would give away a sub- vealing what we learn about subjects, as in the ject’s identity. The key principle is to respect the case of data collection. When writing up the re- privacy of those we study.

Main Points Linking Theory and Analysis ❏ Qualitative analysis involves a continual inter- Introduction play between theory and analysis. In analyzing ❏ Qualitative analysis is the nonnumerical ex- qualitative data, we seek to discover patterns amination and interpretation of observations. 440 CHAPTER 13 QUALITATIVE DATA ANALYSIS

such as changes over time or possible causal Key Terms links between variables. axial coding memoing ❏ Examples of approaches to the discovery and case-oriented analysis open coding explanation of such patterns are Grounded concept mapping qualitative analysis Theory Method (GTM), semiotics, and conver- constant comparative method selective coding sation analysis (CA). conversation analysis (CA) semiotics cross-case analysis variable-oriented Qualitative Data Processing Grounded Theory Method (GTM) analysis ❏ The processing of qualitative data is as much art as science. Three key tools for preparing data for analysis are coding, memoing, and Review Questions concept mapping. ❏ In contrast to the standardized units used in 1. Review Goffman’s examination of gender coding for statistical analyses, the units to be advertising, and collect and analyze a set of coded in qualitative analyses may vary within advertising photos from magazines or news- a document. Although codes may be derived papers. What is the relationship between from the theory being explored, more often gender and status in the materials you found? researchers use open coding, in which codes 2. Review the discussion of homosexuality in are suggested by the researchers’ examination the Book of Leviticus. How might the exami- and questioning of the data. nation be structured as a cross-case analysis? ❏ Memoing is appropriate at several stages of 3. Imagine you were conducting a cross-case data processing and serves to capture code analysis of revolutionary documents such meanings, theoretical ideas, preliminary con- as the Declaration of Independence and the clusions, and other thoughts that will be useful Declaration of the Rights of Man and of the during analysis. Citizen (from the French Revolution). What ❏ Concept mapping uses diagrams to explore key concepts might you code in the following relationships in the data graphically. sentence?

Computer Programs for Qualitative Data When in the Course of human events, it be- ❏ Many computer programs, such as NUD*IST comes necessary for one people to dissolve and NVivo, are speciﬁ cally designed to assist the political bands which have connected researchers in the analysis of qualitative data. them with another, and to assume among the Powers of the earth, the separate and The Qualitative Analysis equal station to which the Laws of Nature of Quantitative Data and of Nature’s God entitle them, a decent ❏ Researchers need both qualitative and quanti- respect to the opinions of mankind requires tative analysis for the fullest understanding of that they should declare the causes which social science data. impel them to the separation.

Ethics and Qualitative Data Analysis 4. Go to the World Press Review online (http:// ❏ The subjective element in qualitative data www.wpr.com) and pick a controversial news analysis provides an added challenge to avoid- topic discussed by several newspapers. See ing bias in the interpretation of data. if you can identify characteristics of those newspapers (such as political leaning, region) ❏ Because the qualitative data analyst knows that might explain the different points of view the identity of subjects, taking special steps to expressed on the topic. protect their privacy is crucial. ADDITIONAL READINGS 441

Online Study Resources aspects of qualitative research, in both theory and practice. Glaser, Barney G., and Anselm L. Strauss. 1967. The Discovery of Grounded Theory: Strategies for Qualitative Research. Chicago: Aldine. This is the classic statement of grounded theory, with practical suggestions Go to that are still useful today. http://sociology.wadsworth.com/babbie_basics4e Hutchby, Ian, and Robin Wooffi tt. 1998. Conversation Analysis: Principles, Practices and Applications. Cam- and click on ThomsonNow for access to this bridge, England: Polity Press. An excellent overview powerful online study tool. You will get a per- of the conversation analysis method. The book ex- sonalized study plan based on your responses to amines the theory behind the technique, how to use a diagnostic pretest. Once you have mastered it, and some possible applications. the material with the help of interactive learning Jacobson, David. 1999. “Doing Research in Cyberspace.” tools, you can take a posttest to confi rm that you Field Methods 11:127–45. The use of the Internet for are ready to move on to the next chapter. social research is not limited to surveys and experiments, as Jacobson demonstrates in this examination of computer-mediated communication (CMC). Website for King, Gary, Robert O. Keohane, and Sidney Verba. 1994. The Basics of Social Research Designing Social Inquiry: Scientifi c Inference in Qualita- 4th edition tive Research. Princeton, NJ: Princeton University Press. This controversial book by three political At the book companion website (http://sociology scientists seeks to bring the logic of causal, quantita- .wadsworth.com/babbie_basics4e) you will fi nd tive analysis to bear on qualitative data. Their stated intention is to unify the two approaches. many resources in addition to ThomsonNow to Lewins, Ann, and Christina Silver. 2006. “Choosing a aid you in studying for your exams. For example, CAQDAS Package.” July. http://caqdas.soc.surrey you will fi nd Tutorial Quizzes with feedback, Inter- .ac.uk/. This excellent working paper gives a detailed net Exercises, Flashcards, and Chapter Tutorials, as examination of the features to look for in a qualita- well as Extended Projects, InfoTrac College Edition tive data analysis program and discusses some of the search terms, Social Research in Cyberspace, GSS more popular programs available. Data, Web Links, and primers for using various McCormack, Coralie. 2004. “Storying Stories: A Narra- data analysis software such as SPSS and NVivo. tive Approach to In-Depth Interview Conversations.” International Journal of Social Research Methodology 7 (3): 219–36. The in-depth interviews common to Additional Readings qualitative fi eld research can result in lengthy narrative accounts that can pose daunting challenges for Berg, Bruce. 1998. Qualitative Research Methods for the analysts. This article details a set of procedures for Social Sciences. Boston: Allyn and Bacon. Here’s a organizing the analysis of such stories, with a special comprehensive and readable review of the tech- concern for the ethical dimension. niques for collecting and analyzing qualitative data, Strauss, Anselm, and Juliet Corbin. 1998. Basics of with a special sensitivity to research ethics. Qualitative Research: Techniques and Procedures for Denzin, Norman K., and Yvonna S. Lincoln, eds. 1994. Developing Grounded Theory. Thousand Oaks, CA: Handbook of Qualitative Research. Thousand Oaks, Sage. This updated statement of grounded theory of- CA: Sage. Here’s a rich resource covering many fers special guidance on coding and memoing. 141 HUMANQUANTITATIVE INQUIRY DATA AND ANALYSIS SCIENCE

Image not available due to copyright restrictions Photo credit

What You’ll Learn in thisThis ChapterChapter

Often,We’ll social examine data theare wayconverted people to learn numerical about theirform worldfor statistical and the analyses. mistakes Inthey this makechapter, along we’ll the begin way. with We’ll the also process begin ofto quantifyingsee what makes data, siencethen turn different to analysis. from

otherQuantitative ways of analysis knowing may things. be descriptive or explanatory; it may involve one, two, 2S or several variables. We begin our examination of quantitative analyses with 1S N some simple but powerful ways of manipulating data in order to attain research L conclusions. 442 In this chapter . . . WHAT DO YOU THINK?

Introduction In Chapter 13, we Quantifi cation of Data saw several inher- Developing Code Categories ent shortcomings in Codebook Construction Image not available due to copyright restrictions quantitative data. Data Entry These shortcomings Univariate Analysis centered primarily Distributions on standardization Central Tendency and superfi ciality in Dispersion the face of a social reality that is varied and Continuous and Discrete Variables deep. Can anything meaningful be learned Detail versus Manageability from data that sacrifi ce meaningful detail in order to permit numerical manipulations? Subgroup Comparisons “Collapsing” Response Categories See the “What Do You Think? Revisited” box Handling Don’t Knows toward the end of the chapter. Numerical Descriptions in Qualitative Research

Bivariate Analysis Percentaging a Table To begin, we’ll look at quantiﬁ cation—the pro- Constructing and Reading Bivariate Tables cess of converting data to a numerical format. This Introduction to Multivariate Analysis involves converting social science data into a machine-readable form—a form that can be read and Sociological Diagnostics manipulated by computers and similar machines Ethics and Quantitative Data Analysis used in quantitative analysis. The rest of the chapter will present the logic INTRODUCTION and some of the techniques of quantitative data analysis—starting with the simplest case, univariate analysis, which involves one variable, then In Chapter 13, we saw some of the logic and tech- discussing bivariate analysis, which involves two niques by which social researchers analyze the variables. We’ll move on to a brief introduction to qualitative data they have collected. This chapter multivariate analysis, or the examination of sev- will examine quantitative analysis, or the tech- eral variables simultaneously, such as age, educa- niques by which researchers convert data to a nu- tion, and prejudice, and then we’ll end with a dis- merical form and subject it to statistical analyses. cussion of sociological diagnostics. Before we can do any sort of analysis, we need to quantify our data. Let’s turn now to the basic quantitative analysis The numerical representation and manipulation of observations for the purpose steps involved in converting data into machine- of describing and explaining the phenomena that those readable forms amenable to computer processing observations reﬂ ect. and analysis. 443 444 CHAPTER 14 QUANTITATIVE DATA ANALYSIS

As with content analysis, the task of quantitative coding is to reduce a wide variety of idiosyncratic items of information to a more limited set of attributes composing a variable. Suppose, for example, that a survey researcher asks respondents, “What is your occupation?” The responses to such a question will vary considerably. Although it will be possible to assign each reported occupation a separate numerical code, this procedure will not facilitate analysis, which typically depends on several subjects having the same attribute. The variable occupation has many preestab-

Aaron Babbie lished coding schemes. One such scheme distin- Some students take to statistics more readily than guishes professional and managerial occupations, others. clerical occupations, semiskilled occupations, and so forth. Another scheme distinguishes different QUANTIFICATION OF DATA sectors of the economy: manufacturing, health, education, commerce, and so forth. Still others combine both of these schemes. Using an estab- Today, quantitative analysis is almost always done lished coding scheme gives you the advantage of by computer programs such as SPSS and Micro- being able to compare your research results with Case. For those programs to work their magic, they those of other studies. must be able to read the data you’ve collected in your research. If you’ve conducted a survey, for To learn more about preestablished cod- example, some of your data are inherently numeri- ing schemes, visit the Bureau of Labor cal: age or income, for instance. Whereas the writ- Statistics* to learn about their Standard ing and check marks on a questionnaire are quali- Occupational Classifi cation: http://stats.bls tative in nature, a scribbled age is easily converted .gov/soc/soc_majo.htm. to quantitative data. Other data are also easily quantifi ed: Trans- forming male and female into “1” and “2” is hardly The occupational coding scheme you choose rocket science. Researchers can also easily assign should be appropriate to the theoretical concepts numerical representations to such variables as being examined in your study. For some studies, religious affi liation, political party, and region of the coding all occupations as either white-collar or country. blue-collar might suffi ce. For others, self-employed Some data are more challenging, however. If a and not self-employed might do. Or a peace re- survey respondent tells you that he or she thinks searcher might wish to know only whether the oc- the biggest problem facing Woodbury, Vermont, cupation depended on the defense establishment is “the disintegrating ozone layer,” the computer or not. can’t process that response numerically. You must Although the coding scheme should be tailored translate by coding the responses. We’ve already to meet particular requirements of the analysis, you discussed coding in connection with content anal- should keep one general guideline in mind. If the ysis (Chapter 11) and again in connection with data are coded to maintain a great deal of detail, qualitative data analysis (Chapter 13). Now we look at coding specifi cally for quantitative analysis, which differs from the other two primarily in its *Each time the Internet icon appears, you’ll be given help- goal of converting raw data into numbers. ful leads for searching the World Wide Web. QUANTIFICATION OF DATA 445 code categories can always be combined during TABLE 14-1 Student Responses That Can Be an analysis that does not require such detail. If the Coded “Financial Concerns” data are coded into relatively few, gross catego- Financial Concerns ries, however, you’ll have no way during analysis to recreate the original detail. To keep your options Tuition is too high X open, it’s a good idea to code your data in greater Not enough parking spaces detail than you plan to use in the analysis. Faculty don’t know what they are doing Advisors are never available Not enough classes offered Developing Code Categories Cockroaches in the dorms There are two basic approaches to the coding pro- Too many requirements cess. First, you may begin with a relatively well- Cafeteria food is infected developed coding scheme, derived from your re- Books cost too much X search purpose. Thus, as suggested previously, the Not enough fi nancial aid X peace researcher might code occupations in terms of their relationship to the defense establishment. Or, you may want to use an existing coding scheme so that you can compare your fi ndings with those In more general terms, the fi rst answer can of previous research. also be seen as refl ecting nonacademic concerns. The alternative method is to generate codes This categorization would be relevant if your re- from your data, as discussed in Chapter 13. Let’s search interest included the distinction between say we’ve asked students in a self-administered academic and nonacademic concerns. If that were campus survey to say what they believe is the big- the case, the responses might be coded as shown gest problem facing their college today. Here are a in Table 14-2. few of the answers they might have written in. Notice that I didn’t code the response “Books cost too much” in Table 14-2, because this concern Tuition is too high could be seen as representing both of the catego- Not enough parking spaces ries. Books are part of the academic program, but Faculty don’t know what they are doing their cost is not. This signals the need to refi ne Advisors are never available the coding scheme we’re developing. Depending Not enough classes offered on our research purpose, we might be especially Cockroaches in the dorms interested in identifying any problems that had an Too many requirements academic element; hence we’d code this one “Aca- Cafeteria food is infected demic.” Just as reasonably, however, we might be Books cost too much more interested in identifying nonacademic prob- Not enough fi nancial aid lems and would code the response accordingly. Or, Take a minute to review these responses and as another alternative, we might create a separate see whether you can identify some categories rep- category for responses that involved both aca- resented. Realize that there is no right answer; demic and nonacademic matters. several coding schemes might be generated from As yet another alternative, we might want to these answers. separate nonacademic concerns into those involv- Let’s start with the fi rst response: “Tuition is too ing administrative matters and those dealing with high.” What general areas of concern does that re- campus facilities. Table 14-3 shows how the fi rst sponse refl ect? One obvious possibility is “Financial ten responses would be coded in that event. Concerns.” Are there other responses that would fi t As these few examples illustrate, there are into that category? Table 14-1 shows which of the many possible schemes for coding a set of data. questionnaire responses could fi t. Your choices should match your research pur- 446 CHAPTER 14 QUANTITATIVE DATA ANALYSIS

TABLE 14-2 Student Concerns Coded as “Academic” and “Nonacademic”

Academic Nonacademic

Tuition is too high X Not enough parking spaces X Faculty don’t know what they are doing X Advisors are never available X Not enough classes offered X Cockroaches in the dorms X Too many requirements X Cafeteria food is infected X Books cost too much X Not enough ﬁ nancial aid X

TABLE 14-3 Nonacademic Concerns Coded as “Administrative” or “Facilities”

Academic Administrative Facilities

poses and refl ect the logic that emerges from the If you’re fortunate enough to have assistance in data themselves. Often, you’ll fi nd yourself modify- the coding process, you’ll need to train your coders ing the code categories as the coding process pro- in the defi nitions of code categories and show them ceeds. Whenever you change the list of categories, how to use those categories properly. To do so, ex- however, you must review the data already coded plain the meaning of the code categories and give to see whether changes are in order. several examples of each. To make sure your cod- Like the set of attributes composing a vari- ers fully understand what you have in mind, code able, and like the response categories in a closed- several cases ahead of time. Then ask your coders ended questionnaire item, code categories should to code the same cases without knowing how you be both exhaustive and mutually exclusive. Every coded them. Finally, compare your coders’ work piece of information being coded should fi t into with your own. Any discrepancies will indicate an one and only one category. Problems arise when- imperfect communication of your coding scheme ever a given response appears to fi t equally into to your coders. Even with perfect agreement be- more than one code category or whenever it fi ts tween you and your coders, however, it’s best to into no category: Both signal a mismatch between check the coding of at least a portion of the cases your data and your coding scheme. throughout the coding process. QUANTIFICATION OF DATA 447

POLVIEWS ATTEND We hear a lot of talk these days about liberals and conserva- How often do you attend religious services? tives. I’m going to show you a seven-point scale on which 0. Never the political views that people might hold are arranged from extremely liberal—point 1—to extremely conservative— 1. Less than once a year point 7. Where would you place yourself on this scale? 2. About once or twice a year 1. Extremely liberal 3. Several times a year 2. Liberal 4. About once a month 3. Slightly liberal 5. 2–3 times a month 4. Moderate, middle of the road 6. Nearly every week 5. Slightly conservative 7. Every week 6. Conservative 8. Several times a week 7. Extremely conservative 9. Don’t know, No answer 8. Don’t know 9. No answer

FIGURE 14-1 A Partial Codebook

If you’re not fortunate enough to have assis- and interpreting codes in your data fi le during tance in coding, you should still obtain some verifi - analysis. If you decide to correlate two variables as cation of your own reliability as a coder. Nobody’s a part of your analysis of your data, the codebook perfect, especially a researcher hot on the trail of a tells you where to fi nd the variables and what the fi nding. Suppose that you’re studying an emerging codes represent. cult and that you have the impression that people Figure 14-1 is a partial codebook created from who do not have a regular family will be the most two variables from the General Social Survey. likely to regard the new cult as a family substitute. Though there is no one right format for a code- The danger is that whenever you discover a sub- book, this example presents some of the common ject who reports no family, you’ll unconsciously try elements. to fi nd some evidence in the subject’s comments Notice fi rst that each variable is identifi ed by an that the cult is a substitute for family. If at all pos- abbreviated variable name: POLVIEWS, ATTEND. sible, then, get someone else to code some of your We can determine the religious service attendance cases to see whether that person makes the same of respondents, for example, by referencing AT- assignments you made. TEND. This example uses the format established by the General Social Survey, which has been car- Codebook Construction ried over into SPSS. Other data sets and/or analysis programs might format variables differently. The end product of the coding process in quantita- Some use numerical codes in place of abbreviated tive analysis is the conversion of data items into names, for example. You must, however, have numerical codes. These codes represent attributes some identifi er that will allow you to locate and composing variables, which, in turn, are assigned use the variable in question. locations within a data fi le. A codebook is a document that describes the locations of variables and lists the assignments of codes to the attributes The document used in data processing and composing those variables. codebook analysis that tells the location of different data items in a A codebook serves two essential functions. data fi le. Typically, the codebook identifi es the locations of First, it is the primary guide used in the coding pro- data items and the meaning of the codes used to repre- cess. Second, it is your guide for locating variables sent different attributes of variables. 448 CHAPTER 14 QUANTITATIVE DATA ANALYSIS

Next, every codebook should contain the full complishing this step, depending on the original defi nition of the variable. In the case of a question- form of your data and also the computer program naire, the defi nition consists of the exact wordings you’ll use for analyzing the data. I’ll simply intro- of the questions asked, because, as we’ve seen, duce you to the process here. If you fi nd yourself the wording of questions strongly infl uences the undertaking this task, you should be able to tailor answers returned. In the case of POLVIEWS, you your work to the particular data source and pro- know that respondents were given the several gram you’re using. political categories and asked to pick the one that If your data have been collected by question- best fi t them. naire, you might do your coding on the question- The codebook also indicates the attributes naire itself. Then, data-entry specialists (including composing each variable. In POLVIEWS, for exam- yourself) could enter the data into, say, an SPSS ple, the political categories just mentioned serve data matrix or into an Excel spreadsheet that as these attributes: “Extremely liberal,” “Liberal,” would later be imported into SPSS. “Slightly liberal,” and so forth. Sometimes, social researchers use optical scan Finally, notice that each attribute also has a nu- sheets for data collection. These sheets can be fed meric label. Thus, in POLVIEWS, “Extremely liberal” into machines that will convert the black marks is code category 1. These numeric codes are used into data, which can be imported into the analysis in various manipulations of the data. For example, program. This procedure only works with subjects you might decide to combine categories 1 through who are comfortable using such sheets, and it’s 3 (all the “liberal” responses). It’s easier to do this usually limited to closed-ended questions. with code numbers than with lengthy names. Sometimes, data entry occurs in the process of data collection. In Computer Assisted Telephone Interviewing, for example, the interviewer keys You can visit the GSS codebook online at responses directly into the computer, where the http://webapp.icpsr.umich.edu/GSS/. If you data are compiled for analysis (see Chapter 9). know the symbolic name (e.g., POLVIEWS), Even more effortlessly, online surveys can be con- you can locate it in the Mnemonic listing. structed so that the respondents enter their own Otherwise, you can browse the “Index by answers directly into the accumulating database, Subject” to fi nd all the different questions without the need for an intervening interviewer or that have been asked regarding a particular data-entry person. topic. Once data have been fully quantifi ed and entered into the computer, researchers can begin quantitative analysis. Let’s look at the three cases Data Entry mentioned at the start of this chapter: univariate, In addition to transforming data into quantita- bivariate, and multivariate analyses. tive form, researchers interested in quantitative analysis also need to convert data into a machine- UNIVARIATE ANALYSIS readable format, so that computers can read and manipulate the data. There are many ways of ac- The simplest form of quantitative analysis, univariate analysis, involves describing a case in terms of a single variable—specifi cally, the distri- The analysis of a single variable, univariate analysis bution of attributes that compose it. For example, for purposes of description. Frequency distributions, averages, and measures of dispersion are examples of if sex were measured, we would look at how many univariate analysis, as distinguished from bivariate and of the subjects were men and how many were multivariate analysis. women. UNIVARIATE ANALYSIS 449

Distributions By analogy, suppose your best friend tells you that she drank a six-pack of beer. Is that a little The most basic format for presenting univariate beer or a lot? The answer, of course, depends on data is to report all individual cases, that is, to list whether she consumed the beer in a month, a the attribute for each case under study in terms week, a day, or an hour. In the case of religious of the variable in question. Let’s take as an ex- participation, similarly, we need some basis for ample the General Social Survey (GSS) data on at- assessing the number that represents the people tendance at religious services, ATTEND. Table 14-4 who never attend religious services. presents the results of an SPSS analysis of this One way to assess the number is to calculate variable. the percentage of all respondents who said they Let’s examine the table, piece by piece. First, if never go to religious services. If you were to di- you look near the bottom of the table, you’ll see vide 471 by the 2,801 who gave some answer, you that the sample being analyzed has a total of 2,812 would get 16.8 percent, which appears in the table cases. In the last row above the totals, you’ll see as the “Valid Percent.” Now we can say that about that 11 of the 2,812 respondents either said they 17 percent, or roughly one U.S. adult in six, reports didn’t know (DK) or gave no answer (NA) in re- never attending religious services. sponse to this question. So our assessment of U.S. This result is more meaningful, but does it sug- attendance at religious services in 2004 will be gest that people in the United States are generally based on the 2,801 respondents who answered nonreligious? A further look at Table 14-4 shows the question. that the response category most often chosen was Go back to the top of the table now. You’ll see “Every Week,” with 18.1 percent of the respon- that 471 people said they never went to religious dents giving that answer. Add to that the 8.6 per- services. This number in and of itself tells us noth- cent who report attending religious services more ing about religious practices. It does not, in itself, than once a week, and we ﬁ nd that over a fourth gives us an idea of whether the “average Ameri- (26.7 percent) of U.S. adults say they attend reli- can” attends religious services a little or a lot. gious services at least once a week. As you can

TABLE 14-4 GSS Attendance at Religious Services, 2004

Attend How Often R Attends Religious Services

Value Label Value Frequency Percent Valid Percent Cum Percent

NEVER 0 471 16.7 16.8 16.8 LT ONCE A YEAR 1 198 7.0 7.1 23.9 ONCE A YEAR 2 396 14.1 14.1 38.0 SEVRL TIMES A YR 3 371 13.2 13.2 51.3 ONCE A MONTH 4 191 6.8 6.8 58.1 2–3X A MONTH 5 255 9.1 9.1 67.2 NRLY EVERY WEEK 6 169 6.0 6.0 73.2 EVERY WEEK 7 508 18.1 18.1 91.4 MORE THN ONCE WK 8 242 8.6 8.6 100.0 DK,NA 9 11 0.4 ______Total 2,812 100.0 99.8 Valid cases 2,812 Missing cases 11

Source: General Social Survey, 2004, National Opinion Research Center. 450 CHAPTER 14 QUANTITATIVE DATA ANALYSIS

10 Percent

0 Never Less than Once Several Once 2–3 Nearly Every More once a year times a month times every week than once a year a year a month week a week How often R attends religious services FIGURE 14-2 Bar Chart of GSS ATTEND, 2004 see, each new comparison gives a more complete Central Tendency picture of the data. Beyond simply reporting the overall distribution of A description of the number of times that the values, sometimes called the marginal frequencies various attributes of a variable are observed in a or just the marginals, you may choose to present sample is called a frequency distribution. Some- your data in the form of an average, or measure times it’s easiest to see a frequency distribution in of central tendency. You’re already familiar with the a graph. Figure 14-2 was created by SPSS from the concept of central tendency from the many kinds GSS data on ATTEND. The vertical scale on the left of averages you use in everyday life to express the side of the graph indicates the percentages select- “typical” value of a variable. For instance, in base- ing each of the answers displayed along the hori- ball a batting average of .300 says that a batter gets zontal axis of the graph. Take a minute to notice a hit three out of every ten opportunities on aver- how the percentages in Table 14-4 correspond to age. Over the course of a season, a hitter might go the heights of the bars in Figure 14-2. through extended periods without getting any hits at all and go through other periods when he or she gets a bunch of hits all at once. Over time, though, frequency distribution A description of the number of times the various attributes of a variable are observed the central tendency of the batter’s performance in a sample. The report that 53 percent of a sample were can be expressed as getting three hits in every ten men and 47 percent were women would be a simple chances. Similarly, your grade point average ex- example of a frequency distribution. presses the “typical” value of all your grades taken average An ambiguous term generally suggesting typi- together, even though some of them might be A’s, cal or normal—a central tendency. The mean, median, and others B’s, and one or two might be C’s (I know you mode are specifi c examples of mathematical averages. never get anything lower than a C). An average computed by summing the values of mean Averages like these are more properly called several observations and dividing by the number of observations. If you now have a grade point average of 4.0 the arithmetic mean (the result of dividing the based on 10 courses, and you get an F in this course, your sum of the values by the total number of cases). new grade point (mean) average will be 3.6. The mean is only one way to measure central ten- UNIVARIATE ANALYSIS 451 dency or “typical” values. Two other options are each of the age groups. Hence, it is appropriate to the mode (the most frequently occurring attribute) add 0.5 years to the fi nal calculation, making the and the median (the middle attribute in the ranked mean age 16.37, as indicated in Figure 14-3. distribution of observed attributes). Here’s how the The third measure of central tendency, the me- three averages would be calculated from a set dian, represents the “middle” value: Half are above of data. it, half below. If we had the precise ages of each Suppose you’re conducting an experiment that subject (for example, 17 years and 124 days), we’d involves teenagers as subjects. They range in age be able to arrange all 31 subjects in order by age, from 13 to 19, as indicated in the following table: and the median for the whole group would be the age of the middle subject. Age Number As you can see, however, we do not know precise ages; our data constitute “grouped data” in 13 3 this regard. For example, three people who are not 14 4 precisely the same age have been grouped in the 15 6 category “13-year-olds.” 16 8 Figure 14-3 illustrates the logic of calculating 17 4 a median for grouped data. Because there are 31 18 3 subjects altogether, the “middle” subject would be 19 3 subject number 16 if they were arranged by age— 15 teenagers would be younger and 15 older. Look Now that you’ve seen the actual ages of the 31 at the bottom portion of Figure 14-3, and you’ll see subjects, how old would you say they are in gen- that the middle person is one of the eight 16-year- eral, or “on average”? Let’s look at three different olds. In the enlarged view of that group, we see ways you might answer that question. that number 16 is the third from the left. The easiest average to calculate is the mode, Because we do not know the precise ages of the most frequent value. As you can see, there the subjects in this group, the statistical conven- were more 16-year-olds (eight of them) than any tion here is to assume they are evenly spread along other age, so the modal age is 16, as indicated the width of the group. In this instance, the pos- in Figure 14-3. Technically, the modal age is the sible ages of the subjects go from 16 years and no category “16,” which may include some people days to 16 years and 364 days. Strictly speaking, who are closer to 17 than 16 but who haven’t yet the range, then, is 364/365 days. As a practical reached that birthday. matter, it’s suffi cient to call it one year. Figure 14-3 also demonstrates the calculation If the eight subjects in this group were evenly of the mean. There are three steps: (1) multiply spread from one limit to the other, they would each age by the number of subjects who have be one-eighth of a year apart from each other—a that age, (2) total the results of all those multipli- 0.125-year interval. Look at the illustration and cations, and (3) divide that total by the number of you’ll see that if we place the fi rst subject half the subjects. In the case of age, a special adjustment is needed. As indicated in the discussion of the mode An average representing the most frequently mode, those who call themselves “13” actually observed value or attribute. If a sample contains 1,000 Protestants, 275 Catholics, and 33 Jews, “Protestant” is range from exactly 13 years old to those just short the modal category. of 14. It is reasonable to assume, moreover, that median An average representing the value of the as a group the “13-year-olds” in the country are “middle” case in a rank-ordered set of observations. If the evenly distributed within that one-year span, mak- ages of fi ve men are 16, 17, 20, 54, and 88, the median ing their average age 13.5 years. This is true for would be 20. (The mean would be 39.) Age Number

16 Mode = 16 Most frequent 17

Age Number

13 13 × 3 = 39

14 14 × 4 = 56

15 15 × 6 = 90

16 16 × 8 = 128

17 17 × 4 = 68 Mean = 16.37 Arithmetic average 18 18 × 3 = 54

19 19 × 3 = 57 ÷ 31 = 15.87 + 0.50 = 16.37 492 Age Number (Total) (Cases)

13 1–3 Median = 16.31 14 4–7 Midpoint

15 8–13 14 15 16 17 18 19 20 21

17 22–25 16.06 16.19 16.31 16.44 16.56 16.69 16.81 16.94

18 26–28

19 29–31

FIGURE 14-3 Three “Averages” UNIVARIATE ANALYSIS 453 interval from the lower limit and add a full interval Dispersion to the age of each successive subject, the fi nal one Averages offer readers the advantage of reducing is half an interval from the upper limit. the raw data to the most manageable form: A sin- What we’ve done is calculate, hypothetically, the gle number (or attribute) can represent all the de- precise ages of the eight subjects, assuming their tailed data collected in regard to the variable. This ages were spread out evenly. Having done this, we advantage comes at a cost, of course, because the merely note the age of the middle subject—16.31— reader cannot reconstruct the original data from an and that is the median age for the group. average. Summaries of the dispersion of responses Whenever the total number of subjects is an can somewhat alleviate this disadvantage. even number, of course, there is no middle case. Dispersion refers to the way values are dis- To get the median, you merely calculate the mean tributed around some central value, such as an of the two values on either side of the midpoint average. The simplest measure of dispersion is the in the ranked data. Suppose, for example, that range: the distance separating the highest from there was one more 19-year-old in our sample, the lowest value. Thus, besides reporting that our giving us a total of 32 cases. The midpoint would subjects have a mean age of 15.87, we might also then fall between subjects 16 and 17. The median indicate that their ages range from 13 to 19. would therefore be calculated as (16.31 16.44)/ A more sophisticated measure of dispersion is 2 16.38. the standard deviation. This measure was briefl y As you can see in Figure 14-3, the three mea- mentioned in Chapter 7 as the standard error of sures of central tendency produce three different a sampling distribution. Essentially, the standard values for this set of data, which is often (but not deviation is an index of the amount of variability necessarily) the case. Which measure, then, best in a set of data. A higher standard deviation means represents the “typical” value? More generally, that the data are more dispersed; a lower stan- which measure of central tendency should you dard deviation means that they are more bunched prefer? The answer depends on the nature of your together. Figure 14-4 illustrates the basic idea. data and the purpose of your analysis. For exam- Notice that the professional golfer not only has a ple, whenever means are presented, you should lower mean score but is also more consistent— be aware that they are susceptible to extreme val- represented by the smaller standard deviation. The ues—a few very large or very small numbers. As duffer, on the other hand, has a higher average but only one example, the (mean) average person in Redmond, Washington, has a net worth in excess of a million dollars. If you were to visit Redmond, however, you would not fi nd that the “average” dispersion The distribution of values around some resident lives up to your idea of a millionaire. The central value, such as an average. The range is a simple very high mean refl ects the infl uence of one ex- example of a measure of dispersion. Thus, we may report treme case among Redmond’s 40,000 residents— that the mean age of a group is 37.9, and the range is from 12 to 89. Bill Gates of Microsoft, who has a net worth (at the A measure of dispersion around time this is being written) of tens of billions of dol- standard deviation the mean, calculated so that approximately 68 percent lars. Clearly, the median wealth would give you a of the cases will lie within plus or minus one standard more accurate picture of the residents of Redmond deviation from the mean, 95 percent will lie within plus as a whole. or minus two standard deviations, and 99.9 percent will This example should illustrate the need to lie within three standard deviations. Thus, for example, if the mean age in a group is 30 and the standard deviation choose carefully among the various measures of is 10, then 68 percent have ages between 20 and 40. The central tendency. A course or textbook in statistics smaller the standard deviation, the more tightly the values will give you a fuller understanding of the variety are clustered around the mean; if the standard deviation is of situations in which each is appropriate. high, the values are widely spread out. 454 CHAPTER 14 QUANTITATIVE DATA ANALYSIS a. High standard deviation = spread-out values scores ranging from 60 to 90, the report might say that the interquartile range was from 90 to 120 (or Amateur Golfer’s Scores 30 points) with a mean score of, let’s say, 102.

Continuous and Discrete Variables

The preceding calculations are not appropriate for

Number of games Mean = 100 all variables. To understand this point, we must distinguish between two types of variables: con- 68% of values tinuous and discrete. A continuous variable (or ratio variable) increases steadily in tiny fractions. An example is age, which increases steadily with b. Low standard deviation = tightly clustered values each increment of time. A discrete variable Professional Golfer’s Scores jumps from category to category without intervening steps. Examples include sex, military rank, or year in college (you go from being a sophomore to a junior in one step). In analyzing a discrete variable—a nominal or ordinal variable, for example—some of the techniques discussed previously do not apply. Strictly

speaking, modes should be calculated for nominal Number of games of Number data, medians for interval data, and means for ratio data, not for nominal data (see Chapter 5). If the Mean = 70 variable in question is sex, for example, raw num- 68% of values bers (23 of the cross-dressing outlaw bikers in our sample are women) or percentages (7 percent are FIGURE 14-4 High and Low Standard women) can be appropriate and useful analyses, Deviations but neither a median nor a mean would make any sense. Calculating the mode would be legitimate, is also less consistent: sometimes doing much bet- though not very revealing, because it would only ter, sometimes much worse. tell us “most were men.” However, the mode for There are many other measures of dispersion. In data on religious affi liation might be more interest- reporting intelligence test scores, for example, re- ing, as in “most people in the United States are searchers might determine the interquartile range, Protestant.” the range of scores for the middle 50 percent of subjects. If the top one-fourth had scores ranging from 120 to 150, and if the bottom one-fourth had Detail versus Manageability In presenting univariate and other data, you’ll be continuous variable A variable whose attributes constrained by two goals. On the one hand, you form a steady progression, such as age or income. Thus, should attempt to provide your reader with the the ages of a group of people might include 21, 22, 23, fullest degree of detail regarding those data. On 24, and so forth and could even be broken down into the other hand, the data should be presented in fractions of years. a manageable form. As these two goals often di- discrete variable A variable whose attributes are separate from one another, or discontinuous, as in the rectly confl ict, you’ll fi nd yourself continually seek- case of sex or religious affi liation. In other words, there is ing the best compromise between them. One use- no progression from male to female in the case of sex. ful solution is to report a given set of data in more SUBGROUP COMPARISONS 455 than one form. In the case of age, for example, you made legal?” In response, 33.4 percent said it might report the distribution of ungrouped ages should and 66.6 percent said it shouldn’t. Table 14-5 plus the mean age and standard deviation. presents the responses given to this question by As you can see from this introductory discus- respondents in different age categories. sion of univariate analysis, this seemingly simple Notice that the subgroup comparisons tell us matter can be rather complex. In any event, the how different groups in the population responded lessons of this section pave the way for a consid- to this question. You can undoubtedly see a pat- eration of subgroup comparisons and bivariate tern in the results, though possibly not exactly analyses. what you expected; we’ll return to that in a moment. First, let’s see how another set of subgroups SUBGROUP COMPARISONS answered this question. Table 14-6 presents attitudes toward legalizing marijuana by different political subgroups, based Univariate analyses describe the units of analysis on whether respondents characterized themselves of a study and, if they are a sample drawn from as conservative or liberal. Before looking at the some larger population, allow us to make descrip- table, you might try your hand at hypothesizing tive inferences about the larger population. Bivari- what the results are likely to be and why. Notice ate and multivariate analyses are aimed primar- that I have changed the direction of percentaging ily at explanation. Before turning to explanation, this table, to make it easier to read. To compare the however, we should consider the case of subgroup subgroups in this case, you would read down the description. columns, not across them. Often it’s appropriate to describe subsets of Before examining the logic of causal analysis, cases, subjects, or respondents. Here’s a simple let’s consider another example of subgroup com- example from the General Social Survey. In 2004, parisons—one that will let us address some table- respondents were asked, “Should marijuana be formatting issues.

TABLE 14-5 Marijuana Legalization by Age of Respondents, 2004

Under 21 21–35 36–54 55 and Older

Should be legalized 27% 40% 37% 24% Should not be legalized 73 60 63 76 100% (34) (238) (338) (265)

Source: General Social Survey, 2004, National Opinion Research Center.

TABLE 14-6 Marijuana Legalization by Political Orientation, 2004

Should Should ؍ Legalize Not Legalize 100%

Extremely liberal 77% 23 (30) Liberal 49% 51 (75) Slightly liberal 35% 65 (92) Moderate 33% 67 (326) Slightly conservative 32% 68 (136) Conservative 25% 75 (155) Extremely conservative 16% 84 (37)

Source: General Social Survey, 2004, National Opinion Research Center. 456 CHAPTER 14 QUANTITATIVE DATA ANALYSIS

“Collapsing” Response Categories the United States were the most positive (46 percent) about the UN’s performance, followed closely “Textbook examples” of tables are often simpler by France (45 percent), with Britain (39 percent) than you’ll typically fi nd in published research re- less positive than any of those three and Japan (11 ports or in your own analyses of data, so this sec- percent) the least positive of all. tion and the next one address two common prob- This procedure is inappropriate in that it ig- lems and suggest solutions. nores all those respondents who gave the most Let’s begin by turning to Table 14-7, which re- positive answer of all: “very good job.” In a situa- ports data collected in a multinational poll con- tion like this, you should combine or “collapse” the ducted by the New York Times, CBS News, and the two ends of the range of variation. In this instance, Herald Tribune in 1985, concerning attitudes about combine “very good” with “good” and “very poor” the United Nations. The question reported in Table with “poor.” If you were to do this in the analysis 14-7 deals with general attitudes about the way of your own data, it would be wise to add the raw the UN was handling its job. frequencies together and recompute percentages Here’s the question: How do people in the fi ve for the combined categories, but in analyzing a nations reported in Table 14-7 compare in their published table such as this one, you can simply support for the kind of job the UN was doing? As add the percentages, as illustrated by the results you review the table, you may fi nd there are simply shown in Table 14-8. so many numbers that it’s hard to see any mean- With the collapsed categories illustrated in Ta- ingful pattern. ble 14-8, we can now rather easily read across the Part of the problem with Table 14-7 lies in the several national percentages of people who said relatively small percentages of respondents select- the UN was doing at least a good job. Now the ing the two extreme response categories: the UN is United States appears the most positive; Germany, doing a very good or a very poor job. Furthermore, Britain, and France are only slightly less positive although it might be tempting to read only the sec- and are nearly indistinguishable from one another; ond line of the table (those saying “good job”), that and Japan stands alone in its quite low assessment would be improper. Looking at only the second of the UN’s performance. Although the conclusions row, we would conclude that West Germany and

Text not available due to copyright restrictions

TABLE 14-8 Collapsing Extreme Categories

West Germany Britain France Japan United States

Good job or better 48% 46% 47% 12% 51% Poor job or worse 27 37 25 48 40 Don’t know 26 17 28 41 10 SUBGROUP COMPARISONS 457 to be drawn now do not differ radically from what Here’s an easy way to recalculate percentages, we might have concluded from simply reading the with the “don’t knows” excluded. Look at the fi rst second line of Table 14-7, we should note that Brit- column of percentages in Table 14-8: West Germa- ain now appears relatively more supportive. ny’s answers to the question about the UN’s perfor- Here’s the risk I’d like to spare you. Suppose you mance. Notice that 26 percent of the respondents had hastily read the second row of Table 14-7 and said they didn’t know. This means that those who noted that the British had a somewhat lower as- said “good” or “bad” job—taken together—repre- sessment of the job the UN was doing than was sent only 74 percent (100 minus 26) of the whole. true of people in the United States, West Germany, If we divide the 48 percent saying “good job or bet- and France. You might feel obliged to think up an ter” by 0.74 (the proportion giving any opinion), we explanation for why that was so—possibly creat- can say that 65 percent “of those with an opinion” ing an ingenious psychohistorical theory about the said the UN was doing a good or very good job painful decline of the once powerful and dignifi ed (48%/0.74 65%). British Empire. Then, once you had touted your Table 14-9 presents the whole table with the “theory” about, someone else might point out that “don’t knows” excluded. Notice that these new a proper reading of the data would show the Brit- data offer a somewhat different interpretation than ish were actually not really less positive than the do the previous tables. Specifi cally, it would now other three nations. This is not a hypothetical risk. appear that France and West Germany were the Errors like these happen frequently, but they can most positive in their assessments of the UN, with be avoided by collapsing answer categories where the United States and Britain a bit lower. Although appropriate. Japan still stands out as lowest in this regard, it has moved from 12 percent to 20 percent positive. Handling “Don’t Knows” At this point, having seen three versions of the data, you may be asking yourself, Which is the right Tables 14-7 and 14-8 illustrate another common one? The answer depends on your purpose in ana- problem in the analysis of survey data. It’s usually lyzing and interpreting the data. For example, if it a good idea to give people the option of saying is not essential for you to distinguish “very good” “don’t know” or “no opinion” when asking for their from “good,” it makes sense to combine them, be- opinions on issues. But what do you do with those cause it’s easier to read the table. answers when you analyze the data? Whether to include or exclude the “don’t Notice there is a good deal of variation in the knows” is harder to decide in the abstract. It may national percentages saying “don’t know” in this be a very important fi nding that such a large per- instance, ranging from only 10 percent in the centage of the Japanese had no opinion—if you United States to 41 percent in Japan. The presence wanted to fi nd out whether people were familiar of substantial percentages saying they don’t know with the work of the UN, for example. On the other can confuse the results of a tables like these. For hand, if you wanted to know how people might example, were the Japanese so much less likely to vote on an issue, it might be more appropriate to say the UN was doing a good job simply because exclude the “don’t knows” on the assumption that so many didn’t express any opinion? they wouldn’t vote or that ultimately they would be

TABLE 14-9 Omitting the “Don’t Knows”

West Germany Britain France Japan United States

Good job or better 65% 55% 65% 20% 57% Poor job or worse 35% 45% 35% 81% 44% 458 CHAPTER 14 QUANTITATIVE DATA ANALYSIS likely to divide their votes between the two sides of the issue. IN THE REAL WORLD In any event, the truth contained within your YOUR LIFE AND THE NUMBERS data is that a certain percentage said they didn’t know and the remainder divided their opinions in You’ll likely be analyzing quantitative data in whatever manner they did. Often, it’s appropriate most aspects of your life for as long as you to report your data in both forms—with and with- live. Doing it well is a really good idea. Do out the “don’t knows”—so your readers can also you keep running short of money? A rigor- draw their own conclusions. Of course, you your- ous budgeting system may be the solution, self will be a reader of such tables, drawn up by telling you where your money is going and others, and knowing the logic behind constructing giving you the possibility of addressing prob- them will help you be a savvy consumer of quan- lems. Being comfortable with quantitative titative data. See the box “Your Life and the Num- analyses can also be a big help in connec- bers” for more on this. tion with sports—whether for fun or profi t. And, who knows, you may have children Numerical Descriptions one day who need help with their math. Be- in Qualitative Research ing able to assist them in that regard may impress them and make up for what they Although this chapter deals primarily with quan- regard as your defi ciencies in dress and mu- titative research, the discussions are also relevant sical taste. to qualitative studies. Numerical testing can often verify the fi ndings of in-depth, qualitative studies. Thus, for example, when David Silverman wanted to compare the cancer treatments received by pa- data indeed did show that the former were tients in private clinics with those in Britain’s Na- almost twice as long as the latter (20 minutes tional Health Service, he primarily chose in-depth as against 11 minutes) and that the difference analyses of the interactions between doctors and was statistically highly signifi cant. However, I patients: recalled that, for special reasons, one of the NHS clinics had abnormally short consultations. I felt My method of analysis was largely qualitative a fairer comparison of consultations in the two and . . . I used extracts of what doctors and sectors should exclude this clinic and should patients had said as well as offering a brief only compare consultations taken by a single ethnography of the setting and of certain behav- doctor in both sectors. This subsample of cases ioural data. In addition, however, I constructed revealed that the difference in length between a coding form which enabled me to collate a NHS and private consultations was now reduced number of crude measures of doctor and patient to an average of under 3 minutes. This was still interactions. — (1993:163) statistically signifi cant, although the signifi cance Not only did the numerical data fi ne-tune Silver- was reduced. Finally, however, if I compared man’s impressions based on his qualitative ob- only new patients seen by the same doctor, NHS servations, but his in-depth understanding of the patients got 4 minutes more on the average—34 situation allowed him to craft an ever more ap- minutes as against 30 minutes in the private propriate quantitative analysis. Listen to the in- clinic. — (SILVERMAN 1993:163–64) teraction between qualitative and quantitative ap- This example further demonstrates the special proaches in this lengthy discussion: power that can be gained from a combination of My overall impression was that private consul- approaches in social research. The combination of tations lasted considerably longer than those qualitative and quantitative analyses can be espe- held in the NHS clinics. When examined, the cially potent. BIVARIATE ANALYSIS 459

BIVARIATE ANALYSIS of Table 14-10 might be taken from Charles Glock’s Comfort Hypothesis as discussed in Chapter 2:

In contrast to univariate analysis, subgroup com- 1. Women are still treated as second-class citi- parisons involve two variables. In this respect, zens in U.S. society. subgroup comparisons constitute a kind of bi- 2. People denied status gratifi cation in the secular variate analysis—that is, an analysis of two vari- society may turn to religion as an alternative ables simultaneously. However, as with univariate source of status. analysis, the purpose of subgroup comparisons is 3. Hence, women should be more religious than largely descriptive. Most bivariate analysis in so- men. cial research adds another element: determining The data presented in Table 14-10 confi rm this relationships between the variables themselves. reasoning. Thirty-one percent of the women at- Thus, univariate analysis and subgroup compari- tend religious services weekly, as compared with sons focus on describing the people (or other units 22 percent of the men. of analysis) under study, whereas bivariate anal- Adding the logic of causal relationships among ysis focuses on the variables and their empirical variables has an important implication for the con- relationships. struction and reading of percentage tables. One of Table 14-10 could be regarded as an instance the chief bugaboos for new-data analysts is decid- of subgroup comparison: It independently de- ing on the appropriate “direction of percentaging” scribes the attendance of men and women at refor any given table. In Table 14-10, for example, I’ve ligious services, as reported in the 2004 General divided the group of subjects into two subgroups— Social Survey. It shows—comparatively and de- men and women—and then described the behav- scriptively—that the women under study attended ior of each subgroup. That is the correct method religious services more often than did the men. for constructing this table. Notice, however, that However, the same table, seen as an explanatory we could—however inappropriately—construct the bivariate analysis, tells a somewhat different story. table differently. We could fi rst divide the subjects It suggests that the variable sex has an effect on into different degrees of religious attendance and the variable religious service attendance. That is, we then describe each of those subgroups in terms of can view the behavior as a dependent variable that the percentage of men and women in each. This is partially determined by the independent vari- method would make no sense in terms of expla- able, sex. nation, however. Table 14-10 suggests that your Explanatory bivariate analyses, then, involve sex will affect your frequency of religious service the “variable language” introduced in Chapter 1. attendance. Had we used the other method of con- In a subtle shift of focus, we are no longer talking struction, the table would suggest that your reli- about men and women as different subgroups but gious service attendance affects whether you are about sex as a variable: one that has an infl uence a man or a woman—which makes no sense. Your on other variables. The theoretical interpretation behavior cannot determine your sex. A related problem complicates the lives of new- TABLE 14-10 Religious Attendance Reported data analysts. How do you read a percentage ta- by Men and Women in 2004 ble? There is a temptation to read Table 14-10 as Men Women follows: “Of the women, only 31 percent attended

Weekly 22% 31% Less often 78 69 bivariate analysis The analysis of two variables simul- 100% (1,276) (1,525) taneously, for the purpose of determining the empirical relationship between them. The construction of a simple Source: General Social Survey, 2004, National Opinion Research percentage table or the computation of a simple correla- Center. tion coeffi cient are examples of bivariate analyses. 460 CHAPTER 14 QUANTITATIVE DATA ANALYSIS religious services weekly, and 69 percent said they Percentaging a Table attended less often; therefore, being a woman Figure 14-5 reviews the logic by which we create makes you less likely to attend religious services percentage tables from two variables. I’ve used as frequently.” This is, of course, an incorrect read- variables sex and attitudes toward equality for men ing of the table. Any conclusion that sex—as a vari- and women. able—has an effect on religious service attendance Here’s another example. Suppose we’re in- must hinge on a comparison between men and terested in learning something about newspaper women. Specifi cally, we compare the 31 percent editorial policies regarding the legalization of mar- with the 22 percent and note that women are more ijuana. We undertake a content analysis of edito- likely than men to attend religious services weekly. rials on this subject that have appeared during a The comparison of subgroups, then, is essential in given year in a sample of daily newspapers across reading an explanatory bivariate table. the nation. Each editorial has been classifi ed as fa- In constructing and presenting Table 14-10, vorable, neutral, or unfavorable toward the legal- I’ve used a convention called percentage down. ization of marijuana. Perhaps we wish to examine This term means that you can add the percent- the relationship between editorial policies and the ages down each column to total 100 percent. You types of communities in which the newspapers are read this form of table across a row. For the row published, thinking that rural newspapers might be labeled “Weekly,” what percentage of the men at- more conservative in this regard than urban ones. tend weekly? What percentage of the women at- Thus, each newspaper (hence, each editorial) has tend weekly? been classifi ed in terms of the population of the The direction of percentaging in tables is arbi- community in which it is published. trary, and some researchers prefer to percentage Table 14-11 presents hypothetical data describ- across, as I did in Table 14-6. They would orga- ing the editorial policies of rural and urban news- nize Table 14-10 so that “Men” and “Women” were papers. Note that the unit of analysis in this exam- shown on the left side of the table, identifying the ple is the individual editorial. Table 14-11 tells us two rows, and “Weekly” and “Less often” would that there were 127 editorials about marijuana in appear at the top to identify the columns. The ac- our sample of newspapers published in communi- tual numbers in the table would be moved around ties with populations under 100,000. (Note that this accordingly, and each row of percentages would cutting point is chosen for simplicity of illustration total 100 percent. In that case, you would read the and does not mean that rural refers to a commu- table down a column, still asking what percentage nity of less than 100,000 in any absolute sense.) of men and women attended frequently. The logic Of these, 11 percent (14 editorials) were favorable and the conclusion would be the same in either toward legalization of marijuana, 29 percent were case; only the form would differ. neutral, and 60 percent were unfavorable. Of the In reading a table that someone else has constructed, therefore, you need to fi nd out in which direction it has been percentaged. Usually this will TABLE 14-11 Hypothetical Data Regarding Newspaper Editorials on the Legalization of be labeled or be clear from the logic of the vari- Marijuana ables being analyzed. As a last resort, however, you should add the percentages in each column Community Size Editorial Policy and each row. If each of the columns totals 100 Toward Legalizing Under Over percent, the table has been percentaged down. If Marijuana 100,000 100,000 the rows total 100 percent each, it has been per- Favorable 11% 32% centaged across. The rule, then, is as follows: Neutral 29 40 1. If the table is percentaged down, read across. Unfavorable 60 28 2. If the table is percentaged across, read down. 100% (127) (438) a. Some men and women who either favor (+) gender equality or don’t (–) favor it.

+ – + + + + – + – + + – + + – + + + – +

b. Separate the men and the women (the independent variable).

+ + + – + + + + – + Women

+ – + + – – + + – + Men

c. Within each gender group, separate those who favor equality from those who don’t (the dependent variable).

+ + + + + + + + – – Women

+ + + + + + – – – – Men

d. Count the numbers in each cell of the table.

8 + + + + + + + + 2 – – Women

6 4 + + + + + + – – – – Men

e. What percentage of the women favor equality? f. What percentage of the men favor equality?

80% + + + + + + + + 60% + + + + + +

– – – – – –

g. Conclusions. Women Men While a majority of both men and women favored gender equality, Favor equality 80% 60% women were more likely than men to do so. Don’t favor equality 20 40 Thus, gender appears to be one of the causes of attitudes toward sexual equality. Total100% 100% FIGURE 14-5 Percentaging a Table 462 CHAPTER 14 QUANTITATIVE DATA ANALYSIS

438 editorials that appeared in our sample of news- 3. Men and women are compared in terms of the papers published in communities of more than percentages approving of gender equality. 100,000 residents, 32 percent (140 editorials) were In the example of editorial policies regarding favorable toward legalizing marijuana, 40 percent the legalization of marijuana, size of community is were neutral, and 28 percent were unfavorable. the independent variable, and a newspaper’s edito- When we compare the editorial policies of ru- rial policy the dependent variable. The table would ral and urban newspapers in our imaginary study, be constructed as follows: we ﬁ nd—as expected—that rural newspapers are less favorable toward the legalization of marijuana 1. Divide the editorials into subgroups according than are urban newspapers. We determine this by to the sizes of the communities in which the noting that a larger percentage (32 percent) of the newspapers are published. urban editorials were favorable than the percent- 2. Describe each subgroup of editorials in terms age of rural ones (11 percent). We might note as of the percentages favorable, neutral, or unfa- well that more rural than urban editorials were un- vorable toward the legalization of marijuana. favorable (60 percent compared with 28 percent). 3. Compare the two subgroups in terms of the Note that this table assumes that the size of a com- percentages favorable toward the legalization munity might affect its newspapers’ editorial poli- of marijuana. cies on this issue, rather than that editorial policy Bivariate analyses typically have an explanatory might affect the size of communities. causal purpose. These two hypothetical examples have hinted at the nature of causation as social Constructing and Reading scientists use it. Bivariate Tables Tables such as the ones we’ve been examining are commonly called contingency tables: Values Let’s now review the steps involved in the con- of the dependent variable are contingent on (destruction of explanatory bivariate tables: pend on) values of the independent variable. Al- though contingency tables are common in social 1. The cases are divided into groups according to science, their format has never been standardized. the attributes of the independent variable. As a result, you’ll ﬁ nd a variety of formats in re- 2. Each of these subgroups is then described in search literature. As long as a table is easy to read terms of attributes of the dependent variable. and interpret, there’s probably no reason to strive 3. Finally, the table is read by comparing the infor standardization. However, there are several dependent variable subgroups with each other guidelines that you should follow in the presenta- in terms of a given attribute of the dependent tion of most tabular data: variable. 1. A table should have a heading or a title that Let’s repeat the analysis of sex and attitude on succinctly describes what is contained in the gender equality following these steps. For the rea- table. sons outlined previously, sex is the independent 2. The original content of the variables should variable; attitude toward gender equality consti- be clearly presented—in the table itself if at tutes the dependent variable. Thus, we proceed as all possible or in the text with a paraphrase in follows: the table. This information is especially critical 1. The cases are divided into men and women. when a variable is derived from responses to 2. Each gender subgrouping is described in terms an attitudinal question, because the meaning of approval or disapproval of gender equality. of the responses will depend largely on the wording of the question. 3. The attributes of each variable should be contingency table A format for presenting the rela- clearly indicated. Though complex categories tionships among variables as percentage distributions. will have to be abbreviated, their meaning INTRODUCTION TO MULTIVARIATE ANALYSIS 463

should be clear in the table and, of course, the TABLE 14-12 Multivariate Relationship: full description should be reported in the text. Religious Service Attendance, Sex, and Age 4. When percentages are reported in the table, “How often do you attend religious services?” the base on which they are computed should be indicated. It’s redundant to present all the Under 40 40 and Older raw numbers for each category, because these Men Women Men Women could be reconstructed from the percentages About weekly* 16% 24% 26% 35% and the bases. Moreover, the presentation of Less often 84 76 74 65 both numbers and percentages often confuses 100% (495) (602) (781) (923) a table and makes it more diffi cult to read. 5. If any cases are omitted from the table because *About weekly “More than once a week,” “Weekly,” and of missing data (“no answer,” for example), “Nearly every week.” their numbers should be indicated in the table. Source: General Social Survey, 2004, National Opinion Research Center. Although I have introduced the logic of causal, Let’s return to the example of religious atten- bivariate analysis in terms of percentage tables, dance. Suppose we believe that age would also many other formats are appropriate to this topic. affect such behavior (Glock’s Comfort Hypothesis Scatterplot graphs are one possibility, providing a suggests that older people are more religious than visual display of the relationship between two vari- younger people). As the fi rst step in table construc- ables. For an engaging example of this, you might tion, we would divide the total sample into sub- check out the GapMinder software available on the groups based on the attributes of both indepen- web. Using countries as the unit of analysis, you dent variables simultaneously: younger men, older can examine the relationship between birthrate men, younger women, and older women. Then and infant mortality, for example. In fact, you can the several subgroups would be described in terms watch the relationship develop over time. of the dependent variable, religious service attendance, and comparisons would be made. Table You can fi nd GapMinder at http://tools 14-12, from an analysis of the 2004 General Social .google.com/gapminder/. Survey data, is the result. Table 14-12 has been percentaged down and therefore should be read across. The interpretation of this table warrants several conclusions: INTRODUCTION 1. Among both men and women, older people TO MULTIVARIATE ANALYSIS attend religious services more often than do younger people. Among women, 24 percent of those under 40 and 35 percent of those 40 The logic of multivariate analysis, or the analysis and older attend religious services weekly. of more than two variables simultaneously, can be Among men, the respective fi gures are 16 and seen as an extension of bivariate analysis. Specifi - 26 percent. cally, we can construct multivariate tables on the 2. Within each age group, women attend slightly basis of a more complicated subgroup description more frequently than men. Among those re- by following essentially the same steps outlined for spondents under 40, 24 percent of the women bivariate tables. Instead of one independent vari- attend weekly, compared with 16 percent of able and one dependent variable, however, we’ll the men. Among those 40 and over, 35 percent have more than one independent variable. Instead of explaining the dependent variable on the basis multivariate analysis The analysis of the simulta- of a single independent variable, we’ll seek an ex- neous relationships among several variables. Examining planation through the use of more than one inde- simultaneously the effects of age, sex, and social class on pendent variable. religiosity would be an example of multivariate analysis. 464 CHAPTER 14 QUANTITATIVE DATA ANALYSIS

of the women and 26 percent of the men at- on which the percentages are based. Thus, for extend weekly. ample, the reader knows there are 602 women un- 3. As measured in the table, age appears to have der 40 years of age in the sample, and 24 percent a greater effect on attendance at religious of them attend religious services weekly. We can services than does sex. calculate from this that 147 of those 602 women 4. Age and sex have independent effects on attend weekly and that the other 455 younger religious service attendance. Within a given women (or 76 percent) attend less frequently. This attribute of one independent variable, different new table is easier to read than the former one, attributes of the second still affect behaviors. and it does not sacrifi ce any detail. 5. Similarly, the two independent variables have a cumulative effect on behaviors. Older SOCIOLOGICAL DIAGNOSTICS women attend the most often (35 percent), and younger men attend the least often (16 percent). The multivariate techniques we are now exploring can serve as powerful tools for diagnosing social Before I conclude this section, it will be useful problems. They can be used to replace opinions to note an alternative format for presenting such with facts and to settle ideological debates with data. Several of the tables presented in this chap- data analysis. ter are somewhat ineffi cient. When the dependent For an example, let’s return to the issue of gen- variable, religious attendance, is dichotomous (hav- der and income. Many explanations have been ad- ing exactly two attributes), knowing one attribute vanced to account for the long-standing pattern of permits the reader to reconstruct the other easily. women in the labor force earning less than men. Thus, if we know that 24 percent of the women One explanation is that, because of traditional under 40 attend religious services weekly, then we family patterns, women as a group have partici- know automatically that 76 percent attend less of- pated less in the labor force and many only begin ten. So reporting the percentages who attend less working outside the home after completing certain often is unnecessary. child-rearing tasks. Thus, women as a group will On the basis of this recognition, Table 14-12 probably have less seniority at work than will men, could be presented in the alternative format of and income increases with seniority. An important Table 14-13. In Table 14-13, the percentages of 1984 study by the Census Bureau showed this rea- people saying they attend religious services about soning to be partly true, as Table 14-14 shows. weekly are reported in the cells representing the Table 14-14 indicates, fi rst of all, that job tenure intersections of the two independent variables. did indeed affect income. Among both men and The numbers presented in parentheses below women, those with more years on the job earned each percentage represent the number of cases more. This is seen by reading down the fi rst two columns of the table. TABLE 14-13 A Simplifi cation of Table 14-12 The table also indicates that women earned less Percent Who than men, regardless of job seniority. This can be Attend about Weekly seen by comparing average wages across the rows Men Women of the table, and the ratio of women-to-men wages is shown in the third column. Thus, years on the Under 40 16 24 job was an important determinant of earnings, but (495) (602) seniority did not adequately explain the pattern of 40 and Older 26 35 women earning less than men. In fact, we see that (781) (923) women with 10 or more years on the job earned

Source: General Social Survey, 2004, National Opinion Research substantially less ($7.91/hour) than men with less Center. than two years ($8.46/hour). SOCIOLOGICAL DIAGNOSTICS 465

TABLE 14-14 Gender, Job Tenure, and Income, 1984 (Full-time workers 21–64 years of age)

Years Working with Average Hourly Income Current Employer Men Women Women/Men Ratio

Less than 2 years $8.46 $6.03 0.71 2 to 4 years $9.38 $6.78 0.72 5 to 9 years $10.42 $7.56 0.73 10 years or more $12.38 $7.91 0.64

Source: U.S. Bureau of the Census, Current Population Reports, Series P-70, No. 10, Male- Female Differences in Work Experience, Occupation, and Earning, 1984 (Washington, DC: U.S. Government Printing Ofﬁ ce, 1987), 4.

Although years on the job did not fully explain Each of the variables listed here might reason- the difference between men’s and women’s pay, ably affect earnings and, if women and men differ there are other possible explanations: level of edu- in these regards, could help to account for male/fe- cation, child care responsibilities, and so forth. The male income differences. When all these variables researchers who calculated Table 14-14 also ex- were taken into account, the researchers were amined some of the other variables that might rea- able to account for 60 percent of the discrepancy sonably explain the differences in pay without rep- between the incomes of men and women. The resenting gender discrimination, including these: remaining 40 percent, then, is a function of other “reasonable” variables and/or prejudice. This kind • Number of years in the current occupation of conclusion can be reached only by examining • Total years of work experience (any the effects of several variables at the same time— occupation) that is, through multivariate analysis. • Whether they have usually worked full time I hope this example shows how the logic im- • Marital status plicit in day-to-day conversations can be repre- • Size of city or town they live in sented and tested in a quantitative data analysis • Whether covered by a union contract like this. See the box “Salary Discrimination against • Type of occupation Women” for more on this issue. • Number of employees in the ﬁ rm As another example of multivariate data analy- • Whether private or public employer sis in real life, consider the common observation • Whether they left previous job involuntarily that minority group members are more likely to • Time spent between current and previous job be denied bank loans than are white applicants. A • Race counterexplanation might be that the minority ap- • Whether they have a disability plicants in question were more likely to have had • Health status a prior bankruptcy or that they had less collateral • Age of children to guarantee the requested loan—both reasonable • Whether they took an academic curriculum in bases for granting or denying loans. However, the high school kind of multivariate analysis we’ve just examined • Number of math, science, and foreign could easily resolve the disagreement. language classes in high school Let’s say we look only at those who have not had • Whether they attended private or public high a prior bankruptcy and who have a certain level of school collateral. Are whites and minorities equally likely • Educational level achieved to get the requested loan? We could conduct the • Percentage of women in the occupation same analysis in subgroups determined by level • College major of collateral. If whites and minorities were equally 466 CHAPTER 14 QUANTITATIVE DATA ANALYSIS

ETHICS AND QUANTITATIVE IN THE REAL WORLD DATA ANALYSIS SALARY DISCRIMINATION AGAINST WOMEN In Chapter 13, I pointed out that the subjectivity present in qualitative data analysis increases the The data in the text pointed to salary dis- risk of biased analyses, which experienced re- crimination against women in 1984, but searchers learn to avoid. Some think, however, hasn’t that been remedied? Not really, as in- that quantitative analyses are not susceptible to dicated by more recent data. subjective biases. Unfortunately, this isn’t so. Even In 2003 the average full-time, year-round the most mathematically explicit analysis yields male worker earned $53,039. The average ample room for defi ning and measuring variables full-time, year-round female worker earned in ways that encourage one fi nding over another, $37,197, or about 70 percent as much as her and quantitative analysts need to guard against male counterpart (U.S. Bureau of the Census this. Sometimes, the careful specifi cation of hy- 2006:467). But does that difference represent potheses in advance can offer protection, although gender discrimination, or does it refl ect le- this can also inhibit a full exploration of what data gitimate factors? can tell us. For example, some people argue that education affects income and that, in the past, women have gotten less education than men. We might start, therefore, by checking WHAT DO YOU THINK? REVISITED whether educational differences explain why women today earn less, on average, than men. This chapter began with a question about Table 14-15, which we discussed briefl y in whether anything meaningful or useful Chapter 11, offers data to test this hypothesis. could be learned from the analysis of data As the table shows, at each level of com- that have been stripped of many details in parable education, women earn substan- order to permit statistical manipulation. The tially less than men. Clearly, education does answer, we’ve seen, is an unqualifi ed “yes.” not explain the discrepancy. Quantitative analysis can be a tool for This is the kind of analysis you are now social change. For instance, calculating the equipped to undertake. average incomes of men and women or of whites and minorities can demonstrate the inequalities that exist for people do- likely to get their loans in each of the subgroups, ing exactly the same job. Such quantitative we would need to conclude that there was no eth- analyses can overpower anecdotal evidence nic discrimination. If minorities were still less likely about particular women or minorities earn- to get their loans, however, that would indicate ing large salaries. We’ve also seen that quan- that bankruptcy and collateral differences were titative analyses of qualitative phenomena, not the explanation—strengthening the case that such as voting intentions, can be done with discrimination was at work. precision and utility. All this should make clear that social research The key lesson is that both qualitative can play a powerful role in serving in the human and quantitative research are legitimate and community. It can help us determine the current powerful approaches to understanding so- state of affairs and can often point the way to cial life. They are particularly useful, more- where we want to go. over, when used together. Welcome to the world of sociological diagnostics! MAIN POINTS 467

TABLE 14-15 Average Earnings of Year-Round, Full-Time Workers, 2003

Ratio of Women/ Men Women Men Earnings

All workers 53,039 37,197 0.70 Less than 9th grade 23,972 20,979 0.88 9th–12th grades 29,100 21,426 0.74 HS graduates 38,331 27,956 0.73 Some college 46,332 31,655 0.68 Associate degree 48,683 36,528 0.75 Bachelors or more 81,007 53,215 0.66

Source: U.S. Bureau of the Census, Statistical Abstract of the United States (Washington, DC: U.S. Government Printing Ofﬁ ce, 2006), Table 686, p. 467. You can also access this table online at http:// www.census.gov/prod/2005pubs/06statab/income.pdf.

The quantitative analyst has an obligation to The protection of subjects’ privacy is as impor- report formal hypotheses and less formal expecta- tant in quantitative analysis as in qualitative analy- tions that didn’t pan out. Let’s suppose you think sis. However, with quantitative methods it’s often that a particular variable will prove a powerful easier to collect and record data in ways that make cause of gender prejudice, but your data analysis subject identifi cation more diffi cult. However, the contradicts that expectation. You should report fi rst time public offi cials demand that you reveal the the lack of correlation, because such information names of student subjects who reported using ille- is useful to others who conduct research on this gal drugs in a survey, this issue will take on more topic. Although it would be more satisfying to dis- salience. (Don’t reveal the names, by the way. If cover what causes prejudice, it’s quite important to necessary, burn the questionnaires—accidentally.) know what doesn’t cause it.

Main Points ❏ Researchers may use existing coding schemes, such as the Census Bureau’s categorization of occupations, or develop their own coding Introduction categories. In either case, the coding scheme ❏ Most data are initially qualitative: They must must be appropriate to the nature and objec- be quantiﬁ ed to permit statistical analysis. tives of the study. ❏ Quantitative analysis involves the techniques ❏ A codebook is the document that describes the by which researchers convert data to a identiﬁ ers assigned to different variables and numerical form and subject it to statistical the codes assigned to represent the attributes analyses. of those variables.

Quantiﬁ cation of Data Univariate Analysis ❏ Some data, such as age and income, are intrin- ❏ Univariate analysis is the analysis of a single sically numerical. variable. Because univariate analysis does not ❏ Often, quantiﬁ cation involves coding into involve the relationships between two or more categories that are then given numerical variables, its purpose is descriptive rather than representations. explanatory. 468 CHAPTER 14 QUANTITATIVE DATA ANALYSIS

❏ Several techniques allow researchers to sum- Ethics and Quantitative Data Analysis marize their original data to make them more ❏ Unbiased analysis and reporting is as much an manageable while maintaining as much of the ethical concern in quantitative analysis as in original detail as possible. Frequency distribu- qualitative analysis. tions, averages, grouped data, and measures ❏ Subjects’ privacy must be protected in quanti- of dispersion are all ways of summarizing data tative data analysis and reporting. concerning a single variable.

Subgroup Comparisons Key Terms ❏ Subgroup comparisons can be used to describe average mean similarities and differences among subgroups bivariate analysis median with respect to some variable. codebook mode ❏ Collapsing response categories and handling contingency table multivariate analysis “don’t knows” are two techniques for present- continuous variable quantitative analysis ing and interpreting data. discrete variable standard deviation dispersion univariate analysis frequency distribution Bivariate Analysis ❏ Bivariate analysis focuses on relationships between variables rather than comparisons of Review Questions groups. Bivariate analysis explores the statistical association between the independent vari- 1. How might the various majors at your college able and the dependent variable. Its purpose be classiﬁ ed into categories? Create a coding is usually explanatory rather than merely system that would allow you to categorize descriptive. them according to some meaningful variable. Then create a different coding system, using ❏ The results of bivariate analyses often are a different variable. presented in the form of contingency tables, which are constructed to reveal the effects of 2. How many ways could you be described in the independent variable on the dependent numerical terms? What are some of your variable. intrinsically numerical attributes? Could you express some of your qualitative attributes in Introduction to Multivariate Analysis quantitative terms? ❏ Multivariate analysis is a method of analyzing 3. How would you construct and interpret a the simultaneous relationships among several contingency table from the following infor- variables. It may also be used to understand mation: 150 Democrats favor raising the the relationship between two variables more minimum wage, and 50 oppose it; 100 Repub- fully. licans favor raising the minimum wage, and ❏ The logic and techniques of quantitative 300 oppose it? research can be valuable to qualitative 4. Using the hypothetical data in the following researchers. table, how would you construct and interpret tables showing these three relationships? Sociological Diagnostics a. The bivariate relationship between age ❏ Sociological diagnostics is a quantitative and attitude toward abortion analysis technique for determining the nature b. The bivariate relationship between politi- of social problems such as ethnic or gender cal orientation and attitude toward discrimination. abortion ADDITIONAL READINGS 469

c. The multivariate relationship linking age, Additional Readings political orientation, and attitude toward abortion Babbie, Earl, Fred Halley, and Jeanne Zaino. 2000. Attitude Adventures in Social Research. Newbury Park, CA: Political toward Pine Forge Press. This book introduces you to the Age Orientation Abortion Frequency analysis of social research data through SPSS for Windows. Several of the basic statistical techniques Young Liberal Favor 90 used by social researchers are discussed and Young Liberal Oppose 10 illustrated. Young Conservative Favor 60 Bernstein, Ira H., and Paul Havig. 1999. Computer Lit- Young Conservative Oppose 40 eracy: Getting the Most from Your PC. Thousand Oaks, Old Liberal Favor 60 CA: Sage. Here’s a quick overview of the various Old Liberal Oppose 40 ways social scientists use computers, including many Old Conservative Favor 20 common applications programs. Old Conservative Oppose 80 Davis, James. 1971. Elementary Survey Analysis. Engle- wood Cliffs, NJ: Prentice-Hall. An extremely well- written and well-reasoned introduction to analysis. Online Study Resources In addition to covering the materials just presented in this chapter, Davis’s book is well worth reading in terms of measurement and statistics. Ferrante, Joan, and Angela Vaughn. 1999. Let’s Go Sociol- ogy: Travels on the Internet. Belmont, CA: Wadsworth. Go to This accessible little book gives an excellent introduction to the Internet and suggests many websites http://sociology.wadsworth.com/babbie_basics4e of interest to social researchers. Lewis-Beck, Michael. 1995. Data Analysis: An Introduc- and click on ThomsonNow for access to this tion. Volume 103 in the Quantitative Application powerful online study tool. You will get a per- in the Social Sciences series. Thousand Oaks, CA: sonalized study plan based on your responses to Sage. This popular short book makes statistical a diagnostic pretest. Once you have mastered language accessible to the novice. You should enjoy the material with the help of interactive learning the clarity of explanations and the thorough use of tools, you can take a posttest to confi rm that you examples. are ready to move on to the next chapter. Nardi, Peter. 2006. Interpreting Data: A Guide to Under- standing Research. Boston: Pearson. This excellent little book offers an accessible guide to understand- Website for ing commonly used statistical analyses in the social The Basics of Social Research sciences. 4th edition Newton, Rae R., and Kjell Erik Rudestam. 1999. Your Statistical Consultant: Answers to Your Data Analy- At the book companion website (http://sociology sis Questions. Thousand Oaks, CA: Sage. Excellent .wadsworth.com/babbie_basics4e) you will fi nd reader-friendly manual that will answer all sorts of many resources in addition to ThomsonNow to questions you have or will have as soon as you begin to analyze quantitative data. aid you in studying for your exams. For example, Ziesel, Hans. 1957. Say It with Figures. New York: Harper you will fi nd Tutorial Quizzes with feedback, Inter- & Row. An excellent discussion of table construction net Exercises, Flashcards, and Chapter Tutorials, as and other elementary analyses. Though many years well as Extended Projects, InfoTrac College Edition old, this is still perhaps the best available presenta- search terms, Social Research in Cyberspace, GSS tion of that specifi c topic. It is eminently readable Data, Web Links, and primers for using various and understandable and has many concrete data analysis software such as SPSS and NVivo. examples. 151 HUMANREADING INQUIRY AND WRITING AND SCIENCE SOCIAL RESEARCH Photo credit John Henley/CORBIS

What You’ll Learn in thisThis ChapterChapter

SocialWe’ll research examine is theuseless way unlesspeople it’slearn communicated about their world effectively and the to mistakesothers. There they are makespecial along skills the involved way. We’ll in reading also begin the research to see what of others makes and sience writing different about from

otheryour own. ways of knowing things. 2S 1S N L 470 In this chapter . . . WHAT DO YOU THINK?

Introduction

Reading Social Research Organizing a Review of the Literature Journals versus Books Image not available due to copyright restrictions Evaluation of Research Reports Using the Internet Wisely

Writing Social Research Some Basic Considerations The Internet seems like a great place to get Organization of the Report information for term papers, but some of your Guidelines for Reporting Analyses professors may object, saying the quality of Going Public data on the Internet can’t be trusted. What The Ethics of Reading and Writing should you do? First, read this chapter. Social Research Then, see the “What Do You Think? Revis- ited” box toward the end of the chapter. INTRODUCTION

in order to focus on the resources that will help Meaningful scientifi c research is inextricably wed you the most. Then, when you actually sit down to communication, but it’s not always an easy to read them, you’ll need certain skills for doing or comfortable marriage. Scientists—social and so effi ciently. Finally, you should know how to fi nd other—are not necessarily good at communicating and assess sources on the Internet. their methods and fi ndings. Thus, it’s often hard to read and understand the research of others, and you may also fi nd it diffi cult to write up your own Organizing a Review of the Literature research in ways that communicate your ideas ef- With the exception of some grounded theory fectively. This fi nal chapter addresses these two methodologists, most social researchers begin the problems. design of a research project with a review of the We’ll begin with reading social research, then literature, as indicated in Chapter 4. Most original we’ll turn to writing it. Although I’ll offer guidance research is seen as an extension of what has previ- on both topics, you’ll fi nd that mastering each lies in ously been learned about a particular topic. A re- practice. The more you read social science research, view of the literature is the way we learn what’s the easier it gets, and the same is true of writing it. already known and not known. In most cases, you should organize your search of the literature around the key concepts you wish READING SOCIAL RESEARCH to study; alternatively, you may want to study a certain population: Iraqi War veterans, computer “Reading” is not as simple a task as it may seem, hackers, Catholic priests, gay athletes, and so especially when it involves social research. First, forth. In any case, you’ll identify a set of terms that you need to organize a review of the literature represent your core interests. 471 472 CHAPTER 15 READING AND WRITING SOCIAL RESEARCH

Your college or university library will probably using two separate words. You might also add have several search routines you can use at the “public opinion” to the request to narrow the fi eld library or online. Let’s say you’re interested in de- of possible resources. In general, online searches signing a study of attitudes toward capital punish- tend to turn up huge numbers of entries, most of ment. If your library provides access to InfoTrac which will not help you much. You’ll need some College Edition or a similar program, you might time to separate the wheat from the chaff. Later in discover, as I just did, 8,735 newspaper references this chapter, I’ll give you more-detailed guidelines and 5,489 periodical references to capital punish- for searching the web. ment. In such situations, InfoTrac College Edition No matter how you start the literature review is indexed to allow narrowing the search, and I process, you should always consider a technique soon discovered 249 entries for “public opinion” akin to snowball sampling, discussed in Chapter on capital punishment. Some of the entries were 7. Once you identify a particularly useful book or bibliographic citations and some were full-text ar- article, note which publications its author cites. ticles I could read online. Some of these will likely be useful. In fact, you’ll Another resource available to everyone is the probably discover some citations that appear Library of Congress, easily accessed online at again and again, suggesting that they’re core ref- http://catalog.loc.gov/. Clicking on “Basic Search” erences within the subject matter area you’re ex- or “Guided Search” will open up a vast resource for ploring. This last point is important, because the you. When I specifi ed the keyword as “capital pun- literature review is not about providing “window ishment” and limited the search to English-lan- dressing” in the form of a few citations. Rather, guage books published between 2000 and 2005, it’s about digging into the body of knowledge that the site listed 3,674 entries, such as the following: previous researchers have generated—and taking advantage of that knowledge as you design your • Abolition of the death penalty : SAHRDC’s own inquiry. submission to the National Commission for the Once you’ve identifi ed some potential re- Review of the Working of the Constitution. sources, you must read them and fi nd anything of • America’s experiment with capital punishment value to your project. Here are some guidelines for : refl ections on the past, present, and future of reading research publications. the ultimate penal sanction / [edited by] James R. Acker. • Beyond repair? : America’s death penalty / Journals versus Books edited by Stephen P. Garvey. As you might have guessed, you don’t read a so- • Capital punishment : a bibliography / C. Cliff, cial research report the way you’d read a novel. editor. You can, of course, but it’s not the most effective • Death penalty : infl uences and outcomes / approach. Journal articles and books are laid out edited by Austin Sarat. somewhat differently, so here are some initial Sometimes a simple web search is a useful way guidelines for reading each. to begin. Use a search engine such as Google, Hot- Bot, or Yahoo to look for web resources on “capi- Reading a Journal Article In most journals, each tal punishment” or “death penalty.” Be sure to use article begins with an abstract. Read it fi rst. It quotation marks to look for a phrase rather than should tell you the purpose of the research, the methods used, and the major fi ndings. In a good detective or spy novel, the suspense builds throughout the book and is resolved in some abstract A summary of a research article. The abstract usually begins the article and states the purpose of the kind of surprise ending. This is not the effect most research, the methods used, and the major fi ndings. scholarly writers are going for. Social research READING SOCIAL RESEARCH 473 is purposely anticlimactic. Rather than stringing the reader along, dragging out the suspense over whether X causes Y, social researchers willingly give away the punch line in the abstract. The abstract serves two main functions. First, it gives you a good idea as to whether you’ll want to read the rest of the article. If you’re reviewing the literature for a paper you’re writing, the abstract tells you whether that particular article is relevant. Second, the abstract establishes a framework within which to read the rest of the article. It may raise questions in your mind regarding method or conclusions, thereby creating an agenda to pursue in your reading. (It’s not a bad idea to jot those questions down, to be sure you get answers to them.) After you’ve read the abstract, you might go directly to the summary and/or conclusions at the end of the article. That will give you a more detailed picture of what the article is all about. (You can also do this with detective and spy novels; it makes reading them a lot faster but maybe not as much fun.) Jot down any new questions or observations that occur to you. Earl Babbie Next, skim the article, noting the section head- There’s nothing like sinking your teeth into a good book. ings and any tables or graphs. You don’t need to study any of these things in your skimming, though it’s okay to dally with anything that catches your listen. If you can explain it coherently to someone attention. By the end of this step, you should start who has no prior contact with the subject mat- feeling familiar with the article. You should be ter, however, you’ll have an absolute lock on the pretty clear on the researcher’s conclusions and material. have a general idea of the methods used in reaching them. Reading a Book The approach for reading articles Now, when you carefully read the whole article, can be adapted to reading a book-length report, you’ll have a good idea of where it’s heading and sometimes also called a research monograph. how each section fi ts into the logic of the whole These longer research reports cover the same ba- article. Keep taking notes. Mark any passages you sic terrain and structure. Instead of an abstract, the think you might like to quote later on. preface and opening chapter of the book should After carefully reading the article, it’s a good lay out the purpose, method, and main fi ndings of idea to skim it quickly one more time. This way the study. The preface tends to be written more in- you get back in touch with the forest after having formally and to be easier to understand than an focused on the trees. abstract. If you want to fully grasp what you’ve just read, fi nd someone else to explain it to. If you’re do- research monograph A book-length research report, ing the reading in connection with a course, you either published or unpublished. This is distinguished from should have no trouble fi nding someone willing to a textbook, a book of essays, a novel, and so forth. 474 CHAPTER 15 READING AND WRITING SOCIAL RESEARCH

As with an article, it’s useful to skim through the quoted in the report’s review of the literature book, getting a sense of its organization; its use of and elsewhere may offer a clue. tables, graphs, or other visuals; and so forth. You • On the other hand, is the author attempting to should come away from this step feeling some- refute some paradigm or theory? what familiar with the book. And, as I suggested • Is a theory or hypothesis being tested? in connection with reading an article, you should • In what way has the theoretical orientation take notes as you go along, writing down things shaped the methodology used in the study, you observe and questions that are raised. such as the data-collection technique and the As you settle in to read the book more care- choice of which data were collected and which fully, you should repeat this same process with were ignored? each chapter. Read the opening paragraphs to • Is the methodology used appropriate to the get a sense of what’s to come, and then skip to theoretical issues involved? the concluding paragraphs for the summary. Skim the chapter to increase your familiarity with it, and then read more deliberately, taking notes as Research Design you go. • What was the purpose of the study: ex- It’s sometimes okay to skip portions of a schol- ploration, description, explanation, or a arly book, unlike the way you were taught to read combination? and appreciate literature. This all depends on your • Who conducted the research? Who paid for purpose in reading it in the fi rst place. Perhaps it, if anyone? What motivated the study? If the there are only a few portions of the book that are study’s conclusions happen to correspond to relevant to your purposes. However, realize that if the interests of the sponsor or researcher, this you’re interested in the researcher’s fi ndings, you doesn’t disqualify the conclusions, but you’ll must pay some attention to the methods used (for want to be especially wary. instance, who was studied, how, when?) in order • What was the unit of analysis? Was it appropri- to judge the quality of the conclusions offered by ate to the purpose of the study? Are the con- the author. clusions drawn from the research appropriate to the unit of analysis? For example, have the Evaluation of Research Reports researchers studied cities and ended up with assertions about individuals? In this section, I provide sets of questions you • Is this a cross-sectional or a longitudinal might ask in reading and evaluating a research study? Be especially wary of longitudinal report. I’ve organized these questions to parallel assertions being made on the basis of cross- some of the preceding chapters in this book, to fa- sectional observations. cilitate your getting more details on a topic if nec- • If longitudinal data have been collected, have essary. Although hardly exhaustive, I hope these comparable measurements been made at each questions will help you grasp the meanings of re- point in time? In the case of survey data, have search reports you read and alert you to potential the same questions been asked each time? problems in them. If the report compares, say, crime or poverty rates, are they defi ned the same way each Theoretical Orientations time? (Defi nitions of poverty, for example, • Is there a theoretical aspect to the study, or do change frequently.) no references to theory appear? • If a panel study has been conducted, how • Can you identify the researcher’s chief para- many people dropped out over the course of digm or theoretical orientation? Authors the study? READING SOCIAL RESEARCH 475

Measurement atic sampling, or cluster sampling? Has the • What are the names of the concepts under researcher stratifi ed the sampling frame prior study? to sampling? Have the stratifi cation variables • Has the researcher delineated different dimen- been chosen wisely? That is, are they relevant sions of the variables? Do the analysis and to the variables under study? reporting maintain those distinctions? • How large a sample was selected? What per- • What indicators—either qualitative or quantita- centage of the sample responded? Are there tive—have been chosen as measures of those any likely differences between those who dimensions and concepts? Is each indica- responded and those who didn’t? tor a valid measure of what it’s intended to • Even assuming that the respondents are repre- measure? What else could the indicator be a sentative of those selected in the sample, what measure of? Is it a reliable measure? Has the sampling error do you expect from a sample of reliability been tested? this size? • What is the level of measurement of each vari- • Has the researcher tested for representative- able: nominal, ordinal, interval, or ratio? Is it ness: comparing the gender distribution of the the appropriate level? population and of respondents, for example, or • Have composite measurements (indexes, their ages, ethnicity, education, or income? scales, or typologies) been used? If so, are they • Ultimately, do the studied individuals (or other appropriate to the purpose of the study? Have units of analysis) represent the larger popula- they been constructed correctly? tion from which they were chosen? That is, do conclusions drawn about the sample tell Sampling us anything about meaningful populations or • Was it appropriate to study a sample, or should about life in general? all elements have been studied? Remember, • If probability sampling and statistical represen- it’s not always feasible to select a random tation were not appropriate to the study—in a sample. qualitative study, for example—have subjects • If sampling was called for, were probability and observations been selected in such a way sampling methods appropriate, or would a as to provide a broad overview of the phenom- purposive, snowball, or quota sample have enon being examined? Has the researcher paid been better? Has the appropriate sample de- special attention to deviant or disconfi rming sign been used? cases? • What population does the researcher want to draw conclusions about? Experiments • What is the researcher’s purpose? If it’s sta- • What is the primary dependent variable in the tistical description, then rigorous probability experiment? What effect is the experimenter sampling methods are called for. trying to achieve, for example? • If a probability sample has been selected, what • What is the experimental stimulus? sampling frame has been used? Does it appro- • What other variables are relevant to the ex- priately represent the population that interests periment? Have they been measured? the researcher? What elements of the popula- • How has each variable been defi ned and mea- tion have been omitted from the sampling sured? What potential problems of validity and frame, and what extraneous elements have reliability do these defi nitions and measure- been included? ments raise? • What specifi c sampling techniques have been • Has a proper control group been used? Have employed: simple random sampling, system- subjects been assigned to the experimental 476 CHAPTER 15 READING AND WRITING SOCIAL RESEARCH

and control groups through random selection respondents being asked to agree or disagree or by matching? Has it been done properly? with two ideas, when they might like to agree Has the researcher provided any evidence of with one and disagree with the other? the initial comparability of experimental and • Do the questions contain negative terms? If so, control-group subjects? respondents may have misunderstood them • Have there been pre- and posttest measure- and answered inappropriately. ments of the dependent variable? • Is there a danger of social desirability in any • What is the chance of a placebo (or “Haw- of the questions? Is any answer so right or so thorne”) effect in the experiment? Has any wrong that respondents may have answered attention been given to the problem? Does on the basis of what people would think of the study employ a double-blind design, for them? example? • How would you yourself answer each item? As • Are there any problems of internal invalidity: a general rule, test all questionnaire items by history, maturation, testing, instrumentation, asking yourself how you would answer. Any statistical regression, selection bias, experi- diffi culty you might have in answering might mental mortality, or demoralization? also apply to others. Then, try to assume dif- • Are there issues of external invalidity? How ferent points of view (for example, liberal and has the experimenter ensured that the labora- conservative, religious and unreligious) and tory fi ndings will apply to life in the real world? ask how the questions might sound to someone with each point of view. Survey Research • Has the researcher conducted a secondary • Does the study stand up to all the relevant analysis of previously collected data? If so, questions regarding sampling? determine the quality of the research that • What questions were asked of respondents? produced the data originally. Also, are the What was the precise wording of the ques- data available for analysis appropriate to the tions? Be wary of researcher reports that pro- current purposes? Do the questions originally vide only paraphrases of the questions. asked refl ect adequately on the variables now • If closed-ended questions were asked, were being analyzed? the answer categories provided appropriate, exhaustive, and mutually exclusive? Field Research • If open-ended questions were asked, how • What theoretical paradigm has informed the have the answers been categorized? Has the researcher’s approach to the study? researcher guarded against his or her own bias • Has the research set out to test hypotheses creeping in during the coding of open-ended or generate theory from the observations? responses? Or is there no concern for theory in the • Are all the questions clear and unambigu- study? ous? Could they have been misinterpreted by • What are the main variables in this study? How respondents? If so, could the answers given have they been defi ned and measured? Do you mean something other than what the re- see any problems of validity? searcher has assumed? • How about reliability? Would another re- • Were the respondents capable of answering searcher, observing the same events, classify the questions asked? If not, they may have things the same way? answered anyway, but their answers might not • Is there any chance that the classifi cation of mean anything. observations has been infl uenced by the way • Are any of the questions double-barreled? those classifi cations will affect the research Look for conjunctions (such as and, or ). Are fi ndings and/or the researcher’s hypotheses? READING SOCIAL RESEARCH 477

• If descriptive conclusions have been drawn— Analyzing Existing Statistics for example, “the group’s standards were quite • Who originally collected the data being reana- conservative”—what are the implicit standards lyzed? Were there any fl aws in the data-collec- being used? tion methods? What was the original purpose • How much can the study’s fi ndings be gen- of the data collection? Would that have af- eralized to a broader sector of society? What fected the data that was collected? claims has the researcher made in this regard? • What was the unit of analysis of the data? Is it What is the basis for such claims? appropriate to the current research question • If people have been interviewed, how were and the conclusions being drawn? Is there a they selected? Do they represent all appropri- danger of the ecological fallacy? ate types? • When were the data collected? Are they still • How much did the researcher participate in the appropriate to present concerns? events under study? How might that participa- • What are the variables being analyzed in the tion have affected the events themselves? present research? Were the defi nitions used by • Did the researcher reveal his or her identity as the original researchers appropriate to present a researcher? If so, what infl uence could that interests? revelation have had on the behavior of those being observed? Comparative and Historical Research • Does the research indicate any personal feel- • Is this a descriptive or an explanatory study? ings—positive or negative—about those being Does it involve cross-sectional comparisons or observed? If so, what effect might these feel- changes over time? ings have had on the observations that were • What is the unit of analysis in this study (for made and the conclusions that were drawn example, country, social movement)? from them? • What are the key variables under study? If it is • How has the researcher’s own cultural identity an explanatory analysis, what causal relation- or background affected the interpretation of ships are examined? what has been observed? • Does the study involve the use of other research techniques, such as existing statistics, Content Analysis content analysis, surveys, or fi eld research? • What are the key variables in the analysis? Use the guidelines elsewhere in this section to Are they appropriate to the research question assess those aspects of the study. being asked? • Is the range of data appropriate to the analysis: • What is the source and form of data being for example, the units being compared or the analyzed? Are they appropriate to the research number of observations made for the purpose questions being asked? of characterizing units? • Is the time frame of the data being analyzed • If historical or other documents are used as a appropriate to the research question? data source, who produced them and for what • What is the unit of analysis? purposes? What biases might be embedded in • If a quantitative analysis has been conducted, them? Diaries kept by members of the gentry, (1) has an appropriate sample been selected for example, will not refl ect the life of peasants from the data source and (2) have the appro- of the same time and country. priate statistical techniques been used? • If a qualitative analysis has been conducted, (1) Evaluation Research has an appropriate range of data been exam- • What is the social intervention being analyzed? ined and (2) are the researcher’s conclusions How has it been measured? Are there any logically consistent with the data presented? problems of validity or reliability? 478 CHAPTER 15 READING AND WRITING SOCIAL RESEARCH

• Have the appropriate people (or other units of • If tests of statistical signifi cance were used, analysis) been observed? have they been interpreted correctly? Has • How has “success” been defi ned? Where would statistical signifi cance been confused with the success be manifested—in individuals, in substantive signifi cance? organizations, in crime rates? Has it been measured appropriately? Reporting • Has the researcher judged the intervention • Has the researcher placed this particular a success or a failure? Is the judgment well project in the context of previous research on founded? the topic? Does this research add to, modify, • Who paid for the research, and who actu- replicate, or contradict previous studies? ally conducted it? Can you be confi dent of • In general, has the researcher reported the the researcher’s objectivity? Did the sponsor details of the study design and execution fully? interfere in any way? Are there parts of the report that seem particularly vague or incomplete in the reporting of Data Analysis details? • Did the purpose and design of the study call for • Has the researcher reported any fl aws or short- a qualitative or a quantitative analysis? comings in the study design or execution? Are • How have nonstandardized data been coded? there any suggestions for improving research This question applies to both qualitative and on the topic in the future? quantitative analysis. To what extent were the codes (1) based on prior theory or (2) gener- I hope this section will prove useful to you in ated by the data? reading and understanding social research. The • Has the researcher undertaken all relevant exercises at the end of this chapter will walk you analyses? Have all appropriate variables been through the reading of two journal articles: one identifi ed and examined? Could the correlation qualitative and one quantitative. As I said earlier, observed between two variables have been you’ll fi nd that your profi ciency in reading social caused by a third, antecedent variable, making research reports will mature with practice. the observed relationship spurious? Before discussing how to go about creating so- • Does a particular research fi nding really cial research reports for others to read, let’s look matter? Is an observed difference between at how to read and evaluate data from an increas- subgroups, for example, a large or meaningful ingly popular source of information—the Internet. one? Are there any implications for action? • Has the researcher gone beyond the ac- Using the Internet Wisely tual fi ndings in drawing conclusions and implications? In the closing decade of the twentieth century, • Are there logical fl aws in the analysis and the World Wide Web developed into a profoundly interpretation of data? valuable tool for social research. As it expands • Have the empirical observations of the study exponentially, the web is becoming the mind of revealed new patterns of relationships, provid- humanity, the repository of human knowledge, ing the bases for grounded theories of social opinions, and beliefs—carrying with it intellectual life? Has the researcher looked for discon- insights, misconceptions, and outright bigotry. fi rming cases that would challenge the new Clearly, it will continue to evolve as an ever more theories? powerful entity. As with gunpowder and television, • Are the statistical techniques used in the the power of the technology does not guarantee analysis of data appropriate to the levels of that it will always be used wisely. As I write this, measurement of the variables involved? a substantial number of faculty still prohibit their READING SOCIAL RESEARCH 479 students from using web materials. I have opted to • Statistical Resources on the Web, University of encourage use of the web rather than opposing it, Michigan but I am mindful of the problems that make many http://www.lib.umich.edu/govdocs/stats.html of my colleagues more cautious. • USA Statistics in Brief In this section of the chapter, I share websites http://www.census.gov/compendia/statab/ useful to social researchers and give some gen- brief.html eral advice on searching the web. Then I address • U.S. Bureau of the Census the major problems inherent in using the web and http://www.census.gov/ suggest ways to avoid them. • Yahoo Social Sciences http://dir.yahoo.com/Social_Science/

Some Useful Websites The website associated Now, let’s assume you need some information with this book has up-to-date links to useful social that you suspect is somewhere on the web, but you research websites. I’ve placed these materials on don’t know where. Here are some ideas about be- the web instead of in an appendix, so they can be coming a web detective. revised and updated before the next textbook revision. Nevertheless, I want to mention a few key Searching the Web I won’t estimate the number websites here and, more importantly, offer advice of pages of information on the World Wide Web; on how to search the web. its growth curve is so dramatic that any number I The fi rst website I’ll mention is the one cre- might give now would be embarrassingly low by ated to support this textbook and is mentioned at the time you read this. Let’s just say there are mil- the end of each chapter. You should consider it as lions and millions of pages. Similarly, estimating an extension of the book: http://sociology.wads the number of “facts” or pieces of data on the web worth.com/babbie_basics4e. In addition to tutor- would be impossible, but most of the factual ques- ing you on this book and coaching you in your re- tions you might have can be answered on the web. search methods course, the website also provides Finding them involves skill, however. a great many links that will take you to other use- Let’s say you want to know who was the thir- ful resources to aid you in both learning and doing teenth president of the United States. That’s easily social research. learned in several ways. The most straightforward Here are just a few generally useful websites would be to open one of the many search engines that you might like to check out: available to you; let’s say you use Google, found at http://www.google.com. When I searched for • Computer Assisted Qualitative Data Analysis “thirteenth president,” my responses began with Software, University of Surrey, England those shown in Figure 15-1. (Realize that if you http://caqdas.soc.surrey.ac.uk/ replicate this procedure, you may get somewhat • General Social Survey different responses, because the content of the http://webapp.icpsr.umich.edu/GSS/ web is continuously evolving.) • GSS Resource materials, Queen’s College The fi rst response in the list gives us the answer: http://www.soc.qc.edu/QC_Software/GSS Millard Fillmore. In this case, it’s not even neces- .html sary to follow up on any of the web links given, un- • QUALPAGE: Resources for Qualitative less we want to know something more about him. Research Notice that we have the same answer from three http://www.qualitativeresearch.uga .edu/QualPage/ • Social Sciences Virtual Library search engine A computer program designed to http://www.dialogical.net/socialsciences/ locate where specifi ed terms appear on websites through- index.html out the World Wide Web. 480 CHAPTER 15 READING AND WRITING SOCIAL RESEARCH

Millard Fillmore - Thirteenth President of the United States Biographical fast facts about Millard Fillmore, the thirteenth President of the United States. americanhistory.about.com/library/fastfacts/blffpresl3.htm - 28k - Cached - Similar pages Millard Fillmore - Thirteenth President of the United States Learn all about Millard Fillmore, the Thirteenth president of the United States. americanhistory.about.com/od/millardﬁ llmore/ - 16k - Cached - Similar pages Amazon.com: Millard Fillmore: Thirteenth President of the United ... Amazon.com: Millard Fillmore: Thirteenth President of the United States (Encyclopedia of Presidents): Books by Jane Clark Casey. www.amazon.com/exec/obidos/tg/detail/-/051601353X?v=glance - 53k - Cached - Similar pages President Ronald R. Thomas, Thirteenth President of the University ... The University of Puget Sound is honored to welcome Ronald R. Thomas as its thirteenth president, and only its ﬁ fth since 1913. ... www2.ups.edu/inauguration/inaugurationPresident.html - 11k - Cached - Similar pages Wilson, Woodrow Wilson, [Thomas] Woodrow (1856-1924), thirteenth president of Princeton, was born December 29, 1856, in Staunton, Virginia, the son of Joseph Ruggles Wilson ... etc. princeton.edu/CampusWWW/Companion/wilson_woodrow.html - 13k - Cached - Similar pages The New ASU Story: Leadership John W. Schwada was ASU’s thirteenth president, serving from 1971 to 1981. He presided over one of the largest growth periods in university history, ... www.asu.edu/lib/archives/asustory/pages/18lead.htm - 4k - Sep 14, 2005 - Cached - Similar pages Millard Fillmore, Thirteenth President Of The United States reference - author, title, language for ISBNO51601353X Millard Fillmore, Thirteenth President Of The United States. my.linkbaton.com/isbn/051601353X - 3k - Cached - Similar pages Thirteenth President of the United States Millard Fillmore - Books ... journal articles on: Thirteenth President of the United States Millard Fillmore ... Take Millard Fillmore, who gave his...money, our thirteenth president. ... www.questia.com/search/Thirteenth-President-of-the-United-States-Millard-Fillmore - 41k - Cached - Similar pages The Library of Congress Shop > Presidents, First Ladies > Millard ... Millard Fillmore, Thirteenth President of the United States. Millard Fillmore, Thirteenth President of the United States Click on image to enlarge ... www.loc.gov/shop/index.php?action=cCatalog.showItem&cid=33&scid=229&iid=1019 - 12k - Sep 14, 2005 - Cached - Similar pages

FIGURE 15-1 Finding the “Thirteenth President”

©2005 Google. Downloaded September 15, 2005, 12:25 P.M. different websites—each adding to our confi dence your request inside quotation marks, as I just did, that we have the right answer. Notice also that the the search engine will look for that exact phrase fourth and fi fth answers refl ect the ambiguity of our instead of reporting websites that happen to have request in not specifying president of “what.” all three words. Figure 15-2 presents the initial re- Here’s a more elaborate example. Let’s say you sults I received. want to examine differences in the infant mortal- The fourth web link is to the CIA’s World Fact- ity rates of countries around the world. You may book, a reference that draws on data from a variety already know some websites that are likely to have of sources. The third is from the United Nations; that information, but let’s assume you don’t. the others range from government or commercial Go back to Google or another search engine data sources to news articles. Realize that Figure and search for “infant mortality rate.” If you put 15-2 only presents the fi rst few websites returned Map & Graph: Countries by Health: Infant mortality rate Our infant mortality rate is driven by our high accidental death rate and ... But our relatively high infant mortality rate relative to per capita income is ... www.nationmaster.com/graph-T/hea_inf_mor_rat - 99k - Sep 13, 2005 Cached - Similar pages

GeographyIQ - World Atlas - Rankings - Infant mortality rate (All ... Worldwide Infant mortality rate (All Ascending) ranking information. www.geographyiq.com/ranking/ranking_Infant_Mortality_Rate_aall.htm - 94k - Cached - Similar pages

United Nations Statistics Division - Millennium Indicators Indicator. 14. Infant mortality rate (UNICEF-WHO) ... MDG, 1230, Infant mortality rate (0-1 year) per 1000 live births (UNICEF estimates) - View data ... millenniumindicators.un.org/unsd/mi/mi_indicator_xrxx.asp?ind_code=14 - 16k - Cached - Similar pages

CIA - The World Factbook — Rank Order- Infant mortality Infant mortality rate (deaths/1000 live births). Date of Information. 1. Angola, 187.49, 2005 est. 2. Afghanistan, 163.07, 2005 est. ... www.cia.gov/cia/publications/factbook/rankorder/2091rank.html - 92k - Cached - Similar pages

State Rankings—Statistical Abstract of the United States—Infant ... rankings of states for infant mortality rate.... INFANT MORTALITY RATE — 2001. [When states share the same rank, the next lower rank is omitted. ... www.census.gov/statab/ranks/rank17.html - 15k - Sep 14, 2005 - Cached - Similar pages

Human Development Reports Infant mortality rate (per 1000 live births) The probability of dying between birth and exactly one year of age, expressed per 1000 live births. ... www.undp.org/hdr2003/indicator/indic_289.html - 83k - Cached - Similar pages

Health, Cuba Reports Record Low Infant Mortality Rate: Cuba News ... Health, Cuba Reports Record Low Infant Mortality Rate: Cuba News, Cuba Travel, cultural, business news.. Cuba Travel eXPlorer. www.cubaxp.com/modules/news/article-447.html - 45k - Cached - Similar pages

Infant mortality rate - deaths per 1000 live births - Flags, Maps ... Infant mortality rate - deaths per 1000 live births - Flags, Maps, Economy, Geography, Climate, Natural Resources, Current Issues, International Agreements, ... www.photius.com/wfb1999/rankings/infant_mortality_0.html - 52k - Cached - Similar pages

FIGURE 15-2 Search for “Infant Mortality Rate”

CIA - The World Factbook — Rank Order - Infant mortality rate Top banner The World Factbook Banner ... Infant mortality rate (deaths/1000 live births). Date of Information. 1. Angola, 187.49, 2005 est. ... www.cia.gov/cia/publications/factbook/rankorder/2091rank.html - 92k - Cached - Similar pages

CIA - The World Factbook — United States Buoyed by victories in World Wars I and II and the end of the Cold War in 1991, ... Infant mortality rate:. Deﬁ nition - Field Listing - Rank Order ... www.cia.gov/cia/publications/factbook/geos/us.html - 101k - Sep 13, 2005 - Cached - Similar pages

Global Geografi a - World, Demographic statistics: Infant Mortality ... www.globalgeografi a.com - Website about geography. www.globalgeografi a.com/world/infant_mortality_rate.htm - 6k - Cached - Similar pages

GeographyIQ - World Atlas - Rankings - Infant mortality rate ... Worldwide Infant mortality rate (Bottom 25) ranking information. www.geographyiq.com/ranking/ranking_Infant_Mortality_Rate_bottom25.htm - 28k - Cached - Similar pages

Infant mortality - Wikipedia, the free encyclopedia World infant mortality rate declined from 198 in 1960 to 83 in 2001. However, IMR remained higher in LDCs. In 2001, the Infant Mortality Rate for Less ... en.wikipedia.org/wiki/Infant_mortality_rate - 20k - Cached - Similar pages

List of countries by infant mortality rate (2005) - Wikipedia, the ... This is a list of countries by infant mortality rate, based on The World Factbook, 2005 estimates.[1]. The infant mortality rate (IMR) is reported as number ... en.wikipedia.org/wiki/List_of_countries_by_infant_mortality_rate_(2005) - 35k - Cached - Similar pages

FIGURE 15-3 Search for “World ‘Infant Mortality Rate’”

by the Google search. Google reported that it had one attempt. In this case, I added the word world to found about 1,630,000 websites that seemed to the request: world “infant mortality rate.” have the information we were seeking. Like many other search engines, Google inter- Notice that several of the web links are probably prets this as a request to fi nd websites that con- more specifi c than we want—one deals only with tain the word world plus the exact phrase infant Cuba, another gives data only on the United States. mortality rate. Figure 15-3 presents the fi rst set of Often an effective web search requires more than results. READING SOCIAL RESEARCH 483

This time, the fi rst two web links are to the CIA’s 1. Who/what is the author of the website? The World Factbook. The sixth and seventh links are two biggest risks you face in getting informa- to Wikipedia, a free encyclopedia compiled by the tion from the web are (1) bias and (2) sloppi- web community. Although commercial websites ness. The democratic beauty of the web is its and almanacs can be useful sources of informa- accessibility to such a large proportion of the tion, you should, wherever possible, use data pre- population and the lack of censorship. These sented by those who collect and compile it. In this pluses also present dangers, in that just about case, you might want to search further for links to anyone can put just about anything on the the respected Population Reference Bureau or to web. The fi rst thing you should note, therefore, the United Nations sites we saw in Figure 15-2. is who the author of the website is: either an Conducting this search on your own and visit- organization or an individual. ing the web links that result is a useful exercise. 2. Is the site advocating a particular point of view? You’ll fi nd that some of the sites are discussions of Many of the sites on the World Wide Web have the topic rather than tables of data. Others present been created to support a particular political, a limited set of data (“selected countries”). Thus, religious, nationalistic, social, or other point of compiling a list of web links like this is a step along view. This fact does not necessarily mean that the way to obtaining relevant data, but it is not the the data they present are false, though that’s fi nal step. sometimes the case. Beyond outright lying, however, you can be relatively sure that the Evaluating the Quality of Internet Materials You website will only present data supporting its now know enough about web searches to begin particular point of view. You can usually tell learning through experience. You’ll quickly learn whether a website is reasonably objective or that fi nding data on the web is relatively easy. has an ax to grind, and you should be wary Evaluating what you’ve found is a bit more diffi - of those that go overboard to convince you of cult, however. I’ve already alluded to the matter of something. quality, but there’s much more to be said on the 3. Does the website give accurate and complete ref- topic. In fact, many other people have said many erences? When data are presented, can you tell other things about it. What do you suppose is your where they come from—how they were cre- best source of such advice? If you said, “The web,” ated? If the website is reporting data collected you got it. by someone else, are you given suffi cient Open up a search engine and ask it to fi nd web- guidance to locate the original researchers? sites having to do with “evaluating web sites.” (Us- Or, if the data were compiled by the website ing alternate spellings can yield more results; for authors, do they provide you with suffi ciently example, you could also enter “evaluating web- detailed descriptions of their research meth- sites” and get a similar yet different set of entries.) ods? If data are presented without such clarifi - Figure 15-4 gives you some idea of the extent of cations, you should move on. advice available to you. 4. Are the data up-to-date? Another common As you can tell from the “.edu” in the addresses problem on the web is that materials may be of most of these sites, this is a topic of concern for posted and forgotten. Hence, you may fi nd colleges and universities. Although each of the data reporting crime rates, chronicles of peace various sites approaches the topic differently, the negotiations, and so forth that are out-of-date. guidance they offer has some elements in common. Be sure that the data you obtain are timely for You would do well to study one or more of the sites your purposes. in depth. In the meantime, here’s an overview of 5. Are the data offi cial? It’s often a good idea to the most common questions and suggestions for fi nd data at offi cial government research sites, evaluating the data presented on websites. such as the Bureau of the Census (http:// 484 CHAPTER 15 READING AND WRITING SOCIAL RESEARCH

Evaluating Web Pages: Techniques to Apply & Questions to Ask Includes checklist form (PDF) that can be used to analyze web sites and pages www.lib.berkeley.edu/TeachingLib/Guides/Internet/Evaluate.html - 46k - Cached - Similar pages

Kathy Schrock’s Guide for Educators - Critical Evaluation Surveys ...... a great site which looks at the different types of pages; Evaluating Web Sites ...a rubric and ... Evaluating Web Sites: What Makes a Web Site Good? ... school.discovery.com/schrockguide/eval.html - 42k - Cached - Similar pages

Evaluating Web Sites The User Context: The most important factor when evaluating Web sites is your search, your needs. What are you using the Web for? Entertainment? ... www.library.cornell.edu/olinuris/ref/research/webeval.html - 11k - Sep 13, 2005 - Cached - Similar pages

Five criteria for evaluating Web pages Evaluation of Web documents, How to interpret the basics. 1. Accuracy of Web Documents. Who wrote the page and can you contact him or her? ... www.library.cornell.edu/okuref/research/webcrit.html - 7k - Cached - Similar pages

Evaluating Web Sites Lesley is a multi-site University with more than 150 locations throughout the continental United States. www.lesley.edu/library/guides/research/evaluating_web.html - 25k - Sep 13, 2005 - Cached - Similar pages

Evaluation Criteria from “The Good, The Bad & The Ugly: or, Why ... A easy to use guide for web evaluation. Lists evaluation criteria with links to actual pages that illustrate each point. The Examples page can be used by ... lib.nmsu.edu/instruction/evalcrit.html - 10k - Cached - Similar pages

Evaluating Web Sites for Educational Uses This site contains a list of articles from librarians and other information specialists on Web evaluations. In addition, a checklist for evaluating a Web ... www.unc.edu/cit/guides/irg-49.html - 14k - Cached - Similar pages

Evaluating Web Sites for Accessibility Goals for evaluating Web sites vary, and require different approaches to meet those goals:. Preliminary review can:. identify general kinds of barriers on a ... www.w3.org/WAI/eval/Overview.html - 31k - Cached - Similar pages

FIGURE 15-4 Search for “Evaluating Web Sites”

www.census.gov/), the Bureau of Labor launching point for ﬁ nding data among some Statistics (http://www.bls.gov/home 100 federal research agencies. As we saw in .htm), the National Center for Health Statis- Chapter 11, data presented by ofﬁ cial agencies tics (http://www.cdc.gov/nchs/), and others. are not necessarily “The Truth,” but they are FedStats (http://www.fedstats.gov/) is a good grounded in a commitment to objectivity and READING SOCIAL RESEARCH 485

have checks and balances to support them in Your instructor may prefer a specifi c format for achieving that goal. web citations. However, here are the elements 6. Is it a university research site? Like government commonly suggested for inclusion: research agencies, university research centers and institutes are usually safe resources, com- • The URL or web address. For example, mitted to conducting professional research http://www.fedstats.gov/qf/states/50000 and having checks and balances (such as .html provides demographic data for compar- peer review) to support their achieving that. ing Vermont with the United States as a whole. Throughout this book, I’ve mentioned the So if I tell you that Vermont grew 8.2 percent General Social Survey (http://webapp.icpsr during the 1990s, you can go directly to the .umich.edu/GSS/), conducted regularly by source of my data. the National Opinion Research Center at the • The date and time when the site was accessed. University of Chicago. You could use data Many, like the one just cited, do not change, presented here with confi dence: confi dence in but many others do. It may be useful for the the legitimacy of the data and confi dence that reader to know when you visited the site in your instructor will not question your use of question. Some editing guides say to include that resource. this, whereas others say not to. When in 7. Do the data seem consistent with data from other doubt, you would check with your instructor or sites? Verify (cross-check) data wherever pos- publisher. It’s usually better to have too much sible. We’ve already seen that a web search is information than too little. likely to turn up more than one possible source • If you’re citing text materials, there may very of data. Take the time to compare what they well be an author and title, as well as publish- present. If several websites present essen- ing information. These should be cited the tially the same data, you can use any of those same way you would cite printed materials: sources with confi dence. for example, John Doe. 2003. “How I Learned to Love the Web.” Journal of Web Worship 5 (3): As with so many things, your effective use of 22–45. the web will improve with practice. Moreover, the • Sometimes, you’ll use the web to read a pub- web itself will be evolving alongside your use of it. lished journal article, locating it with InfoTrac College Edition or another vehicle. Such Citing Internet Materials If you use materials materials may be presented in a print format, from the web, you must provide a bibliographic ci- with page numbers. If so, cite the appropriate tation that allows your reader to locate the original page number. Lacking that, you may be able materials—to see them in context. This also pro- to cite the section where the materials in ques- tects you from the serious problem of plagiarism, tion appeared. The goal in all this is to help discussed a little later in this chapter. your reader locate the original web materials There are many standardized formats for bib- you’re using. Although you sometimes cannot liographic citations, such as those established by give a precise location in an article posted to a the Modern Language Association, the American website, most browsers allow users to search Psychological Association, and the American Soci- the site for a specifi ed word or phrase and thus ological Association. Web materials, unfortunately, locate the materials being cited. don’t fi t any of those familiar formats. Fortunately, each of these organizations—and many, many others—have risen to the challenge of web citations. If you don’t believe me, go to your URL Web address, typically beginning with “http://”; favorite search engine and look for “web citations.” stands for “uniform resource locator” or “universal You’ll fi nd plenty of guidance. resource locator.” 486 CHAPTER 15 READING AND WRITING SOCIAL RESEARCH

WRITING SOCIAL RESEARCH IN THE REAL WORLD COMMUNICATION IS THE KEY Unless research is properly communicated, all the efforts devoted to the various procedures discussed No matter what you do with your life— throughout this book will go for naught. This whether in social research or some other means, fi rst and foremost, that good social report- worthy pursuit—you’re likely to fi nd yourself ing requires good English or Spanish or whatever regularly using the skills discussed in this language you use. Whenever we ask the fi gures “to chapter. When colleges and universities ask speak for themselves,” they tend to remain mute. employers for suggestions on how we can Whenever we use unduly complex terminology or better prepare graduates, the most common construction, communication suffers. response, regardless of professional fi eld, My fi rst advice to you is to read and reread (at tends to be the same: Teach them to write. approximately three-month intervals) an excel- Whatever career you choose, you’ll ben- lent small book by William Strunk, Jr., and E. B. efi t greatly from the ability to read a body of White, The Elements of Style (1999; see also Birch- literature or a set of data and write coher- fi eld 1998). If you do this faithfully, and if even 10 ently about it. Moreover, if you’re typical of percent of the contents rub off, you stand a good recent college cohorts, you’re likely to have chance of making yourself understood and your several different careers. The ability to read fi ndings appreciated. and write effectively will serve you well in all Next, you need to understand that scientifi c of them. reporting has several functions. First, your report should communicate a body of specifi c data and ideas. You should provide those specifi cs clearly and with suffi cient detail to permit an informed readers. If the report is written for the former, you evaluation by others. Second, you should view your can make certain assumptions about their exist- report as a contribution to the general body of sci- ing knowledge and therefore summarize certain entifi c knowledge. While remaining appropriately points rather than explain them in detail. Similarly, humble, you should always regard your research you can use more technical language than would report as an addition to what we know about so- be appropriate for a general audience cial behavior. Finally, the report should stimulate At the same time, remain aware that any science and direct further inquiry. See the box “Commu- has its factions and cults. Terms, assumptions, and nication Is the Key” for more on the importance of special techniques familiar to your immediate col- knowing how to read and write well. leagues might only confuse other scientists. The sociologist of religion writing for a general sociol- Some Basic Considerations ogy audience, for example, should explain previous fi ndings in more detail than he or she would if Despite these general guidelines, different reports addressing an audience of sociologists of religion. serve different purposes. A report appropriate for one purpose might be wholly inappropriate for Form and Length of Report My comments here another. This section deals with some of contexts apply to both written and oral reports. Each form, that affect choices in writing. however, affects the nature of the report. It’s useful to think about the variety of reports Audience Before drafting your report, ask your- that might result from a research project. To begin, self who you hope will read it. Normally you should you may wish to prepare a short research note for make a distinction between scientists and general publication in an academic or technical journal. WRITING SOCIAL RESEARCH 487

Such reports are approximately one to fi ve pages Probably the most popular research report is long (typed, double-spaced) and should be concise the article published in an academic journal. Again, and direct. In a small amount of space, you can’t lengths vary, and you should examine the lengths present the state of the fi eld in any detail, so your of articles previously published by the journal in methodological notes must be abbreviated. Basi- question. As a rough guide, however, 25 typed cally, you should tell the reader why you feel your pages is a good length. A subsequent section on fi ndings justify a brief note, then tell what those the organization of the report is based primarily fi ndings are. on the structure of a journal article, so I’ll say no Often researchers must prepare reports for the more at this point except to indicate that student sponsors of their research. These reports can vary term papers should follow this model. As a general greatly in length. In preparing such a report, you rule, a term paper that would make a good journal should bear in mind your audience—scientifi c or article also makes a good term paper. lay—and their reasons for sponsoring the proj- A book, of course, represents the most pres- ect in the fi rst place. It’s both bad politics and bad tigious form of research report. It has the length manners to bore the sponsors with research fi nd- and detail of a working paper but is more polished. ings that have no interest or value to them. At the Because publishing research fi ndings as a book same time, it may be useful to summarize how the lends them greater substance and worth, you have research has advanced basic scientifi c knowledge a special obligation to your audience. Although (if it has). some colleagues may provide comments, possibly Working papers are another form of research re- leading you to revise your ideas, other readers may porting. In a large and complex project especially, be led to accept your fi ndings uncritically. you’ll fi nd comments on your analysis and the interpretation of your data useful. A working paper Aim of the Report Earlier in this book, we con- constitutes a tentative presentation with an im- sidered the different purposes of social research plicit request for comments. Working papers can projects. In preparing your report, keep these dif- also vary in length, and they may present all of the ferent purposes in mind. research fi ndings of the project or only a portion Some reports focus primarily on the explora- of them. Because your professional reputation is tion of a topic. As such, their conclusions are ten- not at stake in a working paper, feel free to present tative and incomplete. If you’re writing this sort of tentative interpretations that you can’t altogether report, clearly indicate to your audience the explor- justify—identifying them as such and asking for atory aim of the study and present the shortcom- evaluations. ings of the particular project. An exploratory report Many research projects result in papers deliv- points the way to more-refi ned research on the ered at professional meetings. Often, these serve topic. the same purpose as working papers. You can Most research reports have a descriptive ele- present fi ndings and ideas of possible interest to ment refl ecting the descriptive purpose of the stud- your colleagues and ask for their comments. Al- ies they document. In yours, carefully distinguish though the length of such professional papers var- those descriptions that apply only to the sample ies, depending on the organization of the meet- and those that apply to the population. Give your ings, it’s best to say too little rather than too much. audience some indication of the probable range of Although a working paper may ramble somewhat error in any inferential descriptions you make. through tentative conclusions, conference partici- Many reports have an explanatory aim: point- pants should not be forced to sit through an oral ing to causal relationships among variables. De- unveiling of the same. Interested listeners can al- pending on your probable audience, carefully de- ways ask for more details later, and uninterested lineate the rules of explanation that lie behind your ones can gratefully escape. computations and conclusions. Also, as in the case 488 CHAPTER 15 READING AND WRITING SOCIAL RESEARCH of description, give your readers some guide to the tent that this form of reporting gives an accurate relative certainty of your conclusions. picture of the research process, it has considerable If your intention is to test a hypothesis based in instructional value. Nevertheless, many readers theory, you should make that hypothesis clear and may not be interested in following your entire re- succinct. Specify what will constitute acceptance search account, and not knowing the purpose and or rejection of the hypothesis and how either of general conclusions in advance may make it dif- those refl ects on the theoretical underpinnings. fi cult for them to understand the signifi cance of the Finally, some research reports propose action. study. For example, if you’ve studied prejudice, you may An old forensic dictum says, “Tell them what suggest in your report how prejudice can be re- you’re going to tell them; tell them; and tell them duced on the basis of your research fi ndings. This what you told them.” You would do well to follow suggestion may become a knotty problem for you, this dictum. however, because your values and orientations may have interfered with your proposals. Although Review of the Literature Next, you must indi- it’s perfectly legitimate for such proposals to be cate where your report fi ts into the general body of motivated by personal values, you must ensure scientifi c knowledge. After presenting the general that the data actually warrant the specifi c actions purpose of your study, you should bring the reader you’ve proposed. Thus, you should be especially up-to-date on the previous research in the area, careful to spell out the logic by which you move pointing to general agreements and disagreements from empirical data to proposed action. among the previous researchers. Your review of the literature should lay the groundwork for your own study, showing why your research may have Organization of the Report value in the larger scheme of things. In some cases, you may wish to challenge pre- Although the various forms and purposes of re- viously accepted ideas. Carefully review the stud- ports somewhat affect the way they are organized, ies that have led to the acceptance of those ideas, knowing a general format for presenting research then indicate the factors that have not been previ- data can be helpful. The following comments ap- ously considered or the logical fallacies present in ply most directly to a journal article, but with some the previous research. modifi cation they apply to most forms of research When you’re concerned with resolving a dis- reports as well. agreement among previous researchers, you should summarize the research supporting one Purpose and Overview It’s always helpful if you view, then summarize the research supporting begin with a brief statement of the purpose of the the other, and fi nally suggest the reasons for the study and the main fi ndings of the analysis. In a disagreement. journal article, as we’ve seen, this overview some- Your review of the literature serves a biblio- times takes the form of an abstract. graphic function for readers by indexing the previ- Some researchers fi nd this diffi cult to do. For ous research on a given topic. This can be over- example, your analysis may have involved consid- done, however, and you should avoid an opening erable detective work, with important fi ndings re- paragraph that runs three pages, mentioning every vealing themselves only as a result of imaginative previous study in the fi eld. The comprehensive bib- deduction and data manipulation. You may wish, liographic function can best be served by a bibli- therefore, to lead the reader through the same ex- ography at the end of the report, and the review citing process, chronicling the discovery process of the literature should focus only on those studies with a degree of suspense and surprise. To the ex- that have direct relevance to the present one. WRITING SOCIAL RESEARCH 489

Avoiding Plagiarism Whenever you’re reporting for initiating the serious study of this important on the work of others, you must be clear about who topic. But what Parkinson failed to perceive, we said what. That is, you must avoid plagiarism: the now enunciate—the general systems analogue theft of another’s words and/or ideas—whether of Parkinson’s Law. intentional or accidental—and the presentation of The System Itself Tends to Grow those words and ideas as your own. Because this At 5 To 6 Percent Per Annum is a common and sometimes unclear problem for Again, this Law is but the preliminary to the college students, especially in regard to the review most general possible formulation, the Big-Bang of the literature, we’ll consider the issue here. Re- Theorem of Systems Cosmology. alize, of course, that these concerns apply to everything you write. Systems Tend To Expand To Fill Here are the ground rules regarding plagiarism: The Known Universe— (GALL 1975:12–14)

• You cannot use another writer’s exact words Now let’s look at some of the acceptable ways you without using quotation marks and giving a might make use of Gall’s work in a term paper. complete citation, which indicates the source Acceptable: of the quotation such that your reader could • John Gall, in his work System- locate the quotation in its original context. As antics, draws a humorous parallel between a general rule, taking a passage of eight or systems and infants: “Systems are like babies: more words without citation is a violation of once you get one, you have it. They don’t go federal copyright laws. away. On the contrary, they display the most • It’s also not acceptable to edit or paraphrase remarkable persistence. They not only persist; another’s words and present the revised ver- they grow.”* Acceptable: sion as your own work. • John Gall warns that systems • Finally, it’s not even acceptable to present are like babies. Create a system and it sticks another’s ideas as your own—even if you around. Worse yet, Gall notes, systems keep use totally different words to express those growing larger and larger.** Acceptable: ideas. • It has also been suggested that systems have a natural tendency to persist, even grow and encroach — (Gall 1975:12). The following examples should clarify what is or is not acceptable in the use of another’s work. Note that the last format requires that you give a complete citation in your bibliography, as I do in The Original Work this book. Complete footnotes or endnotes work Laws of Growth as well. See the publication manuals of various or- Systems are like babies: once you get ganizations such as the APA or the ASA, as well as one, you have it. They don’t go away. On the the Chicago Manual of Style, for appropriate cita- contrary, they display the most remarkable tion formats. persistence. They not only persist; they grow. And as they grow, they encroach. The growth plagiarism Presenting someone else’s words or potential of systems was explored in a tentative, thoughts as though they were your own, constituting preliminary way by Parkinson, who concluded intellectual theft. that administrative systems maintain an average

growth of 5 to 6 percent per annum regardless *John Gall, Systemantics: How Systems Work and Especially of the work to be done. Parkinson was right so How They Fail (New York: Quadrangle, 1975), 12. far as he goes, and we must give him full honors **Ibid. 490 CHAPTER 15 READING AND WRITING SOCIAL RESEARCH

Here now are some unacceptable uses of the Study Design and Execution A research report same material, refl ecting some common errors. containing interesting fi ndings and conclusions will frustrate readers if they can’t determine the • Unacceptable: In this paper, I want to look methodological design and execution of the study. at some of the characteristics of the social The worth of all scientifi c fi ndings depends heavily systems we create in our organizations. First, on the manner in which the data were collected systems are like babies: once you get one, you and analyzed. have it. They don’t go away. On the contrary, In reporting the design and execution of a they display the most remarkable persistence. survey, for example, always include the follow- They not only persist; they grow. [It’s unaccept- ing: the population, the sampling frame, the sam- able to quote someone else’s materials directly pling method, the sample size, the data-collection without using quotation marks and giving a method, the completion rate, and the methods of full citation.] data processing and analysis. Comparable details • Unacceptable: In this paper, I want to look should be given if other methods are used. The ex- at some of the characteristics of the social perienced researcher can report these details in a systems we create in our organizations. First, rather short space, without omitting anything re- systems are a lot like children: once you get quired for the reader’s evaluation of the study. one, it’s yours. They don’t go away; they persist. They not only persist, in fact: they grow. [It’s unacceptable to edit another’s work and Analysis and Interpretation Having set the study present it as your own.] in the perspective of previous research and having • Unacceptable: In this paper, I want to look at described the design and execution of it, you should some of the characteristics of the social sys- then present your data. This chapter momentarily tems we create in our organizations. One thing will provide further guidelines in this regard. For I’ve noticed is that once you create a system, now, a few general comments are in order. it never seems to go away. Just the opposite, The presentation of data, the manipulation of in fact: systems have a tendency to grow. You those data, and your interpretations should be inte- might say systems are a lot like children in that grated into a logical whole. It frustrates the reader respect. [It’s unacceptable to paraphrase some- to discover a collection of seemingly unrelated one else’s ideas and present them as analyses and fi ndings with a promise that all the your own.] loose ends will be tied together later in the report. Every step in the analysis should make sense at the Each of the preceding unacceptable examples is time it is taken. You should present your rationale an example of plagiarism and represents a serious for a particular analysis, present the data relevant offense. Admittedly, there are some “gray areas.” to it, interpret the results, and then indicate where Some ideas are more or less in the public domain, that result leads next. not “belonging” to any one person. Or you may reach an idea on your own that someone else has already put in writing. If you have a question about Summary and Conclusions According to the fo- a specifi c situation, discuss it with your instructor rensic dictum mentioned earlier, summarizing the in advance. research report is essential. Avoid reviewing every I’ve discussed this topic in some detail because, specifi c fi nding, but review all the signifi cant ones, although you must place your research in the con- pointing once more to their general signifi cance. text of what others have done and said, the im- The report should conclude with a statement of proper use of their materials is a serious offense. what you have discovered about your subject mat- Learning to avoid plagiarism is a part of your “com- ter and where future research might be directed. ing of age” as a scholar. Many journal articles end with the statement, “It WRITING SOCIAL RESEARCH 491 is clear that much more research is needed.” This whether it involves participant observation among conclusion is probably always true, but it has lit- heavy-metal groupies, an experiment regarding tle value unless you can offer pertinent sugges- jury deliberation, or any other study format. Recall tions about the nature of that future research. You that replicability is an essential norm of science. A should review the particular shortcomings of your single study does not prove a point; only a series of own study and suggest ways those shortcomings studies can begin to do so. And unless studies can might be avoided. be replicated, there can be no meaningful series of studies. Guidelines for Reporting Analyses Integrate supporting materials. I have previously mentioned the importance of integrating data The presentation of data analyses should provide and interpretations in the report. Here is a more a maximum of detail without being cluttered. You specifi c guideline for doing this. Tables, charts, can accomplish this best by continually examining and fi gures, if any, should be integrated into the your report to see whether it achieves the follow- text of the report—appearing near that portion of ing aims. the text discussing them. Sometimes students de- If you’re using quantitative data, present them scribe their analyses in the body of the report and so the reader can recompute them. In the case of place all the tables in an appendix. This procedure percentage tables, for example, the reader should greatly impedes the reader, however. As a general be able to collapse categories and recompute the rule, it is best to (1) describe the purpose for pre- percentages. Readers should receive suffi cient in- senting the table, (2) present it, and (3) review and formation to permit them to compute percentages interpret it. in the table in the direction opposite from that of Draw explicit conclusions. Although research your own presentation. is typically conducted for the purpose of drawing Describe all aspects of a quantitative analysis general conclusions, you should carefully note the in suffi cient detail to permit a secondary analyst specifi c basis for such conclusions. Otherwise you to replicate the analysis from the same body of may lead your reader into accepting unwarranted data. This means that he or she should be able to conclusions. create the same indexes and scales, produce the Point to any qualifi cations or conditions war- same tables, arrive at the same regression equa- ranted in the evaluation of conclusions. Typically, tions, obtain the same factors and factor loadings, you know best the shortcomings and tentative- and so forth. This will seldom be done, of course, ness of your conclusions, and you should give the but if the report allows for it, the reader will be far reader the advantage of that knowledge. Failure to better equipped to evaluate the report than if it do so can misdirect future research and result in a does not. waste of research funds. Provide details. If you’re doing a qualitative As I said at the outset of this discussion, reanalysis, you must provide enough detail that your search reports should be written in the best pos- reader has a sense of having made the observa- sible literary style. Writing lucidly is easier for some tions with you. Presenting only those data that people than for others, and it’s always harder than support your interpretations is not suffi cient; you writing poorly. You are again referred to the Strunk must also share those data that confl ict with the and White book. Every researcher would do well way you’ve made sense of things. Ultimately, you to follow this procedure: Write. Read Strunk and should provide enough information that the reader White. Revise. Reread Strunk and White. Revise might reach a different conclusion than you did— again. This will be a diffi cult and time-consuming though you can hope your interpretation will make endeavor, but so is science. the most sense. The reader, in fact, should be in A perfectly designed, carefully executed, and position to replicate the entire study independently, brilliantly analyzed study will be altogether worth- 492 CHAPTER 15 READING AND WRITING SOCIAL RESEARCH less unless you can communicate your fi ndings WHAT DO YOU THINK? REVISITED to others. This chapter has attempted to provide some guidelines toward that end. The best guides are logic, clarity, and honesty. Ultimately, there is There is a vast amount of information avail- no substitute for practice. able on the Internet, but it is not all equally trustworthy and usable in scholarly research. Going Public The chapter suggested guidelines for sort- ing the wheat from the chaff. For example, Though I have written this chapter with a particu- data provided on government websites or lar concern for the research projects you may be on those of university research centers and called on to undertake in your research methods institutes, although not perfect, are usually course, you should realize that graduate and even dependable. Clarity of how data were col- undergraduate students are increasingly present- lected is a good sign; so are clear citations ing the results of their research as professional pa- to any other sources used. Be wary of web- pers or published articles. sites that push a particular point of view or If you would like to explore these possibilities agenda. Their data may be valid and useful, further, you may fi nd state and regional associa- but caution is in order. Never trust websites tions to be more open to students than are na- that are ambiguous about the methods used tional associations, although students may present or about the exact meanings of variables re- papers to the American Sociological Association, ported on. Finally, look for agreement across for example. Some associations have special ses- several websites, if possible. sions and programs for student participants. You In some cases, you may fi nd SourceWatch can learn more about these possibilities by visit- (http://www.sourcewatch.org/) a useful tool ing the associations’ websites to learn of upcom- to help you judge the trustworthiness of web ing meetings and the topics for which papers are sources. Sometimes, you will fi nd that a “re- being solicited. search team” is actually a public relations Typically, you’ll submit your paper to someone fi rm or that an individual “expert” always who has agreed to organize a session with three seems to report fi ndings in support of a par- to fi ve papers on a particular topic. The organizer ticular company or industry. chooses which of the submissions will be accepted for presentation. Oral presentations at scholarly meetings are typically 15–20 minutes long, with the possibility of questions from the audience. nal are circulated among three or so anonymous Some presenters read a printed paper, whereas reviewers, who make comments and recommen- others speak from notes. Increasingly, presenters dations to the journal’s editor. This is referred to as use computer slide shows, though such presenta- the “peer review” process. Sometimes manuscripts tions are still in the minority. are accepted pretty much as submitted, some are To publish an article in a scholarly journal, you returned for revision and resubmission, and still would do well to identify a journal that publishes others are rejected. The whole process from sub- articles on the topic of your research. Again, the mission to a decision to publish or reject may take journals published by state or regional associations a few months, and there will be a further delay be- may be the most accessible to student authors. fore the article is actually published. Each journal will contain instructions for submit- To meet the costs of publication, a journal will ting articles, including instructions for formatting sometimes require that authors pay a small fee on your article. Typically, articles submitted to a jour- acceptance. Typically, authors receive extra copies MAIN POINTS 493 of their article—called “reprints”—to give to friends Because this chapter concludes the main body and family and to satisfy requests from profes- of the book, I hope this fi nal section makes clear sional colleagues. that research ethics constitute not merely a nice thing to consider as long as it doesn’t get in the way, but a fundamental component of social sci- THE ETHICS OF READING ence. Research ethics has not always been recog- AND WRITING SOCIAL RESEARCH nize in this fashion. When I fi rst began writing this textbook, there was some objection to including this topic. It wasn’t so much that researchers ob- I’ve already commented on some ethical issues jected to the ethical treatment of subjects—ethics involved in writing research reports. However, simply wasn’t considered a proper topic for a book there are also some ethical issues at play in terms like this one. Attitudes have changed substantially of reading the research literature. There has al- over the years, however. I hope you benefi t from ways been the risk of reviewing the literature with understanding the crucial role of ethics in your a special eye toward reports that support a point work as well as in your life. of view you may be fond of. Although wonderful This chapter, and indeed this book, has pro- in most respects, the power of the Internet to provided what I hope will be a springboard for you vide fast and expansive searches can allow more to engage in and enjoy the practice of social re- “cherry picking” of supportive research literature. search. The next time you fi nd yourself pondering This places an ever greater burden on research- the cause of prejudice, or observing a political rally, ers to exercise professional honesty in represent- or just plain curious about the latest trends in tele- ing the history of research fi ndings in a particu- vision, I trust you’ll have the tools to explore your lar area. world with a social scientifi c eye.

Main Points ❏ Readers of social science literature should form questions as they go along and take notes. Introduction ❏ The key elements to note in reading a research ❏ Meaningful scientiﬁ c research is inextricably report include theoretical orientation, research wed to communication; knowing how to read design, measurement methods, sampling (if and write it requires practice. any), and other considerations speciﬁ c to the several data-collection methods discussed in Reading Social Research this book. ❏ Social researchers can access many resources, ❏ including the library and the Internet, for orga- The Internet is a powerful tool for social re- nizing a review of the literature. searchers, but it also carries risks. ❏ ❏ Reading scholarly literature is different from Not everything you read on the web is neces- reading other works, such as novels. sarily true. ❏ ❏ In reading scholarly literature, one should Original sources of data are preferred over begin by reading the abstract, skimming the those that take data from elsewhere. piece, and reading the conclusion to get a ❏ In evaluating a web source, one should ask the good sense of what it is about. following: 494 CHAPTER 15 READING AND WRITING SOCIAL RESEARCH

Who/what is the author of the website? The Ethics of Reading Is the site advocating a particular point of and Writing Social Research ❏ view? A review of the literature should not be biased toward a particular point of view. Does the site give accurate and complete ❏ references? Research ethics is a fundamental component of social science, not a nice afterthought. Are the data up-to-date? ❏ Ofﬁ cial data are usually a good source, although they are subject to error. Key Terms

❏ The reader of a report should verify (cross- abstract search engine check) data wherever possible. plagiarism URL ❏ Web citations, like other bibliographic refer- research monograph ences, should be complete—allowing the reader to locate and review the materials cited. Review Questions and Exercises

1. Analyze a quantitative research report: Writing Social Research Stanley Lieberson, Susan Dumais, Shyon ❏ Good social research writing begins with Baumann, “The Instability of Androgynous good writing, which means, among other Names: The Symbolic Maintenance of Gender things, writing to communicate rather than to Boundaries,” American Journal of Sociology impress. 105 (5, March 2000): 1249 (can be accessed ❏ Being mindful of one’s audience and one’s in print or online through InfoTrac College purpose in writing the report is important. Edition, for example). Use the following ques- ❏ Avoiding plagiarism—that is, presenting some- tions as your guide: one else’s words or thoughts as though they a. What are the theoretical underpinnings of were one’s own—is essential. Whenever using the study? someone else’s exact words, writers must be b. How are some of the key variables such as sure to use quotation marks or some other androgynous, racial, and gender segregation indication that they are quoting. In paraphras- conceptualized and operationalized? ing someone else’s words or ideas, writers must provide a full bibliographic citation of the c. What data is this research based on? source. d. Are there controlling variables? ❏ The research report should include an account e. What is the unit of analysis? of the study design and execution. f. What type of analysis was done? ❏ The analysis of a report should be clear at each g. What did the authors ﬁ nd? step, and its conclusion should be speciﬁ c but h. What are the strengths and weaknesses in not overly detailed. this study? ❏ To write good reports, researchers need to 2. Analyze a qualitative research report: Dingxin provide details, integrate supporting materials, Zhao, “State-Society Relations and the and draw explicit conclusions. Discourses and Activities of the 1989 Bei- ❏ Increasingly, students are presenting papers at jing Student Movement,” American Journal professional meetings and publishing articles of Sociology 105 (6, May 2000): 1592 (can be in scholarly journals. accessed in print or online through InfoTrac ADDITIONAL READINGS 495

College Edition, for example). Use the follow- Website for ing questions as your guide: The Basics of Social Research a. What is the author’s main research ques- 4th edition tion? At the book companion website (http://sociology b. What theoretical frameworks does he refer .wadsworth.com/babbie_basics4e) you will fi nd to, and which ones did he use? many resources in addition to ThomsonNow to c. What methodology is the author us- aid you in studying for your exams. For example, ing? What type of data collection did he you will fi nd Tutorial Quizzes with feedback, choose? What is the unit of analysis? Internet Exercises, Flashcards, and Chapter Tutori- d. Does the author have an hypothesis? If so, als, as well as Extended Projects, InfoTrac College what is it? Edition search terms, Social Research in Cyber- e. How does the author conceptualize key space, GSS Data, Web Links, and primers for terms such as state, state-society, tradition- using various data analysis software such as alism? What new ideal types of states does SPSS and NVivo. he bring to the fi eld? f. What are his fi ndings? Additional Readings g. What is the signifi cance of this study? Were you convinced by the author, or do you see Alexander, Jan, and Marsha Ann Tate. 1999. Web Wis- weaknesses in the study? dom. Mahwah, NJ: Erlbaum. A guide to the evaluation of web materials. Online Study Resources Birchfi eld, R. W. 1998. The New Fowler’s Modern English Usage. 3rd ed. New York: Oxford University Press. H. W. Fowler’s concise and witty Modern English Usage has been the chief resource and fi nal word on “proper” English since it was fi rst published in 1926. The third edition ensures that the advice is Go to “modern.” http://sociology.wadsworth.com/babbie_basics4e Strunk, William, Jr., and E. B. White. 1999. The Elements of Style. 4th ed. New York: Macmillan. This marvelous and click on ThomsonNow for access to this little book provides specifi c guidance as to grammar powerful online study tool. You will get a per- and spelling, but it’s primary power is its ability to sonalized study plan based on your responses to inspire good writing. a diagnostic pretest. Once you have mastered Walker, Janice R., and Todd Taylor. 1998. The Columbia the material with the help of interactive learning Guide to Online Style. New York: Columbia University Press. A guide to citing web materials in a scholarly tools, you can take a posttest to confi rm that you report. have successfully completed this chapter. This page intentionally left blank APPENDIXES

A Using the Library

B Random Numbers

C Distribution of Chi Square

D Normal Curve Areas

E Estimated Sampling Error

497 APPENDIX A

Using the Library

INTRODUCTION umes exist to offer a guide to the information that’s available.

We live in a world filled with social science re- Books in Print This volume lists all the books search reports. Our daily newspapers, magazines, currently in print in the United States—listed sepa- professional journals, alumni bulletins, club news- rately by author and by title. Out-of-print books can letters—virtually everything we pick up to read— often be found in older editions of Books in Print. may carry reports dealing with a particular topic. For formal explorations of a topic, of course, the Readers’ Guide to Periodical Literature This an- best place to start is still a good college or univer- nual volume with monthly updates lists articles sity library. Today, there are two major approaches published in many journals and magazines. Be- to finding library materials: the traditional paper cause the entries are organized by subject mat- system and the electronic route. Because I don’t ter, this is an excellent source for organizing your know what will be available to you, we’ll begin reading on a particular topic. Figure A-1 presents a with the traditional method and then examine the sample page from the Readers’ Guide. electronic option. In addition to these general reference volumes, you’ll find a great variety of specialized references. GETTING HELP Here are just a few:

• Sociological Abstracts When you want to find something in the library, • Psychological Abstracts your best friend is the reference librarian, who is • Social Science Index specially trained to find things in the library. Some • Social Science Citation Index libraries have specialized reference librarians—for • Popular Guide to Government Publications the social sciences, humanities, government docu- • New York Times Index ments, and so forth. Find the librarian who special- • Facts on File izes in your field. Make an appointment. Tell the • Editorial Research Reports librarian what you’re interested in. He or she will • Business Periodicals Index probably put you in touch with some of the many • Monthly Catalog of Government Publications available reference sources. • Public Affairs Information Service Bulletin • Education Index REFERENCE SOURCES • Applied Science and Technology Index • A Guide to Geographic Periodicals • General Science Index You’ve probably heard the expression “information • Biological and Agricultural Index explosion.” Your library is one of the main battle- • Nursing and Applied Health Index fields. Fortunately, a large number of reference vol- • Nursing Studies Index 498 Text not available due to copyright restrictions 500 APPENDIX A USING THE LIBRARY

2 3 ADOLESCENCE 301.43 Eagan, Andrea Boroff E Why am I so miserable if these 7 are the best years of my life? A survival guide for the young woman; with an introduction by Ellen Frankfort. Lippincott 1976 251p illus 6 4 5

FIGURE A-2 Sample Subject Catalog Card Source: Lilian L. Shapiro, Teaching Yourself in Libraries (New York: H. W. Wilson, 1978), 3–4. Used by permission.

• Index to Little Magazines you have only a general subject area of interest, • Popular Periodical Index you should thumb through the subject catalog. Fig- • Biography Index ure A-2 presents a sample card in the card catalog. • Congressional Quarterly Weekly Report Notice the following elements: • Library Literature • Bibliographic Index 1. Subject heading (always in capital letters) 2. Author’s name (last name, first name) 3. Title of the book USING THE STACKS 4. Publisher 5. Date of publication Serious research usually involves using the stacks, 6. Number of pages in the book plus other infor- where most of the library’s books are stored. This mation (such as whether the book contains section provides information about find ing books illustrations) in the stacks. 7. Call number needed to find a nonfiction book on the library shelves; fiction is generally found in alphabetical order by the author’s name The Card Catalog

In the traditional paper system, the card catalog is Library of Congress Classification the main reference system for finding out where books are stored. Each book is described on three Here’s a useful strategy to use when you’re re- separate 3-by-5 cards. The cards are then filed searching a topic. Once you’ve identified the call in three alphabetical sets. One set is arranged by au- number for a particular book in your subject area, thor, another by title, and the third by subject matter. go to the stacks, find that book, and look over the If you want to find a particular book, you can other books on the shelves near it. Because the look it up in either the author file or the title file. If books are arranged by subject matter, this method COMPUTERIZED LIBRARY FILES 501 will help you locate relevant books you didn’t K LAW know about. L EDUCATION Alternatively, you may want to go directly to the M MUSIC stacks and look at books in your subject area. In N FINE ARTS most libraries, books are arranged and numbered NA Architecture according to a subject matter classification devel- NB Sculpture oped by the Library of Congress. (Some follow the NC Graphic arts Dewey decimal system.) The following is a selected ND Painting list of Library of Congress categories. NE Engraving NK Ceramics, textiles Library of Congress Classifications P LANGUAGE AND LITERATURE (partial) RE English language PG Slavic language A GENERAL WORKS PJ-PM Oriental language B PHILOSOPHY, PSYCHOLOGY, RELIGION PN Drama, oratory, journalism B-BD Philosophy PQ Romance literature BF Psychology PR English literature BL-BX Religion PS American literature C HISTORY-AUXILIARY SCIENCES PT Germanic literature D HISTORY (except America) Q SCIENCE DA-DR Europe QA Mathematics DS Asia QB Astronomy DT Africa QC Physics E-F HISTORY (America) QD Chemistry E United States QE Geology E51–99 Indians of North America QH-QR Biology E185 Negroes in the United R MEDICINE States RK Dentistry F101–1140 Canada RT Nursing F1201–3799 Latin America S AGRICULTURE—PLANT AND ANIMAL G GEOGRAPHY-ANTHROPOLOGY INDUSTRY G-GF Geography T TECHNOLOGY GC Oceanology and TA-TL Engineering oceanography TR Photography GN Anthropology U MILITARY SCIENCE GV Sports, amusements, V NAVAL SCIENCE games Z BIBLIOGRAPHY AND LIBRARY SCIENCE H SOCIAL SCIENCES H62.B2 The Basics of Social COMPUTERIZED LIBRARY FILES Research HB-HJ Economics and business HM-HX Sociology Increasingly, library materials are catalogued elec- J POLITICAL SCIENCE tronically. While there are different computerized JK United States library systems, here’s a typical example of how JN Europe they work. JQ Asia, Africa Sitting at a computer terminal—in the library, JX International relations at a computer lab, or at home—you can type the 502 APPENDIX A USING THE LIBRARY

AU Kinloch-Graham-C. Tl The Changing Definition and Content of Sociology in Introductory Textbooks, 1894–1981. SO International Review of Modern Sociology. 1984, 14, 1, spring, 89–103. DE Sociology-Education; (D810300). Textbooks; (D863400). AB An analysis of 105 introductory sociology text books published between 1894 & 1981 reveals historical changes in definitions of the discipline & major topics in relation to professional factors & changing societal contexts. Pre dominant views of sociology in each decade are discussed, with the prevailing view being that of a “scientific study of social structure in order to decrease conflict & deviance, thereby increasing social control.” Consistencies in this orientation over time, coupled with the textbooks’ generally low sensitivity to social issues, are explored in terms of their authors’ relative homogeneity in age & educational backgrounds. 1 Table, 23 References. Modified HA.

FIGURE A-3 A Research Summary from Sociological Abstracts

title of a book and in seconds see a video display nal authors—so that you can locate a great many of a catalog card. If you want to explore the book relevant references easily and effectively. As you further, you can type an instruction at the terminal find relevant references, you can track down the and see an abstract of the book. original works and see the full details. The sum- Alternatively, you might type a subject name maries are available in both written and computer- and see a listing of all the books and articles writ- ized forms. ten on that topic. You could skim through the list Figure A-3 contains the abstract of an article and indicate which ones you want to see. obtained in a computer search of Sociological Ab- Many libraries today provide access to peri- stracts. I began by asking for a list of articles deal- odicals and books via the World Wide Web. Your ing with sociology textbooks. After reviewing the library’s computerized system should allow you to list, I asked to see the abstracts of each of the listed see which materials are available online. Some- articles. Here’s an example of what I received sec- times whole dissertations or books can be down- onds later: an article by the sociologist Graham C. loaded. Most likely, your largest local library pro- Kinloch, published in the International Review of vides document delivery services to its members. Modern Sociology. Many college libraries now have access to the In case the meaning of the abbreviations in Educational Resources Information Center (ERIC). Figure A-3 isn’t immediately obvious, I should ex- This computer-based system allows you to search plain that AU is author; TI is title; SO is the source through hundreds of major educational journals to or location of the original publication; DE indicates find articles published in the subject area of your classification codes under which the abstract is interest (within the field of education). As a rule, referenced; and AB is the abstract. The comput- each library website should have a list of the da- erized availability of resources such as Sociologi- tabases by discipline that you can visit, which may cal Abstracts provides a powerful research tool for help you limit the number of titles related to a spe- modern social scientists. You’ll have the option to cific keyword. Make sure you narrow your search download or print, with or without the abstract, by limiting, for instance, language or period of the any title you find through the library’s browsers. publication. Once you identify the articles you’re If a document is not available in the library it- interested in, the computer will print out their self or via the web, you always have the resource abstracts. of interlibrary loans, which often are free. Libraries Of particular value to social science re- don’t own every document or multimedia mate- searchers, the publications Sociological Abstracts rial (CD-ROMs, videocassettes, DVDs, films), but and Psychological Abstracts present summaries of many have loan agreements that can serve your books and articles—often prepared by the origi- needs. You need to be aware of the time you can FULL-TEXT ONLINE RESOURCES 503

FIGURE A-4 Opening Screen for InfoTrac College Edition

expect between your request and actually receiv- in researching. Let’s say you enter “capital punishing the book or article. In the case of a book that ment” in the box. Figure A-5 shows what I got in is located in another library close by, for example, response. (By the time you try this, there may be it may be faster for you to get it directly yourself. different articles in the database.) The key to a good library search is to become well As you can see, there are more than a couple informed. So start networking with librarians, fac- of articles on capital punishment. For most re- ulty, and peers! search purposes, periodicals provide a better source than do newspapers, but sometimes news- FULL-TEXT ONLINE RESOURCES papers are best. Let’s say you’re doing research on capital punishment for juveniles, and you can see that Today, it’s often possible to get the full text of ar- there are 100 periodical articles on that topic. ticles online, and this capacity is growing in leaps Figure A-6 presents a portion of what you should and bounds. InfoTrac College Edition is just one of see if you click the appropriate “View” button. the many ways you can do this. You may have re- (Again, the articles may have changed by the time ceived a subscription to this service with the pur- you try this.) chase of this book. If not, your school library may Finally, let’s take a look at the ﬁrst of these ar- have an institutional subscription that will give you ticles, to see if it would be appropriate for your full access. paper on capital punishment. Figure A-7 presents Once you have connected with InfoTrac College part of this article. Edition, your computer screen should look some- Full-text retrieval services such as InfoTrac Col- thing like Figure A-4. You have several options for lege Edition offer you a powerful research tool. initiating your search. If you’re looking for a par- However, let me close this discussion with two ticular article, you might enter the title (or a sub- cautionary comments. stantial part of it) in the box. If you’re looking for a First, not every article resulting from a search particular author, enter his or her name in the box. like this will be appropriate to your research pur- Or you can enter a topic that you are interested poses, so you may have to read through a great 504 APPENDIX A USING THE LIBRARY

FIGURE A-5 Index of Articles on Capital Punishment

FIGURE A-6 Articles on Capital Punishment for Juveniles many articles to ﬁnd what you need. My own, pro- Second, when using bibliographic materials fessorial bias is that tools such as this should not like these, it’s important to remember our earlier require less effort on your part than other types of discussion of plagiarism. Be sure to quote and cite searches will do but that they will yield a far supe- appropriately. A failure to do is, at best, sloppy and, rior product from the same amount of effort. at worst, dishonest. ADDITIONAL READINGS 505

FIGURE A-7 A Portion of an Article on Juvenile Capital Punishment

With those two cautions, I encourage you to ment and nongovernment sources of data. Special make full use of these kinds of bibliographic re- section on sex roles and women’s studies. sources. It’s also OK to poke around the ﬁles out of Li, Tze-chung. 2000. Social Science Reference Sources: A idle curiosity. You never know what you might ﬁnd. Practical Guide. Westport, CT: Greenwood Press. Lists and describes all types of reference materials, including databases and archives as well as published Additional Readings sources. Organized into two parts: social sciences in general and by discipline. Bart, Pauline, and Linda Frankel. 1986. The Student Richlin-Klonsky, Judith, and Ellen Strenski, eds. 1998. Sociologist’s Handbook. New York: Random House. A A Guide to Writing Sociology Papers. New York: St. survival kit for doing sociological research. Contains Martin’s Press. This is a great little book with good a step-by-step guide for writing research papers; advice on doing research. It’s particularly useful for chapters on periodicals, abstract and indexing ser- those who are new to sociology or other social sci- vices, bibliographies, bibliographical aids, and other ence disciplines and have to learn about the most secondary sources; and a complete guide to govern- rudimentary aspects of research. APPENDIX B

Random Numbers

10480 15011 01536 02011 81647 91646 69179 14194 62590 36207 20969 99570 91291 90700 22368 46573 25595 85393 30995 89198 27982 53402 93965 34095 52666 19174 39615 99505 24130 48360 22527 97265 76393 64809 15179 24830 49340 32081 30680 19655 63348 58629 42167 93093 06243 61680 07856 16376 39440 53537 71341 57004 00849 74917 97758 16379 37570 39975 81837 16656 06121 91782 60468 81305 49684 60672 14110 06927 01263 54613 77921 06907 11008 42751 27756 53498 18602 70659 90655 15053 21916 81825 44394 42880 99562 72905 56420 69994 98872 31016 71194 18738 44013 48840 63213 21069 10634 12952 96301 91977 05463 07972 18876 20922 94595 56869 69014 60045 18425 84903 42508 32307 89579 14342 63661 10281 17453 18103 57740 84378 25331 12566 58678 44947 05585 56941 85475 36857 53342 53988 53060 59533 38867 62300 08158 17983 16439 11458 18593 64952 28918 69578 88231 33276 70997 79936 56865 05859 90106 31595 01547 85590 91610 78188 63553 40961 48235 03427 49626 69445 18663 72695 52180 20847 12234 90511 33703 90322 09429 93969 52636 92737 88974 33488 36320 17617 30015 08272 84115 27156 30613 74952 10365 61129 87529 85689 48237 52267 67689 93394 01511 26358 85104 20285 29975 89868 07119 97336 71048 08178 77233 13916 47564 81056 97735 85977 29372 74461 28551 90707 51085 12765 51821 51259 77452 16308 60756 92144 49442 53900 70960 63990 75601 40719 02368 21382 52404 60268 89368 19885 55322 44819 01188 65255 64835 44919 05944 55157 01011 54092 33362 94904 31273 04146 18594 29852 71585 85030 51132 01915 92747 64951 52162 53916 46369 58586 23216 14513 83149 98736 23495 64350 94738 17752 35156 35749 07056 97628 33787 09998 42698 06691 76988 13602 51851 46104 88916 19509 25625 58104 48663 91245 85828 14346 09172 30168 90229 04734 59193 22178 30421 61666 99904 32812 54164 58492 22421 74103 47070 25306 76468 26384 58151 06646 21524 15227 96909 44592 32639 32363 05597 24200 13363 38005 94342 28728 35806 06912 17012 64161 18296 22851 29334 27001 87637 87308 58731 00256 45834 15398 46557 41135 10367 07684 36188 18510 02488 33062 28834 07351 19731 92420 60952 61280 50001 67658 32586 86679 50720 94953 81525 72295 04839 96423 24878 82651 66566 14778 76797 14780 13300 87074 79666 95725 29676 20591 68086 26432 46901 20849 89768 81536 86645 12659 92259 57102 80428 25280 00742 57392 39064 66432 84673 40027 32832 61362 98947 96067 64760 64584 96096 98253 05366 04213 25669 26422 44407 44048 37397 63904 45766 66134 75470 66520 34693 90449 91921 26418 64117 94305 26766 25940 39972 22209 71500 64568 91402 42416 07844 69618 00582 04711 87917 77341 42206 35126 74087 99547 81817 42607 43808 76655 62028 76630 00725 69884 62797 56170 86324 88072 76222 36086 84637 93161 76038 65855 77919 88006 69011 65795 95876 55293 18988 27354 26575 08625 40801 59920 29841 80150 12777 48501 25976 57948 29888 88604 67917 48708 18912 82271 65424 69774 33611 54262 85963 03547 09763 83473 73577 12908 30883 18317 28290 35797 05998 41688 34952 37888 38917 88050 91567 42595 27958 30134 04024 86385 29880 99730 55536 84855 29080 09250 79656 73211 17955 56349 90999 49127 20044 59931 06115 20542 18059 02008 73708 83517 36103 42791 46503 18584 18845 49618 02304 51038 20655 58727 28168 15475 56942 53389 20562 87338 92157 89634 94824 78171 84610 82834 09922 25417 44137 48413 25555 21246 35509 20468 14577 62765 35605 81263 39667 47358 56873 56307 61607 49518 89656 20103 77490 18062 98427 07523 33362 64270 01638 92477 66969 98420 04880 45585 46565 04102 46880 45709 34914 63976 88720 82765 34476 17032 87589 40836 32427 70002 70663 88863 77775 69348 70060 28277 39475 46473 23219 53416 94970 25832 69975 94884 19661 72828 00102 66794 53976 54914 06990 67245 68350 82948 11398 42878 80287 88267 47363 46634 06541 97809 76072 29515 40980 07391 58745 25774 22987 80059 39911 96189 41151 14222 60697 59583 90725 52210 83974 29992 65831 38857 50490 83765 55657 14361 31720 57375 56228 41546 64364 67412 33339 31926 14883 24413 59744 92351 97473 89286 35931 04110 23726 51900 08962 00358 31662 25388 61642 34072 81249 35648 56891 69352 48373 45578 78547 81788 95012 68379 93526 70765 10592 04542 76463 54328 02349 17247 28865 14777 62730 92277 15664 10493 20492 38391 91132 21999 59516 81652 27195 48223 46751 22923 32261 85653 16408 81899 04153 53381 79401 21438 83035 92350 36693 31238 59649 91754 72772 02338 18629 81953 05520 91962 04739 13092 97662 24822 94730 06496 35090 04822 86774 98289 73115 35101 47498 87637 99016 71060 88824 71013 18735 20286 23153 72924 35165 43040 57491 16703 23167 49323 45021 33132 12544 41035 80780 45393 44812 12515 98931 91202 30405 83946 23792 14422 15059 45799 22716 19792 09983 74353 68668 30429 70735 25499 16631 35006 85900 98275 32388 52390 16815 69298 82732 38480 73817 32523 41961 44437 96773 20206 42559 78985 05300 22164 24369 54224 35083 19687 11052 91491 60383 19746 38935 64202 14349 82674 66523 44133 00697 35552 35970 19124 63318 29686 03387 59846 31624 76384 17403 53363 44167 64486 64758 75366 76554 31601 12614 33072 60332 92325 78919 19474 23632 27889 47914 02584 37680 20801 72152 39339 34806 08930 85001 87820

506 03931 33309 57047 74211 63445 17361 62825 39908 05607 91284 68833 25570 38818 46920 74426 33278 43972 10119 89917 15665 52872 73823 73144 88662 88970 74492 51805 99378 09066 00903 20795 95452 92648 45454 09552 88815 16553 51125 79375 97596 16296 66092 42238 12426 87025 14267 20979 04508 64535 31355 86064 29472 47689 05974 52468 16834 16153 08002 26504 41744 81959 65642 74240 56302 00033 67107 77510 70625 28725 34191 21457 40742 29820 96783 29400 21840 15035 34537 33310 06116 95240 15957 16572 06004 21581 57802 02050 89728 17937 37621 47075 42080 97403 48626 68995 43805 33386 21597 55612 78095 83197 33732 05810 24813 86902 60397 16489 03264 88525 42786 05269 92532 44657 66999 99324 51281 84463 60563 79312 93454 68876 25471 93911 25650 12682 73572 91340 84979 46949 81973 37949 61023 43997 15263 80644 43942 89203 71795 99533 50501 91227 21199 31935 27022 84067 05462 35216 14486 29891 68607 41867 14951 91696 85065 50001 38140 66321 19924 72163 09538 12151 06878 91903 18749 34405 56087 82790 70925 65390 05224 72958 28609 81406 39147 25549 48542 42627 45233 57202 94617 23772 07896 27504 96131 83944 41575 10573 08619 64482 73923 36152 05184 94142 25299 84387 34925 37169 94851 39117 89632 00959 16487 65536 49071 39782 17095 02330 74301 00275 48280 11508 70225 51111 38351 19444 66499 71945 05422 13442 78675 84081 66938 93654 59894 37449 30362 06694 54690 04052 53115 62757 95348 78662 11163 81651 50245 34971 52924 46515 70331 85922 38329 57015 15765 97161 17869 45349 61796 66345 81073 49106 79860 30986 81223 42416 58353 21532 30502 32305 86482 05174 07901 54339 58861 74818 46942 63798 64995 46583 09785 44160 78128 83991 42865 92520 83531 80377 35909 81250 54238 82486 84846 99254 67632 43218 50076 21361 64816 51202 88124 41870 52689 51275 83556 21885 32906 92431 09060 64297 51674 64126 62570 26123 05155 59194 52799 28225 85762 60336 98782 07408 53458 13564 59089 26445 29789 85205 41001 12535 12133 14645 23541 43937 46891 24010 25560 86355 33941 25786 54990 71899 15475 95434 98227 21824 19585 97656 63175 89303 16275 07100 92063 21942 18611 47348 20203 18534 03862 78095 50136 03299 01221 05418 38982 55758 92237 26759 86367 21216 98442 08303 56613 91511 75928 79626 06486 03574 17668 07785 76020 79924 25651 83325 88428 85076 72811 22717 50585 85636 68335 47539 03129 65651 11977 02510 26113 99447 68645 34327 15152 55230 93448 18039 14367 61337 06177 12143 46609 32989 74014 64708 00533 35398 58408 13261 47908 08362 15656 60627 36478 65648 16764 53412 09013 07832 41574 17639 82163 60859 75567 79556 29068 04142 16268 15387 12856 66227 38358 22478 73373 88732 09443 82558 05250 92608 82674 27072 32534 17075 27698 98204 63863 11951 34648 88022 56148 34925 57031 23982 25835 40055 67006 12293 02753 14827 23235 35071 99704 37543 11601 35503 85171 09915 96306 05908 97901 28395 14186 00821 80703 70426 75647 76310 88717 37890 40129 59037 33300 26695 62247 69927 76123 50842 43834 86654 70959 79725 93872 28117 19233 42488 78077 69882 61657 34136 79180 97526 43092 04098 73571 80799 76536 71255 64239 46764 86273 63003 93017 31204 36692 40202 35275 57306 55543 53203 18098 47625 88684 03237 45430 55417 63282 90816 17349 88298 90183 36600 78406 06216 95787 42579 90730 86591 81482 52667 61582 14972 90053 89534 76036 49199 43716 97548 04379 46370 28672 38534 01715 94964 87288 65680 43772 39560 12918 86537 62738 19636 51132 25739 56947

Abridged from Handbook of Tables for Probability and Statistics, 2nd ed., edited by William H. Beyer (Cleveland: The Chemical Rubber Company, 1968). Used by permission of The Chemical Rubber Company. APPENDIX C

Distribution of Chi Square

Text not available due to copyright restrictions

508 Text not available due to copyright restrictions APPENDIX D

Normal Curve Areas

Text not available due to copyright restrictions

510 APPENDIX E

Estimated Sampling Error

How to use this table: Find the intersection be- Example: In the sample of 400 respondents, tween the sample size and the approximate per- 60-percent answer yes and 40 percent answer no. centage distribution of the binomial in the sample. The sampling error is estimated at plus or minus The number appearing at this intersection repre- 4.9 percentage points. The confidence interval, sents the estimated sampling error, at the 95 per- then, is between 55.1 percent and 64.9 percent. cent confidence level, expressed in percentage We would estimate (95 percent confidence) that points (plus or minus). the proportion of the total population who would say yes is somewhere within that interval.

Binomial Percentage Distribution Sample Size 50/50 60/40 70/30 80/20 90/10

100 10.0 9.8 9.2 8.0 6.0 200 7.1 6.9 6.5 5.7 4.2 300 5.8 5.7 5.3 4.6 3.5 400 5.0 4.9 4.6 4.0 3.0 500 4.5 4.4 4.1 3.6 2.7 600 4.1 4.0 3.7 3.3 2.4 700 3.8 3.7 3.5 3.0 2.3 800 3.5 3.5 3.2 2.8 2.1 900 3.3 3.3 3.1 2.7 2.0 1000 3.2 3.1 2.9 2.5 1.9 1100 3.0 3.0 2.8 2.4 1.8 1200 2.9 2.8 2.6 2.3 1.7 1300 2.8 2.7 2.5 2.2 1.7 1400 2.7 2.6 2.4 2.1 1.6 1500 2.6 2.5 2.4 2.1 1.5 1600 2.5 2.4 2.3 2.0 1.5 1700 2.4 2.4 2.2 1.9 1.5 1800 2.4 2.3 2.2 1.9 1.4 1900 2.3 2.2 2.1 1.8 1.4 2000 2.2 2.2 2.0 1.8 1.3

511 This page intentionally left blank GLOSSARY

abstract A summary of a research article. The the computation of a simple correlation coef- abstract usually begins the article and states fi cient are examples of bivariate analyses. See the purpose of the research, the methods used, Chapter 14. and the major fi ndings. See Chapter 15. Bogardus social distance scale A measure- anonymity Anonymity is guaranteed in a ment technique for determining the willing- research project when neither the researchers ness of people to participate in social rela- nor the readers of the fi ndings can identify a tions—of varying degrees of closeness—with given response with a given respondent. See other kinds of people. It is an especially Chapter 3. effi cient technique in that one can summarize attribute A characteristic of a person or a thing. several discrete answers without losing any of See variable and Chapter 1. the original details of the data. See Chapter 6. average An ambiguous term generally suggest- case study The in-depth examination of a single ing typical or normal—a central tendency. instance of some social phenomenon, such The mean, median, and mode are specifi c as a village, a family, or a juvenile gang. See examples of mathematical averages. See Chapter 10. Chapter 14. case-oriented analysis (1) An analysis that axial coding A reanalysis of the results of open aims to understand a particular case or several coding in Grounded Theory Method, aimed at cases by looking closely at the details of each. identifying the important, general concepts. See Chapter 13. (2) A private investigator’s bill- See Chapter 13. ing system. bias (1) That quality of a measurement device closed-ended questions Survey questions in that tends to result in a misrepresentation, in which the respondent is asked to select an a particular direction, of what is being mea- answer from among a list provided by the re- sured. For example, the questionnaire item searcher. These are popular in survey research “Don’t you agree that the president is doing because they provide a greater uniformity of a good job?” would be biased in that it would responses and are more easily processed than generally encourage favorable responses. See open-ended questions. See Chapter 9. Chapter 9. (2) The thing inside you that makes cluster sampling (1) A multistage sampling in other people or groups seem consistently bet- which natural groups (clusters) are sampled ter or worse than they really are. (3) What a initially, with the members of each selected nail looks like after you hit it crooked. (If you group being subsampled afterward. For drink, don’t drive.) example, you might select a sample of U.S. bivariate analysis The analysis of two variables colleges and universities from a directory, get simultaneously, for the purpose of determining lists of the students at all the selected schools, the empirical relationship between them. The then draw samples of students from each. This construction of a simple percentage table or procedure is discussed in Chapter 7. (2) Pawing 513 514 GLOSSARY

around in a box of macadamia nut clusters to pal, operationalization. (2) Sexual reproduction take all the big ones for yourself. among intellectuals. codebook (1) The document used in data confi dence interval (1) The range of values processing and analysis that tells the location within which a population parameter is of different data items in a data fi le. Typically, estimated to lie. A survey, for example, the codebook identifi es the locations of data may show 40 percent of a sample favoring items and the meaning of the codes used to Candidate A (poor devil). Although the best represent different attributes of variables. See estimate of the support existing among all Chapter 14. (2) The document that cost you 38 voters would also be 40 percent, we would not box tops just to learn that Captain Marvelous expect it to be exactly that. We might, there- wanted you to brush your teeth and always tell fore, compute a confi dence interval (such as the truth. (3) The document that allows CIA from 35 to 45 percent) within which the actual agents to learn that Captain Marvelous wants percentage of the population probably lies. them to brush their teeth. Note that we must specify a confi dence level in coding (1) The process whereby raw data are connection with every confi dence interval. See transformed into standardized form suitable Chapter 7. (2) How close you dare to get to an for machine processing and analysis. See alligator. Chapter 11. (2) A strong drug you may take confi dence level (1) The estimated probability when you hab a bad code. that a population parameter lies within a given cohort study A study in which some specifi c confi dence interval. Thus, we might be 95 per- subpopulation, or cohort, is studied over time, cent confi dent that between 35 and 45 percent although data may be collected from different of all voters favor Candidate A. See Chapter 7. members in each set of observations. A study (2) How sure you are that the ring you bought of the occupational history of the class of 1970, from a street vendor for $10 is really a three- in which questionnaires were sent every fi ve carat diamond. years, for example, would be a cohort study. confi dentiality A research project guarantees See Chapter 4 for more on this topic (if you confi dentiality when the researcher can iden- want more). See also longitudinal study, panel tify a given person’s responses but promises study, and trend study. not to do so publicly. See Chapter 3. comparative and historical research The constant comparative method (1) A compo- examination of societies (or other social units) nent of the Grounded Theory Method in which over time and in comparison with one another. observations are compared with one another See Chapter 11. and with the evolving inductive theory. See concept mapping (1) The graphical display Chapter 13. (2) A blind-dating technique. of concepts and their interrelations, useful in construct validity The degree to which a the formulation of theory. See Chapter 13. measure relates to other variables as expected (2) A masculine technique for fi nding loca- within a system of theoretical relationships. tions by logic and will, without asking for See Chapter 5. directions. content analysis The study of recorded human conceptualization (1) The mental process communications, such as books, websites, whereby fuzzy and imprecise notions (con- paintings, and laws. See Chapter 11. cepts) are made more specifi c and precise. content validity The degree to which a mea- So you want to study prejudice. What do you sure covers the range of meanings included mean by prejudice? Are there different kinds within a concept. See Chapter 5. of prejudice? What are they? See Chapter 5, contingency question A survey question in- which is all about conceptualization and its tended for only some respondents, determined GLOSSARY 515

by their responses to some other question. constitute a causal relationship between the For example, all respondents might be asked two variables, but it is one criterion of causal- whether they belong to the Cosa Nostra, and ity. See Chapter 4. (2) Someone you and your only those who said yes would be asked how friend are both related to. often they go to company meetings and pic- criterion-related validity The degree to which nics. The latter would be a contingency ques- a measure relates to some external criterion. tion. See Chapter 9. For example, the validity of the College Board contingency table (1) A format for presenting exams is shown in their ability to predict the the relationships among variables as percent- college success of students. Also called predic- age distributions. See Chapter 14 for several tive validity. See Chapter 5. illustrations and guides to making such tables. cross-case analysis An analysis that involves (2) The card table you keep around in case an examination of more than one case, either your guests bring their seven kids with them to a variable-oriented or case-oriented analysis. See dinner. Chapter 13. continuous variable A variable whose attri- cross-sectional study A study based on butes form a steady progression, such as observations representing a single point in age or income. Thus, the ages of a group of time. Contrasted with a longitudinal study. See people might include 21, 22, 23, 24, and so Chapter 4. forth and could even be broken down into debriefi ng (1) Interviewing subjects to learn fractions of years. Contrast this with discrete about their experience of participation in the variables, such as sex or religious affi liation, project and to inform them of any unrevealed whose attributes form discontinuous chunks. purpose. This is especially important if there’s See Chapter 14. a possibility that they have been damaged by control group (1) In experimentation, a group that participation. See Chapter 3. (2) Pulling of subjects to whom no experimental stimu- someone’s shorts down. Don’t do that. It’s lus is administered and who resemble the not nice. experimental group in all other respects. deduction (1) The logical model in which spe- The comparison of the control group and the cifi c expectations of hypotheses are developed experimental group at the end of the experi- on the basis of general principles. Starting ment points to the effect of the experimental from the general principle that all deans are stimulus. See Chapter 8. (2) American Associa- meanies, you might anticipate that this one tion of Managers. won’t let you change courses. This anticipa- control variable See test variable. tion would be the result of deduction. See also conversation analysis (CA) A meticulous induction and Chapter 1. (2) What the Internal analysis of the details of conversation, based Revenue Service said your good-for-nothing on a complete transcript that includes pauses, moocher of a brother-in-law technically isn’t. hems, and also haws. See Chapter 13. (3) Of a duck. correlation (1) An empirical relationship be- dependent variable (1) A variable assumed to tween two variables such that (a) changes in depend on or be caused by another (called the one are associated with changes in the other independent variable). If you fi nd that income is or (b) particular attributes of one variable are partly a function of amount of formal education, associated with particular attributes of the income is being treated as a dependent vari- other. Thus, for example, we say that education able. See Chapter 1. (2) A wimpy variable. and income are correlated in that higher levels dimension A specifi able aspect of a concept. of education are associated with higher levels “Religiosity,” for example, might be specifi ed in of income. Correlation in and of itself does not terms of a belief dimension, a ritual dimension, 516 GLOSSARY

a devotional dimension, a knowledge dimen- evaluation research Research undertaken for sion, and so forth. See Chapter 5. the purpose of determining the impact of some discrete variable A variable whose attributes social intervention, such as a program aimed are separate from one another, or discontinu- at solving a social problem. See Chapter 12. ous, as in the case of sex or religious affi liation. experimental group In experimentation, a Contrast this with continuous variables, in group of subjects to whom an experimental which one attribute shades off into the next. stimulus is administered. Compare with control Thus, in age (a continuous variable), the at- group. See Chapter 8. tributes progress steadily from 21 to 22 to 23, extended case method A technique developed and so forth, whereas there is no progression by Michael Burawoy in which case study ob- from male to female in the case of sex. See servations are used to discover fl aws in and Chapter 14. to improve existing social theories. See dispersion The distribution of values around Chapter 10. some central value, such as an average. The external invalidity Refers to the possibility that range is a simple example of a measure of conclusions drawn from experimental results dispersion. Thus, we may report that the mean may not be generalizable to the “real” world. age of a group is 37.9, and the range is from 12 See Chapter 8 and also internal invalidity. to 89. See Chapter 14. external validation The process of testing the double-blind experiment An experimental validity of a measure, such as an index or design in which neither the subjects nor the scale, by examining its relationship to other, experimenters know which is the experimental presumed indicators of the same variable. If group and which is the control. See Chapter 8. the index really measures prejudice, for ex- ecological fallacy Erroneously basing conclu- ample, it should correlate with other indicators sions about individuals solely on the observa- of prejudice. See Chapter 6 for a fuller discus- tion of groups. See Chapter 4. sion and illustrations. element That unit of which a population is face validity (1) That quality of an indicator composed and which is selected in a sample. that makes it seem a reasonable measure of Distinguished from units of analysis, which are some variable. That the frequency of atten- used in data analysis. See Chapter 7. dance at religious services is some indication emancipatory research Research conducted of a person’s religiosity seems to make sense for the purpose of benefi ting disadvantaged without a lot of explanation. It has face validity. groups. See Chapter 10. See Chapter 5. (2) When your face looks like EPSEM (equal probability of selection your driver’s license photo (rare and perhaps method) A sample design in which each unfortunate). member of a population has the same chance fi eld experiment A formal experiment con- of being selected into the sample. See ducted outside the laboratory, in a natural Chapter 7. setting. See Chapter 8. ethnography A report on social life that focuses focus group A group of subjects interviewed on detailed and accurate description rather together, prompting a discussion. The tech- than explanation. See Chapter 10. nique is frequently used by market researchers, ethnomethodology An approach to the study who ask a group of consumers to evaluate a of social life that focuses on the discovery of product or discuss a type of commodity, for implicit, usually unspoken assumptions and example. See Chapter 10. agreements; this method often involves the frequency distribution (1) A description of intentional breaking of agreements as a way of the number of times the various attributes revealing their existence. See Chapter 10. of a variable are observed in a sample. The GLOSSARY 517

report that 53 percent of a sample were men a dependent variable. If we discover that and 47 percent were women would be a religiosity is partly a function of sex—women simple example of a frequency distribution. are more religious than are men—sex is the Another example would be the report that 15 independent variable and religiosity is the de- of the cities studied had populations under pendent variable. Note that any given variable 10,000, 23 had populations between 10,000 might be treated as independent in one part of and 25,000, and so forth. See Chapter 14. an analysis and dependent in another part of it. (2) A radio dial. Religiosity might become an independent vari- grounded theory (1) An inductive approach to able in an explanation of crime. See Chapter 1. the study of social life that attempts to gener- (2) A variable that refuses to take advice. ate a theory from the constant comparing of index A type of composite measure that sum- unfolding observations. This differs greatly marizes and rank-orders several specifi c from hypothesis testing, in which theory is observations and represents some more used to generate hypotheses to be tested general dimension. Contrasted with scale. through observations. See Chapter 10. See Chapter 6. (2) A theory that is not allowed to fl y. indicator An observation that we choose to Grounded Theory Method (GTM) An induc- consider as a refl ection of a variable we wish tive approach to research introduced by Bar- to study. Thus, for example, attending religious ney Glaser and Anselm Strauss in which theo- services might be considered an indicator of ries are generated solely from an examination religiosity. See Chapter 5. of data rather than being derived deductively. induction (1) The logical model in which gen- See Chapter 13. eral principles are developed from specifi c Guttman scale (1) A type of composite measure observations. Having noted that Jews and used to summarize several discrete observa- Catholics are more likely to vote Democratic tions and to represent some more-general than are Protestants, you might conclude that variable. See Chapter 6. (2) The device Louis religious minorities in the United States are Guttman weighed himself on. more affi liated with the Democratic party, and hypothesis A specifi ed testable expectation then your task is to explain why. This would about empirical reality that follows from a be an example of induction. See also deduction more general proposition; more generally, an and Chapter 1. (2) The culinary art of stuffi ng expectation about the nature of things derived ducks. from a theory. It is a statement of something informant Someone well versed in the social that ought to be observed in the real world if phenomenon that you wish to study and who the theory is correct. See Chapter 2. is willing to tell you what he or she knows idiographic An approach to explanation in about it. If you were planning participant which we seek to exhaust the idiosyncratic observation among the members of a religious causes of a particular condition or event. sect, you would do well to make friends with Imagine trying to list all the reasons why you someone who already knows about them— chose to attend your particular college. Given possibly a member of the sect—who could all those reasons, it’s diffi cult to imagine your give you some background information about making any other choice. Contrasted with them. Not to be confused with a respondent. nomothetic. See Chapter 1. See Chapter 7. independent variable (1) A variable with val- informed consent A norm in which subjects ues that are not problematical in an analysis base their voluntary participation in research but are taken as simply given. An independent projects on a full understanding of the possible variable is presumed to cause or determine risks involved. See Chapter 3. 518 GLOSSARY institutional ethnography A research tech- research through the use of standardized nique in which the personal experiences response categories in survey questionnaires of individuals are used to reveal power re- to determine the relative intensity of different lationships and other characteristics of the items. Likert items are those using such re- institutions within which they operate. See sponse categories as “strongly agree,” “agree,” Chapter 10. “disagree,” and “strongly disagree.” Such items interest convergence The thesis that majority may be used in the construction of true Likert group members will only support the interests scales as well as other types of composite of minorities when those actions also support measures. See Chapter 6. the interests of the majority group. See longitudinal study A study design involving Chapter 2. data collected at different points in time, as internal invalidity Refers to the possibility that contrasted with a cross-sectional study. See also the conclusions drawn from experimental Chapter 4 and cohort study, panel study, and results may not accurately refl ect what went trend study. on in the experiment itself. See also external macrotheory A theory aimed at understanding invalidity and Chapter 8. the “big picture” of institutions, whole socie- interval measure A level of measurement de- ties, and the interactions among societies. scribing a variable whose attributes are rank- Karl Marx’s examination of the class struggle ordered and have equal distances between is an example of macrotheory. Contrasted with adjacent attributes. The Fahrenheit tempera- microtheory. See Chapter 2. ture scale is an example of this, because the manifest content (1) In connection with con- distance between 17 and 18 is the same as that tent analysis, the concrete terms contained in between 89 and 90. See Chapter 5 and nominal a communication, as distinguished from latent measure, ordinal measure, and ratio measure. content. See Chapter 11. (2) What you have interview A data-collection encounter in which after a manifest bursts. one person (an interviewer) asks questions matching In connection with experiments, of another (a respondent). Interviews may be the procedure whereby pairs of subjects conducted face-to-face or by telephone. See are matched on the basis of their similarities Chapter 9 for more information on interview- on one or more variables, and one member ing as a method of survey research. of the pair is assigned to the experimental item analysis An assessment of whether each group and the other to the control group. of the items included in a composite measure See Chapter 8. makes an independent contribution or merely mean (1) An average computed by summing the duplicates the contribution of other items in values of several observations and dividing the measure. See Chapter 6. by the number of observations. If you now judgmental sampling (1) See purposive sam- have a grade point average of 4.0 based on 10 pling and Chapter 7. (2) A sampling of opinion- courses, and you get an F in this course, your ated people. new grade point (mean) average will be 3.6. latent content (1) In connection with content See Chapter 14. (2) The quality of the thoughts analysis, the underlying meaning of commu- you might have if your instructor did that nications as distinguished from their manifest to you. content. See Chapter 11. (2) What you need to median An average representing the value of make a latent. the “middle” case in a rank-ordered set of Likert scale A type of composite measure observations. If the ages of fi ve men are 16, 17, developed by Rensis Likert in an attempt to 20, 54, and 88, the median would be 20. (The improve the levels of measurement in social mean would be 39.) See Chapter 14. (2) The GLOSSARY 519

dividing line between safe driving and exciting that generally impact a class of conditions or driving. events. Imagine the two or three key fac- memoing Writing memos that become part of tors that determine which colleges students the data for analysis in qualitative research choose, such as proximity, reputation, and so such as grounded theory. Memos can describe forth. Contrasted with idiographic. See also and defi ne concepts, deal with methodological Chapter 1. issues, or offer initial theoretical formulations. nonequivalent control group A control group See Chapter 13. that is similar to the experimental group but microtheory A theory aimed at understanding is not created by the random assignment of social life at the level of individuals and their subjects. This sort of control group does differ interactions. Explaining how the play behavior signifi cantly from the experimental group in of girls differs from that of boys is an example terms of the dependent variable or variables of microtheory. Contrasted with macrotheory. related to it. See Chapter 12. See Chapter 2. nonprobability sampling Any technique in mode (1) An average representing the most fre- which samples are selected in some way not quently observed value or attribute. If a sample suggested by probability theory. Examples contains 1,000 Protestants, 275 Catholics, and include reliance on available subjects as well 33 Jews, “Protestant” is the modal category. as purposive (judgmental), snowball, and quota See Chapter 14 for more thrilling disclosures sampling. See Chapter 7. about averages. (2) Better than apple pie à la null hypothesis (1) In connection with hypoth- median. esis testing and tests of statistical signifi cance, multiple time-series designs The use of more that hypothesis that suggests there is no than one set of data that were collected over relationship among the variables under study. time, as in accident rates over time in several You may conclude that the variables are re- states or cities, so that comparison can be lated after having statistically rejected the null made. See Chapter 12. hypothesis. See Chapter 2. (2) An expectation multivariate analysis The analysis of the about nulls. simultaneous relationships among several open coding The initial classifi cation and label- variables. Examining simultaneously the ef- ing of concepts in qualitative data analysis. In fects of age, sex, and social class on religiosity open coding, the codes are suggested by the would be an example of multivariate analysis. researchers’ examination and questioning of See Chapter 14. the data. See Chapter 13. naturalism An approach to fi eld research based open-ended questions Questions for which on the assumption that an objective social the respondent is asked to provide his or her reality exists and can be observed and reported own answers. In-depth, qualitative interview- accurately. See Chapter 10. ing relies almost exclusively on open-ended nominal measure A variable whose attributes questions. See Chapter 9. have only the characteristics of exhaustiveness operational defi nition The concrete and and mutual exclusiveness. In other words, a specifi c defi nition of something in terms of level of measurement describing a variable the operations by which observations are to that has attributes that are merely different, be categorized. The operational defi nition of as distinguished from ordinal, interval, or ratio “earning an A in this course” might be “cor- measures. Sex is an example of a nominal mea- rectly answering at least 90 percent of the fi nal sure. See Chapter 5. exam questions.” See Chapter 2. nomothetic An approach to explanation in operationalization (1) One step beyond which we seek to identify a few causal factors conceptualization. Operationalization is the 520 GLOSSARY

process of developing operational defi nitions, measured by the number of units to be sub- or specifying the exact operations involved sampled. See Chapter 7. (2) The odds on who in measuring a variable. See Chapter 2. (2) Sur- gets to go fi rst: you or the 275-pound fullback. gery on intellectuals. pretesting The measurement of a dependent ordinal measure A level of measurement variable among subjects before they are ex- describing a variable with attributes we can posed to a stimulus representing an indepen- rank-order along some dimension. An ex- dent variable. See posttesting and Chapter 8. ample is socioeconomic status as composed of probability sampling The general term for the attributes high, medium, low. See Chapter samples selected in accord with probability 5 and nominal measure, interval measure, and theory, typically involving some random- ratio measure. selection mechanism. Specifi c types of prob- panel study A type of longitudinal study, in ability sampling include EPSEM, PPS, simple which data are collected from the same set of random sampling, and systematic sampling. people (the sample or panel) at several points See Chapter 7. in time. See Chapter 4 and cohort, longitudinal, probe A technique employed in interviewing to and trend study. solicit a more complete answer to a question. paradigm (1) A model or framework for ob- It is a nondirective phrase or question used to servation and understanding, which shapes encourage a respondent to elaborate on an an- both what we see and how we understand it. swer. Examples include “Anything more?” and The confl ict paradigm causes us to see social “How is that?” See Chapter 9 for a discussion behavior one way, the interactionist paradigm of interviewing. causes us to see it differently. See Chapter 2. program evaluation/outcome assessment (2) $0.20. The determination of whether a social inter- parameter The summary description of a given vention is producing the intended result. See variable in a population. See Chapter 7. Chapter 12. participatory action research An approach purposive sampling A type of nonprobability to social research in which the people being sampling in which the units to be observed studied are given control over the purpose are selected on the basis of the researcher’s and procedures of the research; intended judgment about which ones will be the most as a counter to the implicit view that research- useful or representative. Also called judgmental ers are superior to those they study. See sampling. See Chapter 7. Chapter 10. qualitative analysis (1) The nonnumerical plagiarism Presenting someone else’s words or examination and interpretation of observa- thoughts as though they were your own, con- tions, for the purpose of discovering underlying stituting intellectual theft. See Chapter 15. meanings and patterns of relationships. This population The theoretically specifi ed aggrega- approach is most typical of fi eld research and tion of the elements in a study. See Chapter 7. historical research. See Chapter 13. (2) A classy posttesting The remeasurement of a dependent analysis. variable among subjects after they’ve been qualitative interview Contrasted with survey exposed to a stimulus representing an inde- interviewing, the qualitative interview is based pendent variable. See pretesting and Chapter 8. on a set of topics to be discussed in depth PPS (probability proportionate to size) (1) rather than the use of standardized questions. This refers to a type of multistage cluster See Chapter 10. sample in which clusters are selected, not quantitative analysis (1) The numerical rep- with equal probabilities (see EPSEM) but with resentation and manipulation of observations probabilities proportionate to their sizes—as for the purpose of describing and explaining GLOSSARY 521

the phenomena that those observations refl ect. peated observations of the same phenomenon. See Chapter 14. (2) A BIG analysis. In the context of a survey, we would expect quasi experiments Nonrigorous inquiries that the question “Did you attend religious ser- somewhat resembling controlled experiments vices last week?” would have higher reliability but lacking key elements such as pre- and than the question “About how many times posttesting and/or control groups. See have you attended religious services in your Chapter 12. life?” This is not to be confused with validity. questionnaire A document containing ques- See Chapter 5. (2) Quality of repeatability in tions and other types of items designed to untruths. solicit information appropriate for analysis. replication The duplication of an experiment to Questionnaires are used primarily in survey expose or reduce error. See Chapter 1. research but also in experiments, fi eld re- representativeness (1) That quality of a sample search, and other modes of observation. See of having the same distribution of charac- Chapter 9. teristics as the population from which it was quota sampling A type of nonprobability sam- selected. By implication, descriptions and pling in which units are selected into a sample explanations derived from an analysis of the on the basis of prespecifi ed characteristics, so sample may be assumed to represent similar that the total sample will have the same dis- ones in the population. Representativeness is tribution of characteristics assumed to exist in enhanced by probability sampling and provides the population being studied. See Chapter 7. for generalizability and the use of inferential random selection A sampling method in which statistics. See Chapter 7. (2) A noticeable qual- each element has an equal chance of selection ity in the presentation-of-self of some mem- independent of any other event in the selection bers of the U.S. Congress. process. See Chapter 7. research monograph A book-length research randomization A technique for assigning report, either published or unpublished. This experimental subjects to experimental and is distinguished from a textbook, a book of es- control groups randomly. See Chapter 8. says, a novel, and so forth. See Chapter 15. ratio measure A level of measurement describ- respondent A person who provides data for ing a variable with attributes that have all analysis by responding to a survey question- the qualities of nominal, ordinal, and inter- naire. See Chapter 9. val measures and in addition are based on a response rate The number of people participat- “true zero” point. Age is an example of a ratio ing in a survey divided by the number selected measure. See Chapter 5 and interval measure, in the sample, in the form of a percentage. nominal measure, and ordinal measure. This is also called the completion rate or, in reactivity The problem that the subjects of self-administered surveys, the return rate: the social research may react to the fact of being percentage of questionnaires sent out that are studied, thus altering their behavior from what returned. See Chapter 9. it would have been normally. See Chapter 10. sampling error The degree of error to be reductionism (1) A fault of some researchers: expected in probability sampling. The formula a strict limitation (reduction) of the kinds of for determining sampling error contains three concepts to be considered relevant to the phe- factors: the parameter, the sample size, and nomenon under study. See Chapter 4. (2) The the standard error. See Chapter 7. cloning of ducks. sampling frame That list or quasi list of units reliability (1) That quality of measurement composing a population from which a sample method that suggests that the same data is selected. If the sample is to be representative would have been collected each time in re- of the population, it is essential that the sam- 522 GLOSSARY

pling frame include all (or nearly all) members semiotics The study of signs and the meanings of the population. See Chapter 7. associated with them. This is commonly asso- sampling interval The standard distance (k) ciated with content analysis. See Chapter 13. between elements selected from a population simple random sampling (1) A type of prob- for a sample. See Chapter 7. ability sampling in which the units composing sampling ratio The proportion of elements a population are assigned numbers. A set of in the population that are selected to be in a random numbers is then generated, and the sample. See Chapter 7. units having those numbers are included in sampling unit That element or set of elements the sample. Although probability theory and considered for selection in some stage of sam- the calculations it provides assume this basic pling. See Chapter 7. sampling method, it’s seldom used, for practi- scale (1) A type of composite measure com- cal reasons. An equivalent alternative is the posed of several items that have a logical or systematic sample (with a random start). See empirical structure among them. Examples of Chapter 7. (2) A random sample with a low IQ. scales include Bogardus social distance, Gutt- snowball sampling (1) A nonprobability- man, Likert, and Thurstone scales. Contrasted sampling method, often employed in fi eld with index. See also Chapter 6. (2) One of the research, whereby each person interviewed less appetizing parts of a fi sh. may be asked to suggest additional people for search engine A computer program designed interviewing. See Chapter 7. (2) Picking the icy to locate where specifi ed terms appear on ones to throw at your methods instructor. websites throughout the World Wide Web. See social artifact Any product of social beings or Chapter 15. their behavior. Can be a unit of analysis. See secondary analysis (1) A form of research in Chapter 4. which the data collected and processed by one social indicators Measurements that refl ect the researcher are reanalyzed—often for a differ- quality or nature of social life, such as crime ent purpose—by another. This is especially rates, infant mortality rates, number of physi- appropriate in the case of survey data. Data cians per 100,000 population, and so forth. archives are repositories or libraries for the Social indicators are often monitored to deter- storage and distribution of data for second- mine the nature of social change in a society. ary analysis. See Chapter 9. (2) Estimating See Chapter 12. the weight and speed of an opposing team’s spurious relationship A coincidental statistical linebackers. correlation between two variables, shown to selective coding In Grounded Theory Method, be caused by some third variable. For example, this analysis builds on the results of open there is a positive relationship between the coding and axial coding to identify the central number of fi re trucks responding to a fi re and concept that organizes the other concepts that the amount of damage done: the more trucks, have been identifi ed in a body of textual mate- the more damage. The third variable is the rials. See Chapter 13. size of the fi re. They send lots of fi re trucks to a semantic differential A questionnaire for- large fi re and a lot of damage is done because mat in which the respondent is asked to rate of the size of the fi re. For a little fi re, they just something in terms of two, opposite adjectives send a little fi re truck, and not much damage (e.g., rate textbooks as “boring” or “exciting”), is done because it’s a small fi re. Sending more using qualifi ers such as “very,” “somewhat,” fi re trucks does not cause more damage. For a “neither,” “somewhat,” and “very” to bridge given size of fi re, in fact, sending more trucks the distance between the two opposites. See would reduce the amount of damage. See Chapter 6. Chapter 4. GLOSSARY 523 standard deviation (1) A measure of dispersion Thurstone scale A type of composite mea- around the mean, calculated so that approxi- sure, constructed in accord with the weights mately 68 percent of the cases will lie within assigned by “judges” to various indicators of plus or minus one standard deviation from the some variables. See Chapter 6. mean, 95 percent will lie within plus or minus time-series design A research design that two standard deviations, and 99.9 percent will involves measurements made over some pe- lie within three standard deviations. Thus, for riod, such as the study of traffi c accident rates example, if the mean age in a group is 30 and before and after lowering the speed limit. See the standard deviation is 10, then 68 percent Chapter 12. have ages between 20 and 40. The smaller the trend study A type of longitudinal study in standard deviation, the more tightly the values which a given characteristic of some popu- are clustered around the mean; if the standard lation is monitored over time. An example deviation is high, the values are widely spread would be the series of Gallup Polls showing the out. See Chapter 14. (2) Routine rule-breaking. electorate’s preferences for political candidates statistic The summary description of a variable over the course of a campaign, even though in a sample, used to estimate a population different samples were interviewed at each parameter. See Chapter 7. point. See Chapter 4 and cohort, longitudinal, stratifi cation The grouping of the units compos- and panel study. ing a population into homogeneous groups (or typology (1) The classifi cation (typically nomi- strata) before sampling. This procedure, which nal) of observations in terms of their attributes may be used in conjunction with simple ran- on two or more variables. The classifi cation of dom, systematic, or cluster sampling, improves newspapers as liberal-urban, liberal-rural, the representativeness of a sample, at least in conservative-urban, or conservative-rural terms of the variables used for stratifi cation. would be an example. See Chapter 6. See Chapter 7. (2) Apologizing for your neckwear. study population That aggregation of elements units of analysis The what or whom being from which a sample is actually selected. See studied. In social science research, the most Chapter 7. typical units of analysis are individual people. systematic sampling (1) A type of probability See Chapter 4. sampling in which every kth unit in a list is se- univariate analysis The analysis of a single lected for inclusion in the sample—for exam- variable, for purposes of description. Fre- ple, every 25th student in the college directory quency distributions, averages, and measures of students. You compute k by dividing the size of dispersion are examples of univariate of the population by the desired sample size; k analysis, as distinguished from bivariate and is called the sampling interval. Within certain multivariate analysis. See Chapter 14. constraints, systematic sampling is a func- unobtrusive research Methods of studying so- tional equivalent of simple random sampling cial behavior without affecting it. This includes and usually easier to do. Typically, the fi rst unit content analysis, analysis of existing statistics, is selected at random. See Chapter 7. (2) Pick- and comparative and historical research. See ing every third one whether it’s icy or not. See Chapter 11. snowball sampling (2). URL Web address, typically beginning with theory A systematic explanation for the obser- “http://”; stands for “uniform resource locator” vations that relate to a particular aspect of life: or “universal resource locator.” See Chapter 15. juvenile delinquency, for example, or perhaps validity A term describing a measure that ac- social stratifi cation or political revolution. See curately refl ects the concept it is intended to Chapter 1. measure. For example, your IQ would seem a 524 GLOSSARY

more valid measure of your intelligence than variable-oriented analysis An analysis that would the number of hours you spend in the describes and/or explains a particular variable. library. Though the ultimate validity of a mea- See Chapter 13. sure can never be proven, we may agree to its weighting Assigning different weights to cases relative validity on the basis of face validity, that were selected into a sample with differ- criterion-related validity, content validity, con- ent probabilities of selection. In the simplest struct validity, internal validation, and external scenario, each case is given a weight equal to validation. This must not be confused with reli- the inverse of its probability of selection. When ability. See Chapter 5. all cases have the same chance of selection, variable A logical grouping of attributes. The no weighting is necessary. See Chapter 7. variable sex is made up of the attributes male and female. See Chapter 1. REFERENCES

Abdollahyan, Hamid, and Taghi Azadarmaki. 2000. Auster, Carol J. 1985. “Manuals for Socialization: Sampling Design in a Survey Research: The Sampling Examples from Girl Scout Handbooks 1913–1984.” Practice in Iran. Paper presented at the meeting of the Qualitative Sociology 8 (4): 359–67. American Sociological Association, Washington, DC, Babbie, Earl. 1966. “The Third Civilization.” Review of Au gust 12–16. Religious Research, Winter, pp. 101–21. Abdulhadi, Rabab. 1998. “The Palestinian Women’s 1967.“A Religious Profi le of Episcopal Church- Autonomous Movement: Emergence, Dynamics, and women.” Pacifi c Churchman, Janu ary, Challenges.” Gender and Society 12 (6): 649–73. pp. 6–8, 12. Adams, Cecil. 2006. April 8. http://www.straightdope 1970. Science and Morality in Medicine. Berkeley: .com/columns/010202.html. University of California Press. Anderson, Eric. 2005. “Orthodox and Inclusive Masculin- 1982. Social Research for Consumers. Belmont, CA: ity: Competing Masculinities among Heterosexual Wadsworth. Men in a Feminized Terrain.” Sociological Perspectives 1985. You Can Make a Difference. New York: 48 (3): 337–55. St. Martin’s Press. Anderson, Walt. 1990. Reality Isn’t What It Used to Be: 2004. “Laud Humphreys and Research Ethics.” Inter- Theatrical Politics, Ready-to-Wear Religion, Global national Journal of Sociology and Social Policy 24 Myths, Primitive Chic, and Other Wonders of the Post- (3/4/5): 12–18. modern World. San Francisco: Harper & Row. Bailey, William C. 1975. “Murder and Capital Punish- Andorka, Rudolf. 1990. “The Importance and the Role ment.” In Criminal Law in Action, edited by William J. of the Second Economy for the Hungarian Economy Chambliss. New York: Wiley. and Society.” Quarterly Journal of Budapest University Bailey, William C., and Ruth D. Peterson. 1994. “Murder, of Economic Sciences 12 (2): 95–113. Capital Punishment, and Deterrence: A Review of Aneshensel, Carol S., Rosina Becerra, Eve Fielder, the Evidence and an Examination on Police Killings.” and Roberleigh Schuler. 1989. “Participation of Journal of Social Issues 50:53–74. Mexican American Female Adolescents in a Lon- Ball-Rokeach, Sandra J., Joel W. Grube, and Milton gitudinal Panel Survey.” Public Opinion Quarterly Rokeach. 1981. “Roots: The Next Generation—Who 53:548–62. Watched and with What Effect.” Public Opinion Quar- Asch, Solomon. 1958. “Effects of Group Pressure upon terly 45:58–68. the Modifi cation and Distortion of Judgments.” Barker, Kriss, and Miguel Sabido, eds. 2005. Soap Operas Pp. 174–83 in Readings in Social Psychology, 3rd ed., for Social Change to Prevent HIV/AIDS: A Training edited by Eleanor E. Maccoby et al. New York: Holt, Guide for Journalists and Media Personnel. Shelburne, Rinehart & Winston. VT: Population Media Center. Asher, Ramona M., and Gary Alan Fine. 1991. “Fragile Bart, Pauline, and Patricia O’Brien. 1985. Stopping Ties: Sharing Research Relationships with Women Rape: Successful Survival Strategies. New York: Married to Alcoholics.” Pp. 196–205 in Experienc- Pergamon. ing Fieldwork: An Inside View of Qualitative Research, Becker, Penny Edgell. 1998. “Making Inclusive Communi- edited by William B. Shaffi r and Roberta A. Stebbins. ties: Congregations and the ‘Problem’ of Race.” Social Thousand Oaks, CA: Sage. Problems 45 (4): 451–72.

525 526 REFERENCES

Bednarz, Marlene. 1996. “Push Polls Statement.” Report Blair, Johnny, Shanyang Zhao, Barbara Bickart, and to the AAPORnet listserv, April 5. http://www.aapor Ralph Kuhn. 1995. Sample Design for Household .org/ethics/pushpoll.html. Telephone Surveys: A Bibliography 1949–1995. College Belenky, Mary Field, Blythe McVicker Clinchy, Nancy Rule Park: Survey Research Center, University of Maryland. Goldberger, and Jill Mattuck Tarule. 1986. Women’s Blaunstein, Albert, and Robert Zangrando, eds. 1970. Ways of Knowing: The Development of Self, Voice, and Civil Rights and the Black American. New York: Wash- Mind. New York: Basic Books. ington Square Press. Bell, Derrick A. 1980. “Brown v. Board of Education and Blumenthal, Mark M. 2005. “Toward an Open-Source the Interest-Convergence Dilemma.” Harvard Law Methodology: What We Can Learn from the Blog- Review 93:518–33. sphere.” Public Opinion Quarterly [Special Issue] 69 Bellah, Robert N. 1957. Tokugawa Religion. Glencoe, IL: (5): 655–69. Free Press. Bolstein, Richard. 1991. “Comparison of the Likelihood to 1967. “Research Chronicle: Tokugawa Religion.” Vote among Preelection Poll Respondents and Nonre- Pp. 164–85 in Sociologists at Work, edited by spondents.” Public Opinion Quarterly 55:648–50. Phillip E. Hammond. Garden City, NY: Anchor Bottomore, T. B., and Maximilien Rubel, eds. [1843] 1956. Books. Karl Marx: Selected Writings in Sociology and Social 1970. “Christianity and Symbolic Realism.” Journal for Philosophy. Translated by T. B. Bottomore. New York: the Scientifi c Study of Religion 9:89–96. McGraw-Hill. 1974. “Comment on the Limits of Symbolic Real- Brown v. Board of Education of Topeka, 347 U.S. 483 ism.” Journal for the Scientifi c Study of Religion (1954). 13:487–89. Burawoy, M., A. Burton, A. A. Ferguson, K. J. Fox, J. Gam- Benton, J. Edwin, and John L. Daly. 1991. “A Question son, N. Gartrell, L. Hurst, C. Kurzman, L. Salzinger, Order Effect in a Local Government Survey.” Public J. Schiffman, and S. Ui, eds. 1991. Ethnography Un- Opinion Quarterly 55:640–42. bound: Power and Resistance in the Modern Metropolis. Berbrier, Mitch. 1998. “‘Half the Battle’: Cultural Reso- Berkeley: University of California Press. nance, Framing Processes, and Ethnic Affectations Campbell, Donald, and Julian Stanley. 1963. Experimental in Contemporary White Separatist Rhetoric.” Social and Quasi-Experimental Designs for Research. Chicago: Problems 45 (4): 431–50. Rand McNally. Berg, Bruce L. 1989. Qualitative Research Methods for the Campbell, M. L. 1998. “Institutional Ethnography and Ex- Social Sciences. Boston: Allyn and Bacon. perience as Data.” Qualitative Sociology 21 (1): 55–73. Beveridge, W. I. B. 1950. The Art of Scientifi c Investigation. Carpini, Michael, X. Delli, and Scott Keeter. 1991. “Stabil- New York: Vintage Books. ity and Change in the U.S. Public’s Knowledge of Bian, Yanjie. 1994. Work and Inequality in Urban China. Politics.” Public Opinion Quarterly 55:583–612. Albany: State University of New York Press. Carr, C. Lynn. 1998. “Tomboy Resistance and Conformity: Biddle, Stuart J. H., David Markland, David Gilbourne, Agency in Social Psychological Gender Theory.” Nikos L. D. Chatzisarantis, and Andrew C. Sparkes. Gender and Society 12 (5): 528–53. 2001. “Research Methods in Sport and Exercise Psy- Census Bureau See U.S. Bureau of the Census chology: Quantitative and Qualitative Issues.” Journal Chirot, Daniel, and Jennifer Edwards. 2003. “Making of Sports Sciences 19 (10): 777–809. Sense of the Senseless: Understanding Genocide.” Bielby, William T., and Denise Bielby. 1999. “Organiza- Contexts 2 (2): 12–19. tional Mediation of Project-Based Labor Markets: Chossudovsky, Michel. 1997. The Globalization of Poverty: Talent Agencies and the Careers of Screenwriters.” Impacts of IMF and World Bank Reforms. London: American Sociological Review 64:64–85. Zed Books. Birchfi eld, R. W. 1998. The New Fowler’s Modern English Clark, Roger, Rachel Lennon, and Leana Morris. 1993. Usage. 3rd ed. New York: Oxford University Press. “Of Caldecotts and Kings: Gendered Images in Recent Bishop, George, and Andrew Smith. 2001. “Response- American Children’s Books by Black and Non-Black Order Effects and the Early Gallup Split-Ballots.” Illustrators.” Gender and Society 7 (2): 227–45. Public Opinion Quarterly 65:479–505. CNN. 2006. http://www.cnn.com/2006/POLITICS/ Black, Donald. 1970. “Production of Crime Rates.” Ameri- 05/10/congress.poll/index.html. Accessed can Sociological Review 35 (Au gust): 733–48. May 11. REFERENCES 527

Coates, Rodney D. 2006. “Towards a Simple Typology of Replacement in Forty-two General Social Survey Racial Hegemony.” Societies without Borders 1:69–91. Items, 1972–1989.” Public Opinion Quarterly Coleman, James. 1966. Equality of Educational Oppor- 56:261–306. tunity. Washington, DC: U.S. Government Printing De Coster, Stacy. 2005. “Depression and Law Violation: Offi ce. Gendered Responses to Gendered Stresses.” Socio- Collins, G. C., and Timothy B. Blodgett. 1981. “Sexual Ha- logical Perspectives 48 (2): 155–87. rassment . . . Some See It . . . Some Won’t.” Harvard Defl em, Mathieu. 2002. Policing World Society: Historical Business Review, March–April, pp. 76–95. Foundations of International Police Cooperation. New Comstock, Donald. 1980. “Dimensions of Infl uence in York: Oxford University Press. Organizations.” Pacifi c Sociological Review, Janu ary, DeFleur, Lois. 1975. “Biasing Infl uences on Drug Arrest pp. 67–84. Records: Implications for Deviance Research.” Ameri- Conrad, Clifton F. 1978. “A Grounded Theory of Aca- can Sociological Review 40:88–103. demic Change.” Sociology of Education 51:101–12. Delgado, Richard. 2002. “Explaining the Rise and Fall of Cook, Thomas D., and Donald T. Campbell. 1979. Quasi- African American Fortunes—Interest Convergence Experimentation: Design and Analysis Issues for Field and Civil Rights Gains.” Harvard Civil Rights–Civil Settings. Chicago: Rand McNally. Liberties Law Review 37:369–87. Cooper-Stephenson, Cynthia, and Athanasios The- Dillman, Don A. 1978. Mail and Telephone Surveys: ologides. 1981. “Nutrition in Cancer: Physicians’ The Total Design Method. New York: Wiley. Knowledge, Opinions, and Educational Needs.” 1999. Mail and Telephone Surveys: The Tailored Journal of the American Dietetic Association, May, Design Method. 2nd ed. New York: Wiley. pp. 472–76. Doyle, Sir Arthur Conan. [1891] 1892. “A Scandal in Couper, Mick P. 2001. “Web Surveys: A Review of Issues Bohemia.” First published in The Strand, July 1891. and Approaches.” Public Opinion Quarterly 64:464–94. Reprinted in The Original Illustrated Sherlock Holmes, Craig, R. Stephen. 1992. “The Effect of Television Day pp. 11–25. Secaucus, NJ: Castle. Part on Gender Portrayals in Television Commercials: DuBois, W. E. B. 1903. The Souls of Black Folk. Chicago: A Content Analysis.” Sex Roles 26 (5/6): 197–211. McClurg. Crawford, Kent S., Edmund D. Thomas, and Jeffrey J. Durkheim, Emile. [1893] 1964. The Division of Labor in Fink. 1980. “Pygmalion at Sea: Improving the Society. Translated by George Simpson. New York: Work Effectiveness of Low Performers.” Journal Free Press. of Applied Behavioral Science, Oc tober–December, [1897] 1951. Suicide. Glencoe, IL: Free Press. pp. 482–505. Eastman, Crystal. 1910. “Work-Accidents and Employers’ Curtin, Richard, Stanley Presser, and Eleanor Singer. Liability.” The Survey, Sep tember 3, pp. 788–94. 2005. “Changes in Telephone Survey Nonresponse Ellison, Christopher G., and Darren E. Sherkat. 1990. over the Past Quarter Century.” Public Opinion Quar- “Patterns of Religious Mobility among Black Ameri- terly 69 (1): 87–98. cans.” Sociological Quarterly 31 (4): 551–68. Danieli, Ardha, and Carol Woodhams. 2005. “Eman- Emerson, Robert M., Kerry O. Ferris, and Carol Brooks cipatory Research Methodology and Disability: A Gardner. 1998. “On Being Stalked.” Social Problems Critique.” International Journal of Social Research 8 (4): 45 (3): 289–314. 281–96. Farquharson, Karen. 2005. “A Different Kind of Snowball: Davern, Michael, Todd H. Rockwood, Randy Sherrod, Identifying Key Policymakers.” International Journal of and Stephen Campbell. 2003. “Prepaid Monetary In- Social Research 8 (4): 345–53. centives and Data Quality in Face-to-Face Interviews: Fausto-Sterling, Anne. 1992. “Why Do We Know So Little Data from the 1996 Survey of Income and Program about Human Sex?” Discover Archives, June. http:// Participation Incentive Experiment.” Public Opinion cas.bellarmine.edu/tietjen/Human%20Nature%20S Quarterly 67:139–47. %201999/why_do_we_know_so_little_about_h.htm. Davis, Fred. 1973. “The Martian and the Convert: Onto- Festinger, L., H. W. Reicker, and S. Schachter. 1956. logical Polarities in Social Research.” Urban Life 2 (3): When Prophecy Fails. Minneapolis: University of 333–43. Minnesota Press. Davis, James. 1992. “Changeable Weather in a Cooling Fielding, Nigel. 2004. “Getting the Most from Archived Climate atop the Liberal Plateau: Conversion and Qualitative Data: Epistemological, Practical and 528 REFERENCES

Professional Obstacles.” International Journal of Social ing the Monopoly with Participatory Action-Research, Research Methodology 7 (1): 97–104. edited by O. Fals-Borda and M. A. Rahman. New Ford, David A. 1989. “Preventing and Provoking Wife York: Apex Press. Battery through Criminal Sanctioning: A Look at the Geertz, Clifford. 1973. The Interpretation of Cultures. New Risks.” Sep tember, unpublished manuscript. York: Basic Books. Ford, David A., and Mary Jean Regoli. 1992. “The Preven- Glaser, Barney, and Anselm Strauss. 1967. The Discovery tive Impacts of Policies for Prosecuting Wife Batter- of Grounded Theory: Strategies for Qualitative Research. ers.” Pp. 181–208 in Domestic Violence: The Changing Chicago: Aldine. Criminal Justice Response, edited by E. S. Buzawa and Glock, Charles Y., Benjamin B. Ringer, and Earl R. Bab- C. G. Buzawa. New York: Auburn. bie. 1967. To Comfort and to Challenge. Berkeley: Foschi, Martha, G. Keith Warriner, and Stephen D. Hart. University of California Press. 1985. “Standards, Expectations, and Interpersonal Goffman, Erving. 1961. Asylums: Essays on the Social Situ- Infl uence.” Social Psychology Quarterly 48 (2): 108–17. ation of Mental Patients and Other Inmates. Chicago: Fox, Katherine J. 1991. “The Politics of Prevention: Aldine. Ethnographers Combat AIDS among Drug Users.” 1963. Stigma: Notes on the Management of a Spoiled Pp. 227–49 in Ethnography Unbound: Power and Resis- Identity. Englewood Cliffs, NJ: Prentice-Hall. tance in the Modern Metropolis, edited by M. Burawoy, 1974. Frame Analysis. Cambridge, MA: Harvard Uni- A. Burton, A. A. Ferguson, K. J. Fox, J. Gamson, N. versity Press. Gartrell, L. Hurst, C. Kurzman, L. Salzinger, J. Schiff- 1979. Gender Advertisements. New York: Harper & man, and S. Ui. Berkeley: University of California Row. Press. Gottlieb, Bruce. 1999. “Cooking the School Books: How Frankel, Mark S., and Sanyin Siang. 1999. “Ethical and U.S. News Cheats in Picking Its ‘Best American Legal Aspects of Human Subjects Research on the Colleges.’” Slate, Au gust 31. http://www.slate.com/ Internet: A Report of a Workshop.” Washington, DC: crapshoot/99-08-31/crapshoot.asp. American Association for the Advancement of Sci- Graham, Laurie, and Richard Hogan. 1990. “Social Class ence, No vember. http://www.aaas.org/spp/dspp/ and Tactics: Neighborhood Opposition to Group sfrl/projects/intres/main.htm. Homes.” Sociological Quarterly 31 (4): 513–29. Gall, John. 1975. Systemantics: How Systems Work and Greenwood, Peter W., et al. 1994. Three Strikes and You’re Especially How They Fail. New York: Quadrangle. Out: Estimated Benefi ts and Costs of California’s New Gamson, William A. 1992. Talking Politics. New York: Mandatory-Sentencing Law. Santa Monica, CA: Rand Cambridge University Press. Corporation. Gans, Herbert. 1971. The Uses of Poverty: The Poor Pay Greenwood, Peter W., C. Peter Rydell, and Karyn Model. All. Social Policy, July–Au gust, pp. 20–24. 1996. Diverting Children from a Life of Crime: Mea- 2002. “More of Us Should Become Public Sociolo- suring Costs and Benefi ts. Santa Monica, CA: Rand gists.” Footnotes, July–Au gust (Washington, DC: Corporation. American Sociological Association). http://www Griffi th, Alison I. 1995. “Mothering, Schooling, and .asanet.org/footnotes/julyaugust02/fn10.html. Children’s Development.” Pp. 108–21 in Knowledge, Garant, Carol. 1980. “Stalls in the Therapeutic Process.” Experience, and Ruling Relations: Studies in the Social American Journal of Nursing, De cember, pp. 2166–67. Organization of Knowledge, edited by M. Campbell Gard, Greta, ed. 1993. Ecofeminism: Women, Animals, and A. Manicom. Toronto, Canada: University of Nature. Philadelphia: Temple University Press. Quoted Toronto Press. in Linda J. Rynbrandt and Mary Jo Deegan. 2002. “The Gubrium, Jaber F., and James A. Holstein. 1997. The New Ecofeminist Pragmatism of Caroline Bartlett Crane, Language of Qualitative Method. New York: Oxford 1896–1935.” American Sociologist 33 (3): 60. University Press. Garfi nkel, H. 1967. Studies in Ethnomethodology. Engle- Hawking, Stephen. 2001. The Universe in a Nutshell. New wood Cliffs, NJ: Prentice-Hall. York: Bantam Books. Gaventa, J. 1991. “Towards a Knowledge Democracy: Hedrick, Terry E., Leonard Bickman, and Debra J. Rog. Viewpoints on Participatory Research in North 1993. Applied Research Design: A Practical Guide. America.” Pp. 121–31 in Action and Knowledge: Break- Thousand Oaks, CA: Sage. REFERENCES 529

Hempel, Carl G. 1952. “Fundamentals of Concept Forma- Irwin, John, and James Austin. 1997. It’s About Time: tion in Empirical Science.” International Encyclopedia America’s Imprisonment Binge. Belmont, CA: of United Science II, no. 7. Wadsworth. Heritage, J. 1984. Garfi nkel and Ethnomethodology. Cam- Isaac, Larry W., and Larry J. Griffi n. 1989. “A Historicism bridge: Polity Press. in Time-Series Analyses of Historical Process: Heritage, John, and David Greatbatch. 1992. “On the Critique, Redirection, and Illustrations from U.S. Institutional Character of Institutional Talk.” In Talk at Labor History.” American Sociological Association Work, edited by P. Drew and J. Heritage. Cambridge, 54:873–90. England: Cambridge University Press. Jackman, Mary R., and Mary Scheuer Senter. 1980. Higginbotham, A. Leon, Jr. 1978. In the Matter of Color: “Images of Social Groups: Categorical or Qualifi ed?” Race and the American Legal Process. New York: Public Opinion Quarterly 44:340–61. Oxford University Press. Jackson, Jonathan. 2005. “Validating New Measures Hill, Lewis. 2000. Yankee Summer: The Way We Were: of the Fear of Crime.” International Journal of Social Growing up in Rural Vermont in the 1930s. Blooming- Research 8 (4): 297–315. ton, IN: First Books Library. Jacobs, Bruce A., and Jody Miller. 1998. “Crack Dealing, Hilts, Philip J. 1981. “Values of Driving Classes Disputed.” Gender, and Arrest Avoidance.” Social Problems 45 San Francisco Chronicle, June 25, p. 4. (4): 550–69. Hogan, Richard, and Carolyn C. Perrucci. 1998. “Produc- Jasso, Guillermina. 1988. “Principles of Theoretical ing and Reproducing Class and Status Differences: Analysis.” Sociological Theory 6:1–20. Racial and Gender Gaps in U.S. Employment Jensen, Arthur. 1969. “How Much Can We Boost IQ and and Retirement Income.” Social Problems 45 (4): Scholastic Achievement?” Harvard Educational Review 528–49. 39:273–74. Howard, Edward N., and Darlene M. Norman. 1981. Jobes, Patrick C., Andra Aldea, Constantin Cernat, Ioana- “Measuring Public Library Performance.” Library Minerva Icolisan, Gabriel Iordache, Sabastian Lazeru, Journal, Feb ruary, pp. 305–8. Catalin Stoica, Gheorghe Tibil, and Eugenia Udangiu. Howell, Joseph T. 1973. Hard Living on Clay Street. Garden 1997. “Shopping as a Social Problem: A Grounded City, NY: Doubleday Anchor. Theoretical Analysis of Experiences among Roma- Huberman, A. Michael, and Matthew B. Miles. 1994. nian Shoppers.” Journal of Applied Sociology 14 (1): “Data Management and Analysis Methods.” Pp. 428– 124–46. 44 in Handbook of Qualitative Research, edited by Johnson, Jeffrey C. 1990. Selecting Ethnographic Infor- Norman K. Denzin and Yvonna S. Lincoln. Thousand mants. Thousand Oaks, CA: Sage. Oaks, CA: Sage. Johnston, Hank. 1980. “The Marketed Social Movement: Hughes, Michael. 1980. “The Fruits of Cultivation Analy- A Case Study of the Rapid Growth of TM.” Pacifi c sis: A Reexamination of Some Effects of Television Sociological Review, July, pp. 333–54. Watching.” Public Opinion Quarterly 44:287–302. Johnston, Hank, and David A. Snow. 1998. “Subcultures Humphreys, Laud. 1970. Tearoom Trade: Impersonal Sex and the Emergence of the Estonian Nationalist Op- in Public Places. Chicago: Aldine. position 1945–1990.” Sociological Perspectives 41 (3): Hurst, Leslie. 1991. “Mr. Henry Makes a Deal.” Pp. 183– 473–97. 202 in Ethnography Unbound: Power and Resistance in Jones, James H. 1981. Bad Blood: The Tuskegee Syphilis the Modern Metropolis, edited by M. Burawoy, A. Bur- Experiments. New York: Free Press. ton, A. A. Ferguson, K. J. Fox, J. Gamson, N. Gartrell, Kaplan, Abraham. 1964. The Conduct of Inquiry. San L. Hurst, C. Kurzman, L. Salzinger, J. Schiffman, and Francisco: Chandler. S. Ui. Berkeley: University of California Press. Kasl, Stanislav V., Rupert F. Chisolm, and Brenda Eske- Iannacchione, Vincent G., Jennifer M. Staab, and David nazi. 1981. “The lmpact of the Accident at Three Mile T. Redden. 2003. “Evaluating the Use of Residential Island on the Behavior and Well-Being of Nuclear Mailing Addresses in a Metropolitan Household Sur- Workers.” American Journal of Public Health, May, vey.” Public Opinion Quarterly 67:202–10. pp. 472–95. Ibrahim, Saad Eddin. 2003. “Letter from Cairo.” Contexts Kasof, Joseph. 1993. “Sex Bias in the Naming of Stimulus 2 (2): 68–72. Persons.” Psychological Bulletin 113 (1): 140–63. 530 REFERENCES

Kebede, Alemseghed, and J. David Knottnerus. 1998. Lee, Motoko Y., Stephen G. Sapp, and Melvin C. Ray. “Beyond the Pales of Babylon: The Ideational Com- 1996. “The Reverse Social Distance Scale.” Journal of ponents and Social Psychological Foundations of Social Psychology 136 (1): 17–24. Rastafari.” Sociological Perspectives 42 (3): 499–517. Lengermann, Patricia Madoo, and Jill Niebrugge- Keeter, Scott. 2006. “The Impact of Cell Phone Noncover- Brantley. 2002. “Back to the Future: Settlement age Bias on Polling in the 2004 Presidential Election.” Sociology, 1885–1930.” American Sociologist 33 (3): Public Opinion Quarterly 70 (1): 88–98. 5–20. Kentor, Jeffrey. 2001. “The Long Term Effects of Global- Lever, Janet. 1986. “Sex Differences in the Complexity of ization on Income Inequality, Population Growth, Children’s Play and Games.” Pp. 74–89 in Structure and Economic Development.” Social Problems 48 (4): and Process, edited by Richard J. Peterson and Char- 435–55. lotte A. Vaughan. Belmont, CA: Wadsworth. Khayatt, Didi. 1995. “Compulsory Heterosexuality: Lewins, Ann, and Christina Silver. 2006. “Choosing a Schools and Lesbian Students.” Pp. 149–63 in Knowl- CAQDAS Package.” July. http://caqdas.soc.surrey edge, Experience, and Ruling Relations: Studies in the .ac.uk/. Social Organization of Knowledge, edited by M. Camp- Libin, A., and J. Cohen-Mansfi eld. 2000. “Individual bell and A. Manicom. Toronto, Canada: University of versus Group Approach for Studying and Interven- Toronto Press. ing with Demented Elderly Persons: Methodological Kilburn, John C., Jr. 1998. “It’s a Matter of Defi nition: Di- Perspectives.” Gerontologist, Oc tober 15, p. 105. lemmas of Service Delivery and Organizational Struc- Linton, Ralph. 1937. The Study of Man. New York: ture in a Growing Voluntary Organization.” Journal of D. Appleton-Century. Applied Sociology 15 (1): 89–103. Literary Digest. 1936a. “Landon, 1,293,669: Roosevelt, Kinnell, Ann Marie, and Douglas W. Maynard. 1996. “The 972,897.” Oc tober 31, pp. 5–6. Delivery and Receipt of Safer Sex Advice in Pretest 1936b. “What Went Wrong with the Polls?” Novem- Counseling Sessions for HIV and AIDS.” Journal of ber 14, pp. 7–8. Contemporary Ethnography 24:405–37. Lofl and, John. 2003. Demolishing a Historic Hotel. Davis, Kinsey, Alfred C., et al. 1948. Sexual Behavior in the Hu- CA: Davis Research. man Male. Philadelphia: W. B. Saunders. Lofl and, John, and Lyn H. Lofl and. 1995. Analyzing Social 1953. Sexual Behavior in the Human Female. Philadel- Settings: A Guide to Qualitative Observation and Analy- phia: W. B. Saunders. sis. 3rd. ed. Belmont, CA: Wadsworth. Kish, Leslie. 1965. Survey Sampling. New York: Wiley. Lofl and, John, David Snow, Leon Anderson, and Lyn H. Krueger, Richard A. 1988. Focus Groups. Thousand Oaks, Lofl and. 2006. Analyzing Social Settings: A Guide to CA: Sage. Qualitative Observation and Analysis. 4th ed. Belmont, Kubrin, Charis E. 2005. “I See Death around the Corner: CA: Wadsworth. Nihilism in Rap Music.” Sociological Perspectives 48 Lynd, Robert S., and Helen M. Lynd. 1929. Middletown. (4): 433–59. New York: Harcourt, Brace. Kubrin, Charis E., and Ronald Weitzer. 2003. “Retaliatory 1937. Middletown in Transition. New York: Harcourt, Homicide: Concentrated Disadvantage and Neighbor- Brace. hood Culture.” Social Problems 50 (2): 157–80. Madison, Anna-Marie. 1992. “Primary Inclusion of Kuhn, Thomas. 1970. The Structure of Scientifi c Revolu- Culturally Diverse Minority Program Participants in tions. Chicago: University of Chicago Press. the Evaluation Process.” New Directions for Program Kvale, Steinar. 1996. InterViews: An Introduction to Evaluation, no. 53, pp. 35–43. Qualitative Research Interviewing. Thousand Oaks, CA: Mahoney, James, and Dietrich Rueschemeyer, eds. 2003. Sage. Comparative Historical Analysis in the Social Sciences. Lakoff, George. 2002. Moral Politics: How Liberals and New York: Cambridge University Press. Conservatives Think. Chicago: University of Chicago Manning, Peter K., and Betsy Cullum-Swan. 1994. “Nar- Press. rative, Content, and Semiotic Analysis.” Pp. 463–77 in Laumann, Edward O., John H. Gagnon, Robert T. Michael, Handbook of Qualitative Research, edited by Norman and Stuart Michaels. 1994. The Social Organization of K. Denzin and Yvonna S. Lincoln. Thousand Oaks, Sexuality. Chicago: University of Chicago Press. CA: Sage. REFERENCES 531

Marshall, Catherine, and Gretchen B. Rossman. 1995. Morgan, David L., ed. 1993. Successful Focus Groups: Ad- Designing Qualitative Research. Thousand Oaks, CA: vancing the State of the Art. Thousand Oaks, CA: Sage. Sage. Morgan, Lewis H. 1870. Systems of Consanguinity and Af- Marx, Karl. [1867] 1967. Capital. New York: International fi nity. Washington, DC: Smithsonian Institution. Publishers. Moskowitz, Milt. 1981. “The Drugs That Doctors Order.” [1880] 1956. Revue Socialist, July 5. Reprinted in Karl San Francisco Chronicle, May 23, p. 33. Marx: Selected Writings in Sociology and Social Phi- Moynihan, Daniel. 1965. The Negro Family: The Case for losophy, edited by T. B. Bottomore and Maximilien National Action. Washington, DC: U.S. Government Rubel. New York: McGraw-Hill. Printing Offi ce. McAlister, Alfred, Cheryl Perry, Joel Killen, Lee Ann Myrdal, Gunnar. 1944. An American Dilemma. New York: Slinkard, and Nathan Maccoby. 1980. “Pilot Study Harper & Row. of Smoking, Alcohol, and Drug Abuse Prevention.” Naisbitt, John, and Patricia Aburdene. 1990. Megatrends American Journal of Public Health, July, pp. 719–21. 2000: Ten New Directions for the 1990’s. New York: McGrane, Bernard. 1994. The Un-TV and the 10 mph Car: Morrow. Experiments in Personal Freedom and Everyday Life. Nature Conservancy. 2005. “The Nature of Science on the Fort Bragg, CA: The Small Press. Sacramento River.” Nature Conservancy [newsletter], Meadows, Donella H., Dennis L. Meadows, and Jørgen Spring–Summer. Randers. 1992. Beyond the Limits: Confronting Global Neuman, W. Lawrence. 1998. “Negotiated Meanings and Collapse, Envisioning a Sustainable Future. Post Mills, State Transformation: The Trust Issue in the Progres- VT: Chelsea Green. sive Era.” Social Problems 45 (3): 315–35. Meadows, Donella H., Dennis L. Meadows, Jørgen Rand- Nicholls, William L., II, Reginald P. Baker, and Jean Mar- ers, and William W. Behrens, III. 1972. The Limits to tin. 1996. “The Effect of New Data Collection Technol- Growth. New York: Signet Books. ogy on Survey Data Quality.” In Survey Measurement Menjívar, Cecilia. 2000. Fragmented Ties: Salvadoran Im- and Process Quality, edited by L. Lyberg, P. Biemer, M. migrant Networks in America. Berkeley: University of Collins, C. Dippo, N. Schwarz, and D. Trewin. New California Press. York: Wiley. Merton, Robert K. 1938. “Social Structure and Anomie.” “1 in 5 in New Survey Express Some Doubt about the American Sociological Review 3:672–82. Holocaust.” 1993. New York Times, April 2, p. A12. Milgram, Stanley. 1963. “Behavioral Study of Obedience.” O’Neill, Harry W. 1992. “They Can’t Subpoena What You Journal of Abnormal Social Psychology 67:371–78. Ain’t Got.” AAPOR News 19 (2): 4, 7. 1965. “Some Conditions of Obedience and Onwuegbuzie, Anthony J., and Nancy L. Leech. 2005. “On Disobedience to Authority.” Human Relations Becoming a Pragmatic Researcher: The Importance 18:57–76. of Combining Quantitative and Qualitative Research Miller, Delbert. 1991. Handbook of Research Design and Methodologies.” International Journal of Research Social Measurement. Thousand Oaks, CA: Sage. Methodology 8 (5): 375–87. Mirola, William A. 2003. “Asking for Bread, Receiving a Överlien, Carolina, Karin Aronsson, and Margareta Stone: The Rise and Fall of Religious Ideologies in Hydén. 2005. “The Focus Group Interview as an Chicago’s Eight-Hour Movement.” Social Problems 50 In-Depth Method? Young Women Talking about (2): 273–93. Sexuality.” International Journal of Social Research 8 Mitchell, Richard G., Jr. 1991. “Secrecy and Disclosure in (4): 331–44. Field Work.” Pp. 97–108 in Experiencing Fieldwork: An Perinelli, Phillip J. 1986. “Nonsuspecting Public in TV Inside View of Qualitative Research, edited by William Call-in Polls.” New York Times, Feb ruary 14, letter to B. Shaffi r and Robert A. Stebbins. Thousand Oaks, the editor. CA: Sage. Perrow, Charles. 2002. Organizing America: Wealth, 2002. Dancing at Armageddon: Survivalism and Chaos Power, and the Origins of Corporate Capitalism. Prince- in Modern Times. Chicago: University of Chicago ton, NJ: Princeton University Press. Press. Petersen, Larry R., and Judy L. Maynard. 1981. “Income, Mitofsky, Warren J. 1999. “Miscalls Likely in 2000.” Public Equity, and Wives’ Housekeeping Role Expectations.” Perspective 10 (5): 42–43. Pacifi c Sociological Review, Janu ary, pp. 87–105. 532 REFERENCES

Picou, J. Steven. 1996a. “Compelled Disclosure of Rogers, Everett M., Peter W. Vaughan, Ramadhan M. Scholarly Research: Some Comments on High Stakes A. Swalehe, Nagesh Rao, and Suruchi Sood. 1996. Litigation.” Law and Contemporary Problems 59 (3): “Effects of an Entertainment-Education Radio Soap 149–57. Opera on Family Planning and HIV/AIDS Prevention 1996b. “Sociology and Compelled Disclosure: Pro- Behavior in Tanzania.” Report presented at a techni- tecting Respondent Confi dentiality.” Sociological cal briefi ng on the Tanzania Entertainment-Education Spectrum 16 (3): 207–38. Project, Rockefeller Foundation, New York, March 27. Plutzer, Eric, and Michael Berkman. 2005. “The Graying Roper, Burns. 1992. “. . . But Will They Give the Poll Its of America and Support for Funding the Nation’s Sup- Due?” AAPOR News 19 (2): 5–6. port.” Public Opinion Quarterly 69:66–86. Rosenberg, Morris. 1968. The Logic of Survey Analysis. Polivka, Anne E., and Jennifer M. Rothgeb. 1993. “Re- New York: Basic Books. designing the CPS Questionnaire.” Monthly Labor Rosenthal, Robert, and Lenore Jacobson. 1968. Pygma- Review 116 (9): 10–28. lion in the Classroom. New York: Holt, Rinehart & “Poll on Doubt of Holocaust Is Corrected.” 1993. New Winston. York Times, July 8, p. A7. Ross, Jeffrey Ian. 2004. “Taking Stock of Research Meth- Population Communications International. 1996. Interna- ods and Analysis on Oppositional Political Terrorism.” tional Dateline [Feb ruary]. New York: Author. American Sociologist 35 (2): 26–37. Powell, Elwin H. 1958. “Occupation, Status, and Suicide: Rossman, Gabriel. 2002. “The Qualitative Infl uence of Toward a Redefi nition of Anomie.” American Sociolog- Ownership on Media Content: The Case of Movie ical Review 23 (4): 131–39. Reviews.” Paper presented to the American Socio- Presser, Stanley, and Johnny Blair. 1994. “Survey Pretest- logical Association, Chicago. Reported in Contexts 2 ing: Do Different Methods Produce Different Results?” (2, Spring 2003): 7. Pp. 73–104 in Sociological Methodology 1994, edited Rothman, Ellen K. 1981. “The Written Record.” Journal of by Peter Marsden. San Francisco: Jossey-Bass. Family History, Spring, pp. 47–56. Prewitt, Kenneth. 2003. “Partisan Politics in the 2000 U.S. Rubin, Herbert J., and Riene S. Rubin. 1995. Qualitative Census.” Population Reference Bureau, No vember. Interviewing: The Art of Hearing Data. Thousand Oaks, http://www.prb.org/Template.cfm?SectionPRB&te CA: Sage. mplate/Content/ContentGroups/Articles/03/ Sacks, Jeffrey J., W. Mark Krushat, and Jeffrey Newman. Partisan_Politics_in_the_2000_U_S__Census.htm. 1980. “Reliability of the Health Hazard Appraisal.” Quoss, Bernita, Margaret Cooney, and Terri Longhurst. American Journal of Public Health, July, pp. 730–32. 2000. “Academics and Advocates: Using Participatory Sanders, William B. 1994. Gangbangs and Drive-bys: Action Research to Infl uence Welfare Policy.” Journal Grounded Culture and Juvenile Gang Violence. New of Consumer Affairs 34 (1): 47–61. York: Aldine De Gruyter. Ragin, Charles C., and Howard S. Becker. 1992. What Is Scarce, Rik. 1990. Ecowarriors: Understanding the Radical a Case? Exploring the Foundations of Social Inquiry. Environmental Movement. Chicago: Noble Press, 1990. Cambridge, England: Cambridge University Press. 1999. “Good Faith, Bad Ethics: When Scholars Go the Rasinski, Kenneth A. 1989. “The Effect of Question Word- Distance and Scholarly Associations Do Not.” ing on Public Support for Government Spending.” Law and Social Inquiry: Journal of the American Public Opinion Quarterly 53:388–94. Bar Foundation 24 (4): 977–86. Redfi eld, Robert. 1941. The Folk Culture of Yucatan. Chi- Schifl ett, Kathy L., and Mary Zey. 1990. “Comparison of cago: University of Chicago Press. Characteristics of Private Product Producing Organi- Reinharz, Shulamit. 1992. Feminist Methods in Social zations and Public Service Organizations.” Sociologi- Research. New York: Oxford University Press. cal Quarterly 31 (4): 569–83. Riecken, Henry W., and Robert F. Boruch. 1974. Social Schmitt, Frederika E., and Patricia Yancey Martin. 1999. Experimentation: A Method for Planning and Evaluating “Unobtrusive Mobilization by an Institutionalized Social Intervention. New York: Academic Press. Rape Crisis Center: ‘All We Do Comes from Victims.’” Roethlisberger, F. J., and W. J. Dickson. 1939. Manage- Gender and Society 13 (3): 364–84. ment and the Worker. Cambridge, MA: Harvard Schutz, Alfred. 1967. The Phenomenology of the Social University Press. World. Evanston, IL: Northwestern University Press. REFERENCES 533

1970. On Phenomenology and Social Relations. Chi- Legislation in Illinois.” Criminal Justice Policy Review 4 cago: University of Chicago Press. (1): 1–18. Sense, Andrew J. 2006. “Driving the Bus from the Rear Srole, Leo. 1956. “Social Integration and Certain Corol- Passenger Seat: Control Dilemmas of Participative laries: An Exploratory Study.” American Sociological Action Research.” International Journal of Social Re- Review 21:709–16. search Methodology 9 (1): 1–13. Stark, Rodney. 1997. The Rise of Christianity: How the Shafﬁ r, William B., and Robert A. Stebbins, eds. 1991. Obscure, Marginal Jesus Movement Became the Domi- Experiencing Fieldwork: An Inside View of Qualitative nant Religious Force in the Western World in a Few Research. Thousand Oaks, CA: Sage. Centuries. San Francisco: HarperCollins. Shea, Christopher. 2000. “Don’t Talk to the Humans: Stearns, Cindy A. 1999. “Breastfeeding and the Good The Crackdown on Social Science Research.” Lingua Maternal Body.” Gender and Society 13 (3): 308–26. Franca 10 (6): 27–34. Strang, David, and James N. Baron. 1990. “Categorical Sherif, Muzafer. 1935. “A Study of Some Social Factors in Imperatives: The Structure of Job Titles in Califor- Perception.” Archives of Psychology 27:1–60. nia State Agencies.” American Sociological Review Silverman, David. 1993. Interpreting Qualitative Data: 55:479–95. Methods for Analyzing Talk, Text, and Interaction. Strauss, Anselm, and Juliet Corbin. 1994. “Grounded Thousand Oaks, CA: Sage. Theory Methodology: An Overview.” Pp. 273–85 in Silverman, George. 2005. “Qualitative Research: Face- Handbook of Qualitative Research, edited by Norman to-Face Focus Groups, Telephone Focus Groups, K. Denzin and Yvonna S. Lincoln. Thousand Oaks, Online Focus Groups.” http://www.mnav.com/ CA: Sage. qualitative_research.htm. Accessed June 1. 1998. Basics of Qualitative Research: Techniques and Singhal, Arvind, and Elizabeth Rattine-Flaherty. 2006. Procedures for Developing Grounded Theory. Thou- “Pencils and Photos as Tools of Communicative sand Oaks, CA: Sage. Research and Praxis: Analyzing Minga Peru’s Quest Strunk, William, Jr., and E. B. White. 1999. The Elements for Social Justice in the Amazon.” International Com- of Style. 4th ed. Boston: Allyn and Bacon. munication Gazette 68 (4): 313–30. Survey Sampling, Inc. 2000. “Increase Response Rates Skedsvold, Paula. 2002. “New Developments Concerning of Online Sampling.” November, http://www.world Public Use Data Files.” Footnotes 30 (1): 3. opinion.com/the_frame/frame4.html. Skocpol, Theda. 2003. Diminished Democracy: From Swalehe, Ramadhan, Everett M. Rogers, Mark J. Gil- Membership to Management in American Civic Life. board, Krista Alford, and Rima Montoya. 1995. A Norman: Oklahoma University Press. Content Analysis of the Entertainment-Education Radio Smith, Andrew E., and G. F. Bishop. 1992. The Gallup Se- Soap Opera ‘Twende na Wakati’ (Let’s Go with the cret Ballot Experiments: 1944–1988. Paper presented at Times) in Tanzania. Arusha, Tanzania: Population the annual conference of the American Association Family Life and Education Programme (POFLEP), for Public Opinion Research, St. Petersburg, FL, May. Ministry of Community Development, Women Affairs, Smith, Dorothy E. 1978. The Everyday World as Problem- and Children, No vember 15. atic. Boston: Northeastern University Press. Takeuchi, David. 1974. “Grass in Hawaii: A Structural Smith, Vicki. 1998. “The Fractured World of the Tempo- Constraints Approach.” M.A. thesis, University of rary Worker: Power, Participation, and Fragmentation Hawaii. in the Contemporary Workplace.” Social Problems 45 Tan, Alexis S. 1980. “Mass Media Use, Issue Knowledge (4): 411–30. and Political Involvement.” Public Opinion Quarterly Snow, David A., and Anderson, Leon. 1987. “Identity 44:241–48. Work among the Homeless: The Verbal Construction Tandon, Rajesh, and L. Dave Brown. 1981. “Organiza- and Avowal of Personal Identities.” American Journal tion-Building for Rural Development: An Experi- of Sociology 92:1336–71. ment in India.” Journal of Applied Behavioral Science, Sorokin, Pitirim A. 1937–1940. Social and Cultural Dynam- April–June, pp. 172–89. ics. 4 vols. Englewood Cliffs, NJ: Bedminster Press. Taylor, Humphrey, and George Terhanian. 1999. “Heady Spohn, Cassie, and Julie Horney. 1990. “A Case of Un- Days Are Here Again: Online Polling Is Rapidly Com- realistic Expectations: The Impact of Rape Reform ing of Age.” Public Perspective 10 (4): 20–23. 534 REFERENCES

Thomas, W. I., and Florian Znaniecki. 1918. The Polish Warner, W. Lloyd. 1949. Democracy in Jonesville. New Peasant in Europe and America. Chicago: University of York: Harper. Chicago Press. Webb, Eugene J., Donald T. Campbell, Richard D. Trials of War Criminals before the Nuremberg Military Tri- Schwartz, and Lee Sechrest. 2000. Unobtrusive Mea- bunals under Control Council Law No. 10. Nuremberg, sures. Rev. ed. Thousand Oaks, CA: Sage. October 1946–April 1949. 1949–1953. Washington, Weber, Max. [1905] 1958. The Protestant Ethic and the DC: U.S. Government Printing Offi ce. http://www Spirit of Capitalism. Translated by Talcott Parsons. .ushmm.org/research/doctors/indiptx.htm. New York: Scribner. Tuckel, Peter S., and Barry M. Feinberg. 1991. “The [1925] 1946. “Science as a Vocation.” Pp. 129–56 in Answering Machine Poses Many Questions for Tele- From Max Weber: Essays in Sociology, edited and phone Survey Researchers. Public Opinion Quarterly translated by Hans Gerth and C. Wright Mills. New 55:200–217. York: Oxford University Press. Tuckel, Peter, and Harry O’Neill. 2002. “The Vanishing [1934] 1951. The Religion of China. Translated by Hans Respondent in Telephone Surveys.” Journal of Adver- H. Gerth. New York: Free Press. tising Research, Sep tember–October, pp. 26–48. [1934] 1952. Ancient Judaism. Translated by Hans H. Turk, Theresa Guminski. 1980. “Hospital Support: Gerth and Don Martindale. New York: Free Press. Urban Correlates of Allocation Based on Organi- [1934] 1958. The Religion of India. Translated by Hans zational Prestige.” Pacifi c Sociological Review, July, H. Gerth and Don Martindale. New York: Free pp. 315–32. Press. Union of Concerned Scientists. 2005. “Political Interfer- Weiss, Carol. 1972. Evaluation Research. Englewood ence in Science.” Oc tober 23. http://www.ucsusa Cliffs, NJ: Prentice-Hall. .org/scientifi c_integrity/interference/. Weitzman, Lenore J., Deborah Eifl er, Elizabeth Hokada, United Nations. 1995. “Human Development Report and Catherine Ross. 1972. “Sex-Role Socialization 1995.” New York: United Nations Development Pro- in Picture Books for Preschool Children.” American gram. [Summarized in Population Communications Journal of Sociology 77:1125–50. International. 1996. International Dateline, Feb ruary, Wharton, Amy S., and James N. Baron. 1987. “So Happy pp. 1–4.] Together? The Impact of Gender Segregation on Men U.S. Bureau of the Census. 2001. Statistical Abstract of at Work.” American Sociological Review 52:574–87. the United States. Washington, DC: U.S. Government White, William S. 1997. Communication to APP-SOC list- Printing Offi ce. serv ([email protected]) from wwite@jaguar1 2006. Statistical Abstract of the United States. Washing- .usouthal.edu, Oc tober 11. ton, DC: U.S. Government Printing Offi ce. Whyte, W. F. 1943. Street Corner Society. Chicago: Univer- U.S. News and World Report. 1999. “America’s Best Col- sity of Chicago Press. leges.” Au gust 30. Whyte, W. F., D. J. Greenwood, and P. Lazes. 1991. “Partici- Veroff, Joseph, Shirley Hatchett, and Elizabeth Douvan. patory Action Research: Through Practice to Science 1992. “Consequences of Participating in a Longitu- in Social Research.” Pp. 19–55 in Participatory Action dinal Study of Marriage.” Public Opinion Quarterly Research, edited by W. F. Whyte. New York: Sage. 56:325–27. Wieder, D. L. 1988. Language and Social Reality: The Case Votaw, Carmen Delgado. 1979. Women’s Rights in the of Telling the Convict Code. Landman, MD: University United States. United States Commission on Civil Press of America. Rights, Inter-American Commission on Women. Wilson, Camilo. 1999. Private email, Sep tember 8. Washington, DC: Clearinghouse Publications. Wilson, Edward O. 1975. Sociobiology: The New Synthesis. Walker, Jeffery T. 1994. “Fax Machines and Social Sur- Cambridge, MA: Harvard University Press. veys: Teaching an Old Dog New Tricks.” Journal of Yammarino, Francis J., Steven J. Skinner, and Terry L. Quantitative Criminology 10 (2): 181–88. Childers. 1991. “Understanding Mail Survey Response Walker Research. 1988. Industry Image Study. 8th ed. Behavior: A Meta-Analysis.” Public Opinion Quarterly Indianapolis, IN: Walker Research. 55:613–39. Wallace, Walter. 1971. The Logic of Science in Sociology. Yinger, J. Milton, et al. 1977. Middle Start: An Experiment Chicago: Aldine-Atherton. in the Educational Enrichment of Young Adolescents. Ward, Lester. 1906. Applied Sociology. Boston: Ginn. London: Cambridge University Press. INDEX

A Authority, 8 Abstracts, 472 Autoethnography, 322 Accuracy in measurement, 156–157 Auto-kinetic effects, 44 Ackroyd, Dan, 260 Average, 450 Aggregates, 15–16, 104 Axial coding, 423 American Almanac, The, 367 American Democracy (Skocpol), 374 B Aminzade, Ron, 375–376 Babbie, Earl R., 50, 69, 84, 86 Analysis of data, 122, 124–125 Bailey, William, 406–407 ethics, 73–74 Ball-Rokeach, Sandra, 264 qualitative data analysis. See Qualitative data analysis. Baron, James, 180 quantitative data analysis. See Quantitative data Bart, Pauline, 395 analysis. Basics of Qualitative Research (Strauss/Corbin), 418 reading social research, 478 Becker, Howard, 326 Analysis of existing statistics, 362–369 Belenky, Mary Field, 41 Durkheim’s study of suicide, 362–364 Bell, Derrick, 42 globalization, consequences of, 364–365 Bellah, Robert, 319, 372–373, 375, 378 reading social research, 477 Benton, J. Edwin, 281 reliability, 366 Berg, Bruce, 169, 353, 357, 359 sources of existing statistics, 366–369 Berkman, Michael, 114 units of analysis, 365 Best American Colleges, 181 validity in, 365 Beveridge, W. I. B., 48 Analytic induction, 359 Bian, Yanjie, 111, 274 Analyzing Social Settings (Loﬂ and), 315 Bias Anderson, Eric, 322 in sampling, 208–209 Anderson, Leon, 321, 322 survey research, 277–278 Anderson, W., 10 Biddle, Stuart, 26 Andorka, Rudolf, 386 Bielby, Denise, 153 Anomic suicide, 364 Bielby, William, 153 Anomie, 142–144, 364 Bin Laden, Osama, 39 Anonymity, 69–70 Birthrates, 15–16 Applied research, 27–28 Bishop, G. F., 274 Aronsson, Karin, 339 Bivariate analysis, 459–463 Articles, 487 Bivariate relationships, 174–177 Asch Experiment, 42–43 Black, Donald, 366 Asher, Ramona, 112 Blair, Johnny, 283, 302 Attributes, 17, 149 Blodgett, Timothy, 237 Attribution process, 260 Blumenthal, Mark, 301 Auster, Carol, 359 Bogardus, Emory, 187

535 536 INDEX

Bogardus social distance scale, 186–187 Cohort studies, 113–114 Bollen, Kenneth, 175 Coleman, James, 83–84 Bolstein, Richard, 289 Collins, G. C., 237 Book of Leviticus, 424–426, 429–433 Comfort Hypothesis, 50–51, 57, 69, 459, 463 Books in Print, 498 Communication. See Reading social research. Boruch, Robert, 388 Comparative and historical research, 369–378 Branch Davidians, 322 analytical techniques, 376–378 Brown, L. Dave, 395 defi ned, 369 Brown vs. Board of Education of Topeka, 42, 83 examples of, 369–374 Budget in research proposals, 125 reading social research, 477 Burawoy, Michael, 326 sources of, 374–376 Burneson, Ray, 402 Computer-assisted personal interviewing (CAPI), 299 Bush, George W., 200, 201 Computer-assisted self-interviewing (CASI), 299 Computer-assisted telephone interviewing (CATI), 297, C 298 Calvin, John, 372 Computerized library fi les, 501–503 Campbell, Donald, 253, 254, 256, 257, 258, 259 Computerized Physician Order Entry (CPOE) systems, 7 Campbell, M. L., 328 Computerized self-administered questionnaire (CSAQ), CAPI, 299 299 Carpini, Michael, 112 Computer programs for qualitative data analysis, 428–438 Carr, C. Lynn, 116 Computer self-administered questionnaires, 299 Carroll, Lewis, 46 Computer simulations, 407–408 Case-oriented analysis, 417 Comte, Au guste, 36, 39, 42, 369, 370 Case studies, 326–328 Conception measurements, 133–134 CASI, 299 Concept mapping, 427–428 CATI, 297, 298 Concept measurements, 133–136 Causal reasoning, 6 Conceptualization, 131–132, 136 Causation, 19 anomie, 142–144 Cause and effect indicators, 175 in content analysis, 357 Census, U. S., 85–86 creating conceptual order, 140–142 Childers, Terry, 289 defi nitions in descriptive and explanatory studies, Chirot, Daniel, 138 145–147 Chisolm, Rupert, 263 dimensions, 137–139 Chi square distribution, 508–509 example of, 142–144 Chossudovsky, Michel, 37 indicators, 136–137, 139 Church involvement study, 50–51, 57, 69, 459, 463 nominal defi nitions, 140 Clark, Roger, 108 operational defi nitions, 140 Closed-ended questions, 272–273 in project design, 120 Cluster sampling, 231–238, 355 real defi nitions, 140 Codebooks, 447–448 Confi dence intervals, 219–221 Code notes, 427 Confi dence levels, 219–221 Codes of ethics, 77–79 Confi dentiality, 70–72 Coding Confl ict paradigm, 37 axial, 423 Conrad, Clifton F., 325 in content analysis, 355–359 Constant comparative method, 418 open, 423 Constructed knowledge, 41 in qualitative data analysis, 422–426 Constructs, 135 selective, 424 Construct validity, 161 Coeffi cient of reproducibility, 192 Content analysis, 350–362 Cohen-Mansfi eld, J., 23 coding, 355–359 INDEX 537

conceptualization, 357 quantitative data analysis. See Quantitative data counting and record keeping, 357–359 analysis. defi ned, 350 reading social research, 478 examples of, 359–361 Data processing, 121–122, 124 latent content, 356–357 Davern, Michael, 289 manifest content, 356 Davis, Fred, 318 qualitative data analysis, 359 Davis, James, 113–114, 305 reading social research, 477 Death penalty as deterrence, 406–407 sampling, 352–355 Debriefi ng, 73 strengths and weaknesses of, 361–362 Deception, 72–73 topics in, 350–351 De Coster, Stacy, 180 units of analysis, 352–355 Deduction, 23–25 Content validity, 161 induction compared, 48–54 Contingency questions, 279–280 theory construction, 54–56 Contingency tables, 462 Defl em, Mathieu, 373 Continuous variables, 454 DeFleur, Lois, 366 Control groups, 115, 248–249 Delgado, Richard, 42 Controlled experiments. See Experiments. Democracy in America (de Tocqueville), 374 Control variable. See Variables. Demographic Yearbook, 368 Conversation analysis (CA), 421 DeNuzzo, Rinaldo V., 221 Cook, Thomas, 254, 256 Dependent variables, 19, 247 Cooley, Charles Horton, 38, 260 Description research, 99, 145–147 Cooney, Margaret, 330 Design of research. See Research design. Copernicus, 35, 401 De Tocqueville, Alexis, 374 Corbin, Juliet, 324, 416, 418, 423, 426 Dewey, Thomas E., 202, 203 Correlation, 100 Diagnostics, sociological, 464–466 Cost-benefi t studies, 385 Dialectics, 22–28 Couper, Mick, 299, 300 applied research, 27–28 Cox, James, 201 deduction. See Deduction. CPOE systems, 7 idiographic explanations, 22–23, 24 Craig, R. Stephen, 359–360 induction. See Induction. Crawford, Kent, 397 nomothetic explanations, 22–23, 24 Creation myth, 44 pure research, 27–28 Criterion-related validity, 161 qualitative data, 25–27 Critical race theory, 42 quantitative data, 25–27 Cross-case analysis, 416 Dickson, W. J., 249 Cross-sectional studies, 111–112 Dillman, Don, 289 CSAQ, 299 Dimensions, 137–139, 148–149 Cullum-Swan, Betsy, 419 Direct observables, 134–135 “Cult of the individual,” 35 Disconfi rmability, 47 Curtin, Richard, 297 Discovery of Grounded Theory, The (Glaser/Strauss), 324 Discrete variables, 454 D Dispersion, 453–454 Daly, John, 281 Distributive justice, 54–56 Danieli, Ardha, 331 Double-barreled questions, 273–274, 275 Dannemeyer, William, 85 Double-blind experiments, 249–250 Darwin, 35 Douvan, Elizabeth, 115 Data analysis, 122, 124–125 Doyle, Sir Arthur Conan, 53 ethics, 73–74 Dual consciousness, 42 qualitative data analysis. See Qualitative data analysis. DuBois, W. E. B., 42 538 INDEX

Dunlap, Riley E., 48 Ethnography, 321 Durkheim, Emile, 40, 53, 142, 144, 349, 362–364, 365, 369 autoethnography, 322 institutional, 322, 328–329 E virtual ethnography, 322 Eastman, Crystal, 28 Ethnomethodology, 38–39, 322–324 Ecological fallacy, 109–110 Etic perspective, 319 Economic Ethic of the World Religions, The (Bellah), 372 Evaluation research, 383–409 Ecowarriors: Understanding the Radical Environmental computer simulations, 407–408 Movement (Scarce), 71 defi ned, 384 Edwards, Jennifer, 138 ethics in, 408–409 Einstein, 35 experimental designs, 390–391 Elements in sampling, 211 interventions, 388–389 Elements of Style, The (Strunk/White), 486 logistics, 397–399 Emancipatory research, 331 measurements, 386–390 Emerson, Robert, 38 multiple time-series designs, 394–395 Emic perspective, 319 nonequivalent control groups, 394 Empirical support for reality, 6 operationalizing success and failure, 389–390 Epistemology, 6 outcomes, specifying, 387–388 Equal probability of selection method (EPSEM), populations, 389 210–211 qualitative evaluations, 395–397 Eskenazi, Brenda, 263 quasi experiments, 391–395 Estimated sampling error, 511 reading social research, 477–478 Ethics, 22, 65–81 results, use of, 400–405 analysis, 73–74 Sabido methodology, 405 anonymity, 69–70 social indicators research, 406–408 confi dentiality, 70–72 time-series design, 392–393 deception, 72–73 topics in, 385–386 defi ned, 67 Tuskegee syphilis program, 409 in evaluation research, 408–409 Expectations communication model, 260 in experiments, 265 Experiments, 245–265 harm to participants, 68–69, 75 classical experiments, 246–250 homosexual behavior studies, 79–80 control groups, 248–249 human obedience observations, 80–81 dependent variables, 247 human sexuality research, 74–75 double-blind experiments, 249–250 institutional review boards, 74–77 ethics in, 265 of measurement, 164 in evaluation research, 390–391 professional codes, 77–79 example of, 259–262 in qualitative data analysis, 438–439 experimental groups, 115, 248–249 in qualitative fi eld research, 345 external invalidity, 257–259 in quantitative data analysis, 466–467 fi eld experiments, 259 in reading social research, 493 independent variables, 247 reporting, 73–74 internal invalidity, 254–257 in research design, 125 matching, 251–253 sampling, 238–239 “natural” experiments, 246, 263–264 survey research and, 307 posttesting, 247–248 Tearoom Trade (Humphreys), 79–80 preexperimental research designs, 253–254 theory and, 60 pretesting, 247–248 in unobtrusive research, 378–379 probability sampling, 250–251 voluntary participation, 67–68 randomization, 251, 252–253 in writing social research, 493 reading social research, 475–476 INDEX 539

strengths and weaknesses of method, 264–265 Gender Empowerment Measure (GEM), 185, 186 subject selection, 250–253 Gender-related Development Index (GDI), 185 topics appropriate to, 246 General Social Survey (GSS), 20 validity, 254–259 Genocide defi ned, 138–139 web-based, 262 Glaser, Barney, 56, 324, 359, 418, 423 Explanation research, 99, 145–147 Globalization, consequences of, 364–365 Explanatory social research, 22 Glock, Charles, 50, 459, 463 Exploration research, 97–99 Goffman, Erving, 56–57, 419–421 Exploratory social research, 21–22 Gottlieb, Bruce, 181 Extended case method, 326–328 Greatbatch, David, 39 External invalidity, 257–259 Greenwood, Peter, 400 External validation, 184 Griffi n, Larry, 378 Exxon Valdez oil spill, 70–71 Griffi th, Alison, 328–329 Grounded theory, 56, 324–326, 417–418 F Groups as units of analysis, 106, 107 Face validity, 160, 173 Groupthink, 339 Fair Lady, My, 259 Grube, Joel, 264 Farquharson, Karen, 205 Guttman, Louis, 190 Fausto-Sterling, Anne, 85 Guttman scale, 190–193 Feinberg, Barry, 297 Feminist paradigms, 40–42 H Ferris, Kerry, 38 Haley, Alex, 264 Festinger, Leon, 112 Harding, Warren, 201 Field experiments, 259 Harm to participants, 68–69, 75 Fielding, Nick, 306–307 Hart, Stephen, 260 Field research, 56. See also Qualitative fi eld research. Hatchett, Shirley, 115 Fine, Gary, 112 Hawking, Stephen, 45 Fink, Jeffrey, 397 Hawthorne effect, 249 Fisher, Patricia, 146 Helms, Jesse, 85 Focus groups, 97, 338–339 Hempel, Carl, 140 Ford, David, 403–404 Heritage, John, 39 Foschi, Martha, 260 Hermeneutic circle, 141 Foundations of social science. See Social science. Higginbotham, Leon Jr., 375 Fox, Katherine, 327–328 Hill, Lewis, 9 Freire, Paulo, 332 Hilts, Philip, 402 Frequency distributions, 450 Historical research. See Comparative and historical research. G “History,” 44 Gall, John, 489 Holmes, Sherlock, 53–54 Gallup, George, 202, 203, 205, 270 Homosexual behavior studies, 79–80, 84–85 Gambler’s fallacy, 9 Horney, Julie, 402–403 Gamson, William, 339 Howard, Edward, 387 Gans, Herbert, 59, 60, 88 Howell, Joseph, 343 GapMinder software, 463 Huberman, A. Michael, 416–417 Gard, Greta, 40 Human obedience observations, 80–81 Gardner, Carol Brooks, 38 Human sexuality research ethics, 74–75 Garfi nkel, Harold, 38, 323 Humphreys, Laud, 79, 84 GDI, 185 Hurst, Leslie, 326–327 Geertz, Clifford, 162 Hussein, Saddam, 39 Gender Advertisements (Goffman), 419 Hydén, Margareta, 339 540 INDEX

Hypotheses, 45, 48–49, 101–102 Internal invalidity, 254–257 Hypothesis testing, 48 International Monetary Fund (IMF), 37 International Policing (Defl em), 373 I Internet research, 478–485 Iannacchione, Vincent, 231 Interval measures, 150–151, 152 Ibrahim, Saad Eddin, 27 Interviews, qualitative, 335–338 Idealistic point of view, 370 Interview surveys, 291–295 Ideal types, 377 appearance and demeanor, 292 Idiographic explanations, 22–23, 24 computer-assisted personal interviewing, 299 Illogical reasoning, 9–10 computer-assisted self-interviewing, 299 Inaccurate observations, 8 computer-assisted telephone interviewing, 297, 298 Independent variables, 19, 247 coordination and control, 294–295 Indexes, 169–186 familiarity with questionnaire, 292–293 bad indexes, 184–185 following question wording, 293 bivariate relationships, 174–177 guidelines for, 292–294 cause and effect indicators, 175 probing for responses, 293–294 construction example, 185–186 recording responses, 293 defi ned, 171 role of interviewer, 291–292 empirical relationships, examination of, 174–179 Iran, sampling, 232 external validation, 184 IRB, 74–77, 125 face validity, 173 Isaac, Larry, 378 generality, 173–174 Item analysis, 183–184 item analysis, 183–184 item selection, 173–174 J missing data, handling, 180–182 Jackson, Jonathan, 137 multivariate relationships, 177–179 Jacobson, Lenore, 260 scales compared, 170–173 Jasso, Guillermina, 54–55 scoring, 179–180 Jensen, Arthur, 84 specifi city, 173–174 Jobes, Patrick C., 325 unidimensionality, 173 Johnson, Jeffrey, 206–207 validation, 182–185 Jones, Bill, 400 validators, bad, 184–185 Judgmental sampling, 204–205 variance, 174 Indicators, 136–137 K interchangeability of, 139 Kaplan, Abraham, 134, 135 single or multiple, 154 Kasl, Stanislav, 263 social, 406–408 Kasof, Joseph, 278 Indirect observables, 135 Kebede, Alemseghed, 58 Individual rights, 35 Keeter, Scott, 112, 223 Individuals as units of analysis, 105–106, 107 Kentor, Jeffrey, 364 Induction, 23–25 Kerry, John, 223 analytic, 359 Khayatt, Didi, 329 deduction compared, 48–54 Kim IL Sung, 27 theory construction, 56–58 Kinnell, Ann Marie, 421 Informants, 206–207 Kinsey, Alfred, 84 Informed consent, 69, 74 Knottnerus, J. David, 58 InfoTrac College Edition, 472 Koppel, Ross, 7 Institutional ethnography, 322, 328–329 Koresh, David, 322 Institutional review boards (IRB), 74–77, 125 Krueger, Richard, 339 Interest convergence, 42 Krushat, W. Mark, 158 INDEX 541

Kubrin, Charis, 105, 360–361 Lynd, Helen, 326 Kuhn, Thomas, 35 Lynd, Robert, 326 Kvale, Steinar, 336, 338 M L Machine-readable form, 443 Lakoff, George, 140 Macrotheory, 36 Landon, Alf, 199, 201, 271 Madison, Anna-Marie, 387 Language and Social Reality: The Case of Telling the Convict Mahoney, James, 369 Code (Wieder), 323 Mail distribution and return of questionnaires, 286–287 Laslett, Barbara, 375–376 Manifest content, 356 Latent content, 356 Manning, Peter, 419 Laumann, Edward O., 85 Marijuana smoking, 57–58 Lazes, Peter, 330 Marsden, Peter V., 305 Lee, Motoko, 187 Marshall, Catherine, 140, 317, 318 Leech, Nancy, 27 Martin, Patricia Yancey, 59 Lennon, Rachel, 108 Marx, Karl, 33, 37, 38, 270, 369, 371 Lever, Janet, 41 Matching in experiments, 251–253 Leviticus example, 424–426, 429–433 Matrix questions in questionnaires, 280–281 Libin, A., 23 Maynard, Douglas, 421 Library usage, 498 McGrane, Bernard, 39 Books in Print, 498 McVeigh, Timothy, 322 card catalogs, 500 Mead, George Herbert, 33, 38, 41, 260 computerized library fi les, 501–503 Meadows, Dennis, 408 Library of Congress classifi cation system, 500–501 Meadows, Donella, 408 online full-text resources, 503–505 Mean, 450 Readers’ Guide to Periodical Literature, 498–500 Measurement, 131–133 reference librarians, 498 accuracy, 156–157 reference sources, 498–500 conceptions, 133–134 using the “stacks,” 500–501 concepts, 133–136 Likert, Rensis, 188 ethics of, 164 Likert scale, 188–189 in evaluation research, 386–390 Linking theory and qualitative data analysis, 416–421 interval measures, 150–151, 152 Linton, Ralph, 40 nominal measures, 149–150, 152 Literature reviews operationalization, 149–153 organizing, 471–472 ordinal measures, 150, 152 in research proposals, 124 precision, 156–157 writing social research, 488 quality of criteria, 156–164 Lofl and, John, 315, 319, 331, 337, 342, 345, 416 ratio measures, 151, 152 Logical reasoning in analysis of existing statistics, 365 reading social research, 475 Logical support for reality, 6 reality, 133–134 Logistics in evaluation research, 397–399 reliability, 157–159, 163–164 Longhurst, Terri, 330 split-half method, 159 Longitudinal studies, 112–118 test-retest method, 158–159 approximating, 115–116 validity of, 160–164 cohort studies, 113–114 Median, 451 comparing types of, 115 Megatrends 2000 (Naisbitt/Aburdene), 351 panel studies, 114–115 Memoing, 426–427 strategy examples, 117 Menjivar, Cecilia, 335 trend studies, 112–113 Merton, Robert, 142 “Looking-glass-self,” 38 Meta-analysis, 306 542 INDEX

Methodology, 6 purposive sampling, 204–205 Microtheory, 36 quota frame, 206 Miles, Matthew, 416–417 quota sampling, 202, 205–206 Milgram, Stanley, 73, 80–81 snowball sampling, 205 Miller, Delbert, 289 Normal curve, 217 Mitchell, Richard, 274, 322 Normal curve areas, 510 Mitofsky, Warren, 300 Norman, Darlene, 387 Mode, 451 NUD*IST, 429–433 Modern view of reality, 10–11 Null hypothesis, 49 Monitoring studies, 385 NVivo, 433–438 Morgan, David, 339 Morgan, Lewis, 369 O Morris, Leana, 108 Objectivity, 43–44, 82–86 Moynihan, Daniel, 84 O’Brien, Patricia, 395 Multiple time-series designs, 394–395 Observations, 46–47, 121 Multistage cluster sampling, 231–238 One-group pretest-posttest design, 253, 255 Multivariate analysis, 463–464 O’Neill, Harry, 72, 289 Multivariate relationships, 177–179 One-shot case study, 253, 255 Murphy, Eddie, 260 Online full-text resources, 503–505 My Lai massacre, 80 Onwuegbuzie, Anthony, 27 Myrdal, Gunnar, 83 Open coding, 423 Open-ended questions, 272 N Operational defi nitions, 46, 140 Naive realism, 10 Operationalization, 45–46, 131–132, 147 “Natural” experiments, 246, 263–264 attributes, defi ning, 149 Naturalism, 321–322 choices of, 154–155 Nazi medical experiments, 65 dimensions, 148–149 Necessary cause, 102–103 extremes, variations between, 148 Needs assessment studies, 385 indicators, single or multiple, 154 Negative case testing, 359 interval measures, 150–151, 152 Neuman, W. Lawrence, 58 measurement of, 149–153 Newman, Jeffrey, 158 nominal measures, 149–150, 152 Newton, 35 ordinal measures, 150, 152 Nihilism, 361 in project design, 120–121 Nominal defi nitions, 140 ratio measures, 151, 152 Nominal measures, 149–150, 152 variables, defi ning, 149 Nomothetic explanations, 22–23, 24, 99–100 variation, range of, 147–148 causal analysis and hypothesis testing, 101–102 Operational notes, 427 complete causation, 102 Ordinal measures, 150, 152 correlation, 100 Organizations as units of analysis, 106–107 criteria for nomothetic causality, 100–101 Outcome assessment, 385 exceptional cases, 102 Overgeneralization, 8–9 false criteria, 102 Överlien, Carolina, 339 spurious relationships, 100–101 time order, 100 P Nonequivalent control groups, 394 Panel studies, 114–115 Nonprobability sampling, 203–207 Paradigms, 33–48 available subjects, reliance on, 203–204 confl ict paradigm, 37 informants, 206–207 critical race theory, 42 judgmental sampling, 204–205 early positivism, 36–37 INDEX 543

ethnomethodology, 38–39 confi dence intervals, 219–221 feminist paradigms, 40–42 confi dence levels, 219–221 macrotheory, 36 elements, 211 microtheory, 36 equal probability of selection method, 210–211 rational objectivity, 42–45 experiments, 250–251 scientifi c method, 45–48 normal curve, 217 structural functionalism, 39–40 parameters, 212 symbolic interactionism, 37–38 population, 211 Parameters, 212 probability theory, 212–213 Participant observation, 317 random-digit dialing, 212 Participants in studies, harm to, 68–69, 75 random selection, 200, 211–212 Participatory action research (PAR), 82, 329–332 representativeness, 210 Percentage down, 460 sampling distributions, 212–218 Percentaging a table, 460–462 sampling error, 216–218, 220 Perinelli, Phillip, 209 sampling unit, 212 Periodicity in sampling, 225 statistics, 216 Perrow, Charles, 373 study population, 211 Perspectives, 11 Probing for interview responses, 293–294 Peterson, Ruth, 407 Procedural knowledge, 41 Picou, Steven J., 71 Professional codes of ethics, 77–79 Placebo, 249 Professional papers, 487 Plagiarism, 489–490 Program evaluation, 385 Plutzer, Eric, 114 Project design, 117–123 Points of view, 11 analysis, 122 Politics of social research, 81–88 conceptualization, 120 little “p” politics, 86–87 conclusions and reports, 122 objectivity and ideology, 82–86 data processing, 121–122 in perspective, 87–88 method choice, 120 Population, 121 observations, 121 Populations in sampling, 211, 221–223, 230 operationalization, 120–121 Porter, Stephen, 301 population for study, 121 Positivism, 36–37, 44 sampling, 121 Postmodern view of reality, 11–12 triangulation, 123 Posttesting, 247–248 Proposals, research. See Research proposals. Posttest-only control-group design, 259 Protestant Ethic and the Spirit of Capitalism, The (Weber), Powell, Elwin, 142–143 372 PPS sampling, 235–236 Pure research, 27–28 Precision in measurement, 156–157 Purposive sampling, 204–205 Predestination, 372 Pygmalion (Shaw), 259 Preexperimental research designs, 253–254 Pygmalion effect, 260 “Pregnant chads,” 160 Premodern view of reality, 10 Q Presser, Stanley, 283, 297 QDA programs, 428–429 Pretesting, 247–248, 283 Qualitative data, 25–27 Prewitt, Kenneth, 85 Qualitative data analysis, 359, 415–439 Primary group, 38 case-oriented analysis, 417 Probabilistic reasoning, 6 coding, 422–426 Probability proportionate to size (PPS) sampling, 235–236 computer programs for, 428–438 Probability sampling, 200, 207–221, 238 concept mapping, 427–428 bias, 208–209 constant comparative method, 418 544 INDEX

conversation analysis, 421 don’t know/no opinion, 457–458 cross-case analysis, 416 ethics, 466–467 defi ned, 415 multivariate analysis, 463–464 ethics in, 438–439 numerical descriptions in qualitative research, 458 grounded theory method, 417–418 qualitative analysis of, 438 Leviticus example, 424–426, 429–433 quantifi cation of data, 443–448 linking theory and analysis, 416–421 sociological diagnostics, 464–466 memoing, 426–427 subgroup comparisons, 455–458 NUD*IST, 429–433 univariate analysis, 448–455. See also Univariate numerical descriptions in, 458 analysis. NVivo, 433–438 Quasi experiments, 391–395 patterns, discovering, 416–417 Questionnaires, 272, 278–286 processing, 421–428 contingency questions, 279–280 QDA programs, 428–429 format, 278–279 of quantitative data, 438 instructions, 282–283 semiotics, 419–421 matrix questions, 280–281 variable-oriented analysis, 416 order of items, 281–282 Qualitative evaluations, 395–397 pretesting, 283 Qualitative fi eld research, 313–345 sample, 283–286 case studies, 326–328 self-administered. See Self-administered questionnaires. defi ned, 314 Quoss, Bernita, 330 ethics in, 345 Quota frame, 206 ethnomethodology, 322–324 Quota sampling, 202, 205–206 extended case method, 326–328 focus groups, 338–339 R grounded theory, 324–326 Race and social research, 83–84 institutional ethnography, 322, 328–329 Radcliffe-Brown, Alfred, 144 interviews, 335–338 Ragin, Charles, 326 naturalism, 321–322 Random-digit dialing, 212 observer roles, 317–319 Randomization, 251, 252–253 participatory action research, 329–332 Random numbers table, 506–507 preparation, 333–335 Random selection, 200, 211–212 reading social research, 476–477 Rape reform legislation, 402–403 recording observations, 340–342 Rasinski, Kenneth, 277 reliability, 344–345 Rastafarianism, 58–59 strengths and weaknesses, 342–345 Ratio measures, 151, 152 subjects, relation to, 319–321 Rational objectivity, 42–45 topics in, 314–316 Rattine-Flaherty, Elizabeth, 332 validity, 343–344 Ray, Melvin, 187 Qualitative interview, 335–338 Reactivity, 317 Quantifi cation of data, 443–448 Readers’ Guide to Periodical Literature, 498–500 Quantitative data, 25–27 Reading social research, 471–485 Quantitative data analysis, 443–467 analyzing existing statistics, 477 bivariate analysis, 459–463 books, 473–474 codebook construction, 447–448 comparative and historical research, 477 code category development, 445–447 content analysis, 477 collapsing response categories, 456–457 data analysis, 478 data entry, 448 design of research, 474 defi ned, 443 ethics, 493 INDEX 545

evaluation research, 477–478 method choice, 120 experiments, 475–476 necessary cause, 102–103 fi eld research, 476–477 nomothetic explanations. See Nomothetic Internet usage, 478–485 explanations. journal articles, 472–473 observations, 121 literature reviews, organizing, 471–472 operationalization in project design, 120–121 measurements, 475 panel studies, 114–115 reporting, 478 population for study, 121 reports, evaluation of, 474–478 project design, 117–123 sampling, 475 proposals. See Research proposals. survey research, 476 purposes of, 97–99 theoretical orientations, 474 sampling, 121 Real defi nitions, 140 strategy examples, 117 Reality, 6–12 suffi cient cause, 102–103 authority, 8 trend studies, 112–113 inquiry, errors in, 8–10 triangulation, 123 inquiry, ordinary human, 6–7 units of analysis. See Units of analysis. measurement, 133–134 Research monographs, 473 modern view, 10–11 Research proposals, 123 postmodern view, 11–12 analysis, 124–125 premodern view, 10 budget, 125 selective observation, 9 data-collection, 124 tradition, 7–8 institutional review boards, 125 Received knowledge, 41 literature review, 124 Redden, David, 231 measurements, 124 Redfi eld, Robert, 369 objective of, 124 Reductionism, 110–111 problem to be studied, 124 Refl exivity, 320 schedule, 125 Regoli, Mary Jean, 403–404 subjects for study, 124 Reicker, Henry, 112 Respondents in survey research, 270–271 Reifi cation, 136 Response rates to interviews surveys, 297–299 Reliability Response rates to questionnaires, 288–289 in analysis of existing statistics, 366 Response variable, 387 in measurement, 157–159, 163–164 Riecken, Henry, 388 of qualitative fi eld research, 344–345 Ringer, Benjamin, 50 Replication, 9, 365 Rise of Christianity, The (Stark), 373 Reports Roethlisberger, F. J., 249 ethics, 73–74 Rokeach, Milton, 264 evaluations of, 474–478 Roosevelt, Franklin D., 199, 202, 271 Representativeness, 210 Roots: The Next Generation, 264 Research design, 96–125 Roots (Haley), 264 analysis of data, 122, 124–125 Roper, Burns, 72 cohort studies, 113–114 Rosenberg, Morris, 109 conceptualization in project design, 120 Rosenthal, Robert, 260 conclusions and reports, 122 Ross, Jeffrey, 96 cross-sectional studies, 111–112 Rossman, Gabriel, 351 data processing, 121–122, 124 Rossman, Gretchen, 140, 317, 318 ethics of, 125 Rothman, Ellen, 374 longitudinal studies. See Longitudinal studies. Rubin, Herbert, 335, 337 546 INDEX

Rubin, Riene, 335, 337 stratifi cation, 227, 234–235 Rueschemeyer, Dietrich, 369 stratifi ed sampling, 227–230 Ryerson, William N., 399 study population, 211 systematic sample with a random start, 224, 229 S systematic sampling, 224–227, 229–230 Sabido, Miguel, 405 types of sampling methods, 203, 223–231 Sabido methodology, 405 weighting, 236–238 Sacks, Jeffrey, 158 Sanders, William, 99 Sampling, 121, 199–239 Sapp, Stephen, 187 available subjects, reliance on, 203–204 Scales, 169–170, 186–193 bias, 208–209 Bogardus social distance scale, 186–187 cluster sampling, 231–238, 355 defi ned, 171 confi dence intervals, 219–221 Guttman scale, 190–193 confi dence levels, 219–221 indexes compared, 170–173 in content analysis, 352–355 Likert scale, 188–189 elements, 211 semantic differential, 189–190 equal probability of selection method, 210–211 Srole scale, 143 estimated sampling error, 511 Thurstone scale, 187–188 ethics of, 238–239 “Scandal In Bohemia, A” (Conan Doyle), 53–54 example of, 230 Scarce, Rik, 71 history of, 199–203 Schachter, Stanley, 112 informants, 206–207 Schedule in research proposals, 125 Iran, 232 Schmitt, Frederika E., 59 judgmental sampling, 204–205 Schutz, Alfred, 323 modifi cation of, 230–231 “Science as a Vocation” (Weber), 82 multistage cluster sampling, 231–238 “Science,” defi nition of, 2 nonprobability sampling, 203–207 Scientifi c method, 45–48 normal curve, 217 Search engines, 479 parameters, 212 Secondary analysis, 270, 304–307 periodicity, 225 Selective coding, 424 populations, 211, 221–223, 230 Selective observation, 9 probability proportionate to size sampling, 235–236 Self-administered questionnaires, 286–290 probability sampling, 200, 207–221, 238, 250–251 case study of, 289–290 probability theory, 212–213 computer self-administered questionnaire, 299 purposive sampling, 204–205 follow-up mailings, 288 quota frame, 206 mail distribution and return, 286–287 quota sampling, 202, 205–206 monitoring returns, 287–288 random-digit dialing, 212 response rates, 288–289 random selection, 200, 211–212 Selvin, Hanan, 213 reading social research, 475 Semantic differential, 189–190 representativeness, 210 Semiotics, 419–421 sampling distributions, 212–218 Sensate point of view, 370 sampling error, 216–218, 220, 232–233 Sense, Andrew J, 329 sampling frames, 202, 221–223, 230 Separate but equal, 42, 83 sampling interval, 224 September 11th, 201 sampling ratio, 224 Sexual research, 79–80, 84–85 sampling unit, 212 Shaffi r, William, 319 simple random sampling, 224, 225 Shaw, George Bernard, 259, 260 snowball sampling, 205 Shea, Christopher, 77 statistics, 216 Sherif, Muzafer, 44 INDEX 547

Signs, 419 Statistical Abstract of the United States, 366–368 Silence, 41 Statistics, 216 Silverman, David, 421, 458 analysis of existing statistics. See Analysis of existing Silverman, George, 339 statistics. Simmel, Georg, 37, 38 statistical analysis. See Quantitative data analysis. Simple random sampling, 224, 225 statistical signifi cance in nomothetic explanations, 102 Singer, Eleanor, 297 Status, 420 Singhal, Arvind, 332 Stebbins, Robert, 319 Skinner, Steven, 289 Steel, Paul, 396–397 Skocpol, Theda, 374 Stein, Gertrude, 11 Smith, Andrew, 274 Stratifi cation, 227, 234–235 Smith, Dorothy, 328 Stratifi ed sampling, 227–230 Smith, Tom W., 305 Strauss, Anselm, 56, 324, 359, 416, 418, 423, 426 Snow, David, 321, 322 Street Corner Society (Whyte), 321 Snowball sampling, 205 Structural functionalism, 39–40 Social artifacts, 108–109 Structure of Scientifi c Revolutions, The (Kuhn), 35 Social indicators, 406–408 Strunk, William Jr., 486 Social interactions as units of analysis, 108 Study of Man, The (Linton), 40 Social intervention, 385 Study population, 211 Social Organization of Sexuality, The (Laumann), 85 Subgroup comparisons, 455–458 Social regularities, 13–15 Subjective knowledge, 41 Social science, 12–22 Suffi cient cause, 102–103 aggregates, 15–16 Suicide (Durkheim), 142, 144, 362–364 attributes, 17 Summer, William Graham, 83 dialectics. See Dialectics. Survey research, 269–307 ethics. See Ethics. bias, 277–278 paradigms. See Paradigms. brevity of items, 276 purposes of social research, 21–22 clarity of items, 273 social regularities, 13–15 closed-ended questions, 272–273 theory, 13, 14 competency to answer, 274 variables, 16–21 double-barreled questions, 273–274, 275 “Social Structure and Anomie” (Merton), 142 ethics and, 307 Sociobiology, 111 interview surveys. See Interview surveys. Sociological diagnostics, 464–466 method comparisons, 302 Sociologie, 36 negativity of items, 276–277 Solomon four-group design, 258–259 open-ended questions, 272 Sorokin, Pitirim, A., 370 question asking guidelines, 271–278 Souls of Black Folk, The (DuBois), 42 questionnaires. See Questionnaires. SourceWatch, 492 reading social research, 476 Split-half method, 159 relevancy of questions, 274–276 Spock, Benjamin, 2 respondents, 270–271 Spohn, Cassie, 402–403 secondary analysis, 304–307 Spurious relationships, 100–101 social desirability of questions and answers, 277 Srole, Leo, 143–144 strengths and weaknesses of, 303–304 Srole scale, 143 technology, 299–301 Staab, Jennifer, 231 telephone surveys, 295–299 Standard deviation, 453 topics, 270–271 Stanley, Julian, 253, 256, 257, 258, 259 willingness to answer, 274 Stark, Rodney, 373 Symbolic interactionism, 37–38 Static-group comparison, 254, 255 Symbolic realism, 319 548 INDEX

Systematic sample with a random start, 224, 229 Units of analysis, 104–111 Systematic sampling, 224–227, 229–230 aggregates, 104 in analysis of existing statistics, 365 T in content analysis, 352–355 Tables ecological fallacy, 109–110 constructing and reading bivariate tables, elements compared, 211 462–463 faulty reasoning, 109–111 percentaging, 460–462 groups, 106, 107 Takeuchi, David, 57, 59 individuals, 105–106, 107 Tandon, Rajesh, 395 organizations, 106–107 Taylor, Humphrey, 300 reductionism, 110–111 TDE, 299 social artifacts, 108–109 Tearoom Trade (Humphreys), 79–80 social interactions, 108 Telephone surveys, 295–299 Univariate analysis, 448–455 Terhanian, George, 300 central tendency, 450–453 Test-retest method, 158–159 continuous variables, 454 Theoretical notes, 427 defi ned, 448 Theoretical sampling, 325 detail versus manageability, 454–455 Theory, 13, 14, 45 discrete variables, 454 ethics and, 60 dispersion, 453–454 importance of in “real world,” 59–60 distributions, 449–450 research and, 58–59 Unobtrusive Measures (Webb), 350 Thomas, Edmund, 397 Unobtrusive research, 345–379 Thomas, W. I., 374 analysis of existing statistics. See Analysis of existing Three Mile Island, 263–264 statistics. “Three strikes” laws, 400–401 comparative and historical research. See Comparative Through the Looking Glass (Carroll), 46 and historical research. Thurstone scale, 187–188 content analysis. See Content analysis. Time-series designs, 392–393 ethics in, 378–379 multiple, 394–395 Unobtrusive Research (Webb), 350 Tokugawa Religion (Bellah), 372 URL, 485 Total Design Method, 289 Touchtone data entry (TDE), 299 V Trading Places, 260 Validity Tradition, 7–8 in analysis of existing statistics, 365 Traditional model of science, 45–48 of experiments, 254–259 Trend studies, 112–113 of measurements, 160–164 Triangulation, 123 of qualitative fi eld research, 343–344 Trivariate relationships, 177–179 Value-free sociology, 82 Truman, Harry, 203 Variable-oriented analysis, 416 Tuckel, Peter, 289, 297 Variables, 16–21, 247 Tuskegee syphilis program, 409 continuous variables, 454 Twende na Wakati, 383–384, 399, 405 dependent variables, 19, 247 Typologies, 193–195 discrete variables, 454 independent variables, 19, 247 U operationalization, defi ning, 149 U.S. census, 85–86 Variance in index construction, 174 Understanding America (Perrow), 373–374 Veroff, Joseph, 115 Unidimensionality, 173 Virtual ethnography, 322 Uniform Crime Reports, 366 Voice Capture technology, 298 INDEX 549

Voice recognition (VR), 299 Wilson, Edward O., 111 Voluntary participation, 67–68 Women’s Ways of Knowing (Belenky), 41 Woodhams, Carol, 331 W Working papers, 487 Walker, Jeffrey, 299 World Bank, 37 Wallace, Walter, 53 World Factbook, 480, 483 Ward, Lester, 59, 60 Writing social research, 471, 486–493 Warner, W. Lloyd, 326 aim of report, 487–488 Warner, William Lloyd, 144 analyses, guidelines for, 491–492 Warriner, G. Keith, 260 analysis and interpretation, 490 Weaver, Randy, 322 audience, 486 Webb, Eugene, 350 ethics, 493 Web-based experiments, 262 form of report, 486–487 Weber, Max, 82, 371–372, 377 “going public,” 492–493 Web research, 478–485 length of report, 486–487 Webster’s New World Dictionary, 67 literature reviews, 488 Weighting in sampling, 236–238 organization of report, 488–491 Weiss, Carol, 394 overview, 488 Weitzer, Ronald, 105 plagiarism, 489–490 Weitzman, Lenore, 108 purpose of report, 488 Wells, Ida B., 351 study design and execution, 490 Wharton, Amy, 180 summary and conclusions, 490–491 Wheel of Science, 53 When Prophecy Fails (Festinger/Reicker/Schachter), 112 Y Whitcomb, Michael, 301 Yammarino, Francis, 289 White, E. B., 486 Yinger, Milton, 252 White, William, 60 Whyte, William Foote, 321 Z Wieder, D. L., 323–324 Zerbib, Sandrine, 341, 432, 433–438 Wife battering prevention, 403–404 Zhang, Ray, 26 Wilson, Camilo, 300 Znaniecki, Florian, 374 PHOTO CREDITS

Part and Chapter Openers

Pages 1, 3: Bonnie Kamin/PhotoEdit 32: John-Claude LeJeune 64: Hazel Hanken/Stock Boston 92: Kathy Sloane/Photo Researchers, Inc. 94: Photog- raphers Library LTD/eStock Photography 130: David M. Grossman/Photo Researchers, Inc. 168: Stock Solution/Index Stock Imagery 198: Bob Daemmrich/ PhotoEdit 242: Paul Conklin/PhotoEdit 244: Jeff Greenburg/PhotoEdit 268: Don Pitcher/Stock Boston 312: Paul Conklin/PhotoEdit 348: Dion Ogust/The Image Works 382: Wojnarowicz/The Image Works 412: David Falconer/ Folio, Inc. 414: Jim Whitmer/Stock Boston 442: David Falconer/Folio, Inc. 470: John Henley/CORBIS.

In-text Photos

Pages 4, 5, 33, 35, 65, 67, 82, 95, 97, 131, 135, 169, 173, 199, 207, 213, 245, 246, 269, 313, 318, 349, 383, 415, 443, 471, 473: Earl Babbie 444: Aaron Babbie.

550