DOCSLIB.ORG

Workshop on Probability-Based and Nonprobability Survey Research

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Workshop on Probability-Based and Nonprobability Survey Research Workshop on Probability-Based and Nonprobability Survey Research Collaborative Research Center SFB 884 University of Mannheim June 25-26, 2018 Keynote: Jon A. Krosnick (Stanford University) Scientific Committee: Carina Cornesse Alexander Wenz Annelies Blom Location: SFB 884 – Political Economy of Reforms B6, 30-32 68131 Mannheim Room 008 (Ground Floor) Schedule Monday, June 25 08:30 – 09:10 Registration and coffee 09:10 – 09:30 Conference opening 09:30 – 10:30 Session 1: Professional Respondents and Response Quality o Professional respondents: are they a threat to probability-based online panels as well? (Edith D. de Leeuw) o Response quality in nonprobability and probability-based online panels (Carina Cornesse and Annelies Blom) 10:30 – 11:00 Coffee break 11:00 – 12:30 Session 2: Sample Accuracy o Comparing complex measurement instruments across probabilistic and non-probabilistic online surveys (Stefan Zins, Henning Silber, Tobias Gummer, Clemens Lechner, and Alexander Murray-Watters) o Comparing web nonprobability based surveys and telephone probability-based surveys with registers data: the case of Global Entrepreneurship Monitor in Luxembourg (Cesare A. F. Riillo) o Does sampling matter? Evidence from personality and politics (Mahsa H. Kashani and Annelies Blom) 12:30 – 13:30 Lunch 1 13:30 – 15:00 Session 3: Conceptual Issues in Probability-Based and Nonprobability Survey Research o The association between population representation and response quality in probability-based and nonprobability online panels (Alexander Wenz, Carina Cornesse, and Annelies Blom) o Probability vs. nonprobability or high-information vs. low- information? (Andrew Mercer) o Non-probability based online panels: market research practitioners perspective (Wojciech Jablonski) 15:00 – 15:30 Coffee break 15:30 – 17:00 Session 4: Practical Considerations in Online Panel Research o Replenishment of the Life in Australia Panel (Benjamin Phillips and Darren W. Pennay) o Terms of agreement: the inclusion of Muslim minorities (Elisabeth Ivarsflaten and Paul Sniderman) o Do you get what you asked for? On the implementation of a survey in nonprobability online panels (Daniela Ackermann-Piek and Annelies Blom) 2 Tuesday, June 26 09:30 – 10:30 Keynote An update on the accuracy of probability sample surveys and non- probability sample surveys (Jon A. Krosnick) 10:30 – 11:00 Coffee break 11:00 – 12:30 Session 5: Variance Estimation and Weighting Adjustments o Precision of estimates based on non-probability online panels (Marek Fuchs and Tobias Baier) o Improving estimates from non-probability online surveys (Dina Neiger, Andrew C. Ward, Darren W. Pennay, Paul J. Lavrakas, and Benjamin Phillips) o Weighting and estimation strategies for probability-based and nonprobability panel research (Christian Bruch, Barbara Felderer, and Annelies Blom) 12:30 – 13:30 Lunch 13:30 – 15:00 Session 6: Combining Probability-Based and Nonprobability Samples o Rationale for conducting and methods for calibrating hybrid probability/non-probability surveys (David Dutwin) o Estimating the size of the LGB population with a random telephone/Internet survey and a joint Internet volunteer survey and taking advantage of the two to increase the analytical LGB subsample (Stéphane Legleye and Géraldine Charrance) o Blending probability and nonprobability samples for survey inference under a Bayesian framework (Joseph W. Sakshaug, Arkadiusz Wisniowski, Diego Perez-Ruiz, and Annelies Blom) 3 15:00– 15:30 Coffee break 15:30– 17:00 Session 7: Nonprobability Survey Research and Big Data o How research on probability and nonprobability panels can inform passive data collection studies (Bella Struminskaya) o Probability, nonprobability sampling, and Big Data (Andreas Quatember) o Digital trace data: just another nonprobability sample? (Josh Pasek) 4 Abstracts Session 1: Professional Respondents and Response Quality Professional respondents: are they a threat to probability-based online panels as well? Edith D. de Leeuw A major concern about the quality of non-probability online panels centers on the presence of ‘professional’ respondents. In reaction to criticism on non-probability panels, probability- based online panels are established. However, probability-based panels suffer initial nonresponse during panel formation with the danger of selective nonresponse. Are probability-based panels a safeguard against professional respondents? In the Netherlands a large study (NOPVO) of 19 (opt-in) online panels reports on professional respondents. We partly replicated their study in two Dutch probability based online panels: a probability sample of the general population of the Netherlands (LISS panel), and a probability sample of the four largest ethnic minority groups in the Netherlands (LISS immigrant panel). In the probability-based Dutch online panels, the number of panel memberships was lower than in the NOPVO panels: 84.5% of the LISS members and 80.3% of the immigrant panel did not belong to other panels, while in the NOPVO-panel study only 38% was not a member of multiple panels. In the NOPVO-study on average more than 80% of the respondents reported to have completed more than one questionnaire in the past 4 weeks, while in the two probability-based panels this was less than 40 %. Response quality in nonprobability and probability-based online panels Carina Cornesse and Annelies Blom The ongoing debate about the quality of nonprobability online panels predominantly discusses whether or not these panels have representative sets of respondents. While the number of publications on nonprobability panel representativeness is increasing, less attention has so far been paid to potential measurement errors in nonprobability as compared to probability panels. In our paper, we investigate whether there are differences in satisficing across probability and nonprobability online panels using three indicators to operationalize survey satisficing (item nonresponse and non-substantive answers, straight-lining in grids, and mid-point selection in a visual design experiment). These indicators are included in a questionnaire module that was implemented across nine online panels in Germany: one academic probability online panel that includes the offline population, one commercial probability online panel, and seven nonprobability online panels, all differing with respect to their sampling and recruitment methods. Our analyses show significantly less straight-lining in probability than in nonprobability online panels, but no significant differences regarding 5 mid-point selection. With respect to non-substantive answers, we find that significantly more respondents in the probability than in the nonprobability panels say that they don’t know who they voted for in the last general election or refuse to report their height and body weight. Session 2: Sample Accuracy Comparing complex measurement instruments across probabilistic and non- probabilistic online surveys Stefan Zins, Henning Silber, Tobias Gummer, Clemens Lechner, and Alexander Murray-Watters The quality of non-probabilistic samples is often only determined by the comparison of estimated frequency distribution of items with that of benchmarks distributions, e.g. obtained from official statistics or probabilistic samples (Yaeger et al., 2011). However, simple frequencies are often not of primary interest, but dependencies between the measured variables are at the core of theory building and testing in behavioural sciences. In order to compare possible method effects of different non-probabilistic sampling designs with respect to the joint distribution of the measured variables, an established measuring instrument (BFI- 2-S instrument, see, Soto and John, 2017) with measurements from two nonprobabilistic samples and one probabilistic sample are evaluated. The probabilistic mixed-mode sample serves as a reference. With an already carried out study, the sample type of a non-probabilistic online access panel will be examined. As a second non-probabilistic sample type we plan to analyse a so called river sample. River sampling is a relatively uncontrolled recruitment of respondents via ads on various websites. This type of sampling has not yet been taken up in any comparative study of this kind in Germany and rarely internationally. The main goal of this method comparison study is to compare the ability to use the different types of samples to model the latent variables of the measuring instrument. For this we will evaluate two alternative methods to model the latent variables. One will be the standard method that uses factor analysis and for the other we will use a causal search algorithm (FOFC). 6 Comparing web nonprobability based surveys and telephone probability- based surveys with registers data: the case of Global Entrepreneurship Monitor in Luxembourg Cesare A. F. Riillo Failure to predict Brexit and the US election outcome has called into question poll and survey methodology. This study contributes to this debate by assessing the Total Survey Error of two surveys on entrepreneurship. One survey is probability-based and is conducted by fix-line phone. The other survey is based on an opt-in web panel and it is nonprobability based. The same questions are administered in both surveys. The study assess with survey resembles better official register data in terms of distribution of socio demographic characteristics of respondents and in terms of variable of interest (entrepreneurial activity). Research is based on the Global Entrepreneurship Monitor (GEM)
Load more

Recommended publications
  • Combining Probability and Nonprobability Samples to Form Efficient Hybrid Estimates: an Evaluation of the Common Support Assumption Jill A
    Combining Probability and Nonprobability Samples to form Efficient Hybrid Estimates: An Evaluation of the Common Support Assumption Jill A. Dever RTI International, Washington, DC Proceedings of the 2018 Federal Committee on Statistical Methodology (FCSM) Research Conference Abstract Nonprobability surveys, those without a defined random sampling scheme, are becoming more prevalent. These studies can offer faster results at less cost than many probability surveys, especially for targeting important subpopulations. This can be an attractive option given the continual challenge of doing more with less, as survey costs continue to rise and response rates to plummet. Nonprobability surveys alone, however, may not fit the needs (purpose) of Federal statistical agencies where population inference is critical. Nonprobability samples may best serve to enhance the representativeness of certain domains within probability samples. For example, if locating and interviewing a required number of subpopulation members is resource prohibitive, data from a targeted nonprobability survey may lower coverage bias exhibited in a probability survey. In this situation, the question is how to best combine information from both sources. Our research searches for an answer to this question through an evaluation of hybrid estimation methods currently in use that combine probability and nonprobability data. Methods that employ generalized analysis weights (i.e., one set of weights for all analyses) are the focus because they enable other survey researchers and policy makers to analyze the data. The goal is to identify procedures that maximize the strength of each data source to produce hybrid estimates with the low mean square error. The details presented here focus on the propensity score adjusted (PSA) nonprobability weights needed prior to combining the data sources, and the common support assumption critical to hybrid estimation and PSA weighting.
    [Show full text]
  • A Critical Review of Studies Investigating the Quality of Data Obtained with Online Panels Based on Probability and Nonprobability Samples1
    Callegaro c02.tex V1 - 01/16/2014 6:25 P.M. Page 23 2 A critical review of studies investigating the quality of data obtained with online panels based on probability and nonprobability samples1 Mario Callegaro1, Ana Villar2, David Yeager3,and Jon A. Krosnick4 1Google, UK 2City University, London, UK 3University of Texas at Austin, USA 4Stanford University, USA 2.1 Introduction Online panels have been used in survey research as data collection tools since the late 1990s (Postoaca, 2006). The potential great cost and time reduction of using these tools have made research companies enthusiastically pursue this new mode of data collection. However, 1 We would like to thank Reg Baker and Anja Göritz, Part editors, for their useful comments on preliminary versions of this chapter. Online Panel Research, First Edition. Edited by Mario Callegaro, Reg Baker, Jelke Bethlehem, Anja S. Göritz, Jon A. Krosnick and Paul J. Lavrakas. © 2014 John Wiley & Sons, Ltd. Published 2014 by John Wiley & Sons, Ltd. Callegaro c02.tex V1 - 01/16/2014 6:25 P.M. Page 24 24 ONLINE PANEL RESEARCH the vast majority of these online panels were built by sampling and recruiting respondents through nonprobability methods such as snowball sampling, banner ads, direct enrollment, and other strategies to obtain large enough samples at a lower cost (see Chapter 1). Only a few companies and research teams chose to build online panels based on probability samples of the general population. During the 1990s, two probability-based online panels were documented: the CentER data Panel in the Netherlands and the Knowledge Networks Panel in the United States.
    [Show full text]
  • Options for Conducting Web Surveys Matthias Schonlau and Mick P
    Statistical Science 2017, Vol. 32, No. 2, 279–292 DOI: 10.1214/16-STS597 © Institute of Mathematical Statistics, 2017 Options for Conducting Web Surveys Matthias Schonlau and Mick P. Couper Abstract. Web surveys can be conducted relatively fast and at relatively low cost. However, Web surveys are often conducted with nonprobability sam- ples and, therefore, a major concern is generalizability. There are two main approaches to address this concern: One, find a way to conduct Web surveys on probability samples without losing most of the cost and speed advantages (e.g., by using mixed-mode approaches or probability-based panel surveys). Two, make adjustments (e.g., propensity scoring, post-stratification, GREG) to nonprobability samples using auxiliary variables. We review both of these approaches as well as lesser-known ones such as respondent-driven sampling. There are many different ways Web surveys can solve the challenge of gen- eralizability. Rather than adopting a one-size-fits-all approach, we conclude that the choice of approach should be commensurate with the purpose of the study. Key words and phrases: Convenience sample, Internet survey. 1. INTRODUCTION tion and invitation of sample persons to a Web sur- vey. No complete list of e-mail addresses of the general Web or Internet surveys1 have come to dominate the survey world in a very short time (see Couper, 2000; population exists from which one can select a sample Couper and Miller, 2008). The attraction of Web sur- and send e-mailed invitations to a Web survey. How- veys lies in the speed with which large numbers of ever, for many other important populations of interest people can be surveyed at relatively low cost, using (e.g., college students, members of professional asso- complex instruments that extend measurement beyond ciations, registered users of Web services, etc.), such what can be done in other modes (especially paper).
    [Show full text]
  • Lesson 3: Sampling Plan 1. Introduction to Quantitative Sampling Sampling: Definition
    Quantitative approaches Quantitative approaches Plan Lesson 3: Sampling 1. Introduction to quantitative sampling 2. Sampling error and sampling bias 3. Response rate 4. Types of "probability samples" 5. The size of the sample 6. Types of "non-probability samples" 1 2 Quantitative approaches Quantitative approaches 1. Introduction to quantitative sampling Sampling: Definition Sampling = choosing the unities (e.g. individuals, famililies, countries, texts, activities) to be investigated 3 4 Quantitative approaches Quantitative approaches Sampling: quantitative and qualitative Population and Sample "First, the term "sampling" is problematic for qualitative research, because it implies the purpose of "representing" the population sampled. Population Quantitative methods texts typically recognize only two main types of sampling: probability sampling (such as random sampling) and Sample convenience sampling." (...) any nonprobability sampling strategy is seen as "convenience sampling" and is strongly discouraged." IIIIIIIIIIIIIIII Sampling This view ignores the fact that, in qualitative research, the typical way of IIIIIIIIIIIIIIII IIIII selecting settings and individuals is neither probability sampling nor IIIII convenience sampling." IIIIIIIIIIIIIIII IIIIIIIIIIIIIIII It falls into a third category, which I will call purposeful selection; other (= «!Miniature population!») terms are purposeful sampling and criterion-based selection." IIIIIIIIIIIIIIII This is a strategy in which particular settings, persons, or activieties are selected deliberately in order to provide information that can't be gotten as well from other choices." Maxwell , Joseph A. , Qualitative research design..., 2005 , 88 5 6 Quantitative approaches Quantitative approaches Population, Sample, Sampling frame Representative sample, probability sample Population = ensemble of unities from which the sample is Representative sample = Sample that reflects the population taken in a reliable way: the sample is a «!miniature population!» Sample = part of the population that is chosen for investigation.
    [Show full text]
  • Final Abstracts in Order of Presentation
    Final Abstracts in Order of Presentation Sunday, September 20, 2015 9:30-11:30 a.m. Paper Session I Interactions of Survey Error and Ethnicity I Session Chair: Sunghee Lee Invited Presentation: Ethnic Minorities in Surveys: Applying the TSE Paradigm to Surveys Among Ethnic Minority Groups to Assess the Relationship Between Survey Design, Sample Frame and Survey Data Quality Joost Kappelhof1 Institute for Social Research/SCP1 Minority ethnic groups are difficult to survey mainly because of cultural differences, language barriers, socio-demographic characteristics and a high mobility (Feskens, 2009). As a result, ethnic minorities are often underrepresented in surveys (Groves & Couper, 1998; Stoop, 2005). At the same time, national and international policy makers need specific information about these groups, especially on issues such as socio-economic and cultural integration. Using the TSE framework, we will integrate existing international empirical literature on survey research among ethnic minorities. In particular, this paper will discuss four key topics in designing and evaluating survey research among ethnic minorities for policy makers. First of all, it discusses the reasons why ethnic minorities are underrepresented in survey. In this part an overview of the international empirical literature on reasons why it is difficult to conduct survey research among ethnic minorities will be placed in the TSE framework. Secondly, it reviews measures that can be undertaken to increase the representation of minorities in surveys and it discusses the consequences of these measures. In particular the relationship with survey design, sample frame and trade-off decisions in the TSE paradigm is discussed in combination with budget and time considerations.
    [Show full text]
  • STANDARDS and GUIDELINES for STATISTICAL SURVEYS September 2006
    OFFICE OF MANAGEMENT AND BUDGET STANDARDS AND GUIDELINES FOR STATISTICAL SURVEYS September 2006 Table of Contents LIST OF STANDARDS FOR STATISTICAL SURVEYS ....................................................... i INTRODUCTION......................................................................................................................... 1 SECTION 1 DEVELOPMENT OF CONCEPTS, METHODS, AND DESIGN .................. 5 Section 1.1 Survey Planning..................................................................................................... 5 Section 1.2 Survey Design........................................................................................................ 7 Section 1.3 Survey Response Rates.......................................................................................... 8 Section 1.4 Pretesting Survey Systems..................................................................................... 9 SECTION 2 COLLECTION OF DATA................................................................................... 9 Section 2.1 Developing Sampling Frames................................................................................ 9 Section 2.2 Required Notifications to Potential Survey Respondents.................................... 10 Section 2.3 Data Collection Methodology.............................................................................. 11 SECTION 3 PROCESSING AND EDITING OF DATA...................................................... 13 Section 3.1 Data Editing ........................................................................................................
    [Show full text]
  • Ch7 Sampling Techniques
    7 - 1 Chapter 7. Sampling Techniques Introduction to Sampling Distinguishing Between a Sample and a Population Simple Random Sampling Step 1. Defining the Population Step 2. Constructing a List Step 3. Drawing the Sample Step 4. Contacting Members of the Sample Stratified Random Sampling Convenience Sampling Quota Sampling Thinking Critically About Everyday Information Sample Size Sampling Error Evaluating Information From Samples Case Analysis General Summary Detailed Summary Key Terms Review Questions/Exercises 7 - 2 Introduction to Sampling The way in which we select a sample of individuals to be research participants is critical. How we select participants (random sampling) will determine the population to which we may generalize our research findings. The procedure that we use for assigning participants to different treatment conditions (random assignment) will determine whether bias exists in our treatment groups (Are the groups equal on all known and unknown factors?). We address random sampling in this chapter; we will address random assignment later in the book. If we do a poor job at the sampling stage of the research process, the integrity of the entire project is at risk. If we are interested in the effect of TV violence on children, which children are we going to observe? Where do they come from? How many? How will they be selected? These are important questions. Each of the sampling techniques described in this chapter has advantages and disadvantages. Distinguishing Between a Sample and a Population Before describing sampling procedures, we need to define a few key terms. The term population means all members that meet a set of specifications or a specified criterion.
    [Show full text]
  • Chapter 7 "Sampling"
    This is “Sampling”, chapter 7 from the book Sociological Inquiry Principles: Qualitative and Quantitative Methods (index.html) (v. 1.0). This book is licensed under a Creative Commons by-nc-sa 3.0 (http://creativecommons.org/licenses/by-nc-sa/ 3.0/) license. See the license for more details, but that basically means you can share this book as long as you credit the author (but see below), don't make money from it, and do make it available to everyone else under the same terms. This content was accessible as of December 29, 2012, and it was downloaded then by Andy Schmitz (http://lardbucket.org) in an effort to preserve the availability of this book. Normally, the author and publisher would be credited here. However, the publisher has asked for the customary Creative Commons attribution to the original publisher, authors, title, and book URI to be removed. Additionally, per the publisher's request, their name has been removed in some passages. More information is available on this project's attribution page (http://2012books.lardbucket.org/attribution.html?utm_source=header). For more information on the source of this book, or why it is available for free, please see the project's home page (http://2012books.lardbucket.org/). You can browse or download additional books there. i Chapter 7 Sampling Who or What? Remember back in Chapter 1 "Introduction" when we saw the cute photo of the babies hanging out together and one of them was wearing a green onesie? I mentioned there that if we were to conclude that all babies wore green based on the photo that we would have committed selective observation.
    [Show full text]
  • CHAPTER 5 Sampling
    05-Schutt 6e-45771:FM-Schutt5e(4853) (for student CD).qxd 9/29/2008 11:23 PM Page 148 CHAPTER 5 Sampling Sample Planning Nonprobability Sampling Methods Define Sample Components and the Availability Sampling Population Quota Sampling Evaluate Generalizability Purposive Sampling Assess the Diversity of the Population Snowball Sampling Consider a Census Lessons About Sample Quality Generalizability in Qualitative Sampling Methods Research Probability Sampling Methods Sampling Distributions Simple Random Sampling Systematic Random Sampling Estimating Sampling Error Stratified Random Sampling Determining Sample Size Cluster Sampling Conclusions Probability Sampling Methods Compared A common technique in journalism is to put a “human face” on a story. For instance, a Boston Globe reporter (Abel 2008) interviewed a participant for a story about a housing pro- gram for chronically homeless people. “Burt” had worked as a welder, but alcoholism and both physical and mental health problems interfered with his plans. By the time he was 60, Burt had spent many years on the streets. Fortunately, he obtained an independent apartment through a new Massachusetts program, but even then “the lure of booze and friends from the street was strong” (Abel 2008:A14). It is a sad story with an all-too-uncommon happy—although uncertain—ending. Together with one other such story and comments by several service staff, the article provides a persuasive rationale for the new housing program. However, we don’t know whether the two participants interviewed for the story are like most program participants, most homeless persons in Boston, or most homeless persons throughout the United States—or whether they 148 Unproofed pages.
    [Show full text]
  • Chapter 7. Sampling an Example Concepts Continued
    What are the two types of sampling Chapter 7. Sampling methods? Two types of sampling methods Probability sampling: selection of “random” sample. Nonprobability sampling In the sense that every observation in the population Reliance on available subjects has an equal chance to be selected. Judgmental sampling This is the desirable sampling method because it provides Snow-ball sampling precise statistical descriptions of large populations. Quota sampling Nonprobability sampling: when probability sampling Probability sampling principles are not feasible. This is the less desirable method, but Probability sampling methods nevertheless commonly used because of practical Simple random sampling nevertheless commonly used because of practical Systematic sampling difficulties with using probability sampling. Stratified sampling Nonprobability sampling cannot guarantee that the sample Multistage cluster sampling observed is representative of the whole population. What are the types of nonprobability What are the concepts and sampling? terminology in probability sampling? Reliance on available subjects Theoretical Study Sample Examples: Stop people at the mall, University student sample Population Population Problems: no sample representativeness Sampling Purposive or judgmental sampling Frame Examples: friends, colleagues, community leaders Usually used for preliminary testing of questionnaire, and field research Elements EachSampling SLC familyUnits Observation on the list Units Snowball sampling Ask people to introduce researcher to more people for interviews Quota sampling Parameters Statistics Step1. Creating quota matrix: Ex. Gender and age Step 2. Decide on # of observations needed in each quota Sampling Error Step 3. Find subjects with these characteristics to form the sample. Confidence Level Confidence Interval An Example Concepts continued All SLC All SLC families 1000 SLC families in the phone book families Theoretical population List of families in the phone book The theoretically specified aggregation of study elements.
    [Show full text]
  • Session 6 Slides
    SAMPLING Week 6 Slides ScWk 240 1 Purpose of Sampling Why sampling? - to study the whole populaon? A major reason studying samples rather than the whole group is that the whole group is so large that studying it is not feasible. Example- college students in CA. If we can study the whole populaon, we do not need to go through the sampling procedures. Much research is based on samples of people. Representa5veness - how representave the selected cases are? Then, can knowledge gained from selected cases be considered knowledge about a whole group of people? The answer depends on whether those selected cases are representa(ve of larger group. Newsmagazine ar>cles about public opinion: How can we be sure that the results reflect the public’s true opinion, in other words, how much they can represent views of all Americans. The ul>mate purpose of sampling is to get accurate representa(veness. The important consideraon about samples is how representave they are of the populaon from which we draw them. Casual vs. scienfic sampling In both daily life and prac>ce, we are involved in sampling decisions - movies, car purchases, class selec>ons, etc; to get feedbacks about service sasfac>on from clients – what is said in community or agency mee>ng. How much of this informaon is representave? The informaon can be misleading or biased - The people who aend or are the most vocal at a mee>ng may be the most sasfied (or most dissasfied). If a sample size is too small, informaon can be biased as well.
    [Show full text]
  • Nonprobability Samples: Problems & Approaches to Inference
    Nonprobability Samples: Problems & Approaches to Inference Richard Valliant University of Michigan & University of Maryland Washington Statistical Society 25 Sep 2017 (UMich & UMD) 1 / 30 Outline 1 Probability vs. nonprobability sampling 2 Inference problem 3 Methods of Inference Quasi-randomization Superpopulation Models for y’s 4 Numerical example 5 Conclusion 6 References (UMich & UMD) 2 / 30 Probability vs. nonprobability sampling Two Classes of Sampling Probability sampling: Presence of a sampling frame linked to population Every unit has a known probability of being selected Design-based theory focuses on random selection mechanism Probability samples became touchstone in surveys after [Neyman, JRSS 1934] Nonprobability sampling: Investigator does not randomly pick sample units with KNOWN probabilities No population sampling frame available (desired) Underlying population model is important Review paper: [Elliott & Valliant, StatSci 2017] [Vehover Toepoel & Steinmetz, 2016] (UMich & UMD) 3 / 30 Probability vs. nonprobability sampling Types of Nonprobability Samples AAPOR panel on nonprob samples defined three types [Baker. et al., AAPOR 2013]: Convenience sampling—mall intercepts, volunteer samples, river samples, observational studies, snowball samples Sample matching—members of nonprobability sample selected to match set of important population characteristics Network sampling—members of some population asked to identify other members of pop with whom they are somehow connected (UMich & UMD) 4 / 30 Probability vs. nonprobability sampling Examples of Data Sources Twitter Facebook Snapchat Mechanical Turk SurveyMonkey Web-scraping Billion Prices Project @ MIT, http://bpp.mit.edu/ Price indexes for 22 countries based on web-scraped data Google flu and dengue fever trends Pop-up surveys Data warehouses Probabilistic matching of multiple sources see, e.g., [Couper, SRM 2013] (UMich & UMD) 5 / 30 Probability vs.
    [Show full text]