Statistical Sampling: An Overview for Criminal Justice Researchers April 28, 2016 Stan Orchowsky: Good afternoon everyone. My name is Stan Orchowsky, I'm the research director for the Justice Research and Statistics Association. It's my pleasure to welcome you this afternoon to the next in our Training and Technical Assistance Webinar Series on Statistical Analysis for Criminal Justice Research. We want to thank the Bureau of Justice Statistics, which is helping to support this webinar series. Stan Orchowsky: Today's topic will be on statistical sampling, we will be providing an overview for criminal justice research. And I'm going to turn this over now to Dr. Farley, who's going to give you the objectives for the webinar. Erin Farley: Greetings everyone. Erin Farley: Okay, so the webinar objectives. Sampling is a very important and useful tool in social science criminal justice research, and that is because in many, if not most, situations accessing an entire population to conduct research is not feasible. Oftentimes it would be too time-consuming and expensive. As a result, the use of sampling methodology, if utilized properly, can save time, money, and produce sample estimates that can be utilized to make inferences about the larger population. Erin Farley: The objectives of this webinar include: describing the different types of probability and non-probability sampling designs; discussing the strengths and weaknesses of these techniques, as well as discussing the importance of sampling to the external validity of experimental designs and statistical analysis. Erin Farley: This slide presents a simple diagram illustrating the process and goal of sampling. Using the appropriate technique, a sample is drawn from the larger and known population, and if this sample is representative of the population we can take what we have learned about the sample and make an inference about the entire population. Erin Farley: What is missing from this diagram are the important questions that need to be answered prior to drawing a sample. In fact, the answers to these questions will likely impact the type of sampling method that is selected. For example, what is the nature of the study? Is it exploratory, descriptive, or analytical? What are the variables of interest? What is the target population? What is the data collection mode? Is it survey, or interviews? What is the sampling frame, and what is the target sample size? Besides thinking about these keys questions, it is also useful to be familiar with the terminology associated with sampling. So let's take a moment to review a few key terms. Justice Research and Statistics Association Webinar Page 1 of 13 Statistical Sampling: An Overview for Criminal Justice Researchers April 28, 2016 Erin Farley: Target population is the collection of elements about which we wish to make an inference. While we often speak of populations as very large, hence the need for sampling, target populations can be quite small. So in one case the citizens of California may be the population of interest. In another case you may be interested in a very specific population, say 16 to 18-year-old incarcerated females with a history of heroin addiction. Erin Farley: Unit or case refers to the elements you are interested in. These elements could be things like people, organizations, or documents. Erin Farley: Sampling frame is the list of all units of the population of interest. For example, if your target population was a local university's students, then your sampling frame would be all students enrolled at the local university. For another example, let's say you want to survey residents of Maine. You could use voter registration records and/or the phone book as your sampling frame, but I'm sure many of you realize that voter registration and phone books are imperfect and incomplete lists. More resources can be added in an attempt to cover more of the population. For example, in addition to voter registration and phone books one could add DMV records. But even then this would still exclude people, and this highlights a common challenge. In almost all cases the sampling frame will not perfectly match with the target population, and this leads to errors of coverage. This error, referred to as sampling error, means that the sampling does not accurately represent the population. Erin Farley: Other ways that this may occur is through non-response. This could be volunteers selectively refusing to participate or participants refusing to answer particular questions. Understanding sampling error is extremely important when working with sample estimates and attempting to infer to the larger population, and Stan will discuss this later in the presentation. Erin Farley: Okay. So sampling techniques that we are going to discuss today. There are two general categories of sampling techniques: probability and non- probability sampling. With probability sampling every element has a known chance of being selected into the sample. This process eliminates selection bias, allowing for the sample estimates to be generalized to the larger population. The specific sampling methods that fall under probability sampling that will be discussed today include: simple random sampling, stratified sampling, systematic sampling, and cluster sampling. Erin Farley: With non-probability sampling every unit of the population does not have a known probability of being included in the sample. Subjects are Justice Research and Statistics Association Webinar Page 2 of 13 Statistical Sampling: An Overview for Criminal Justice Researchers April 28, 2016 selected based on the subjective judgment of the researcher. As a result, this type of sampling methodology is vulnerable to selection bias and sample estimates that cannot be generalized or inferred to the larger population. Examples that we will be discussing today include: convenience, judgmental purposive, snowball, and quota sampling. Erin Farley: To begin with simple random sampling. In simple random sampling every unit of the population has the same known probability of being included in the sample, and the sample is selected from the population randomly. In other words, each individual in the population has an equal opportunity to be selected for the sample. Erin Farley: Say there are 1000 people in your sampling frame. For example, utilizing a book of names and numbers, going back to the college example, and you have 1000 people in your sampling frame. So you've identified your sampling frame, and you would then number each element in the sampling frame from 1 to n, and in this case, this example, it would be 1 to 1000. Then you would utilize a chance mechanism to assist in selecting the random sample. Here I have listed a web resource called the random number generator. Erin Farley: Advantages of this method is that it is very simple and it relies on probability theory, and it reduces bias. However, this method is also very expensive and time-consuming, and is not feasible in many circumstances. Erin Farley: But to provide you an example, say you work for a research agency interested in conducting a study of non-violent felony offenders diverted to drug court for treatment of a heroin or opiate addiction. A very specific population. You want to follow-up with and interview participants who fit this criteria, and who graduated in 2015. The total number of graduates is 50, and let's say you're interested in sampling 20. The courts have provided you with a list of offenders, how do you pick your 20? Erin Farley: First you must number your graduates from 1 to 50, and then go to the random number generator, for example, this is what ... the link that I provided, it will send you to this page, and then you fill this site out. So for example, you answer this list of questions. How many sets of numbers do you want to generate? You just want to generate one. How many numbers per set? 20, since you're interested in selecting 20. What is the number range? 50 graduates equals 1 to 50. Unique value? Yes. And then, generate? This could be whatever is the personal preference, I Justice Research and Statistics Association Webinar Page 3 of 13 Statistical Sampling: An Overview for Criminal Justice Researchers April 28, 2016 usually choose least to greatest. And then you press on, randomize now, and it will produce a list of numbers. Erin Farley: So I received a random list of numbers, and this resulted in 1, 3, 7, 9, so forth, popping up, and this is my random sample of 20 individuals, 20 graduates. Erin Farley: Okay. So moving onto systematic sampling, this involves the selection of elements from an ordered sampling frame. So it is a variation of simple random sampling, and it is often considered more practical and easier to use in field settings. Using this procedure every element in the population has a known probability of selection, but not every element has an equal chance of being chosen like simple random sampling. In systematic sampling, it is best used when given a population that is logically homogeneous because systematic sample units are uniformly distributed over the population. You're using a constant interval to select your sample. So a researcher needs to make sure that there is no hidden pattern, as this would threaten randomness. Erin Farley: So here is an example, again. Say you wanted to conduct a household victimization survey in a town. Now let's say there are 40 houses in this very small town, and you want to sample 10 of them. First, you would want to establish your selection interview. You would do this by dividing 40, the number of houses, by 10, which is the sample you are interested in, and the result is 4.