Introduction to Survey Design and Delivery SOCIAL PROGRAMS SCIENCE TOOLS FOR COASTAL Social Science Tools for Coastal Programs Introduction to Survey Design and Delivery About This Publication Some of the most challenging decisions in coastal management stem from the relationship between people and the environment. NOAA provides technical assistance to coastal management professionals addressing complex human-based problems. This publication, “Introduction to Survey Design and Delivery,” is the first in a series of guides developed to bring information to this audience about the use of social science tools in their field of work. For more information, or to obtain additional copies, contact our office at [email protected]. NOAA’s Office for Coastal Management “Coastal management” is the term used by communities and organizations striving to keep the nation’s coasts safe from storms, rich in natural resources, and economically strong. The national lead for these efforts is NOAA’s Office for Coastal Management, an organization devoted to partnerships, science, and good policy. This agency, housed within the National Ocean Service, oversees major initiatives that include the National Coastal Zone Management Program, Coral Reef Conservation Program, Digital Coast, and National Estuarine Research Reserve System. Table of Contents Introduction ................................................................................................................ 1 Population and Sample Size ........................................................................................ 1 Survey Delivery Mechanisms ...................................................................................... 3 Sampling Techniques .................................................................................................. 5 Constructing Survey Questionnaires........................................................................... 7 The Report ................................................................................................................... 12 Appendix A: Guide to Survey Sampling and Analysis Terms ...................................... 13 Appendix B: Guide to Survey Types ............................................................................ 15 1 Introduction Often, the public’s attitudes, perceptions, and beliefs strongly influence management decision making. Social science tools, such as a survey instrument, can help identify the relationship between the community and natural resources. Surveys are a method of collecting data via telephone, mail, or the Internet, as well as in person, using an ordered list of questions to gather important information in a systematic fashion. This technique allows for the collection of information and opinions from a representative sample of individuals related to specific natural resource management issues. This document provides insight into the various types and methods of survey research. The most important component of any successful survey project is to have a strong Conducting a survey study conceptualization. Conducting a survey is much more complex than just asking is much more complex a group of people a series of questions to try to solve a problem or better understand an than just asking a issue. Rather, a survey is a systematic examination of a population by means of a series of groupof people a series specific, targeted questions. These questions should possess direct application in solving of questions to try to an identified problem or issue. The development of a survey simply for the purpose of solve a problem. determining what people think (e.g., pleased, displeased, concerned, happy, etc.) often generates data that are of limited utility. As with any professional undertaking, the most successful projects are generally a result of professional knowledge and expertise. Survey research is no different. Research within any academic discipline must follow specific, methodic protocols. Once information has been collected, specific statistical data testing methods must be employed to produce usable information. The intent of this document is to provide insight into the various methodological components of survey research. The document serves as an overview of survey-related techniques and as a means for coastal managers to understand basic survey information and terminology (Appendix A) to strengthen awareness and understanding of such projects. Population and Sample Size Determining an adequate sample size is critical in generating usable survey results; however, it is difficult to identify a specific population to study large-scale issues such as beach access. Beach visitors include local residents and people across a particular state, as well as people from other states and nations. In this instance, the number of potential visitors is indefinite and virtually impossible to quantify accurately. The size of a sample, however, is not nearly as important as the sample design. If you are conducting a survey, it is more important to obtain a representative sample than a large sample. A sample size that is larger than necessary will in no way compromise the results 2 of your study, but it will misuse time, effort, and financial resources. It is more useful to identify the population of interest and make efforts to achieve a high response rate rather than administer vast numbers of surveys. For the most rigorous approach to a sampling, it is important to consider such items such as the margin of error, degree of accuracy, and the most practical sampling strategy to use. These and other terms are described in Appendix A. With pilot data in hand, an experienced individual can help you work out the total sample size you need for the degree of accuracy you choose. Information on pilot testing is highlighted in Appendix B. Fortunately, you do not need to be concerned about knowing the size of an exceptionally large population (e.g., a community, township, or user group such as recreational fishermen of the Chesapeake Bay). Laws of mathematical probability tell us in such instances that population size is of much less concern than the sample size, though there are links between the size of your sample and the level of analysis that can be performed following the survey. Statistical analysis can be categorized into two types: descriptive and inferential. Descriptive statistics (e.g., mean, median, standard deviation) are used to summarize or describe the survey results, such as the average number of males and females, the number of people over the age of 45, etc. Inferential statistics are a bit more complex. Inferential statistics allow the researcher to arrive at a certain conclusion that is based on probability. For example, a survey of sea kayakers may suggest that females over the age of 45 exhibit greater pro-environmental behaviors than males under the same conditions. Inferential statistics reveal the degree to which different variables are related. In the previous example, pro-environmental behavior is related to both gender and age. The following is a general example to show how sampling works with a large population. If you survey 500 people, stastically your results are equally significant whether the total population is 10,000 or 100,000. For this example, where a large population whose exact number is unknown, an appropriate minimum sample size is 384. By sampling this number of people (with a 100 percent response rate) you are ensuring a 95 percent confidence level with a confidence interval or margin of error of 5. This confidence level and margin of error is the standard desired level of significance for most survey research. The confidence level reflects how certain you can be that the survey results are accurate. In this instance, the results would prove to be 95 percent accurate, and the 5 percent margin of error is commonly sought. You could therefore state that results of your survey are 95 percent accurate, plus or minus 5 percent of people surveyed. Please remember that this sampling scenario is a general example and is not applicable to all situations. 3 Main Points: • There is much greater benefit in obtaining a representative sample than in obtaining an exceedingly large sample. • If you have the time, contacts, and resources to be more rigorous about a sampling scheme, try to ensure that your responses possess a 95 percent confidence level. • Determining an adequate sample size is best determined by someone professionally trained in survey techniques. Survey Delivery Mechanisms Surveys can be administered in many different ways, which are commonly divided into two categories: interview-based and self-completed. Common interview-based mechanisms include face-to-face interviews and telephone surveys. Self-completed survey instruments commonly include mailback surveys, hand-delivered questionnaires, and Web surveys. The following table is a brief synopsis of each survey type and various positive and negative aspects of each. Description Positives Negatives Face-to-Face Interview A directed, one-on-one + Obtain great level of detail - Analysis of results can conversation ranging from + Ability to observe non- be cumbersome and time casual to highly structured verbal communication along consuming with verbal responses - Limited generalization to a greater population Telephone Survey A directed one-on-one + Highly effective in - Skilled interviewer is telephone interview ranging generating timely responses needed to help guide from casual to highly + Large