Understanding and Evaluating Survey Research
Total Page:16
File Type:pdf, Size:1020Kb
TRANSLATING RESEARCH INTO PRACTICE Section Editors: Jeannine M. Brant, Marilyn L. Haas-Haseman, Steven H. Wei, and Rita Wickham Understanding and Evaluating Survey Research JULIE PONTO, PhD, APRN, AGCNS-BC, AOCNS® From Winona State University, Rochester, Minnesota variety of methodo- describe and explore human behav- logic approaches ex- ior, surveys are therefore frequently Author's disclosures of potential conflicts of interest are found at the end of this article. ist for individuals in- used in social and psychological re- Correspondence to: Julie Ponto, PhD, APRN, terested in conducting search (Singleton & Straits, 2009). AGCNS-BC, AOCNS®, Winona State University, Aresearch. Selection of a research Information has been obtained Graduate Programs in Nursing, 859 30th Avenue approach depends on a number of from individuals and groups through South East, Rochester, MN 55904. E-mail: [email protected] factors, including the purpose of the use of survey research for de- © 2015 Harborside Press® the research, the type of research cades. It can range from asking a few questions to be answered, and the targeted questions of individuals on availability of resources. The pur- a street corner to obtain information pose of this article is to describe related to behaviors and preferences, survey research as one approach to a more rigorous study using mul- to the conduct of research so that tiple valid and reliable instruments. the reader can critically evaluate Common examples of less rigorous the appropriateness of the con- surveys include marketing or politi- clusions from studies employing cal surveys of consumer patterns and survey research. public opinion polls. Survey research has historically SURVEY RESEARCH included large population-based data Survey research is defined as collection. The primary purpose of “the collection of information from this type of survey research was to ob- a sample of individuals through their tain information describing charac- responses to questions” (Check & teristics of a large sample of individu- Schutt, 2012, p. 160). This type of re- als of interest relatively quickly. Large search allows for a variety of methods census surveys obtaining information to recruit participants, collect data, reflecting demographic and personal and utilize various methods of instru- characteristics and consumer feed- mentation. Survey research can use back surveys are prime examples. quantitative research strategies (e.g., These surveys were often provided using questionnaires with numerical- through the mail and were intended ly rated items), qualitative research to describe demographic characteris- strategies (e.g., using open-ended tics of individuals or obtain opinions questions), or both strategies (i.e., on which to base programs or prod- J Adv Pract Oncol 2015;6:168–171 mixed methods). As it is often used to ucts for a population or group. 168 SURVEY RESEARCH TRANSLATING RESEARCH INTO PRACTICE More recently, survey research has developed to accurately draw conclusions about the popu- into a rigorous approach to research, with scientifi- lation, the sample must include individuals with cally tested strategies detailing who to include (rep- characteristics similar to the population. resentative sample), what and how to distribute It is therefore necessary to correctly identify (survey method), and when to initiate the survey the population of interest (e.g., individuals with and follow up with nonresponders (reducing non- lung cancer currently receiving treatment vs. all response error), in order to ensure a high-quality individuals with lung cancer). The sample will research process and outcome. Currently, the term ideally include individuals who reflect the intend- “survey” can reflect a range of research aims, sam- ed population in terms of all characteristics of the pling and recruitment strategies, data collection in- population (e.g., sex, socioeconomic characteris- struments, and methods of survey administration. tics, symptom experience) and contain a similar Given this range of options in the conduct of distribution of individuals with those character- survey research, it is imperative for the consumer/ istics. As discussed by Mady Stovall beginning on reader of survey research to understand the poten- page 162, Fujimori et al. (2014), for example, were tial for bias in survey research as well as the tested interested in the population of oncologists. The techniques for reducing bias, in order to draw ap- authors obtained a sample of oncologists from two propriate conclusions about the information re- hospitals in Japan. These participants may or may ported in this manner. Common types of error in not have similar characteristics to all oncologists research, along with the sources of error and strat- in Japan. egies for reducing error as described throughout Participant recruitment strategies can affect this article, are summarized in the Table. the adequacy and representativeness of the sam- ple obtained. Using diverse recruitment strategies SAMPLING can help improve the size of the sample and help The goal of sampling strategies in survey re- ensure adequate coverage of the intended popula- search is to obtain a sufficient sample that is rep- tion. For example, if a survey researcher intends resentative of the population of interest. It is often to obtain a sample of individuals with breast can- not feasible to collect data from an entire popula- cer representative of all individuals with breast tion of interest (e.g., all individuals with lung can- cancer in the United States, the researcher would cer); therefore, a subset of the population or sample want to use recruitment strategies that would re- is used to estimate the population responses (e.g., cruit both women and men, individuals from ru- individuals with lung cancer currently receiving ral and urban settings, individuals receiving and treatment). A large random sample increases the not receiving active treatment, and so on. Because likelihood that the responses from the sample will of the difficulty in obtaining samples representa- accurately reflect the entire population. In order tive of a large population, researchers may focus Table. Sources of Error in Survey Research and Strategies to Reduce Error Type of error Source of error Strategies to reduce error Coverage error Unknown or zero chance of individuals in the Multimode design population being included in the sample Sampling error Individuals included in the sample do Clearly identified population of interest; diverse not represent the characteristics of the participant recruitment strategies; large, random population sample Measurement Questions/instruments do not accurately Valid, reliable instruments; pretest questions; user- error reflect the topic of interest; questionnaires/ friendly graphics, visual characteristics interviews do not evoke truthful answers Nonresponse Lack of response from all individuals in User-friendly survey design; follow-up procedures error sample for nonresponders Note. Information from Dillman et al. (2014), Singleton & Straits (2009), Check & Schutt (2012). 169 TRANSLATING RESEARCH INTO PRACTICE PONTO the population of interest to a subset of individu- proving the visual appeal and graphics of surveys als (e.g., women with stage III or IV breast can- by using a font size appropriate for the respon- cer). Large census surveys require extremely large dents, ordering items logically without creating samples to adequately represent the characteris- unintended response bias, and arranging items tics of the population because they are intended to clearly on each page can increase the response represent the entire population. rate to electronic questionnaires. Attending to these and other issues in electronic questionnaires DATA COLLECTION METHODS can help reduce measurement error (i.e., lack of Survey research may use a variety of data col- validity or reliability) and help ensure a better re- lection methods with the most common being sponse rate. questionnaires and interviews. Questionnaires Conducting interviews is another approach to may be self-administered or administered by a data collection used in survey research. Interviews professional, may be administered individually or may be conducted by phone, computer, or in per- in a group, and typically include a series of items son and have the benefit of visually identifying the reflecting the research aims. Questionnaires may nonverbal response(s) of the interviewee and sub- include demographic questions in addition to val- sequently being able to clarify the intended ques- id and reliable research instruments (Costanzo, tion. An interviewer can use probing comments Stawski, Ryff, Coe, & Almeida, 2012; DuBenske et to obtain more information about a question or al., 2014; Ponto, Ellington, Mellon, & Beck, 2010). topic and can request clarification of an unclear It is helpful to the reader when authors describe response (Singleton & Strait, 2009). Interviews the contents of the survey questionnaire so that can be costly and time intensive, and therefore are the reader can interpret and evaluate the poten- relatively impractical for large samples. tial for errors of validity (e.g., items or instruments Some authors advocate for using mixed meth- that do not measure what they are intended to ods for survey research when no one method is measure) and reliability (e.g., items or instruments adequate to address the planned research aims, that do not measure a construct consistently). to reduce the potential