sign in sign up
<<
Home , Quantitative research

TRANSLATING INTO PRACTICE

Section Editors: Jeannine M. Brant, Marilyn L. Haas-Haseman, Steven H. Wei, and Rita Wickham

Understanding and Evaluating Research

JULIE PONTO, PhD, APRN, AGCNS-BC, AOCNS®

From Winona State University, Rochester, Minnesota variety of methodo- describe and explore human behav- logic approaches ex- ior, surveys are therefore frequently Author's disclosures of potential conflicts of interest are found at the end of this article. ist for individuals in- used in social and psychological re- Correspondence to: Julie Ponto, PhD, APRN, terested in conducting search (Singleton & Straits, 2009). AGCNS-BC, AOCNS®, Winona State University, Aresearch. Selection of a research Information has been obtained Graduate Programs in Nursing, 859 30th Avenue approach depends on a number of from individuals and groups through South East, Rochester, MN 55904. E-mail: [email protected] factors, including the purpose of the use of survey research for de- © 2015 Harborside Press® the research, the type of research cades. It can range from asking a few questions to be answered, and the targeted questions of individuals on availability of resources. The pur- a street corner to obtain information pose of this article is to describe related to behaviors and preferences, survey research as one approach to a more rigorous study using mul- to the conduct of research so that tiple valid and reliable instruments. the reader can critically evaluate Common examples of less rigorous the appropriateness of the con- surveys include or politi- clusions from studies employing cal surveys of consumer patterns and survey research. public opinion polls. Survey research has historically SURVEY RESEARCH included large population-based data Survey research is defined as collection. The primary purpose of “the collection of information from this type of survey research was to ob- a of individuals through their tain information describing charac- responses to questions” (Check & teristics of a large sample of individu- Schutt, 2012, p. 160). This type of re- als of interest relatively quickly. Large search allows for a variety of methods census surveys obtaining information to recruit participants, collect data, reflecting demographic and personal and utilize various methods of instru- characteristics and consumer feed- mentation. Survey research can use back surveys are prime examples. quantitative research strategies (e.g., These surveys were often provided using with numerical- through the mail and were intended ly rated items), to describe demographic characteris- strategies (e.g., using open-ended tics of individuals or obtain opinions questions), or both strategies (i.e., on which to base programs or prod- J Adv Pract Oncol 2015;6:168–171 mixed methods). As it is often used to ucts for a population or group.

168 SURVEY RESEARCH TRANSLATING RESEARCH INTO PRACTICE

More recently, survey research has developed to accurately draw conclusions about the popu- into a rigorous approach to research, with scientifi- lation, the sample must include individuals with cally tested strategies detailing who to include (rep- characteristics similar to the population. resentative sample), what and how to distribute It is therefore necessary to correctly identify (survey method), and when to initiate the survey the population of interest (e.g., individuals with and follow up with nonresponders (reducing non- lung cancer currently receiving treatment vs. all response error), in order to ensure a high-quality individuals with lung cancer). The sample will research process and outcome. Currently, the term ideally include individuals who reflect the intend- “survey” can reflect a range of research aims, sam- ed population in terms of all characteristics of the pling and recruitment strategies, in- population (e.g., sex, socioeconomic characteris- struments, and methods of survey administration. tics, symptom experience) and contain a similar Given this range of options in the conduct of distribution of individuals with those character- survey research, it is imperative for the consumer/ istics. As discussed by Mady Stovall beginning on reader of survey research to understand the poten- page 162, Fujimori et al. (2014), for example, were tial for bias in survey research as well as the tested interested in the population of oncologists. The techniques for reducing bias, in order to draw ap- authors obtained a sample of oncologists from two propriate conclusions about the information re- hospitals in Japan. These participants may or may ported in this manner. Common types of error in not have similar characteristics to all oncologists research, along with the sources of error and strat- in Japan. egies for reducing error as described throughout Participant recruitment strategies can affect this article, are summarized in the Table. the adequacy and representativeness of the sam- ple obtained. Using diverse recruitment strategies SAMPLING can help improve the size of the sample and help The goal of sampling strategies in survey re- ensure adequate coverage of the intended popula- search is to obtain a sufficient sample that is rep- tion. For example, if a survey researcher intends resentative of the population of interest. It is often to obtain a sample of individuals with breast can- not feasible to collect data from an entire popula- cer representative of all individuals with breast tion of interest (e.g., all individuals with lung can- cancer in the United States, the researcher would cer); therefore, a subset of the population or sample want to use recruitment strategies that would re- is used to estimate the population responses (e.g., cruit both women and men, individuals from ru- individuals with lung cancer currently receiving ral and urban settings, individuals receiving and treatment). A large random sample increases the not receiving active treatment, and so on. Because likelihood that the responses from the sample will of the difficulty in obtaining samples representa- accurately reflect the entire population. In order tive of a large population, researchers may focus

Table. Sources of Error in Survey Research and Strategies to Reduce Error Type of error Source of error Strategies to reduce error Coverage error Unknown or zero chance of individuals in the Multimode design population being included in the sample Sampling error Individuals included in the sample do Clearly identified population of interest; diverse not represent the characteristics of the participant recruitment strategies; large, random population sample Questions/instruments do not accurately Valid, reliable instruments; pretest questions; user- error reflect the topic of interest; questionnaires/ friendly graphics, visual characteristics do not evoke truthful answers Nonresponse Lack of response from all individuals in User-friendly survey design; follow-up procedures error sample for nonresponders Note. Information from Dillman et al. (2014), Singleton & Straits (2009), Check & Schutt (2012).

169 TRANSLATING RESEARCH INTO PRACTICE PONTO

the population of interest to a subset of individu- proving the visual appeal and graphics of surveys als (e.g., women with stage III or IV breast can- by using a font size appropriate for the respon- cer). Large census surveys require extremely large dents, ordering items logically without creating samples to adequately represent the characteris- unintended response bias, and arranging items tics of the population because they are intended to clearly on each page can increase the response represent the entire population. rate to electronic questionnaires. Attending to these and other issues in electronic questionnaires DATA COLLECTION METHODS can help reduce measurement error (i.e., lack of Survey research may use a variety of data col- validity or reliability) and help ensure a better re- lection methods with the most common being sponse rate. questionnaires and interviews. Questionnaires Conducting interviews is another approach to may be self-administered or administered by a data collection used in survey research. Interviews professional, may be administered individually or may be conducted by phone, computer, or in per- in a group, and typically include a series of items son and have the benefit of visually identifying the reflecting the research aims. Questionnaires may nonverbal response(s) of the interviewee and sub- include demographic questions in addition to val- sequently being able to clarify the intended ques- id and reliable research instruments (Costanzo, tion. An interviewer can use probing comments Stawski, Ryff, Coe, & Almeida, 2012; DuBenske et to obtain more information about a question or al., 2014; Ponto, Ellington, Mellon, & Beck, 2010). topic and can request clarification of an unclear It is helpful to the reader when authors describe response (Singleton & Strait, 2009). Interviews the contents of the survey so that can be costly and time intensive, and therefore are the reader can interpret and evaluate the poten- relatively impractical for large samples. tial for errors of validity (e.g., items or instruments Some authors advocate for using mixed meth- that do not measure what they are intended to ods for survey research when no one method is measure) and reliability (e.g., items or instruments adequate to address the planned research aims, that do not measure a construct consistently). to reduce the potential for measurement and non- Helpful examples of articles that describe the sur- response error, and to better tailor the study meth- vey instruments exist in the literature (Buerhaus ods to the intended sample (Dillman et al., 2014; et al., 2012). Singleton & Strait, 2009). For example, a mixed Questionnaires may be in paper form and methods survey research approach may begin with mailed to participants, delivered in an electronic distributing a questionnaire and following up with format via email or an Internet-based program telephone interviews to clarify unclear survey re- such as SurveyMonkey, or a combination of both, sponses (Singleton & Straits, 2009). Mixed meth- giving the participant the option to choose which ods might also be used when visual or auditory method is preferred (Ponto et al., 2010). Using a deficits preclude an individual from completing a combination of methods of survey administration questionnaire or participating in an . can help to ensure better sample coverage (i.e., all individuals in the population having a chance of FUJIMORI et al.: SURVEY RESEARCH inclusion in the sample) therefore reducing cover- Fujimori et al. (2014) described the use of sur- age error (Dillman, Smyth, & Christian, 2014; Sin- vey research in a study of the effect of communi- gleton & Strait, 2009). For example, if a researcher cation skills training for oncologists on oncologist were to only use an Internet-delivered question- and patient outcomes (e.g., oncologist’s perfor- naire, individuals without access to a computer mance and confidence and patient’s distress, sat- would be excluded from participation. Self-ad- isfaction, and trust). A sample of 30 oncologists ministered mailed, group, or Internet-based ques- from two hospitals was obtained and though the tionnaires are relatively low cost and practical for authors provided a power analysis concluding an a large sample (Check & Schutt, 2012). adequate number of oncologist participants to Dillman et al. (2014) have described and tested detect differences between baseline and follow- a tailored design method for survey research. Im- up scores, the conclusions of the study may not

170 SURVEY RESEARCH TRANSLATING RESEARCH INTO PRACTICE

be generalizable to a broader population of on- constructs of interest. Survey research, like all re- cologists. Oncologists were randomized to either search, has the potential for a variety of sources an intervention group (i.e., communication skills of error, but several strategies exist to reduce the training) or a control group (i.e., no training). potential for error. Advanced practitioners aware Fujimori et al. (2014) chose a quantitative of the potential sources of error and strategies to approach to collect data from oncologist and pa- improve survey research can better determine tient participants regarding the study outcome how and whether the conclusions from a survey variables. Self-report numeric ratings were used research study apply to practice. l to measure oncologist confidence and patient distress, satisfaction, and trust. Oncologist confi- Disclosure dence was measured using two instruments each The author has no potential conflicts of inter- using 10-point Likert rating scales. The Hospital est to disclose. Anxiety and Depression Scale (HADS) was used to measure patient distress and has demonstrated References validity and reliability in a number of populations Bjelland, I., Dahl, A. A., Haug, T. T., & Neckelmann, D. (2002). including individuals with cancer (Bjelland, Dahl, The validity of the Hospital Anxiety and Depression Haug, & Neckelmann, 2002). Patient satisfaction Scale: An updated . Journal of Psycho- somatic Research, 52, 69–77. http://dx.doi.org/10.1016/ and trust were measured using 0 to 10 numeric S0022-3999(01)00296-3 rating scales. Numeric observer ratings were used Buerhaus, P. I., DesRoches, C., Applebaum, S., Hess, R., Nor- to measure oncologist performance of commu- man, L. D., & Donelan, K. (2012). Are nurses ready for nication skills based on a videotaped interaction health care reform? A decade of survey research. Nursing , 30, 318–330. with a standardized patient. Participants com- Check, J., & Schutt, R. K. (2012). Survey research. In J. Check pleted the same questionnaires at baseline and & R. K. Schutt (Eds.). Research methods in education. (pp. follow-up. 159–185). Thousand Oaks, CA: Sage Publications. The authors clearly describe what data were Costanzo, E. S., Stawski, R. S., Ryff, C. D., Coe, C. L., & Al- meida, D. M. (2012). Cancer survivors’ responses to daily collected from all participants. Providing addi- stressors: Implications for quality of life. Health Psychol- tional information about the manner in which ogy, 31, 360–370. http://dx.doi.org/10.1037/a0027018 questionnaires were distributed (i.e., electronic, Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, mail), the setting in which data were collected phone, mail, and mixed-mode surveys: The tailored design method (4th ed). Hoboken, NJ: John Wiley & Sons, Inc. (e.g., home, clinic), and the design of the survey DuBenske, L. L., Gustafson, D. H., Namkoong, K. H., Atwood, instruments (e.g., visual appeal, format, content, R. P., Brown, R. L., Chih, M. Y.,…Cleary, J. (2014). CHESS arrangement of items) would assist the reader in improves cancer caregivers’ burden and mood: Results of drawing conclusions about the potential for mea- an eHealth RCT. Health , 33, 1261–1272. http:// dx.doi.org/10.1037/a0034216 surement and nonresponse error. The authors de- Fujimori, M., Shirai, Y., Asai, M., Kubota, K., Katsumata, N., & scribe conducting a follow-up phone call or mail Uchitomi, Y. (2014). Effect of communication skills train- inquiry for nonresponders, using the Dillman et al. ing program for oncologists based on patient preferences (2014) tailored design for survey research follow- for communication when receiving bad news: A random- ized controlled trial. Journal of Clinical Oncology, 32, up may have reduced nonresponse error. 2166–2172. http://dx.doi.org/10.1200/JCO.2013.51.2756 Ponto, J. A., Ellington, L., Mellon, S., & Beck, S. L. (2010). CONCLUSIONS Predictors of adjustment and growth in women with Survey research is a useful and legitimate recurrent ovarian cancer. Oncology Nursing Forum, 37, 357–364. http://dx.doi.org/10.1188/10.ONF.357-364 approach to research that has clear benefits in Singleton, R. A., & Straits, B. C. (2009). Approaches to social helping to describe and explore variables and research (5th ed.). New York: Oxford University Press.

171