Surveys and Questionnaires
Total Page:16
File Type:pdf, Size:1020Kb
The NIHR Research Design Service for Yorkshire & the Humber Surveys and Questionnaires Authors Nigel Mathers Nick Fox Amanda Hunn This Resource Pack is one of a series produced by The NIHR RDS for the East Midlands / The NIHR RDS for Yorkshire and the Humber. This series has been funded by The NIHR RDS EM / YH. This Resource Pack may be freely photocopied and distributed for the benefit of researchers. However it is the copyright of The NIHR RDS EM / YH and the authors and as such, no part of the content may be altered without the prior permission in writing, of the Copyright owner. Reference as: Mathers N, Fox N. and Hunn A. Surveys and Questionnaires. The NIHR RDS for the East Midlands / Yorkshire & the Humber, 2007. Nigel Mathers Academic Unit of Primary Medical Care Community Sciences Centre Northern General Hospital Herries Road Sheffield S5 7AU Nick Fox School of Health and Related Research (ScHARR) University of Sheffield Regent Court, 30 Regent Street Sheffield S1 4DA Amanda Hunn Left COREC at the end of December 2006 Last updated: 2009 The NIHR RDS for the East www.rds-eastmidlands.nihr.ac.uk Midlands Division of Primary Care, Leicester: [email protected] 14th Floor, Tower building University of Nottingham Nottingham: [email protected] University Park Nottingham NG7 2RD Tel: 0115 823 0500 The NIHR RDS for Yorkshire www.rds-yh.nihr.ac.uk & the Humber ScHARR Sheffield: [email protected] The University of Sheffield Leeds: [email protected] Regent Court York: [email protected] 30 Regent Street Sheffield S1 4DA Tel: 0114 222 0828 © Copyright of The NIHR RDS EM / YH (2009) The NIHR RDS for the East Midlands / Yorkshire & the Humber 2009 2 SURVEYS AND QUESTIONNAIRES Table of Contents Page 1. Introduction …………………………………………………………. 4 2. What is a survey? ………..………………………………………… 5 3. Methods of collecting survey data ……………………………… 8 4. Sampling for surveys ……………………………………………… 11 5. Questionnaire design ……………………………………………… 20 6. Using questionnaires in postal surveys ……………………….. 33 7. Data analysis ………………………………………………………... 36 Summary ……………………………………………………………….. 38 Answers to exercises ………………………………………………… 39 Further reading and resources ……………………………………... 42 Glossary ………………………………………………………………… 44 Appendix 1 The Fog Index …………………………………………… 51 NIHR RDS for the East Midlands / Yorkshire & Humber 2009 3 SURVEYS AND QUESTIONNAIRES 1. Introduction The survey is probably the most commonly used research design in health services research and the social sciences. We have all been asked to take part in a survey at some time. As consumers we are asked about our shopping habits, as users of services we are asked for our opinions of services. The survey is a flexible research approach used to investigate a wide range of topics. Surveys often employ the questionnaire as a tool for data collection. This resource pack considers the use of surveys and questionnaires in health and social care research. LEARNING OBJECTIVES Having successfully completed the work in this chapter, you will be able to: 1. Understand why you might want to use a survey. 2. Describe how to select a sample for a survey. 3. Understand why you might want to use a questionnaire. 4. Understand how the method used for data collection influences the design of the questionnaire. 5. Distinguish between a structured questionnaire, semi-structured questionnaire and a topic guide. 6. Design your own questionnaire and coding frame. 7. Distinguish between open-ended and closed questions. 8. List possible ways of increasing your response rate. The NIHR RDS for the East Midlands / Yorkshire & the Humber 2009 4 SURVEYS AND QUESTIONNAIRES 2. What is a survey? Surveys are a very traditional way of conducting research. They are particularly useful for non- experimental descriptive designs that seek to describe reality. So, for instance, a survey approach may be used to establish the prevalence or incidence of a particular condition. Likewise, the survey approach is frequently used to collect information on attitudes and behaviour. Some issues are best addressed by classical experimental design where participants are randomised to either an intervention group or a control group. In the real world it is not always a very practical design. There may be good reasons, either ethical or practical, why participants cannot be randomly assigned to a particular intervention. It may also be impossible to identify a control group. Control over the randomisation process can also be difficult to achieve. Surveys can take many forms. A survey of the entire population would be known as a census. However usually surveys are restricted to a representative sample of the potential group that the researcher is interested in, for reasons of practicality and cost-effectiveness. Most surveys take one of the following forms: Cross-sectional Surveys Surveys that are carried out at a just one point in time are known as a cross-sectional in design. They provide us with a snapshot of what is happening in that group at that particular time. They usually take a descriptive or exploratory form that simply sets out to describe behaviour or attitudes. So for example, if you want to measure some aspect of client satisfaction, then a cross-sectional descriptive survey would be the recommended approach. Likewise, if you wish to establish the prevalence of depression amongst new mothers, a postal survey might be an appropriate approach. Longitudinal Surveys Alternatively surveys can be longitudinal. A longitudinal survey rather than taking a snap-shot, paints a picture of events or attitudes over time. This may be a matter of months or years. There may be only two discrete surveys or there may be many repeated waves over a long period of time. Longitudinal surveys usually take one of two forms: · cohort surveys - which follow the same group of individuals over time, or · trend surveys - which take repeated samples of different people each time but always use the same core questions. Cohort studies are particularly useful in tracking the progress of particular conditions over time, whereas trend studies set out to measure trends in public opinion and behaviour over time. For instance, take the client satisfaction survey which was mentioned earlier. If we wanted to compare levels of client satisfaction year on year, then a longitudinal trend survey would be recommended. With a trend study it is not necessary to interview the same individuals each time. In fact it is probably better to deliberately avoid the same people since the very fact of participating in a survey can raise levels of awareness and change behaviour. This is particularly true if you are trying to measure awareness of a promotion campaign. A particularly well known version of a trend survey is the General Household Survey which is carried out on an annual basis. A cohort study is more difficult to carry out than a trend survey because the same individuals have to be traced over time and inevitably some participants move house, some fall ill and die, and some just refuse to participate. This loss of participants is known as ‘attrition’. Sample size calculations are even more important for cohort surveys because high levels of attrition can result in too small a sample in The NIHR RDS for the East Midlands / Yorkshire & the Humber 2009 5 SURVEYS AND QUESTIONNAIRES the final stages of the survey. Ideally expected levels of attrition should be calculated and allowed for in the initial sample selection. This means that the early stages of the cohort may be unnecessarily large but in turn this means that you will have adequate numbers in the final wave. A fine example of a cohort study is the National Child Development Study, which is based on an initial sample of children born in one week in 1947 and continued to follow them through over many years. Explanatory or Correlational Surveys We have described how some surveys seek only to describe events and attitudes. It is also possible for surveys to take an explanatory or correlational approach. This means that by using survey data the researcher would try to explore causal relationships between two or more variables. Demonstrating a causal relationship using survey data will always be more difficult than using an experimental design. Nevertheless there will always be situations in which an experimental design is just not possible. Using a longitudinal approach may also help in trying to identify a causal relationship. Statistical tests can be used to show statistically significant differences between groups in a survey. Confounding variables can also be controlled for in the data analysis. 2.1 What are the advantages of using a survey? · Surveys have internal and external validity - A survey which is based on some form of random sampling technique will produce a sample which is representative of the particular population under study and will produce findings which may be generalised to the wider population. Randomised clinical trials (RCTs) on the other hand often have very stringent inclusion and exclusion criteria which make generalisations of the findings very difficult to apply in the real world. · Surveys are efficient - Because surveys can use a random sampling technique to recruit participants, relatively small sample sizes can be used to generate findings which can be used to draw conclusions about the whole population. They are thus a very cost-effective way of finding out what people do, think and want. · Surveys can cover geographically spread samples - Surveys can be undertaken using a wide range of techniques including postal questionnaires and telephone interviews. This means that participants who are widely dispersed can be accessed and included in the sample. · Surveys may have ethical advantages - Since most surveys do not expose individuals to possibly invasive techniques or withhold treatment, they may be considered more ethical, since the individuals included in a study will merely be exposed to events that occur in the real world and would have taken place anyway.