UK Data Archive Study Number 5299 - National Evaluation of the New Deal for Communities Programme: Household Survey Data, 2002-2008 New Deal for Communities Household Survey 2008 Research Study Conducted for the Regeneration and Economic Development Analysis Division & Sheffield Hallam University April - October 2008 Legal notice © 2010 Ipsos MORI – all rights reserved. The contents of this report constitute the sole and exclusive property of Ipsos MORI. Ipsos MORI retains all right, title and interest, including without limitation copyright, in or to any Ipsos MORI trademarks, technologies, methodologies, products, analyses, software and know-how included or arising out of this report or used in connection with the preparation of this report. No license under any copyright is hereby granted or implied. The contents of this report are of a commercially sensitive and confidential nature and intended solely for the review and consideration of the person or entity to which it is addressed. No other use is permitted and the addressee undertakes not to disclose all or part of this report to any third party (including but not limited, where applicable, pursuant to the Freedom of Information Act 2000) without the prior written consent of the Company Secretary of Ipsos MORI. Contents Technical Report .............................................................................1 1 Scope of the Survey................................................................................ 1 2 Sample Design........................................................................................ 2 3 Pilot ......................................................................................................... 7 4 Questionnaire Design.............................................................................. 8 5 Fieldwork............................................................................................... 13 6 Response Rates.................................................................................... 15 7 Foreign Language Interviews................................................................ 16 8 Data Analysis ........................................................................................ 17 Appendices....................................................................................29 Guide to SPSS Data Format of Variable Names Derived Variables Weighting Longitudinal Data Comparator Data Guide to Computer Tables Glossary of Terms Comparator Wards Overall Combined Response Rates (Longitudinal and Cross-sectional) Longitudinal Response Rates Cross-sectional Response Rates Panel Mailouts Pre-Survey Advanced Letters and Frequently Asked Questions Sheets Interviewer Instructions Longitudinal Contact Sheet Cross-sectional Contact Sheet 2008 NDC Questionnaires 2008 NDC Showcards 2008 NDC Language Sheet Questions Cut Since 2004 Technical Report 1 © 2010 Ipsos MORI. New Deal for Communities - Technical Report 2008 Technical Report This volume contains the research methodology used in the 2008 New Deal for Communities (NDC) Household Survey, conducted by Ipsos MORI and GfK NOP for the Regeneration and Economic Development Analysis Division (REDA) of Communities and Local Government (CLG). The survey was conducted as part of the national evaluation of the NDC programme, led by Sheffield Hallam University (SHU). 1 Scope of the Survey Ipsos MORI and GfK NOP interviewed a total of 15,838 resident’s aged 16+ in 39 NDC areas throughout England, between 16 May – 15 October 2008. Interviews were conducted face-to-face in-home, using Computer Assisted Personal Interviewing (CAPI). The overall aim of the survey is to provide information to supplement existing secondary administrative data sources on the key outcome areas of the NDC Programme. In particular, the questionnaire focuses on providing evidence on outcomes that could not be obtained from other sources. This is the fourth survey conducted among NDC residents; with previous waves in 2002, 2004 and 2006. The survey is designed to track change over time, including direct change by following-up residents who were interviewed previously. In addition, a comparator sample of 3,100 residents aged 16+ living in deprived areas was interviewed between 16 June – 4 November 2008. This national deprived area survey aims to help understand how NDC areas differ from other deprived areas. Then, following the design of the main survey, it aims to identify differences in how these areas change over time, via follow- up and longitudinal surveys. The design for this element of the survey is discussed separately below. Finally, in this wave, an additional questionnaire module was included in four NDC areas that are also in receipt of Mixed Communities Intiative (MCI) funding: Coventry, Leicester, Knowsley and Sheffield. 1 © 2010 Ipsos MORI. New Deal for Communities - Technical Report 2008 2 Sample Design The survey comprised a combined panel and cross-sectional “top-up” design. This model aims to complete as many interviews as possible at those addresses where an original interview was achieved in 2006 (i.e. either with the original respondent or someone else if they have moved/died), and then “top up” with new cross-sectional sample to a total of 400 interviews in each NDC area. For the longitudinal sample, interviews were attempted with the original respondent. If the original respondent was no longer living at the address, interviewers selected a new respondent i.e. they conducted a cross-sectional interview (see Respondent Selection Section 2.5). The sampling for the top-up cross-sectional element of the NDC household survey involved a number of steps: a random selection of addresses from within each of the NDC partnership areas, excluding the successful addresses from 2006; random selection of one property or dwelling unit at each sampled address; random selection of one household within each selected property; random selection of one adult aged 16+ within each selected household. Following this design, the 2008 sample breaks down as follows: 8,703 longitudinal interviews; 1,605 cross-sectional interviews at longitudinal addresses (i.e previous respondent had either moved or died); and 5,530 cross-sectional top-up interviews. It is worth noting that in 2006 the sample size was reduced from 500 interviews to 400, and this same sample size was used in 2008. In order to ensure that new addresses were included and to minimise the error associated with the overall sample, a random sample of 400 longitudinal respondents was taken from the achieved 2006 sample, i.e. issuing the same number of contacts as our target number of interviews (400). This optimised the number of longitudinal interviews (given the additional value of these to the evaluation), while taking account of changes in the area in order to provide representative cross-sectional data. It also ensured the balance of the sample remained the same, so that each element of the issued sample comprised the same proportion of the total as in 2004 and 2006, which is an important consideration for year-on-year comparisons. 2 © 2010 Ipsos MORI. New Deal for Communities - Technical Report 2008 2.1 Sample Frame In all four survey waves, the REDA provided digital boundary definitions for each of the NDC partnership areas. All addresses within these boundaries were selected using Ordinance Survey’s AddressPoint, which provides an accurate grid reference for each and is based on the Royal Mail’s Postcode Address File (PAF). Further checks were also run to remove addresses identified as businesses, e.g. those containing key words in specific address fields, such as ‘Business Centre’. Prior to each survey wave, these definitions have been checked with partnerships and against those provided previously. They were asked to confirm details of any major changes, for example, evidence of demolitions or new build properties. In particular, the digital boundary definitions for all NDCs were revised before the 2004 survey, and the original boundaries used in 2002 were compared against the new sets and checked for differences. This comparison revealed some discrepancies, although these were relatively minor. A total of 38 of the interviews conducted in 2002 fell outside the boundaries provided in 2004; and 24 of these individuals were re-interviewed in 2004. A flag is included in the relevant SPSS files to indicate who these respondents are. In 2004 the boundaries supplied by the REDA were also checked against boundaries supplied by the Social Disadvantage Research Centre (SDRC) at Oxford University. In 2006 this checking revealed some “holes”, where addresses appeared in the 2004 sample frame but not in that supplied in 2006. In particular, this revealed that the boundary provided for Doncaster NDC was incorrect; a residential area in the south-west quarter was excluded after the Partnership’s original submission for funding was made as they were asked to reduce the size of the area. However, no changes were made to the area definition files supplied, and as a result, just over 100 interviews were conducted outside the NDC area in 2002 and 2004 (as the area outside the boundary consisted of around 20% of the total addresses in the original definition). Doncaster NDC have been provided with revised data outputs on this basis. However, the aggregate data for 2002 and 2004 includes these interviews, given the very marginal impact on the data at this level. The 2006 and 2008 surveys use the revised area definitions for Doncaster. The digital boundary