Maintaining Evaluation Designs in Long Term Heartbeat Wales Case Study
Total Page：16
File Type：pdf, Size：1020Kb
J7ournal of Epidemiology and Community Health 1993; 47: 127-133 Maintaining evaluation designs in long term community based health promotion programmes: Heartbeat Wales case study Don Nutbeam, Christopher Smith, Simon Murphy, John Catford Abstract in influencing a range of predisposing factors. Study objective-To examine the difficulties These include those relevant to the individual of developing and maintaining outcome (such as knowledge, attitudes, and skills), those evaluation designs in long term, community relevant to the wider social group (social and based health promotion programmes. cultural norms), and factors in the wider environ- Design-Semistructured interviews of ment (such as regulation of exposure to hazards, health promotion managers. and access to goods and services). Influencing Setting-Wales and two reference health social norms and changing the wider environment regions in England. generally requires a total community response and Participants-Nine health promotion man- hence the rationale for a community based agers in Wales and 18 in England. approach. More practical reasons include (i) the Measurements and main results-Informa- opportunities to utilise the existing resources and tion on selected heart health promotion networks within communities to provide a struc- activity undertaken or coordinated by health ture for programme delivery; (ii) the enhanced authorities from 1985-90 was collected. The impact from combining complementary indi- Heartbeat Wales coronary heart disease pre- vidual and environmental approaches to achieving vention programme was set up in 1985, and a change; and (iii) the greater potential for creating research and evaluation strategy was estab- reproducible and self sustaining programmes in lished to complement the intervention. A 'real life' settings than those from narrowly substantial increase in the budget occurred defined studies of volunteer populations in con- over the period. In the reference health trolled settings.8 regions in England this initiative was noted These new intervention approaches have posed and rapidly taken up, thus compromising special challenges for developing appropriate and their use as control areas. manageable evaluation designs that can be main- Conclusion-Information on large scale, tained throughout the life of a health promotion community based health promotion pro- programme. This paper uses the evaluation of the grammes can disseminate quickly and inter- Heartbeat Wales programme as a case study to fere with classic intervention/evaluation assess the sorts of difficulties that can be control designs through contamination. encountered and how they may be overcome. Alternative experinental designs for assess- ing the effectiveness of long term interven- tion programmes need to be considered. Heartbeat Wales These should not rely solely on the use of During the early 1 980s several health bodies in the reference populations, but should balance United Kingdom were becoming conscious ofthe the measurement ofoutcome with an assess- need to respond to the high level of cardiovascular ment of the process of change in com- disease in the country through preventive stra- munities. The development and use of tegies. The UK government's Welsh Office and intervention exposure measures together the then national agency for health education, the with well structured and comprehensive pro- Health Education Council, agreed to establish a cess evaluation in both the intervention and community based demonstration project in reference areas is recommended. Wales. This decision was supported in 1983 by a consensus conference, sponsored by the major Health Promotion Epidemiol Community Health 1993; 47: 127-133 UK health organisations, which recommended a Authority for Wales, comprehensive range of actions by government, Cardiff agriculture, education, health services, and the D Nutbeam C Smith The late 1970s and early 1980s saw the develop- media.9 S Murphy ment of a number oflarge scale, community based The 'Welsh Heart Programme' was at first Institute for Health health promotion programmes in the United administered through the University of Wales Promotion, University of Wales Coliege of States and Europe, most of which were directed College ofMedicine and then subsequently by the Medicine, Cardiff towards reducing modifiable risk factors for coro- Health Promotion Authority for Wales. The J Catford nary heart disease. -5 These community based directorate responsible for the management of the programmes were born out of a growing con- programme was Correspondence to: recruited at the end of 1984 and Professor D Nutbeam, sensus on the content, strategy, and methods for the programme was launched publicy as 'Heart- Department of Public the prevention of coronary heart disease which Health, University of beat Wales' in March 1985 for an initial five year Sydney, Sydney, 2006, emphasised the importance of lifestyle and period. The long term goal of the programme was Australia. behavioural change on a population wide basis.6 7 to develop and evaluate as a demonstration pro- Accepted for publication Fundamental to the achievement and mainten- ject, a regional strategy that would contribute to a August 1992 ance of lifestyle and behavioural change is success sustained reduction in coronary heart disease 128 Don Nutbeam, Christopher Snzith, Simon Murphy, 3tohn Catford incidence, morbidity, and mortality in the general These were generally supported by additional population of Wales, particularly among those evaluation studies to improve confidence in the under the age of 65. Desired health outcomes for causal nature of observed net differences between the first five years were defined as improvements in the populations.8 '3 In most examples these non-smoking, healthy nutrition, physical exercise, designs were non-random, as the intervention hypertension control, and cardiopulmonary resus- areas were generally chosen on an opportunistic citation. 10 basis. The supporting studies usually examined When the programme was established there the impact on targeted community networks and were three strategic aims namely: the process ofdiffusion ofthe intervention. As part (i) Leadership to coordinate, support, initiate, of this approach, measures for use in population and monitor action at local and regional levels surveys were developed to determine programme which would encourage improvements in modifi- exposure (that is, awareness and participation) able risk factors and behaviours for coronary heart among the target population.813 disease prevention. The evaluation ofthe North Karelia programme (ii) Demonstration-to stimulate, disseminate, was based on comparisons between a single inter- and assist the development of strategies and vention and single reference community, as too is programmes to promote health and prevent coro- the Pawtucket Heart Health programme.' 16 The nary heart disease throughout the UK and Stanford and Minnesota programmes have used overseas. several intervention and reference commu- (iii) Experimentation to research, develop, nities. 2 317 The basic evaluation design for the and evaluate a range of new projects and iniatives Heartbeat Wales programme was established for heart health promotion, and provide feedback using this quasi-experimental approach. It was on their feasibility and impact; constructed in such a way as to allow both internal These aims gave direction to the development of comparison ofdifferences in outcome between the the evaluation strategy, which also took into nine health districts (each ofwhich was to receive a account the essential elements of the intervention contrasting intervention) and external compa- that sought changes in both personal behaviours risons between Wales and a large single matched and environmental factors.'0 11 Measures to reference area of similar size. At its simplest, the monitor both these changes were built into the intention was to measure change in the population evaluation design, which also recognised the need risks for coronary heart disease (and associated to balance measurement of outcome with investi- predisposing factors) after the 'input' of a coordi- gation of the process of achieving change.'2 nated and relatively well resourced community based intervention, and to compare this with EVALUATION DESIGN PROBLEMS AND SOLUTIONS change in the reference area with no correspond- At the time of constructing the evaluation design ing input, conducting 'business as usual'. there were relatively few well tested models for the The reference area was selected after examin- evaluation of community based programmes. ation of available health, social, and economic Those that existed had been developed from data as being closest in profile to Wales at baseline. relatively small and more tightly defined interven- Full details have been published previously. 18 As a tions, in individual communities, towns, and cities result of the difficulties experienced in the North (populations 100-200 000). Wales is a compara- Karelia programme, which shared a common tively large country, however, with a substantially boundary with its chosen reference area, a geo- greater population (2-8 million), defined by nine graphically separate area was deliberately selected health districts. The experience provided by these to reduce problems of media overspill and more other projects had already indicated a number of general contamination that might occur along a basic problems in applying experimental designs shared