A Health Impact Assessment of the Draft Hawkes Bay District Council S Graffiti Strategy

Process and Impact Evaluation Results: Health Impact Assessment on the implementation of the Hawke’s Bay District Health Board’s Oral Health Strategy in the community of Flaxmere

Velma McClellan & Louise Signal Health, Equity and Wellbeing Impact Assessment Research Unit, University of Otago

March 2010 Acknowledgements The authors of this report would like to thank the people who participated in this evaluation for their time and thoughtful reflections on the HIA process and outcomes.

i TABLE OF CONTENTS EXECUTIVE SUMMARY...... I

BACKGROUND...... I METHODS...... I RESULTS...... I Strengths of the HIA process...... i Constraints on the HIA process...... i Key lessons learnt from this HIA...... ii CONCLUSION...... III SECTION 1: INTRODUCTION...... 1

BACKGROUND TO THE ORAL HEALTH STRATEGY...... 1 A DEFINITION OF HIA...... 1 THE HIA PROCESS AS APPLIED TO IMPLEMENTING THE ORAL HEALTH STRATEGY IN FLAXMERE...... 2 HIA AIM AND OBJECTIVES...... 2 THE EVALUATION OBJECTIVES...... 3 SECTION 2: EVALUATION METHODOLOGY...... 4

OBSERVATION OF HIA PROCESS...... 4 ANALYSIS OF HIA WORKSHOP PARTICIPANTS’ FEEDBACK...... 4 REVIEW OF RELEVANT HIA DOCUMENTATION...... 4 KEY STAKEHOLDER INTERVIEWS...... 5 Interview schedule design and development...... 5 Selection process for interviewees...... 5 DATA ANALYSIS...... 6 ETHICS...... 6 SECTION 3: THE PROCESS EVALUATION RESULTS...... 7

THE HIA SCREENING PHASE...... 7 THE SCOPING PHASE...... 7 Identification of the HIA aim and objectives...... 7 The determinants of health and wellbeing...... 8 The population groups of interest...... 8 Workshop participant evaluation feedback...... 8 Overall evaluation assessment of the scoping workshop process...... 8 THE APPRAISAL PHASE...... 9 Workshop participant evaluation feedback...... 9 The Flaxmere College students discussion...... 10 THE LITERATURE REVIEW...... 10 Key strengths of the review...... 10 Stakeholder evaluation feedback on literature review...... 11 HIA REPORT AND RECOMMENDATIONS...... 11 Stakeholder evaluation feedback on HIA report...... 12 SECTION 4: RESULTS OF THE STAKEHOLDER INTERVIEWS...... 13

WERE THE HIA OBJECTIVES MET?...... 13 HIA objective 1...... 13 HIA objective 2...... 13 HIA objective 3...... 13 HIA objective 4...... 13 HIA objective 5...... 14 LEVEL OF BUY-IN TO HIA AT A SENIOR DHB MANAGEMENT LEVEL...... 14 OVERALL STRENGTHS OF THE HIA PROCESS...... 15 The consultation process...... 15 The internal nature of the HIA...... 15 The literature review...... 15 Workshop facilitation...... 15

ii The HIA report...... 15 OVERALL CONSTRAINTS OF THE HIA PROCESS...... 15 Obtaining consensus on the HIA focus...... 16 Time and resource constraints...... 16 Community stakeholder negativity...... 16 Skills restraints...... 16 SUGGESTED WAYS THAT WOULD HAVE IMPROVED THE HIA PROCESS...... 16 Conduct HIAs earlier in the planning and policy development process...... 17 Establish HIA’s focus in the screening phase...... 17 SECTION 6: DISCUSSION AND CONCLUSION...... 18

WAS THE HIA PROCESS EFFECTIVE?...... 18 A useful consultation and constructive process...... 18 Useful HIA products...... 18 High quality facilitation...... 19 Lack of agreement on a focus for an HIA...... 19 The time and resource constraints...... 19 HOW EMBEDDED IS HIA IN THE DHB’S POLICY FUNDING AND PLANNING PROCESSES?...... 20 CONCLUSION...... 20 SECTION 7: REFERENCES...... 22 APPENDIX 1: WORKSHOP EVALUATION QUESTIONNAIRES...... 23 APPENDIX 2: FINAL ORAL HEALTH INTERVIEW EVALUATION QUESTIONS...... 25

iii EXECUTIVE SUMMARY

Background This report presents the findings of a process and impact evaluation of a Health Impact Assessment (HIA) on the implementation of the Hawke’s Bay District Health Board’s oral health strategy in the Flaxmere community. The HIA was lead by the Hawke’s Bay District Health Board (DHB) HIA team in collaboration with DHB’s Oral Health Steering Group. The evaluation was undertaken by the HIA Research Unit, University of Otago. The HIA was supported with funding from the Ministry of Health HIA Support Unit’s Learning by Doing Fund.

Methods The evaluation was conducted concomitantly with the actual HIA. Data were gathered for the evaluation using a mix of methods including; participant observation, participant workshop evaluation feedback, documentary analysis and interviews with nine key stakeholders.

Results The evaluation results showed all five of the HIA’s objectives were met (refer to page 3 of this for the list of objectives).

Strengths of the HIA process The HIA process proved an effective vehicle for consulting the wider community about the DHB’s largely internally-driven oral health policy. The process attracted the involvement of a good cross section of the community. Despite some initial discontent among local school principals attending the HIA workshops about the lack of earlier opportunity to input into the policy’s development they and the other community stakeholders attending the HIA’s workshops constructively contributed to the HIA process. The consultation aspect of the HIA was identified as one of its key strengths. The extensive knowledge base of the oral health implementation team and their ready availability to the HIA team particularly during the preparation of the HIA report were also commended. The oral health implementation team valued the HIA’s sound and useful literature review, which provided the team with important evidence supporting the new oral health service model. The DHB’s evaluation participants identified the facilitation skills of the external HIA consultant as another of the HIA’s key strengths. Having the DHB’s two HIA team members introduce the HIA workshops was also considered advantageous, given the internal nature of the oral health policy.

Constraints on the HIA process Pinning down a focus for the HIA proved a ‘major sticking point’ during the HIA’s scoping phase. Participants in this phase questioned the value of conducting an HIA on a service model which the Ministry of Health had already both endorsed and provided funding for. Skilled facilitation served to pave the way to re-establishing the focus originally agreed on in the screening session, namely to consult community stakeholders about where best to site the service models’ proposed centralised community clinic. The DHB’s HIA team members and members some of those from its oral health strategy implementation group indicated having often felt under considerable pressure given the significant amount of time required to run an HIA on top of their other work-related

i commitments. The HIA team found it a struggle to juggle the two, given they had three other HIAs ‘on the go’ at the same time. This HIA was the first internally driven HIA of the five HIAs the DHB has conducted with the support of the Learning by Doing fund. Knowledge, at the senior management level and the DHB generally, of the other HIAs focused on external outside policies/strategies was described as very limited. Specific communication approaches were suggested ‘to help spread the word’ about the value of using HIA to inform the DHB’s funding, planning, consultation and implantation processes.

Key lessons learnt from this HIA Overall learning  Applying an HIA process to an existing, largely internally-driven, policy presents challenges for those involved in conducting this type of HIA. Finding and deciding on a focus for the HIA is less straightforward, while there is a potential for stakeholders to feel frustrated about having not been consulted earlier on an already officially endorsed policy that will directly impact upon them.  Literature reviews are an important and essential evidence-gathering component of HIA and a necessary tool for supporting sound policy-making decisions.  New funding makes HIA possible  The relatively high cost of undertaking HIAs suggests it is mainly suited for use on significant policy projects such as the subject of this HIA. HIA process  When HIA is new to an organisation, careful consideration should be given when screening policies to selecting a policy which lends itself to HIA and is likely to be well supported in the community.  Clear and concise PowerPoint presentations are an effective tool for introducing the aims and objectives of the HIA and the policy/ strategy under consideration. Community consultation  There is benefit in taking HIA to the people and holding workshops in environments where your stakeholders feel comfortable. It can assist HIA participants to more effectively participate in the process.  There is a need to collect full contact details from those involved in the HIA workshops – names, address, organisational roles, and email addresses positions – so HIA reports can be sent out to those who participated in the HIA, as well as for evaluation purposes.  Workshop facilitators need a strong foundation in HIA, effective communication skills, an ability to work with a range of different audiences, and the flexibility ‘to think on their feet’ when it becomes obvious changes to the programme as needed.  Outlining the HIA process and the issues under consideration at the beginning of screening, scoping and appraisal workshops enables effective discussion. Presenters should be able to speak to the realities of the issue under consideration especially at the scoping and appraisal workshops.  Use of exercises to enable people to participate in decision-making can increase the engagement of participants in the process e.g. use of post-it notes for identifying health individuals’ health and wellbeing determinants and population group priorities in order to build consensus.

ii  Evaluation of community consultation workshops provides valuable feedback to the process.  Asking participants in the scoping workshop for advice about available data and who to consult can provide valuable information.  It is critical the principles, aims and objectives and concepts underpinning HIA closely align with the principles of the Treaty of Waitangi and Māori health and community development models, particularly in a district such as Flaxmere which has a high Māori population. Training and resources  Training for HIA practitioners is important to enable them to undertake HIAs effectively.  Novice HIA practitioners benefit from mentoring throughout the HIA process. It enables them to build the necessary knowledge and skill base.  New money makes it easier for organisations to undertake HIA as they do not have to find savings in other areas.  HIA concepts put simply would be useful when working with those not trained in HIA.

Conclusion The HIA on the implementation of the Hawke’s Bay DHB’s oral health policy in the Flaxmere community proved challenging mainly because of its application to an already endorsed and funded policy, and secondly because of the limited consultation that had been undertaken during the policy’s development phase. This feature underlines the need to ideally undertake an HIA when a policy is still in its development phase. It also illustrates HIA’s need for quality facilitation skills throughout the HIA process. The HIA process used for the implementation of the oral health HIA proved effective, as indicated by both the workshop evaluation feedback forms and the key informant interviews. The latter lacked a community perspective, because the DHB had yet to feedback to the community at the time the evaluation interviews were carried out. The final HIA report and recommendations were generally well received by all those interviewed. The positive response to the HIA report bodes well for the implementation of its recommendations. The final decision had yet to be announced at time of writing this report. Further evaluation at some later point in time would be valuable to assess the uptake of the HIA’s recommendations and their short-term impact. This HIA was the first on a policy led by the DHB. The HIA team’s engagement of their DHB colleagues in the process was considered one of the key strengths of the HIA process. While this HIA was acknowledged as valuable to those HIA staff directly involved in the process, there is no evidence that it has impacted any further within the DHB. There are a number of reasons why this maybe the case. First it maybe too early to see its full impact, it maybe that one HIA is not enough to convince key decision makers, it maybe that the context of change at the DHB has made it difficult to be innovative at this time, or a combination of reasons. This evaluation indicates some of the potential for HIA to strengthen DHB policy making and community responsiveness, although it is not without costs in time and resources. It seems appropriate to recommend that all DHBs consider using HIA as a tool to assist them to develop effective policy, particularly in significant policy areas. Evaluation of such initiatives is important to build the case for using HIA.

iii SECTION 1: INTRODUCTION The Health Impact Assessment (HIA) on the implementation of the Hawke’s Bay District Health Board’s (DHB) Oral Health Strategy is one of five HIAs recently undertaken by the DHB’s HIA team. It is the only one of the five that has assessed an internally driven strategy/policy. This report presents the results of a process and short-term impact evaluation on that HIA. The evaluation was undertaken by the University of Otago’s Health Equity and Wellbeing Impact Assessment Unit for the Hawke’s Bay DHB’s HIA team. Funding for the Hawke’s Bay DHB’s current HIA programme and their respective evaluations was provided through the Ministry of Health’s Health Impact Assessment Unit’s Learning by Doing fund.

Background to the oral health strategy In August 2006 the government’s Good Oral Health, for All, for Life: The strategic vision for oral health in New Zealand indicated a major change afoot for the school dental health service to address increasing oral health inequalities, between Māori (including Pacific) and non-Māori children and adolescents. The strategy signalled a major overhaul of the traditional school dental health service, which described as ‘hampered by ageing equipment and buildings and models of service delivery that are no longer meeting community needs’ (Ministry of Health 2006). As part of its new strategic vision the government indicated its intention to invest in a significant upgrade of oral health services for young people aged 0-18 years to improve their oral health outcomes. All DHBs were instructed by the Ministry of Health to prepare business cases for the redevelopment of their region’s oral health services for children aged 0-18 years. The Hawke’s Bay DHB submitted its business case as part of the August 2007 funding round. That case was initially based on an upgrade of the Wairoa dental health service. The Ministry of Health’s Oral Health Services Technical Review Group saw benefits in this initial proposal. It requested the DHB extend and cost the service model to cover the entire Hawke’s Bay region. In July 2008 the Hawke’s Bay DHB re-submitted its revised business case - Oral Health Business Case: Investment in child & Adolescent Health Services in Hawke’s Bay (Hawke’s Bay District Health Board 2008). The service model commonly referred to internally as the ‘hub and spoke model’ was subsequently endorsed by the Ministry of Health, with a funding allocation to re-develop and implement the service model. The proposed service model will essentially replace the Hawke’s Bay region’s existing 45 dental school clinics with three community-based clinics (the hubs), six fixed clinics based on school sites, a fleet of four double operator and one single-operator mobile caravans, and a mobile screening van (the spokes). The proposed service model aims to initially provide low cost, responsive, flexible oral health care for children and adolescents in high health needs communities, for example Wairoa and Flaxmere. It also aims to engage families early in their children’s oral health care through prevention and healthy lifestyles education. Primary and intermediate-aged school children will be screened and assessed annually at the mobile clinics, which will visit school sites on a rotational basis. Students requiring follow- up consultations and treatment will be seen at centralised, community-based oral health clinics. Pre-school children will be seen only at the community oral health clinics.

A definition of HIA HIA is defined as “a combination of procedures, methods and tools by which a policy, program or project may be judged by its potential effects on the health of a population, and

1 the distribution of those effects within the population” (European Centre for Health Policy 1999). It is a practical way to ensure that health, wellbeing and equity are considered as part of policy development in all sectors. It also helps facilitate policy-making that is based on evidence, focused on outcomes and encourages collaboration between a range of sectors and stakeholders. HIA is used in many countries, including New Zealand. In recent years policy- level HIA has been increasingly embedded in this country (Signal et al 2006). HIA typically involves a four-stage process, namely: 1. Screening: a selection process where policies are quickly judged for their potential to affect the health, wellbeing and equity of populations, and hence the need (or not) to undertake HIA 2. Scoping: planning the HIA including identifying the aims and objectives, identifying key stakeholders, determining resources needed and identifying possible data sources 3. Appraisal: describing the potential benefits and risks to health and their nature and magnitude and identifying potential changes that could be made to a policy to enhance its positive and mitigate its negative impacts on health, equity and wellbeing 4. Evaluation: assessing the process of the HIA in order to determine how it was done and to provide useful information for future HIAs, assessing the impact of the HIA to determine the extent to which the recommendations were taken on board in the policy decision-making process; and assessing the outcome of the HIA, namely its long-term effects on health, equity and wellbeing (Public Health Advisory Committee 2005).

The HIA process as applied to implementing the Oral Health Strategy in Flaxmere The HIA of the implementation of the Hawke’s Bay DHB’s Oral Health Strategy in the community of Flaxmere underwent all four HIA phases as outlined above. The HIA was led by Ana Apatu and Maree Rohleder, the two members of the Hawke’s Bay DHB’s HIA team. It was undertaken in partnership with DHB’s Oral Health Implementation Steering Group. The HIA team was also assisted by HIA consultant Robert Quigley of Wellington-based Quigley and Watts Limited. The HIA consultancy’s services – facilitation, mentoring and a review of the literature – like this evaluation were funded by the Ministry of Health, HIA Support Unit’s Learning by Doing fund.

HIA aim and objectives

Aim To identify the positive and negative health and wellbeing impacts on implementing Phase One of the Oral Health Strategy in Flaxmere and to provide recommendations to the decision makers about the risks and benefits.

Objectives 1. To engage with stakeholders and selected parts of the community, and develop potential approach for future consultation.

2. To develop a literature evidence base about implementation of oral health services in communities (engagement of families and models of care).

3. To assist the implementation of the Oral Health Strategy for reducing oral health inequalities in Flaxmere.

2 4. To assist the implementation of the Oral Health Strategy for maintaining or improving access and quality of services.

5. To make recommendations to the Oral Health Strategy decision makers about appropriate ways to maximise positive health and wellbeing outcomes from the Oral Health Strategy and minimise or negate negative outcomes.

The evaluation objectives The evaluation of the HIA on the implementation of the Oral Health Strategy in the Flaxmere Community aimed to:  Undertake and report on the process and short-term impact evaluation of the HIA  Assess and report on the acceptance of HIA in the Hawke’s Bay DHB at the completion of the HIA and ways to build HIA into the DHB’s processes.

3 SECTION 2: EVALUATION METHODOLOGY This evaluation was conducted in the spirit of ‘learning by doing’ in line with the funding source criteria for the HIA and its evaluation. While the two-member evaluation team conducted the evaluation independently, they also worked with the HIA team and the HIA consultant as participant observers. In this role they listened, took notes throughout the key HIA-related workshops and meetings, and reviewed the various key draft documents once completed. The evaluators also provided feedback to the HIA team throughout the HIA process and prompted those involved to reflect on their practice. This feedback and reflection provided opportunities for the HIA team to pause and make changes where required. The evaluation was conducted concurrently with the HIA, with the evaluators first formally engaging at the scoping stage. Evaluation data were gathered using the following methods – participant observation, workshop evaluations, documentary analysis and key stakeholder interviews.

Observation of HIA process One of the two-member evaluation team (Velma McClellan) attended three of the HIA’s sessions – the scoping workshop, one of the two appraisal workshops, and a discussion held at the College with a group of 12 years 11 and 12 Flaxmere College students. Her role at these HIA sessions was primarily one of observing. This involved taking note of who attended, recording observations throughout the workshops’ proceedings, reflecting on the process and documents arising from the process, and providing feedback to the evaluation’s principal researcher, the DHB’s HIA team and the HIA consultant. Both evaluation team members participated in the key planning meetings and the telephone conferences with the HIA team. Feedback on documents produced for the HIA was provided via email.

Analysis of HIA workshop participants’ feedback Evaluation questionnaires were distributed to the 11 HIA scoping workshop participants and the 19 appraisal workshop participants. These attendance numbers do not include the HIA team, the HIA consultant, or the evaluator (see Appendix 1 for copies of the workshop evaluation questionnaires). The majority of the two workshops’ participants completed questionnaires. Seven left the latter workshop early due to other work commitments. Consequently they were not given questionnaires. The evaluation responses were subsequently analysed for key themes. Evaluation reports for each workshop were prepared and emailed to the HIA team and the HIA consultant for their information and comment. The reports aimed to inform the HIA’s ongoing process and provide evidence for future HIAs.

Review of relevant HIA documentation All key documents relevant to the HIA were read to:  inform the evaluation by providing background and context to the HIA  provide information on the HIA’s content, process and impact. The documents reviewed included:  The Oral Health Business Case: Investment in Child and Adolescent Oral Health Services in Hawke’s Bay (HBDHB July 2008)  Literature Review the Health Impact Assessment of Hawke’s Bay DHB Oral Health Strategy (Quigley and Watts Ltd 2009)

4  The HIA screening, scoping and appraisal workshop notes and reports prepared by the HIA team, including the final HIA report - Health Impact Assessment Implementation of the Oral Health Strategy: Location of a community clinic in Flaxmere (Rohleder & Apatu 2009)  The minutes from other HIA team meetings and teleconference calls  Good Health for All, for Life: The strategic vision for oral health in New Zealand (Ministry of Health 2006). It should be noted there is no specific Oral Health Strategy document as such. The strategy referred to throughout this document refers to the service model outlined in the Hawke’s Bay DHB’s oral health business case (see discussion above).

Key stakeholder interviews

Interview schedule design and development An interview schedule was developed for the key stakeholders interviews (see Appendix 2). The schedule consisted of both open-ended questions and questions in the form of Likert rating scales.

Selection process for interviewees Following a teleconference discussion, the HIA team and the evaluators agreed it was inappropriate to interview the community stakeholder representatives for this evaluation. Therefore key informants are largely from within the DHB either as members of the HIA team (2) or staff with an interest in this policy. The oral health implementation team considered it needed to inform the community stakeholders first about the outcome of the DHB’s decision-making based on the HIA’s recommendations. The implementation team proposes to distribute the HIA report to the community stakeholders who attended the appraisal workshop, just prior to a specific feedback meeting the implementation team plans to hold in October 2009. At that meeting, its decision regarding the location of a community oral health clinic in Flaxmere will be outlined. The community stakeholders will be interviewed for their views on this HIA as part of the HIA on the redevelopment of the Flaxmere town centre the evaluation interviews. This is because there is considerable overlap between the players involved in both HIAs. Currently, the DHB’s oral health implementation team are in discussions with the Flaxmere redevelopment team to identify a possible site for the proposed oral health clinic in its redevelopment design. The list of stakeholders interviewed was selected by the evaluation team in consultation with the HIA team. The HIA team provided the proposed interviewees’ contact details. The eight stakeholders were sent an email with an evaluation information sheet and an interview consent form attached. The explanatory email indicated the telephone interviews would take between 30-40 minutes. The evaluator subsequently telephoned the proposed interviewees to schedule interview appointment times. Consent forms and an evaluation information form were sent to people prior to the interview. The interviewer checked to see if the participant had read the information sheet prior to commencing the interview. Signed consent forms were returned in most cases. Verbal consent was agreed to by all. If the interviewees indicated lack of familiarity with the information, the interviewer offered to go through it with them. All had read it.

5 Data analysis All data were analysed for key themes specific to the HIA evaluation objectives and any other relevant issues that emerged during analysis.

Ethics Ethics approval was sought from the multi-region ethics committee. The committee noted ethics approval was not required as the evaluation was an audit of work being undertaken by the DHB.

6 SECTION 3: THE PROCESS EVALUATION RESULTS This section of the evaluation report outlines the process used by the Hawke’s Bay DHB’s HIA team on the implementation of the Oral Health Strategy in the Flaxmere Community and presents the evaluation participants’ feedback about the perceived effectiveness of the HIA’s scoping and the appraisal workshops as a process. The findings also include analysed data arising from the evaluator’s participant observations, documentary analysis, and workshop evaluation forms.

The HIA screening phase This initial stage of the HIA process was conducted on 18 August 2008. Four people attended the screening session: the DHB’s two HIA team members, a DHB business analyst and a portfolio manager. Evaluation team members were not required to attend. The following three screening process questions were considered by the screening group: 1. Is there potential for positive or negative health impacts from the proposed change? Consider social, cultural and economic impacts on the health and independence of Māori and their whānau. 2. Is there potential for the proposed policy to increase or decrease existing health inequalities between Māori and non-Māori or other groups? 3. Is there support from the policy-makers involved or political support with the organisation to carry out the HIA? Issues discussed during the screening session with regard to the above questions included:  Transporting students safely to and from the proposed centralised community-based oral health service, following the closure of school dental health facilities, was considered a potential negative impact, though possibly outweighed by the benefits of having access to improved facilities and by an associated increase in procedures.  The choice of the location of the community oral health clinic could be a critical factor in terms of its potential to impact on increasing or decreasing inequalities. Following consideration of the issues raised, the group decided it was appropriate to carry out an HIA on the Oral Health Strategy (Rohleder & Apatu 2009: 14).

The scoping phase The HIA scoping workshop was held at the Hawke’s Bay DHB corporate office in Hastings on 17 September 2008. Fourteen people attended. Most (11) were DHB staff members these included the two HIA team members. This was the first time that most of these DHB staff had participated in an HIA. Getting them to the table appears to have been an important part of the HIA process and critical to its success. This was in part because some of them were integral to the oral health policy development process. A Wairoa-based consultant, the Wellington-based HIA consultant and one member of the evaluation team also attended.

Identification of the HIA aim and objectives The HIA consultant opened the scoping workshop with a succinct and useful outline of the HIA framework, its concepts and underlying principles. During this opening stage several DHB participants queried the value of carrying out an HIA on a policy that was already in the process of implementation. The consultant noted that ideally an HIA should start at the policy planning and development phase, but indicated it could also be usefully used to assess the implementation of a policy.

7 Following a brief outline of the context within which the DHB’s Oral Health Strategy was developed, participants discussed what the HIA could usefully focus on. Following a lengthy discussion participants agreed there could be value in having the HIA focus on ‘whether to develop a community clinic as part of a wider ‘health centre’, possibly in partnership with a community provider, or to develop one within a specific school site’ (Rohleder & Apatu 2009:14).

The determinants of health and wellbeing A list of six determinants of health and wellbeing were considered by the group. Using ‘post- it notes’ participants were asked to select the two determinants they considered most important for the HIA to focus on. The following determinants were identified  Access to services namely, choice of sites in relation to community need and transport access to those sites  Availability of services e.g. the range of services.

The population groups of interest The above exercise was repeated to identify the population groups most likely to be affected by the proposed implementation approach. This exercise initially identified six potentially affected population groups. These were subsequently narrowed down to two:  whānau/families  children. Workshop participants also identified other potential sources of evidence and data to inform the HIA as well as potential stakeholders they considered should be involved in the HIA process.

Workshop participant evaluation feedback Participants other than the HIA team and the evaluator were invited to complete an evaluation form prior to closing the workshop. Nine of the 14 participants completed forms (Appendix 1 presents a copy of the evaluation form). Subsequent analysis of those forms showed the HIA consultant’s facilitation skills and the opportunity for the oral health team to ‘speak freely’ were the workshop’s key strengths. No weaknesses were identified. Four participants suggested possible improvements. Two considered the HIA should have been done earlier in the policy development process, while the remaining two suggested the background HIA information handed out at the beginning of the workshop could have been distributed earlier, to help inform the workshop’s proceedings.

Overall evaluation assessment of the scoping workshop process It became evident soon after the scoping session commenced that it was ‘too late in the day’ for the HIA to help inform the oral health policy development process. Skilful facilitation turned a potentially contentious situation around to a consensus decision, namely to redirect the focus of the HIA session on to implementation of the proposed service model into the Flaxmere community. The scoping workshops participants appeared to appreciate the place and value of HIA in the policy development and implementation process, and the workshop’s skilled facilitation on the day.

8 The appraisal phase A mixed stakeholders’ appraisal workshop was held on 5 March 2009 at the Te Aranga Marae in Flaxmere. The four-hour workshop was attended by 22 stakeholders. Included were several school principals and senior teachers and a range of health service providers. The two HIA team members, the HIA consultant and an evaluation team member were also present. The workshop was opened with a karakia and a welcome by a prominent local kaumātua, followed by a round of introductions. The HIA team then set the scene for the day’s proceedings by:  outlining the range of HIAs either completed or currently being undertaken by the Hawke’s Bay DHB  providing an effective PowerPoint presentation summarising the various phases and applications of the HIA process  outlining the purpose of the HIA’s workshop. The scene setting was concluded by the DHB’s portfolio manager who outlined the Oral Health Strategy’s service model. This was followed by a lengthy discussion about a range of issues for schools arising from the proposed model, issues that included possible access difficulties for mobile clinics to some local school grounds, transportation of students to and from clinics off school sites, and the possible closure of recently upgraded dental clinics. The principals questioned the value of their participation in the HIA, as the service model proposed for Flaxmere, in their eyes, appeared to be ‘a done deal’. The DHB’s oral health service manager responded suggesting that ‘nothing was set in concrete’ and the HIA provided an opportunity for them to air their views and discuss the possibilities. Participants were subsequently split into three groups to discuss the potential positive and negative impacts of the service model, and to identify possible and practical solutions with regards to the latter. Following the group’s written and verbal report back session, the HIA consultant concluded the workshop by briefly describing how the information collected from the day’s workshop, in conjunction with the evidence from the HIA’s literature review, would be utilised to produce its recommendations to the DHB. Evaluation forms were subsequently distributed.

Workshop participant evaluation feedback Of the 19 evaluation forms distributed prior to concluding the appraisal workshop, 14 were subsequently completed. The responses to questions regarding the effectiveness of the HIA process were generally positive. The workshop’s main strengths, as identified by respondents, included the high quality facilitation, the marae as a venue, and the opportunity the HIA gave them to voice their concerns and ‘be heard.’ In terms of weaknesses and possible improvements, five respondents suggested that consultation with the Flaxmere community should have occurred much earlier in the planning and development process. From an external point of view, the HIA can be seen to have provided the Hawke’s Bay DHB with an opportunity to consult the Flaxmere community. Consultation appears to have been fairly limited up to this point. School representatives appeared generally sceptical about their input having any chance of influencing the shape of the proposed service model. However, all participated fully in the HIA’s group work. Nevertheless, doubt remains as to whether the school principals will in the end see the HIA as having been an effective vehicle for their input, given their seeming lack of enthusiasm for the idea of a centralised community health clinic, and the potential difficulties some schools will have in providing mobile health clinics access to school grounds.

9 The Flaxmere College students discussion On the same day as the above appraisal workshop the HIA team also met at the Flaxmere College with 12 year 11 and 12 students. This one-hour workshop was facilitated by the two HIA team members and the HIA consultant. The facilitators focused the first 30 minutes of the discussion on the implementation of the oral services in Flaxmere, while the remaining 30 minutes focused on the redevelopment of the Flaxmere urban community (the latter is the subject of a forthcoming HIA report). The HIA consultant facilitator outlined the way the DHB proposes to deliver oral health services in schools. Most students appeared to like the idea of an urban-based oral health centre. Only a few students at the meeting had visited the mobile oral health service now operating in Flaxmere. Of those who had visited it all indicated having liked it. In general, the 12 students were attentive throughout the HIA discussion session. Female students were assessed by the HIA facilitators as having provided the more constructive input, compared to their male counterparts. They also considered the input valuable because the students’ responses appeared to be ‘open’ and ‘honest’. The one-hour workshop was possibly at the limit of what might be expected of this age group in terms of their attention span. The facilitators did well to establish the necessary rapport and hold their attention as they did.

The literature review The HIA’s literature review was prepared by Quigley and Watts Limited in May 2009. It was completed after the appraisal workshop. The review addressed two specific issues with regards to the oral health strategy namely:  The location of services in the community which facilitates the best access  The identification of strategies to engage families and whānau in terms of increasing their attendance and participation in the prevention and treatment of their children and adolescents’ oral health care. The review’s methodology is clearly outlined. The literature search resulted in 37 journal articles and reports being identified, all published since 2000. Most of the evidence reviewed originated from the United States, Canada and Australia. Just two published journal articles and two unpublished documents originated from New Zealand.

Key strengths of the review The literature reviews main strengths included:  a valuable executive summary of the review’s key findings  a well written report  a concise and useful overview of the oral health services strategy and the proposed service model  a useful historical context outlining how the New Zealand school dental health service developed and operated over the past 80 years, and pointing out consequential attitudinal barriers those implementing the proposed service model in Flaxmere will need to keep in mind  evidence from an evaluation study of a local whānau ora oral health service model.

1010 Stakeholder evaluation feedback on literature review Five of the eight stakeholders interviewed as part of the evaluation rated the literature review ‘very helpful at a policy level,’ in that, ‘it helped provide some clarity around which service model to adopt and promote’. It was also perceived as supporting the approach put up by the Hawke’s Bay DHB in its business case, as well as supporting the HIA’s recommendations. Another described it as ‘very readable’ and ‘useful’, in the sense that, literature reviews were reported as rarely undertaken in the DHB to inform its policy planning and development. Another stakeholder rated the review ‘helpful, in so far as it went’. He saw the review as having ‘serious deficiencies’, primarily its lack of known New Zealand studies. He suggested the reviewers ‘should have contacted the University of Otago’s dental epidemiologist to help steer them down the right path.’ This point was also picked up by another stakeholder who considered the review ‘unhelpful’. Subsequently, Quigley and Watts followed up this concern but found the data mentioned to be outdated and outside the timeframe of the review (2000 to the present). One of the two evaluation interviewees had not received the literature review. Arrangements were made to have the literature review sent to her. The other had not had time to read it.

HIA report and recommendations The report was written by the HIA team in collaboration with members of the oral health service implementation. A draft was submitted to the Oral Health Implementation Team in June 2009. The report was subsequently revised to take into account relevant feedback, including that from the evaluation team. The final HIA report includes eight recommendations. These, and the rationale immediately following each recommendation, appear closely aligned to the international and local evidence that emanated from the HIA’s literature review and input provided by the HIA’s community stakeholders. In summary, the HIA report’s recommendations supported: 1. locating a community clinic in the Flaxmere village and/or co-located with another health provider 2. early establishment and maintenance of community trust in the service 3. implementing an oral health education programme in Flaxmere 4. establishing a collaborative approach to engage schools in any health service delivery and promotional activities 5. addressing workforce issues and succession planning for dental therapists 6. developing and putting a robust information system in place to prevent children being ‘lost’ in the system 7. considering, as part of the planning, transport options for getting children and adolescents to the community clinic 8. ensuring regular communication with the schools and the Flaxmere community occurs throughout the process (Rohleder & Apatu, 2009: pp 9-10). The HIA report is generally well crafted. It includes a succinct and useful executive summary, solid descriptions and overviews of the HIA process, the oral health service model, and informative summaries of the relevant literature review findings and causal pathways. The summary of the causal pathways is particularly helpful as both causal pathways flow charts are relatively complex. The flows charts currently lack appropriate report headings, namely figures 1 and 2.

1111 Stakeholder evaluation feedback on HIA report There was general consensus among all the interviewees who had seen the final HIA report that it is a sound piece of work. It was described as ‘tightly structured’, ‘reads well’, ‘well argued’, and in the long-term ‘will be a useful tool for the implementation team’s continuing work’. On interview we learnt that the HIA report’s findings had been presented to the Hawke’s Bay DHB’s Oral Health Steering Group and its Acting Chief Executive. The report was described as well received as were its recommendations, which were all endorsed (see subsequent discussion).

1212 SECTION 4: RESULTS OF THE STAKEHOLDER INTERVIEWS This section of the report presents the findings from an analysis of data resulting from the eight HIA stakeholders final evaluation interviews. The findings are presented under headings specific to the various interview questions (see appendix 1 for a copy of the interview schedule).

Were the HIA objectives met?

HIA objective 1 To engage with stakeholders and selected parts of the community, and develop a potential approach for future consultation Seven of the eight evaluation interviewees considered the HIA had been either ‘very helpful’ or ‘helpful’ in engaging a ‘good cross section of community stakeholders’. The HIA process was seen to have ‘galvanised the local school principals into a cooperative approach’, whereas prior to the HIA they were reportedly ‘not ready to be proactive’. One interviewee suggested engagement with the community had preceded the HIA. However, others indicated the earlier engagement process had only involved primarily Ministry of Education school property management services and oral health service providers. Two people described the HIA consultative approach as ‘robust’. Others suggested the value of the consultation ‘could have been even greater had it [the consultation] been done earlier’.

HIA objective 2 To develop a literature evidence base about the implementation of oral health services in communities (engagement of families and models of care) Four of the six interviewees who had seen the literature review saw it as having provided ‘a sound evidence base’ for which service model ‘to adopt and promote’ and the placement of services generally. The review’s perceived lack of New Zealand data was the main reason for those who judged it as less that helpful.

HIA objective 3 To assist the implementation of the Oral Health Strategy for reducing health inequalities in Flaxmere Of the six interviewees who were in a position to respond to the question concerning this objective, most (5) considered the HIA had been either very helpful or helpful in getting health inequalities considered in the oral health services implementation plan for the Flaxmere community. The sixth person considered it had been ‘okay’. This person and others saw the Hawke’s Bay DHB’s oral health services business case as ‘very much’ focused on reducing the region’s oral health inequalities (HBDHB 2008). The health inequalities assessment tool, more commonly known as the HEAT tool (Signal et al 2008), was reportedly used right from the outset to assess the ability of the business case to help reduce oral health inequalities.

HIA objective 4 To assist the implementation of the Oral Health Strategy for maintaining or improving access and quality of services All but one of the eight evaluation interviewees were of the mind that the HIA had been either very helpful or helpful in identifying ways to improve and maintain access and quality

1313 of oral health services in the Flaxmere community. The literature review was identified as having provided the most robust evidence about the placement options for a centralised clinic. However, for one interviewee, the HIA had ‘opened up a can of worms’ for the DHB in this department.

HIA objective 5 To make recommendations to the Oral Health Strategy decision-makers about appropriate ways to maximise positive health and wellbeing outcomes from the Oral Health Strategy and minimise or negate negative outcomes Six of the eight interviewees rated the HIA report and its recommendations as either very helpful or helpful in pointing to ways to maximise positive and minimise negative outcomes. One person had not seen the HIA report, while another could not recall the actual wording of the recommendations, but thought at the time of reading the report that ‘they looked okay’. The recommendations, according to two people, were supported by evidence derived from the literature review. Another considered them ‘sound and workable.’ The two HIA team members developed the recommendations in collaboration with members of the DHB’s oral health implementation team. All recommendations were subsequently accepted by the Oral Health Steering Group. Despite having rated the recommendations helpful, one interviewee expressed some nervousness about the wording in recommendation seven as it implies that the DHB will provide transport to get children and their families/whānau to the community. This, they said, was not an option. This same person suggested the DHB would need to work with other community agencies to come up with a workable transport option.

Level of buy-in to HIA at a senior DHB management level This is the first of the DHB’s evaluated HIAs that has focused on an internally driven policy/strategy. The other three have been driven by external agencies, namely local territorial authorities. Two of this HIA evaluation’s interviewees suggested recent HIAs ‘have not really touched DHB land’. An acting senior manager saw the oral health HIA as having ‘provided support’ and ‘legitimised the approach’ at a senior management level. She suggested that in order for HIA to be widely accepted at a senior management level more effective HIA-specific communication within the DHB is required. The manager indicated she was prepared to advocate for HIA’s use to be continued. Another interviewee saw the oral health HIA as ‘likely to win over the funding and planning team … it’s a very useful tool’. Others (2) similarly noted that few at senior management level had been exposed to HIA to date. However, the DHB’s acting chief executive was said to be pleased with this HIA’s process, product, and outcome. HIA was reported to have been recently considered as part of an independent external study. The study’s investigator, Dr Fiona Cram, has been working with the Hawke’s Bay DHB to determine the best way to embed the HEAT tool into its funding and planning processes. As part of this same study, Dr Cram was said to have ‘drilled down into the possible use of HIA’ as another possible decision-making tool – ‘HIA could usefully sit alongside the HEAT tool’, according to two interviewees. A report from the above study is expected soon. On the other hand, another interviewee considered ‘time and resource constraints are the biggest constraints’ likely to limit the regular use of HIA on a routine basis as a funding and planning tool. One interviewee noted a summary of the Hawke’s Bay DHB’s HIAs had recently been presented to the Public Health Advisory Committee (PHAC). A PHAC member had reportedly asked for further HIA reporting to be put on the Committee’s next meeting agenda.

1414 Overall strengths of the HIA process Seven of the eight evaluation interviewees identified what they considered were the HIA’s main strengths. The bracketed numbers indicate the number who identified a particular strength.

The consultation process The community representation and engagement process at the HIA consultation workshops was generally commended. (4) Reputedly it was the first chance given to community stakeholders, particularly school principals, to have their ‘views and voice heard’, ‘to vent their frustrations in a safe environment’, and to have the proposed service model ‘spelt out for them’. The stakeholders attending the HIA appraisal workshop were considered ‘representative’ and ‘a good cross section of the Flaxmere community’.

The internal nature of the HIA The extensive knowledge base of the policy and implementation group driving the oral health strategy was identified as another major strength. Having ready access to that group, was considered very helpful from the HIA team’s point of view, ‘they being the key decision- makers’. The recommendations were developed in collaboration with the implementation group hence ‘they have their endorsement’. (4)

The literature review Two interviewees identified the literature review as a major strength. A member of the implementation group was particularly pleased the review’s evidence supported the proposed service model. Having access to a literature review designed specifically to inform a particular policy or its implementation was considered a major benefit, as these were reportedly rarely undertaken within the DHB.

Workshop facilitation Three people considered the workshop facilitation a major strength of the HIA. From the HIA consultant’s point of view, having the two Hawke’s Bay DHB’s HIA facilitators front the appraisal workshop was advantageous, given the HIA was focused on an internal policy project. He suggested, ‘the DHB would have been more comfortable with having them at the front’. One of the implementation project team’s members held the independence of the HIA process, its external funding source, and its having been externally driven by professionals outside the actual implementation project team was a major strength. Another implementation group member commended the HIA consultant’s ‘great facilitation skills’.

The HIA report Two DHB interviewees identified the HIA’s report as one of the HIA’s main strengths. One described it as a ‘very useful’ document, while the other considered it ‘a clearly written document … unlike a lot of other reports I’ve read’.

Overall constraints of the HIA process The eight interviewees identified the following range of constraints within the overall HIA process.

1515 Obtaining consensus on the HIA focus Six of the eight evaluation interviewees made reference to the difficulties of ‘pinning down’ exactly what aspect of the Oral Health Strategy the HIA could usefully focus on. The fact that the strategy had already been endorsed, by both the DHB and the Ministry of Health, led the HIA’s screening session participants to focus on its implementation phase, with regards to the best location for the proposed centralised community clinic in Flaxmere. However, the original focus kept shifting overtime according to the HIA team members, one of whom suggested … ‘we’ve learnt that we need to be very clear about what the HIA focus is from the outset’. Asking the right questions at the screening session was considered by a second HIA team member to be ‘very important.’ One of the oral health implementation team indicated having never been particularly keen on the HIA idea from the start … ‘I see HIA as a useful pre-project tool, but not for ‘one that is done and dusted’. Her initial hesitation about using an HIA process on the Oral Health Strategy stemmed from her reluctance to go to the community with a strategy that she considered ‘a done deal’. The issues were subsequently ‘talked over’ to her satisfaction and the HIA turned to ‘how it [the strategy] could be usefully presented to the community’. The consensus was to focus the HIA on the best location of a centralised clinic.

Time and resource constraints The competing pressures on the two Hawke’s Bay DHB’s HIA team members presented some difficulties for the team, according to one team member. The time constraints were exacerbated by ‘a lot of waiting in between the HIA phases for the policy people’ who were described as ‘very busy too’. ‘We needed to get to certain decision points in order to progress’ ... these could not be sped up’. Once those decisions were made, however, ‘things’, reportedly, ‘moved fast’. The backlog of yet incomplete HIA work, reportedly, collided with work related to the other three HIAs the team was running concurrently, as well as their regular non-HIA related work commitments.

Community stakeholder negativity Two interviewees saw the HIA process as somewhat dampened by having to contend with ‘a good deal of negativity and angst’ from local schools principals attending the HIA’s appraisal workshop. Both interviewees considered that an HIA, or some other form of community consultation, should have been undertaken during the strategy’s development phase. It was implied that, this would have gone some way to countering the need for the HIA to spend so much time dealing with the principals’ ‘pent up frustrations’. Both interviewees considered the current HIA would possibly have done little to mollify those frustrations. They [the school principals] will be ‘looking for both more input and feedback’. Skills restraints One of the oral health implementation team indicated having spent a good deal of time assisting one of the HIA members to write the HIA report. The report’s first draft required much rewriting according to this interviewee. The HIA report was considered ‘an important document’ hence the interviewee was prepared to put in the time to ensure that it met their idea of ‘a quality product’. The interviewee indicated surprise about the HIA consultant having not been more involved in the quality control of the report writing.

Suggested ways that would have improved the HIA process Interviewees identified the following range of ways they considered would have improved this HIA’s process.

1616 Conduct HIAs earlier in the planning and policy development process The main improvement interviewees suggested largely revolved around the consultation process ‘having occurred after the event’. (5) One interviewee suggested it would have been useful to have both the potential positive and negative impacts identified prior to actually developing the Oral Health Strategy. Most agreed with the notion that the community should have been consulted as part of the strategy’s development phase. No one, raising this particular issue considered this ‘after the fact’ situation as the fault of the HIA. The Ministry of Health was seen by one interviewee as having ‘its own agenda. … They were very specific about what they wanted’. Consequently, the DHB’s business case was geared to meet the Ministry’s expectations. The Ministry’s timeline for submitting business cases reportedly did not allow for wider community consultation at that point of planning and development.

Establish HIA’s focus in the screening phase Both the DHB’s HIA team members and the HIA consultant suggested the HIA required greater clarity at the initial screening phase, and again at the scoping phase, about what the HIA could usefully focus on. Despite apparent consensus at the screening session that the implementation of the strategy in Flaxmere was the logical focus for the HIA, uncertainty and debate took up considerable time in the subsequent scoping workshop. Reaching consensus about the HIA’s focus at the screening phase was considered essential for every HIA. One HIA team member suggested, a ‘robust’ focus-oriented question, in addition to the usual HIA screening questions, was required in order to achieve certainty.

1717 SECTION 6: DISCUSSION AND CONCLUSION This evaluation of the HIA undertaken on the Hawke’s Bay DHB’s Oral Health Strategy is one of four HIA evaluations either completed or currently being undertaken by the DHB HIA’s team. The Ministry of Health, HIA Support Unit’s Learning by Doing fund has supported this and the four other HIAs and their individual evaluations. This was the first of the four HIAs to focus on an internally developed policy/strategy. This HIA provided an opportunity to engage a number of DHB staff in the process of HIA. In doing so, it modelled the process and provided an opportunity for staff to understand the value of HIA. We used a range of research methods to collect the evaluation data: participant observation, documentary analysis, workshop evaluation and key stakeholder interviews. The evaluation aimed firstly to report on the HIA’s process and its short term impact, and secondly to assess and report on the acceptance of HIA in the DHB and determine in what ways to build HIA into the DHB’s processes. As the final decision has yet to be announced about action on the implementation of the policy, and therefore the response to the HIA’s recommendations, the findings of this evaluation should be regarded with a degree of caution. The key informants were not able to comment on the final impact of the HIA nor was it appropriate to interview community stakeholders With this limitation in mind consideration is now given to whether or not these objectives have been achieved.

Was the HIA process effective?

A useful consultation and constructive process Most of the eight stakeholders interviewed indicated having expressed reservations initially about the wisdom of taking an internally endorsed strategy out into the wider community, which largely has had little prior opportunity to express their views on the proposed radical changes to its traditional school dental services. However, most of the HIA’s stakeholders acknowledged that some form of community consultation was warranted. Most saw the HIA process as a particularly suitable vehicle for consulting with the community, given its seeming independence from the DHB’s oral health implementation team. The HIA’s appraisal workshop was considered as having attracted a good cross section of the community. The school principals initially used the opportunity to vent their frustrations about the lack of earlier consultation but settled down to work constructively with their fellow community stakeholders to identify the potential positive and negative impacts of the proposed service model and possible solutions to reduce or eliminate the potentially detrimental impacts. Whether the HIA process has gone anyway towards ameliorating the school principals’ frustrations about having not been consulted earlier about the proposed changes to the dental health system is unable to be answered here. The DHB stakeholders and evaluation team decided it was inappropriate at this time to interview the community stakeholders given the oral health strategy’s implementation has yet to formally provide feedback and the actual report to the HIA’s community stakeholders has not occurred. They plan to do this once they have some idea of where the centralised oral health clink will best be sited. Discussions to this end are currently underway.

Useful HIA products Both the literature review and the HIA were generally commended by the eight interviewees as useful and supportive. While the lack of New Zealand specific literature in the former was considered a major gap by two interviewees they, nevertheless, still saw value in the evidence

1818 provided. They and others were pleased that the evidence clearly supported the Oral Health Strategy’s centralised community oral health service model. Having access to a specifically targeted literature review was considered a major bonus for the oral health implementation members as such reviews were said to be rarely undertaken in the DHB. The actual HIA report and its recommendations each with a clearly supported rationale, is very well written. Members of the oral health implementation team indicated the report would serve to inform and guide them in their ongoing implementation work. As noted earlier, the report was written by the HIA team in collaboration with members of the oral health service implementation. This was generally regarded by those involved as having produced a very sound product, to the point where one stakeholder called it the best internal DHB document he had ever read. All eight of the HIA recommendations were subsequently endorsed by the oral health steering group, another sign of the report’s quality and value. The coinciding of an HIA on the location of a community oral health clinic with an HIA on the urban redevelopment of the Flaxmere town centre was considered fortuitous, and a major bonus, for the oral health implementation team. The team was reportedly currently working with the redevelopment group to consider a possible site in that redevelopment design for the proposed community oral health clinic.

High quality facilitation The high quality of the facilitation of the HIA process was again commended by several interviewees.

Lack of agreement on a focus for an HIA Although the HIAs focus was seemingly agreed by all parties in the HIA’s screening session, debate around the focus subsequently proved contentious in the HIA’s scoping workshop. A lengthy and sometimes heated discussion arose to the point where resolution of the issue, at times, appeared remote. The fait accompli status of the actual Oral Health Strategy proved the main sticking point, namely the proposed service model had already been endorsed by the Ministry of Health’s Oral Health Services Technical Group and funding provided to develop and implement it. Consequently some scoping group participants considered there was no point in using an HIA process to consult the community. Skilled facilitation served to pave the way to re-establishing the focus originally agreed on in the screening session, namely to consult community stakeholders about where best, of three possible options, to site the service models’ proposed centralised community clinic. The HIA’s objectives also shifted somewhat overtime. From an evaluation point of view this proved somewhat frustrating in terms of finalising the evaluation plan and designing the interview questionnaire. The objectives as presented in the revised final HIA report were generally tighter, more process focused and more strongly worded than earlier versions of the recommendations. Working through and reaching agreement in the initial phases of the HIA with specifically designed questions and consideration would help to clarify early on in the process the HIA focus and its objectives.

The time and resource constraints Both the DHB’s HIA team members and those from its oral health strategy implementation group indicated having often felt under considerable pressure given the inordinate amount of time required to run an HIA on top of their already heavy non-HIA related workload. The HIA team found it at times a struggle to juggle the two, given they had three other HIAs ‘on the go’ at the same time. While neither HIA team members could actually provide the evaluation with information about the number of hours they had spent on this and the other HIAs personal experience suggests the amount of time spent on any one HIA is considerable.

1919 How embedded is HIA in the DHB’s policy funding and planning processes? None of the stakeholders interviewed, including one acting senior manager, could provide any evidence that there is any commitment to embedding HIA in the DHB’s funding and planning processes. The DHB was reportedly in a state of flux at the time the evaluation interviews were conducted, what with its endeavours to trim its current budget and future funding requirements. Staff changes at a senior management level also mean people with any real knowledge of HIA are few and far between The acting chief executive was said to pleased with this HIA’s outcome but was considered too busy to usefully contribute to this evaluation. This was the first internally driven HIA of the five HIAs the DHB has conducted with the support of the Learning by Doing fund. Knowledge, at the senior management level and the DHB generally, of the other HIAs focused on external outside policies/strategies was described as very limited. It was suggested that specific communication approaches were required ‘to help spread the word’ about the value of using HIA to inform the DHB’s funding, planning, consultation and implantation processes. A study currently being undertaken by Dr Fiona Cram, to determine ways to embed the HEAT tool into the Hawke’s Bay DHB processes was also reported as having considered HIA as a complementary funding and planning tool. One interviewee suggested HIA was a superior funding and planning tool to the HEAT tool. Dr Cram’s study hopefully will show that both tools have their place in the planning and funding of significant policies as was the case with the Hawke’s Bay DHB’s Oral Health Strategy.

Conclusion This paper reports on a process and impact evaluation of the implementation of the Hawke’s Bay DHB’s Oral Health Strategy in Flaxmere. This HIA was challenging because it was undertaken on a policy that had already been confirmed and on which limited consultation had been undertaken. As a result considerable facilitation skills were needed at both the scoping and appraisal workshops in order to reach consensus on a way forward, i.e. to undertake the HIA on the implementation of the policy and not on the policy itself. This underlies the ideal use of HIA when a policy is still at the proposal stage and on the value of consultation in any policy development. It also illustrates the importance of good facilitation skills by HIA leaders, especially when the HIA is not straight forward. The HIA appears to have been undertaken with an appropriate process and in a way that was seen as largely satisfactory by participants, as indicated in the workshop evaluations and the key informant interviews, although a community perspective on the HIA is missing from the interviews as discussed above. The final HIA report was generally well regarded by the policy makers receiving the report, the Oral Health Implementation Team, although the final version benefitted from considerable redrafting by one implementation team member. This is a reminder that HIA reports are complex documents that need to be written in a manner that is appropriate for the decision makers being asked to consider them. This positive regard of the HIA report bodes well for a positive response to its recommendations. As the decision has yet to be announced at time of writing it is not possible to comment on whether this is the case. This HIA was the first on a policy led by the DHB. The HIA team’s engagement of their DHB colleagues in the process was considered one of the HIA’s key strengths. While this HIA was acknowledged as valuable to those HIA staff involved in the process, there is no evidence that it has impacted any further within the DHB. There are a number of

2020 reasons why this maybe the case. First it maybe too early to see its full impact, it maybe that one HIA is not enough to convince key decision makers, it maybe that the context of change at the DHB has made it difficult to be innovative at this time, or a combination of reasons. This HIA evaluation indicates some of the potential for HIA to strengthen DHB policy making and community responsiveness, although it is not without costs in time and resources. It seems appropriate to recommend that all DHBs consider using HIA as a tool to assist them to develop effective policy, particularly in areas of significant policy; that meets the needs of the people; and therefore provides good value for money. Evaluation of such initiatives will be important to build the case for HIA.

2121 SECTION 7: REFERENCES European Centre for Health Policy. 1999. Health Impact Assessment: main concepts and suggested approach - the Gothenberg consensus paper. Brussels: World Health Organization Regional Office from Europe. Hawke’s Bay District Health Board. 2008. The Oral Health Business Case: Investment in Child and Adolescent Oral Health Services in Hawke’s Bay. Hastings: Hawke’s Bay District Health Board. Ministry of Health. 2006. Good Oral Health for All. For Life: The Strategic vision for oral health in New Zealand. Wellington: Ministry of Health. Public Health Advisory Committee. 2005. A Guide to Health Impact Assessment: A policy tool for New Zealand. Second edition. Wellington: National Advisory Committee on Health and Disability. Quigley and Watts Limited. 2009. Literature Review: The Health Impact Assessment of the Hawke’s Bay DHB oral health strategy. Wellington: Quigley and Watts Limited. Rohleder M & Apatu A. 2009. Health Impact Assessment on the Implementation of the Oral Health Strategy: Location of a community clinic in Flaxmere. With assistance from Quigley and Watts Limited. Hastings: Hawke’s Bay District Health Board. Signal L, Langford B, Quigley R, & Ward M. Strengthening health, wellbeing and equity: embedding policy-level HIA in New Zealand. Social Policy Journal of New Zealand 2006, 29:17-31. Signal L, Martin J, Cram F, et al. 2008. The Health Equity Tool: A user’s guide. Wellington: Ministry of Health.

2222 APPENDIX 1: WORKSHOP EVALUATION QUESTIONNAIRES

Oral Health Strategy Implementation Approach for Flaxmere Health Impact Assessment Scoping Workshop Evaluation Questionnaire

By completing this brief questionnaire you will help the Health Impact Assessment (HIA) Team to keep on improving its processes for collecting stakeholder/community input into its HIAs. No names or other identifying information are required on this form.

Statement Strongly Agree Disagree Strongly agree disagree

1. I understand why HIA is undertaken

2. I understand why we are doing HIA on this topic

3. I understand what the main focus of this HIA will be

4. I felt I was able to contribute to the decisions about what this HIA was going to focus on

5. I want to continue to be involved in future parts of this HIA

6. I will consider the HIA process for other projects that are/ will be occurring

7. The main strengths of the scoping session were?

8. What could have been done to improve the scoping session?

9. Any other comments?

Thank you very much for providing this feedback

2323 Oral Health Strategy Implementation Approach for Flaxmere HIA Evaluation Appraisal Workshop Questionnaire

By completing this brief questionnaire you will help the Health Impact Assessment (HIA) Team to keep on improving its processes for collecting stakeholder/community input into its HIAs. No names or other identifying information are required on this form.

Statement Strongly agree Agree Disagree Strongly disagree

1. Overall, this workshop has been very useful for identifying the intended and unintended effects that this implementation approach might have

2. I now have a greater understanding of the ways this implementation approach might affect the determinants of wellbeing

3. The workshop has been a good opportunity for me to develop or maintain links with people across the sector/s

4. The workshop has been a good opportunity to contribute my views and ideas for implementing the oral health in Flaxmere

5. I will consider the HIA process for other projects that are/ will be occurring

6. The main strengths of this appraisal workshop were?

7. What could have been done to improve the appraisal workshop?

8. Any other comments?

Thank you very much for providing this feedback

2424 APPENDIX 2: FINAL ORAL HEALTH INTERVIEW EVALUATION QUESTIONS

1. Based on your experience, what do you think were the main strengths of the HIA on the implementation of the oral health strategy in the Flaxmere community?

2. Are you able to identify any factors that acted to constrain the HIA process?

3. What might have been done to improve the HIA process?

4. About how much time did you spend working on the oral health HIA ?

4a. How was that time largely spent?

5. Based on your experience, how helpful was the HIA in assisting the DHB to engage with stakeholders and selected parts of the community? Was it …. ?

Very helpful Helpful Okay Unhelpful Very unhelpful Not sure

1 2 3 4 5 6

5a. In what way was it ….. ?

6. How helpful was the HIA’s literature review in informing how oral health services would best be implemented in the Flaxmere community? Was it ……

1 2 3 4 5 6

6a In what way was the literature review …. ?

7. How helpful was the feedback from the HIA’s stakeholders’ workshops for informing how oral health services would best be implemented in the Flaxmere community? Was it … ?

1 2 3 4 5 6

25 25 7a In what way was the feedback … ?

8. How helpful was the HIA in getting health inequalities issues considered in the DHB’s oral health services implementation plan for the Flaxmere community? Was it … ?

1 2 3 4 5 6

8a In what way was it … ?

9. How helpful was the HIA in identifying ways to improve and maintain access and quality oral health service issues in the Flaxmere community? Was it …. ?

1 2 3 4 5 6

9a In what way was it … ?

10. What do you think of the HIA report? Prompts – what are its strengths and weaknesses?

10a. How helpful are the report’s recommendations?

1 2 3 4 5 6

10b In what way are the recommendations ... ?

11. Do you think the HIA has helped to get buy-in into the HIA process and outcomes at the HBDHB senior management level? Yes No If yes, ask in what way? If no, ask why not?

12. Has the DHB made any effort to embed HIA into its policy and service planning and development processes? If yes, ask how? If no, ask can you identify any barriers to its doing so?

26 26 13. How do you think HIA can be built into the DHB processes?

14. What do you think are the key lessons from this HIA for the DHB ?

15. Are there any particular resources that you think could be developed for future DHB HIAs?

16. That’s the end of the set evaluation questions, is there anything we haven’t covered that you’d like to raise before we finish the interview?