Final Report Litter Hotspots Program Final Evaluation

Prepared for Metropolitan Waste and Resource Recovery Group

Litter Hotspots Program Final Evaluation—Final Report

Company information First Person Consulting Pty Ltd ABN 98 605 466 797 www.fpconsulting.com.au Suite 3, Level 2, 190 Queen Street Melbourne VIC 3000

Contact Patrick Gilmour 03 9600 1778 [email protected]

Document details:

Title: Final Report – Litter Hotspots Final Evaluation

Authors: Rebecca Denniss, Danielle Clarke, Patrick Gilmour

Version: Final

Revision date: 5 October 2017

Client: Metropolitan Waste and Resource Recovery Group

Prepared for MWRRG i Litter Hotspots Program Final Evaluation—Final Report

Executive summary Background The Metropolitan Waste Resource Recovery Group (MWRRG) is a Victorian Government statutory body responsible for coordinating and facilitating the delivery of municipal solid waste management across Melbourne’s 31 metropolitan local governments. From 2014-2017, MWRRG delivered the Litter Hotspots Program (the Program) in partnership with the Department of Environment, Land, Water and Planning (DELWP), funded through the Victorian Government's Sustainability Fund. To meet funding requirements, a full final evaluation of the Program has been undertaken in accordance with the Department of Treasury and Finance (DTF) Lapsing Program Guidelines (Guidelines).

This document is the Final Report for the Litter Hotspots Final Evaluation. The Litter Hotspots Program

The Program was developed as a pilot in 2013 by MWRRG. The 2013 pilot attracted strong interest from stakeholders and received an additional $1.9 million in funding to implement a more comprehensive three-year program in 2014—ending on 30 June 2017. The Litter Hotspots Program’s stated objectives are to:

1. support the implementation of best-practice litter prevention projects 2. develop partnerships in order to capacity build litter prevention projects 3. improve personal responsibility for litter disposal and prevention 4. improve the amenity of public spaces that impact upon the water quality of the Yarra River and Port Phillip Bay.

The Litter Hotspots Program is a catchment-wide capacity building program with a strong partnership focus that has supported and funded 35 local government and community grants projects since 2013 (including 10 pilot projects, which are outside the scope of this evaluation). The Program brought together a combination of government and community partners, influencing leading litter prevention practice and yielding strong partnership, community engagement and litter prevention outcomes. The Program aimed to either build or enhance litter management skills, depending on the skills of each grant recipient. Funded projects ranged from designing new waste technology to attempting to eradicate single-use plastic bags. Evaluation objectives and approach

In addition to meeting the requirements of the DTF Guidelines, the objectives of the evaluation were to:

 undertake a full evaluation of the Litter Hotspots Program to assess performance of the Program against key questions outlined in the funding agreement between DELWP and MWRRG  produce a final evaluation report that includes key findings, achievements, outcomes, lessons learned and recommendations to guide future decision making and delivery of future litter prevention initiatives.

Prepared for MWRRG ii Litter Hotspots Program Final Evaluation—Final Report

In meeting these objectives, and building on ongoing evaluation, monitoring and improvement processes throughout the Program, First Person Consulting undertook an evaluation of the Litter Hotspots Program focused on four key areas: effectiveness, program delivery, justification and value, and looking to the future. This involved: a review and gap analysis of existing evaluation data and program management documentation; undertaking 28 semi-structured phone interviews with MWRRG delivery staff (3), key stakeholders (5) and a selection of project officers from the 25 funded projects (20); analysis and synthesis of all data sources in response to evaluation questions under the four key areas. Key findings The key findings emerging from the Litter Hotspots Program Final Evaluation are summarised below. Effectiveness

 The Litter Hotspots Program either achieved or made strong progress towards achieving all four stated program objectives. Significant, demonstrable outcomes were achieved across the areas of best practice litter prevention, developing partnerships to build capacity, increasing responsibility for litter disposal and prevention, and improving the amenity of public spaces impacting on the water quality of the Yarra River and Port Phillip Bay.  Key achievements of the Litter Hotspots Program include: o Funding projects that targeted diverse audiences, for example, trader groups, school students, businesses, industry organisations, local communities, dog-walkers and smokers. o 382 litter hotspot sites targeted with litter interventions, and an overall improvement in the litter ‘Hotshot’ score across hotspot sites. o $332,753 in additional funding contributed by funded organisations and project delivery partners, and $1,138,870 of in-kind contributions leveraged from funded organisations and project delivery partners. o The creation of 267 new partnerships between organisations on litter management initiatives. o Engagement with 13,227 participants across funded initiatives, 492 community events and 355 media articles and publications. o 175,230 kilograms of litter collected, and 3,317 additional litter infrastructure units installed (for example, bins). o Wide demographic and geographic reach of funded projects across the catchment.  At the Program level, consultation, communications and engagement were effective throughout the Litter Hotspots Program. The Program successfully established an ongoing network in the litter prevention space, and broadened and diversified the range of community, council and business organisations partnering on litter prevention initiatives.  The majority of organisations funded through the Program have continued to deliver outcomes in litter management beyond the funding period, sometimes as a direct result of the Litter Hotspots Program.

Prepared for MWRRG iii Litter Hotspots Program Final Evaluation—Final Report

 A key characteristic of the Litter Hotspots Program was its alignment and commitment to improving the implementation of litter prevention in metropolitan Melbourne, based on the Victorian Litter Action Alliance’s (VLAA) best practice model. Program delivery

 The Litter Hotspots Program was generally delivered within scope, on budget and on time.  The Program was delivered with relative efficiency and economy.  MWRRG demonstrated appropriate governance and risk management practices in delivering the Litter Hotspots Program.  Overall, MWRRG is seen to have effectively managed the Litter Hotspots Program between 2014 and 2017. Justification and value

 Overall, the Litter Hotspots Program provided value, evidenced by the outcomes achieved, and contributed to meeting a demonstrable need.  Funding for the Litter Hotspots Program has not been renewed and the Program will be discontinued. However, there is still a need for Victorian Government intervention in litter prevention.  Generally, the Litter Hotspots Program was well aligned with Victorian Government priorities, policies and strategies (including those of MWRRG, DELWP and Sustainability Victoria (SV)). During the course of the Program, alignment was strengthened throughout the portfolio with the release of the Victorian Waste Education Strategy, the Victorian Litter Plan and the Metropolitan Waste and Resource Recovery Implementation Plan. Looking to the future

 The key identified success factor in the Litter Hotspots Program was MWRRG’s management and administration of the Program.  A key success factor was the partnership component of projects, but also the collaborative component of the Program itself.  Through reflecting on lessons learned from the Litter Hotspots Programs, the following success factors were identified as contributing to the most effective litter interventions: o taking a collaborative, catchment-wide approach o effective engagement with the community o maintaining momentum in intervention approaches and engagement with the community and project partners o capacity building and knowledge sharing with other projects through a community of practice o implementing and developing the best practice model, utilising a range of approaches and activities o targeting litter prevention at the source o others also included: establishing buy-in with delivery organisations and partners, particularly stakeholders at senior levels; effective project planning and strong governance mechanisms; targeting and tailoring interventions (for example, using

Prepared for MWRRG iv Litter Hotspots Program Final Evaluation—Final Report

culturally appropriate communications); investing in research; and using positive messaging and consistent communications.  Four key suggestions for Program level improvements emerged through this evaluation— simplifying application and reporting processes, supporting momentum and ongoing funding for litter prevention interventions, providing grants for 18-month durations as opposed to 12 months or three years, and continuing to develop the catchment-wide approach to litter management.  Key ongoing challenges for litter management in metropolitan Melbourne identified through this evaluation include: o balancing support for new entrants in the litter prevention space, while pushing those already engaged to continue improving best practice o challenges in establishing consistent data collection methods o balancing the need for innovation and supporting wider implementation of proven approaches o MWRRG having ongoing capacity to resource and fund litter management o continuing to support momentum in litter prevention and maintaining engagement with the diversity of stakeholders implicated in litter management. Recommendations Based on these findings, we make the following recommendations for MWRRG:

1. Continue to support momentum within Victoria’s litter management system between funding cycles, recognising MWRRG’s role as a key strategic stakeholder—including providing policy advice and contributing to decision-making processes that involve state and local government stakeholders, non-government organisations and community groups 2. Continue to support the ongoing development of best practice litter management in metropolitan Melbourne, which may involve: a. building the capability of both new entrants to the litter prevention space, as well as encouraging continuous improvement and innovation among councils, non- government organisations and community groups already implementing best practice approaches b. supporting and maintaining the existing community of practice on litter prevention by providing opportunities for stakeholders to collaborate, network, learn from each other and build partnerships (such as through forums, networking events, etc.) c. providing direct capacity building support for organisations wanting to continue their litter prevention work beyond grants funding periods (such as offering training, providing delivery support, developing tools and resources etc.). 3. Continue to work with portfolio partners on improving data collection, monitoring and evaluation processes to measure the impact of litter interventions—such as through providing training, advice, data collection tools and resources etc.). There is an opportunity to ensure that key intended outcomes are identified at the start of interventions, and that monitoring and evaluation is planned to most effectively capture those outcomes. This may involve more strategic approaches to measuring behaviour change aspects of litter interventions to ensure that monitoring and evaluation resources are spent most efficiently.

Prepared for MWRRG v Litter Hotspots Program Final Evaluation—Final Report

4. Continue to pursue further funding opportunities from DELWP and other Victorian or Commonwealth agencies to deliver litter management programs in future, building on the success of MWRRG’s delivery of the Litter Hotspots Program.

We make the following recommendations for litter management in Victoria more broadly:

5. Continue to promote a collaborative catchment-wide approach to litter management. This may include: a. supporting a range of both regional and local level litter interventions that align with wider portfolio priorities, policies and strategies b. ensuring the legacy of the network that was strengthened through the Litter Hotspots Program by continuing to engage key stakeholders—including VLAA, KVB, MWRRG, local councils, portfolio partners, non-government organisations and community groups—on litter related policies, programs and decision-making. 6. Continue to support both innovation in the litter management space, as well as provide financial support for the up-scaling of proven successful interventions throughout the state in transferable contexts. 7. Continue to prioritise funding for litter interventions that align with the best practice model for litter prevention and involve the three key elements of education, infrastructure and enforcement. This may include: a. continuing to support the work of VLAA, SV, MWRRG and other key advocates for the best practice model b. continuing to support the ongoing maintenance of the litter management community of practice between funding cycles. 8. Apply the lessons learned from the Litter Hotspots Program to future litter prevention programs—particularly the key success factors that have been identified to contribute to the most effective litter interventions. 9. Refer to the Litter Hotspots Program as an example of a successfully administered government grants program: a. in particular, the MWRRG’s approach of providing a high level of project management support, capacity building and networking opportunities to grant recipients could be applied in other grants programs b. insights from the Litter Hotspots Program on the optimal length for grant funding periods (18 months) for small-to medium scale litter prevention projects could also be more widely applied as appropriate.

Prepared for MWRRG vi Litter Hotspots Program Final Evaluation—Final Report

Contents Executive summary ...... ii Background ...... ii Key findings ...... iii Recommendations ...... v 1 Introduction ...... 1 1.1 Overview ...... 1 1.2 Litter Hotspots Program ...... 1 1.3 Evaluation objectives ...... 2 2 Methodology ...... 3 2.1 Overview ...... 3 2.2 Evaluation framework ...... 3 2.3 Structure of this report ...... 8 3 Key findings and recommendations...... 9 3.1 Key findings ...... 9 3.2 Recommendations ...... 12 4 Effectiveness ...... 14 4.1 Overview ...... 14 4.2 Achievement of outcomes and objectives ...... 14 5 Program delivery ...... 33 5.1 Overview ...... 33 5.2 Scope, budget, timeframes ...... 33 5.3 Governance and risk management ...... 35 5.4 Efficiency and economy in delivery ...... 37 6 Justification and value ...... 39 6.1 Overview ...... 39 6.2 Program value and demonstrable need ...... 39 6.3 Wider context ...... 41 6.4 Alignment to government priorities, policies and strategies ...... 41 7 Looking to the future ...... 43 7.1 Overview ...... 43 7.2 Lessons learned ...... 44 Appendix 1: Methodology in detail ...... 49

Prepared for MWRRG vii Litter Hotspots Program Final Evaluation—Final Report

Inception meeting ...... 49 Document review and gap analysis ...... 49 Primary data collection ...... 49 Data analysis ...... 50 Reporting and presentation ...... 50

Tables Table 1. Litter Hotspots Program funding rounds...... 2 Table 2. Litter Hotspots Program Final Evaluation framework...... 4 Table 3. Structure of the Litter Hotspots Program Final Evaluation Report...... 8 Table 4. Achievement of Litter Hotspots objectives and outcomes...... 18 Table 5. Funded projects, outcomes, alignment to Program objectives and best practice litter prevention components...... 23 Table 6. Litter Hotspots Program funding allocation from the Sustainability Fund...... 34 Table 7. Litter Hotspots Program budget expenditure of Sustainability Fund allocation...... 34 Table 8. Rating rubric for evidence of outcomes achieved through the Litter Hotspots Program...... 51

Figures Figure 1. Litter Hotspots Program funded projects (MWRRG 2017)...... 15 Figure 2. Litter Hotspots Program network partnership map...... 16 Figure 3. Overall success of the Litter Hotspots Program (n=28)...... 21 Figure 4. Extent to which the Litter Hotspots Program has led to improvements in funded organisations’ capacity for best practice litter management, partnerships and engagement (n=20). 22 Figure 5. Extent to which funded organisations felt supported throughout the delivery of their Litter Hotspots Program project (n=20) ...... 30 Figure 6. Victorian Litter Action Alliance best practice model for litter prevention (VLAA 2017)...... 31 Figure 7. Litter Hotspots Program governance structure (MWRRG 2016)...... 35 Figure 8. Satisfaction with MWRRG’s management of the Litter Hotspots Program (n=20)...... 36 Figure 9. Extent to which interviewees believe that there is value in the Victorian Government funding a similar program in future (n=28)...... 40

Boxes Box 1. ‘Love Where You Live’ project outcomes...... 22 Box 2. ‘Plastic Bag Free Warburton’: outcomes beyond the life of the Litter Hotspots Program...... 27 Box 3. Reflections on effectiveness of the CLEAN network...... 29

Prepared for MWRRG viii Litter Hotspots Program Final Evaluation—Final Report

Box 4. Best practice litter prevention: Greater City of Dandenong...... 32

Acronyms DELWP Department of Environment, Land, Water and Planning

DTF Department of Treasury and Finance

FPC First Person Consulting

Guidelines Lapsing Program Guidelines

KEQ Key Evaluation Question

MWRRG Metropolitan Waste and Resource Recovery Group

SV Sustainability Victoria

Prepared for MWRRG ix Litter Hotspots Program Final Evaluation—Final Report

1 Introduction 1.1 Overview The Metropolitan Waste Resource Recovery Group (MWRRG) is a Victorian Government statutory body established under Environment Protection [Amendment] Act 2006. It is responsible for coordinating and facilitating the delivery of municipal solid waste management across Melbourne’s 31 metropolitan local governments.

From 2014-2017, MWRRG delivered the Litter Hotspots Program (the Program) in partnership with the Department of Environment, Land, Water and Planning (DELWP), funded through the Victorian Government's Sustainability Fund. To meet funding requirements, a full final evaluation of the Program has been undertaken in accordance with the Department of Treasury and Finance (DTF) Lapsing Program Guidelines (Guidelines).

This document is the Final Report for the Litter Hotspots Final Evaluation. 1.2 Litter Hotspots Program The Program was developed as a pilot in 2013 by MWRRG in partnership with DELWP and funded through the Sustainability Fund. The 2013 pilot program attracted strong interest from stakeholders and received an additional $1.9 million in funding to implement a more comprehensive three-year program in 2014—ending on 30 June 2017.

The 2014-2017 Litter Hotspots Program is one of the four priority programs identified in the 2016 Victorian Litter Plan. It aligns closely to the waste and litter reduction priorities identified in the Victorian Community and Business Waste Education Strategy, the Metropolitan Waste and Resource Recovery Implementation Plan, and the water quality priorities identified in the Port Phillip Bay Environmental Management Plan. The Program’s objectives are to:

1. support the implementation of best-practice litter prevention projects 2. develop partnerships in order to capacity build litter prevention projects 3. improve personal responsibility for litter disposal and prevention 4. improve the amenity of public spaces that impact upon the water quality of the Yarra River and Port Phillip Bay.

A planning workshop held between key stakeholders in the development stage identified the top three priority issues to address through the Program:

1. poor personal responsibility for litter is damaging the amenity of public spaces 2. high levels of litter carried by stormwater is impacting the amenity and environmental quality of our waterways 3. current litter management practices are expensive and inefficient.

The Litter Hotspots Program is a catchment-wide capacity building program with a strong partnership focus that has supported and funded 35 local government and community grants projects since 2013 (see Table 1). The Program brought together a combination of government and community partners, influencing leading litter prevention practice and yielding strong partnership, community engagement and litter prevention outcomes. The Program aimed to either build or enhance litter management skills, depending on the skills of each grant recipient. Funded projects

Prepared for MWRRG 1 Litter Hotspots Program Final Evaluation—Final Report ranged from designing new waste technology to attempting to eradicate single-use plastic bags (a complete list of funded projects is provided in Section 4.2).

Table 1. Litter Hotspots Program funding rounds.

2013-14 2014-15 2015-16 2016-17 Program components (Phase 1 (Phase 2, (Phase 2, (Phase 2, Total Pilot) Year 1) Year 2) Year 3) Implement a targeted litter hotspot $250,000 $458,000 $654,000 $640,000 $2,150,000 program for the Yarra River and Port Phillip Bay Small grants less than $15,000 Round 2 Round 1 Round 3 (small) Large grants up to $200,000 over Round 2 Round 2 Round 2 three years (large) (large) (large) Develop partnerships and commission targeted strategic projects if indicated

Program monitoring and evaluation

1.3 Evaluation objectives The two key objectives for the Litter Hotspots Program Final Evaluation are to:

 undertake a full evaluation of the Litter Hotspots Program to assess performance of the Program against key questions outlined in the funding agreement between DELWP and MWRRG  produce a final evaluation report that includes key findings, achievements, outcomes and recommendations to guide future decision making and delivery of future litter prevention initiatives.

The scope of this evaluation includes 25 funded projects as part of Rounds 2 and 3, excluding ten pilot projects funded in 2013 through Round 1 which were evaluated separately.

Building on ongoing evaluation, monitoring and improvement processes throughout the Program, the overarching outcomes of this evaluation will enable:

 MWRRG, DELWP and Sustainability Victoria (SV) to have a greater understanding of the value of the Litter Hotspots Program and what it has achieved in terms of meeting its program objectives.  Key stakeholders to have an improved awareness and understanding of Litter Hotspots Program achievements including project strengths, weaknesses, achievements and opportunities for how projects could be adapted in similar situations to reduce litter and improve waterway health.  MWRRG and funding partners to have a greater understanding of what worked well, what didn’t and lessons learned in delivering future programs.

Prepared for MWRRG 2 Litter Hotspots Program Final Evaluation—Final Report

2 Methodology 2.1 Overview This evaluation includes the following key components:

 Inception meeting—the FPC evaluation team met with MWRRG staff to clarify the objectives of the evaluation, involving discussion about the methodology, project plan and evaluation questions, sharing of relevant documents and stakeholder details, and confirming project timelines and milestones.  Project plan and methodology—following the inception meeting, a project plan and methodology was developed to guide the evaluation, building on the Litter Hotspots Monitoring and Evaluation Framework and discussion with MWRRG.  Document review and gap analysis—we then undertook preliminary analysis of the range of existing program documentation and evaluation reports available relevant to the Program. This included a wider literature review and was a crucial step in identifying the areas to be addressed through additional data collection. It was the first stage in synthesising the range of existing data in response to the evaluation questions.  Primary data collection—primary data collection involved undertaking 28 in-depth phone interviews with MWRRG delivery staff (3), key stakeholders (5), and a selection of Project Officers from the 25 funded projects (20). This was complemented by attendance at the Hotspots Program knowledge sharing and celebration event on 15 August 2017.  Data analysis—all data sources were analysed using a combination of both qualitative and quantitative methods, guided by the evaluation questions.  Draft report and presentation—a draft evaluation report (this document) has been prepared and presented to MWRRG and key partners for review and discussion.  Final report and presentation—a final evaluation report will be developed, incorporating feedback and discussion on the draft report from MWRRG. The final report, including key evaluation findings and recommendations will be presented to MWRRG, key partners and other stakeholders.

A detailed description of each stage of the evaluation methodology is provided in Appendix 1. 2.2 Evaluation framework In meeting the objectives and outcomes described in Section 1.2 above, this evaluation report responds to evaluation questions outlined in the funding agreement between DELWP and MWRRG, informed by the DTF Guidelines, under the following four criteria: effectiveness, program delivery, justification and value, and the future.

The evaluation framework in Table 2 on the following pages presents an overview of the evaluation approach and data sources.

Prepared for MWRRG 3 Litter Hotspots Program Final Evaluation—Final Report

Table 2. Litter Hotspots Program Final Evaluation framework.

Criteria Evaluation questions Indictors to consider Data sources Methods

Has the program achieved its  A description and summary of project hotspot sites and achievements Program management Document review stated objectives and  The stated objectives of the program and outcomes it was seeking to documentation, funding Interviews expected outcomes? achieve, and why the program approach was considered the best way agreements and milestone to achieve the outcomes reporting (project and Learning event Effectiveness What unexpected or  Extent of achievement of objectives and outcomes (for the Program as program level) additional outcomes were a whole and for individual projects) based on the Litter Hotspots achieved? Relevant policies, documents Monitoring & Evaluation Framework (see Appendix 1) and DELWP/ and strategies MWRRG funding agreement  Quantifiable outcomes, including economic, social and environmental Previous evaluations benefits Case studies  Demographic and geographic reach of programs Funded project staff  Effectiveness of consultation, communications and engagement activities MWRRG staff  Unexpected or additional outcomes identified and any associated DELWP and SV staff positive or negative outcomes  Outcomes that grants have contributed to beyond the life of projects Delivery stakeholders  Effectiveness of different approaches, activities and interventions, examples of innovation  External factors, providing context for evaluation results around the achievement of outcomes

Prepared for MWRRG 4

Litter Hotspots Program Final Evaluation—Final Report

Has the program been  Original scope and scope changes (including associated funding and Program management Document review delivered within its scope, objective/performance adjustments) documentation, funding Interviews budget, within the expected  Total approved budget (including but separating out from original agreements and milestone timeframe, and in line with approval any agreed scope variations), program costs and major cost reporting (project and Learning event

appropriate governance and drivers program level) Programdelivery risk management practices?  List of the price paid by Government and all additional funding Financial data provided to the program over the past three years (as applicable) Has the department  List the total costs incurred for the delivery of the program (including Best practice framework demonstrated efficiency and disaggregation by expense category) for each of the three previous economy in relation to the Relevant policies, documents years (as applicable) delivery of the program? and strategies  Additional funding/in-kind leveraged Previous evaluations  Identify cost drivers and gaps between estimated and actual costs  Breakdown of program expense categories Funded project staff  Entities which charge expenses to the program cost MWRRG staff  Comparable benchmarking of program costs and processes (where information is available) DELWP and SV staff  Detailed timeline and status of program delivery implementation Delivery stakeholders  If not delivered on time or on budget, a clear explanation of why  Demonstrate why the governance and risk management practices surrounding the program are appropriate (including alignment with Best Practice guides to grant program administration)  Evidence of improved efficiency and economy (demonstrating that the program is being delivered at lowest possible cost without compromising quality, highlighting improvements that have led to downstream efficiencies)

Prepared for MWRRG 5 Litter Hotspots Program Final Evaluation—Final Report

What is the evidence of a  Overall value of the program Program management Document review continued need for the  Clear articulation and demonstration of program impact documentation, funding Interviews program at this time and role  Demonstrated alignment of the program’s stated objectives, outputs agreements and milestone for government in delivering and outcomes, and any relevant government priorities and strategies reporting (project and Learning event this program? (including those of MWRRG, DELWP and SV) program level) Literature review  Extent to which program continues to address a demonstrable need Relevant policies, documents

Justificationand value and is responsive to the needs of Victorians and strategies  Extent MWRRG has investigated other options to address the identified need or problem Previous evaluations  Why the program continues to be the best way to respond to the Case studies problem and deliver the intended outcomes MWRRG staff  How the economic, environmental and social conditions have changed since the program was funded and how continuation of the program DELWP and SV staff will meet these conditions Delivery stakeholders  Extent to which the market place cannot deliver the program  Whether there are similar services being provided by the Victorian External documents and Government, the Commonwealth, or NGO sector that have published research commenced since the program’s inception  Relevant academic thinking and national and international practices to program delivery  The capacity (resources and monetary) and capability of MWRRG to continue the program while responding to any changes found as a result of the evaluation

Prepared for MWRRG 6 Litter Hotspots Program Final Evaluation—Final Report

What are the lessons for  Identified strengths and weaknesses of program components Program management Document review similar programs?  Identified success factors to program uptake and effectiveness documentation, funding

Future Interviews  Identified challenges/barriers to program uptake and effectiveness agreements and milestone If ongoing funding was  Transferability of knowledge, skills and experience associated with reporting (project and Learning event provided, what level of program delivery program level) efficiencies could be realised? Literature review  Replicability of the program Financial data What would be the impact of  Key lessons learned ceasing the program for  What could be done differently to achieve better outcomes Relevant policies, documents example, service impact,  Identified future needs or gaps and strategies community, jobs in both the  Recommended improvements (for program design, management and Previous evaluations public sector and the wider implementation, as well as monitoring, evaluation and reporting Funded project staff community) and what processes) strategies have been  How program partners could successfully exit from delivering the MWRRG staff identified to minimise program if government so desired DELWP and SV staff negative impacts? What  What the impact would be (both internal and external and direct and alternative approaches or indirect) of ceasing the Litter Hotspots Program Delivery stakeholders strategies could provide a fall-  What strategies have been identified to minimise these impacts External documents and back position?  The level of efficiencies ongoing funding could provide including clear published explanation as to how this information was derived

Prepared for MWRRG 7 Litter Hotspots Program Final Evaluation—Final Report

2.3 Structure of this report The following sections of this report address each of the evaluation questions, as described in Table 3 below.

Table 3. Structure of the Litter Hotspots Program Final Evaluation Report.

Criteria Evaluation questions Relevant section of this report

Effectiveness Has the program achieved its stated objectives Section 0 and expected outcomes?

What unexpected or additional outcomes were achieved?

Program Has the program been delivered within its scope, Section 5 delivery budget, within the expected timeframe, and in line with appropriate governance and risk

management practices?

Has the department demonstrated efficiency and economy in relation to the delivery of the program?

Justification and What is the evidence of a continued need for the Section 6 value program at this time and role for government in delivering this program?

Future What are the lessons for similar programs? Section 7

If ongoing funding was provided, what level of efficiencies could be realised?

What would be the impact of ceasing the program for example, service impact, community, jobs in both the public sector and the wider community) and what strategies have been identified to minimise negative impacts? What alternative approaches or strategies could provide a fall-back position?

Prepared for MWRRG 8 Litter Hotspots Program Final Evaluation—Final Report

3 Key findings and recommendations 3.1 Key findings 3.1.1 Effectiveness

 In addition to the ten pilot projects funded in 2013, 25 projects targeting litter hotspots across the catchment were funded between 2014 and 2017.  Key achievements of the Litter Hotspots Program include: o Funding projects that targeted diverse audiences, for example, trader groups, school students, businesses, industry organisations, local communities, dog-walkers and smokers. o 382 litter hotspot sites targeted with litter interventions, and an overall improvement in the litter ‘Hotshot’ score across hotspot sites. o $332,753 in additional funding contributed by funded organisations and project delivery partners, and $1,138,870 of in-kind contributions leveraged from funded organisations and project delivery partners. o The creation of 267 new partnerships between organisations on litter management initiatives. o Engagement with 13,227 participants across funded initiatives, 492 community events and 355 media articles and publications. o 175,230 kilograms of litter collected, and 3,317 additional litter infrastructure units installed (for example, bins). o Wide demographic and geographic reach of funded projects across the catchment.  The Litter Hotspots Program achieved or made strong progress towards achieving all four stated program objectives. Significant, demonstrable outcomes were achieved across the areas of best practice litter prevention, developing partnerships to build capacity, increasing responsibility for litter disposal and prevention, and improving the amenity of public spaces impacting on the water quality of the Yarra River and Port Phillip Bay.  A range of approaches employed in the Litter Hotspots Program ensured it was delivered with the most effective use of resources—in particular, the catchment-wide and partnerships approach, and the capacity building and project support provided by the MWRRG.  The majority of organisations funded through the Program have continued to deliver outcomes in litter management beyond the funding period, sometimes as a direct result of the Litter Hotspots Program.  At the Program level, consultation, communications and engagement were effective throughout the Litter Hotspots Program. The Program successfully established an ongoing network in the litter prevention space, and broadened and diversified the range of community, council and business organisations partnering on litter prevention initiatives.  A key characteristic of the Litter Hotspots Program was its alignment and commitment to improving the implementation of litter prevention in metropolitan Melbourne, based on VLAA’s best practice model.

Prepared for MWRRG 9 Litter Hotspots Program Final Evaluation—Final Report

3.1.2 Program delivery

 The Litter Hotspots Program was generally delivered within scope, on budget and on time.  The Program was delivered with relative efficiency and economy compared to other similar government funded grants programs.  MWRRG demonstrated appropriate governance and risk management practices in delivering the Litter Hotspots Program.  Overall, MWRRG is seen to have effectively managed the Litter Hotspots Program between 2014 and 2017.  Interviewees reflected on the relative effectiveness of Litter Hotspots Program administration compared to other similar government funded grants programs. 3.1.3 Justification and value

 Overall, the Litter Hotspots Program provided value, evidenced by the outcomes achieved, and contributed to meeting a demonstrable need.  Funding for the Litter Hotspots Program has not been renewed and the Program will be discontinued. However, there is still a need for Victorian Government intervention in litter prevention.  Since the commencement of the Program, there have been some key contextual changes influencing litter management in Victoria, including: o Establishment of the Litter Innovation Fund—a $700,000 fund administered by SV that provides grants of up to $20,000 to organisations that provide innovative project proposals to reduce litter or dumping. o The Victorian Litter Strategy lapsed in 2014, and litter was incorporated into the Victorian Waste Education Strategy released in 2016. Despite litter now forming part of the Victorian Government’s wider waste education policy, it is perceived by some stakeholders that litter has declined as a priority—this reflects a gap in understanding among stakeholders about litter funding opportunities and the strengthening of statewide approaches to litter management. o The remit for litter management in Victoria has been broadened to include more of a focus on illegal dumping and micro-plastics in the River and Bay, although this is perceived by some stakeholders to have shifted the focus away from the broader problem of litter behaviour within the community.  Generally, the Litter Hotspots Program was well aligned with Victorian Government priorities, policies and strategies (including those of MWRRG, DELWP and SV). During the course of the Program, alignment was strengthened throughout the portfolio with the release of the Victorian Waste Education Strategy, the Victorian Litter Plan and the Metropolitan Waste and Resource Recovery Implementation Plan. 3.1.4 Looking to the future

 The key identified success factor in the Litter Hotspots Program was MWRRG’s management and administration of the Program.

Prepared for MWRRG 10 Litter Hotspots Program Final Evaluation—Final Report

 A key success factor was the partnership component of projects, but also the collaborative catchment management approach of the Program itself.  Through reflecting on lessons learned from the Litter Hotspots Programs, the following success factors were identified as contributing to the most effective litter interventions: o taking a collaborative, catchment-wide approach o effective engagement with the community o maintaining momentum in intervention approaches and engagement with the community and project partners o capacity building and knowledge sharing with other projects through a community of practice o implementing and developing the best practice model, utilising a range of approaches and activities o targeting litter prevention at the source o others also included: establishing buy-in with delivery organisations and partners, particularly stakeholders at senior levels; effective project planning and strong governance mechanisms; targeting and tailoring interventions (for example, using culturally appropriate communications); investing in research; and using positive messaging and consistent communications.  The two key challenges with the Program identified by interviewees were issues with reporting requirements and the structure of grant rounds. The challenge of managing community or volunteer-run organisations with lower capacity for grants and litter management as well as managing councils with higher capacity was also highlighted.  The two key barriers to success at the individual project level through the Litter Hotspots Program were around staff and volunteer management within funded organisations, and difficulties engaging internal staff for project buy-in as well as the target audience for interventions.  Four key suggestions for Program level improvements emerged through this evaluation— improving application and reporting processes, supporting momentum and ongoing funding for litter prevention interventions, providing grants for 18-month durations as opposed to 12 months or three years, and continuing to develop the catchment-wide approach to litter management.  Key ongoing challenges for litter management in metropolitan Melbourne identified through this evaluation include: o balancing support for new entrants in the litter prevention space, while pushing those already engaged to continue improving best practice o challenges with consistent data collection (particularly in relation to measuring behaviour change components of litter interventions) o balancing the need for innovation and supporting wider implementation of proven approaches o MWRRG’s ongoing capacity for litter management o continuing to support momentum in litter prevention and maintaining engagement with the diversity of stakeholders implicated in litter management.

Prepared for MWRRG 11 Litter Hotspots Program Final Evaluation—Final Report

3.2 Recommendations Based on these findings, we make the following recommendations for MWRRG:

1. Continue to support momentum within Victoria’s litter management system between funding cycles, recognising MWRRG’s role as a key strategic stakeholder—including providing policy advice and contributing to decision-making processes that involve state and local government stakeholders, non-government organisations and community groups. 2. Continue to support the ongoing development of best practice litter management in metropolitan Melbourne, which may involve: a. building the capability of both new entrants to the litter prevention space, as well as encouraging continuous improvement and innovation among councils, non- government organisations and community groups already implementing best practice approaches b. supporting and maintaining the existing community of practice on litter prevention by providing opportunities for stakeholders to collaborate, network, learn from each other and build partnerships (such as through forums, networking events, etc.) c. providing direct capacity building support for organisations wanting to continue their litter prevention work beyond grants funding periods (such as offering training, providing delivery support, developing tools and resources etc.). 3. Continue to work with portfolio partners on improving data collection, monitoring and evaluation processes to measure the impact of litter interventions—such as through providing training, advice, data collection tools and resources etc.). There is an opportunity to ensure that key intended outcomes are identified at the start of interventions, and that monitoring and evaluation is planned to most effectively capture those outcomes. This may involve more strategic approaches to measuring behaviour change aspects of litter interventions to ensure that monitoring and evaluation resources are spent most efficiently. 4. Continue to pursue further funding opportunities from DELWP and other Victorian or Commonwealth agencies to deliver litter management programs in future, building on the success of MWRRG’s delivery of the Litter Hotspots Program.

We make the following recommendations for litter management in Victoria more broadly:

5. Continue to promote a collaborative catchment-wide approach to litter management. This may include: a. supporting a range of both regional and local level litter interventions that align with wider portfolio priorities, policies and strategies b. ensuring the legacy of the network that was strengthened through the Litter Hotspots Program by continuing to engage key stakeholders—including VLAA, KVB, MWRRG, local councils, portfolio partners, non-government organisations and community groups—on litter related policies, programs and decision-making. 6. Continue to support both innovation in the litter management space, as well as provide financial support for the up-scaling of proven successful interventions throughout the state in transferable contexts.

Prepared for MWRRG 12 Litter Hotspots Program Final Evaluation—Final Report

7. Continue to prioritise funding for litter interventions that align with the best practice model for litter prevention and involve the three key elements of education, infrastructure and enforcement. This may include: a. continuing to support the work of VLAA, SV, MWRRG and other key advocates for the best practice model b. continuing to support the ongoing maintenance of the litter management community of practice between funding cycles. 8. Apply the lessons learned from the Litter Hotspots Program to future litter prevention programs—particularly the key success factors that have been identified to contribute to the most effective litter interventions. 9. Refer to the Litter Hotspots Program as an example of a successfully administered government grants program: a. in particular, the MWRRG’s approach of providing a high level of project management support, capacity building and networking opportunities to grant recipients could be applied in other grants programs b. insights from the Litter Hotspots Program on the optimal length for grant funding periods (18 months) for small-to medium scale litter prevention projects could also be more widely applied as appropriate.

Prepared for MWRRG 13 Litter Hotspots Program Final Evaluation—Final Report

4 Effectiveness 4.1 Overview This section responds to the following evaluation questions:

 Has the program achieved its stated objectives and expected outcomes?  What unexpected or additional outcomes were achieved?

Evaluation results presented here demonstrate that:

 The Litter Hotspots Program achieved or made strong progress towards achieving all four stated program objectives. Significant, demonstrable outcomes were achieved across the areas of best practice litter prevention, developing partnerships to build capacity, increasing responsibility for litter disposal and prevention, and improving the amenity of public spaces impacting on the water quality of the Yarra River and Port Phillip Bay.  A range of approaches employed in the Litter Hotspots Program ensured it was delivered with the most effective use of resources—in particular, the catchment-wide and partnerships approach, and the capacity building and project support provided by MWRRG.  The majority of organisations funded through the Program have continued to deliver outcomes in litter management beyond the funding period, sometimes as a direct result of the Litter Hotspots Program.  At the Program level, consultation, communications and engagement were effective throughout the Litter Hotspots Program. The Program successfully established an ongoing network in the litter prevention space, and broadened and diversified the range of community, council and business organisations partnering on litter prevention initiatives.  A key characteristic of the Litter Hotspots Program was its alignment and commitment to improving the implementation of litter prevention in metropolitan Melbourne, based on VLAA’s best practice model. 4.2 Achievement of outcomes and objectives 4.2.1 Litter Hotspots

In addition to the ten pilot projects funded in 2013, 25 projects targeting litter hotspots across the catchment were funded between 2014 and 2017.

As presented in detail throughout the following sections of the report, key achievements of the Litter Hotspots Program include:

 Funding projects that targeted diverse audiences, for example, trader groups, school students, businesses, industry organisations, local communities, dog-walkers and smokers.  382 litter hotspot sites targeted with litter interventions, and an overall improvement in the litter ‘Hotshot’ score across hotspot sites.  $488,659 in additional funding contributed by funded organisations and project delivery partners, and $1,318,870 of in-kind contributions leveraged from funded organisations and project delivery partners.

Prepared for MWRRG 14 Litter Hotspots Program Final Evaluation—Final Report

 The creation of 267 new partnerships between organisations on litter management initiatives.  Engagement with 13,227 participants across funded initiatives, 492 community events and 355 media articles and publications.  175,230 kilograms of litter collected, and 3,317 additional litter infrastructure units installed (for example, bins).  Wide demographic and geographic reach of funded projects across the catchment (see Figure 1 and Figure 2 below).

1. Bayside City Council 13. Friends of Westgate Park Inc. 25. City of Whittlesea 2. Darebin City Council 14. Friends of Williamstown Wetlands 26. Yarra Ranges Shire Council 3. Dolphin Research Institute 15. Hobsons Bay City Council 27. Friends of Kororoit Creek 4. Hobsons Bay City Council 16. Kingston City Council 28. City of Greater Dandenong 5. Kingston City Council 17. Manningham City Council 29. Friends of Williamstown Wetlands 6. Manningham City Council 18. Maribyrnong City Council 30. Hume City Council 7. City of Melbourne 19. Maroondah City Council 31. Maroondah City Council 8. Mornington Peninsula RWMG 20. City of Monash 32. Nillumbik Shire Council 9. City of Port Phillip / EcoCentre 21. City of Melbourne (Laneways) 33. Plastic Bag Free Warburton 10. Yarra City Council 22. The City of Melbourne (Butts) 34. Whittlesea City Council rd 11. 3 Croydon Scout Group 23. Tangaroa Blue Foundation Ltd 35. Wyndham City Council 12. Bellarine Catchment Network 24. Port Phillip EcoCentre Figure 1. Litter Hotspots Program funded projects (MWRRG 2017).

Prepared for MWRRG 15 Litter Hotspots Program Final Evaluation—Final Report

Figure 2. Litter Hotspots Program network partnership map.

4.2.2 Project and Program outcomes

Program level outcomes

The Litter Hotspots Program achieved or made strong progress towards achieving all four stated program objectives. Significant, demonstrable outcomes were achieved across the areas of best practice litter prevention, developing partnerships to build capacity, increasing responsibility for litter disposal and prevention, and improving the amenity of public spaces impacting on the water quality of the Yarra River and Port Phillip Bay.

All 25 funded projects align with the Litter Hotspots Program’s objectives.

Table 4 presents an assessment of the achievement of each Program objective and the key outcomes delivered across the four areas. This builds upon the outcomes and indicators identified

Prepared for MWRRG 16 Litter Hotspots Program Final Evaluation—Final Report for each outcome in the Litter Hotspots Program Monitoring and Evaluation Framework, and outcome achievement ratings are guided by the rubric in Table 8 in Appendix 2.

Prepared for MWRRG 17 Litter Hotspots Program Final Evaluation—Final Report

Table 4. Achievement of Litter Hotspots objectives and outcomes.

Stated objectives Intended outcomes and indicators Outcomes achieved Overall extent of achievement Achieved or Mostly Contributed to Little or no exceeded / achieved / achievement / achievement / evidence of evidence of evidence of little or no significant change some change evidence of change change 1. Support the Identification of key issues and Interventions at 382 identified litter implementation priority hotspots hotspots of best practice Strategic delivery of infrastructure, Delivery of 25 funded projects, including litter prevention education and enforcement infrastructure, education and projects activities enforcement activities Inter/intra organisation co- Improved inter- and intra-organisational operation and working working relationships were cited by  relationships funded organisations, MWRRG and portfolio partners. Professional development, training MWRRG delivered quarterly networking, and implementation of preferred capacity building and knowledge sharing best practices forums and ongoing project support for implementation of best practice litter prevention 2. Develop New partnerships formally 267 new partnerships established partnerships in established (via contracts, between funded organisations and project order to capacity agreements, letters) partners build litter Total project value, cash and in- $488,659 in additional cash contributions prevention kind funds leveraged $1,318,870 of in-kind contributions  projects leveraged Development and delivery of litter 3,317 additional infrastructure units prevention infrastructure and installed resource materials (X bins etc.)

Prepared for MWRRG 18 Litter Hotspots Program Final Evaluation—Final Report

Improved leverage to reach 355 media articles and publications audiences through publication of consistent messages, communications and media 3. Improve Increased awareness of litter and 13,227 participants engaged in litter personal its impacts prevention activities responsibility for Example: 60% increase in awareness litter disposal about the impacts of litter on platypus and prevention. habitats through the ‘Ramp up Litter Prevention’ project at Eltham Skate Park Improved bin disposal behaviour Example: 38% increase in awareness about litter and stormwater through the Ramp up Litter Prevention’ project at Eltham Skate Park, leading to increased bin usage and a 27% reduction in litter Example: Installation of a 1,100 litre bin along Yarra River through the ‘Keeping  Westgate Park Litter Free’ project X% improvement in understanding Example: 40% increase in understanding where stormwater goes about stormwater among traders through the ‘Keep Kingsway Clean’ project X% reduction in dumping at target Example: 54% average reduction in illegal sites dumping incidences across hotspot sites through the ‘Reducing Litter and Dumped Rubbish’ project X% increase in volunteers at clean- Example: 90 new volunteers engaged ups through the ‘Love Where You Live’ project X events with Y people attending 492 community events with 13,227 people attending 4. Improve the X% reduction in amount of litter at Average improvement in the litter  amenity of public project sites ‘Hotshot’ score across hotspot sites, from

Prepared for MWRRG 19 Litter Hotspots Program Final Evaluation—Final Report

spaces that 3 to 2.2. impact upon the X kg litter removed 175,230 kilograms of litter removed water quality of X improvement in water quality Example: observed improvement in water the Yarra River and/or indicator species found quality through the ‘Yarra River Litter and Port Phillip Removal’ project Bay

Prepared for MWRRG 20 Litter Hotspots Program Final Evaluation—Final Report

Overall, the Litter Hotspots Program was considered to be very successful by staff, stakeholders and grant recipients. Through this evaluation, all interviewees stated that that the Program was either successful or very successful—with 17 interviewees (61%) stating that it was very successful (see Figure 3).

18

16

14

12

10 Projects Staff 8 Stakeholders 6

Total Numberofinterviewees 4

2

0 Not at all Somewhat Moderately Very NA successful successful successful successful

Figure 3. Overall success of the Litter Hotspots Program (n=28).

Among funded community organisations and local councils, the Litter Hotspots Program led to improvements in capacity for best practice litter management, partnerships with other organisations and engagement with local communities on litter prevention.

Through interviews, 16 grant recipients (80%) reported either moderate or great improvement in their organisation’s capacity for best practice litter management; 14 grant recipients (70%) reported either moderate or great improvement in their organisation’s partnerships with other organisations; and 17 grant recipients (85%) reported either moderate or great improvement in their organisation’s engagement with local communities on litter prevention (see Figure 4).

“[It has led to a] completely different mindset on litter and it has forced us to do some practices that we are doing to this day, one thing is it forced us to get a proper 1500 litre bin in our compound to dispose of litter and we developed a relationship with the [local] recycling centre to take the recycling from the park which never happened before.” (Funded project representative, interviewee)

In some cases, there was less scope for improvements, depending on the existing capacity of the funded organisation, for example, a local government grant recipient noted that:

“Our organisation’s face-to-face interaction with the community on cigarette butt litter specifically has improved. Our management of litter has remained the same as before the project.” (Funded project representative, interviewee)

Prepared for MWRRG 21 Litter Hotspots Program Final Evaluation—Final Report

12

10

8 Capacity for best practice litter management 6 Partnerships with other organisations

4 Engagement with the local

community on litter prevention Numberofinterviewees

2

0 Not at all Somewhat Moderately Greatly NA

Figure 4. Extent to which the Litter Hotspots Program has led to improvements in funded organisations’ capacity for best practice litter management, partnerships and engagement (n=20).

Project level outcomes

Across Rounds 2 and 3 of the Litter Hotspots Program between 2014 and 2017, all 25 funded projects achieved outcomes that contributed to the achievement of the wider Program objectives. Table 5 presents an overview of each funded project, including an example of the key outcomes achieved by each.

Instances of innovation emerged from the projects—including delivering engagement projects through art (Beautiful Brimbank and Nullumbik Shire), research and design (Manningham), developing a new reporting method plastic resin pellets (Tangaroa Blue). Box 1 also includes a short case study, demonstrating the project-level outcomes of the ‘Love Where You Live’ Drain Project delivered by Beautiful Brimbank and the Friends of Kororoit Creek.

Box 1. ‘Love Where You Live’ project outcomes.

The ‘Love Where You Live’ Drain Project is an example of a successful community engagement project funded through the Litter Hotspots Program. The project increased community awareness regarding the amount of litter entering local waterways. Using art, education and community clean- up days, the project aimed to create long-term behaviour change. The project utilised various methods of engagement to spread the anti-litter message—including the design and implementation of a mural in both the Sunshine Central Business District and Kororoit Creek and the creation of a cigarette butt mascot. The project achieved a 58% reduction in litter at its hotspot sites. This project also created strong relationships within the community and partnerships with local organisations, in particular with Brimbank City Council and the Sunshine Central Business District— an invaluable resource to utilise in future litter projects.

Prepared for MWRRG 22 Litter Hotspots Program Final Evaluation—Final Report

Table 5. Funded projects, outcomes, alignment to Program objectives and best practice litter prevention components.

Organisation Project Best practice Sector or target Litter type Region Key project outcome components group 3rd Croydon ‘Yarra River Litter Education / Youth engagement Small items washed Hotspot sites along the 1500 kilograms of litter collected Scout Group Removal’ Engagement through from the street Wonga Park – Warrandyte Infrastructure litter / vehicle parts / section of the Yarra River, dumped building waste Manningham Friends of ‘Keeping Education / Local residents of General park litter and Hotspot sites around Installation of 5 bins along the Westgate Park Westgate Park Engagement Westgate Park fishing debris Westgate Park, Port Yarra River section of Westgate Litter Free’ Infrastructure Melbourne Park

City of Kingston ‘Filter it Out’ Education / Local life-saving Cigarette butts Hotspots along the Project implemented 200 Engagement clubs Kingston foreshore infrastructure units – a mixture of Infrastructure new bins along the foreshore and the distribution of personal butt bins to smokers. Maribyrnong ‘Footscray Litter Education / Traders in the Litter from traders FBD, Maribyrnong 23.5% litter reduction around the City Council Prevention’ Engagement Footscray Business ending up in FBD District (FBD) stormwater drains and around the FBD vicinity Maroondah City ‘My Mullum Education / Community, Litter from the Mullum Mullum Creek Dog droppings decreased by 81% in Council Mullum Creek’ Engagement especially pet community / dogs section between Suda the targeted area, litter decreased Infrastructure owners and local entering Mullum Avenue, Ringwood and its by 61% Enforcement businesses Mullum Creek headwaters at Highland avenue, Croydon. City of ‘Food Businesses Education / Traders within the Litter from this precinct Chinatown precinct, City Distributed 20 business education Melbourne and Stormwater Engagement Chinatown precinct entering stormwater of Melbourne packs regarding impact of litter in Pollution’ Infrastructure drains stormwater drains Enforcement City of Monash ‘Keep Kingsway Education / 32 traders within Litter from this precinct Target areas in Kingsway 75% decrease in cigarette but litter, Clean’ Engagement particular laneways entering stormwater (Glen Waverly) 40% increase of traders

Prepared for MWRRG 23 Litter Hotspots Program Final Evaluation—Final Report

Organisation Project Best practice Sector or target Litter type Region Key project outcome components group Infrastructure in the municipality drains understanding impact litter has in Enforcement stormwater drains Bellarine ‘Caring for Our Education / Local schools, wider Litter from the Corio Catchment area of Corio 80% decrease in litter across all Catchment Bays’ Engagement community Bay foreshore entering Bay three sites over a three year period Network Infrastructure the seabed Hobsons Bay ‘Cigarette Litter Education / Traders, businesses Cigarette butt litter Six hotspot areas around 20,681 butts collected City Council Reduction’ Engagement and identified Hobsons Bay (three in the Infrastructure recreation facilities west and three in the east) that house hotspot locations Manningham Development of Infrastructure NA – research and Litter that would Six hotspot sites around Development and implementation City Council the ‘RiverSafe’ design focused typically be caught by a Manningham (which had of a cost-effective GPT Low Maintenance GPT the new GPT prototypes Gross Pollutant installed) Trap (GPT) City of ‘Operation Bin Education / Students / hospital Cigarette butt litter 60 hotspot sites around A total of 1,129,946 cigarette butts Melbourne Infrastructure’ Engagement patients and visitors RMIT, Melbourne have been captured in butt bin Infrastructure University, Peter Mac infrastructure for recycling Hospital (both the old and new hospital) and the Royal Children’s Hospital Melbourne. Port Phillip ‘Turn off the Education / Stakeholders in the Microplastics entering 96 sites around the Port 13 cafes around the City of Port EcoCentre Tap—Catchment Engagement microplastic space the waterways Phillip catchment region Phillip have replaced plastic straws to Bay Litter with paper straws, which was Prevention and driven by data collection activities Monitoring’ Tangaroa Blue ‘Operation Clean Education / Plastics and logistics plastic resin pellet Stakeholders targeted Pellet methodology developed Foundation Sweep’ Engagement industry members (PRP) rather than geographical during this project is becoming the

Prepared for MWRRG 24 Litter Hotspots Program Final Evaluation—Final Report

Organisation Project Best practice Sector or target Litter type Region Key project outcome components group Infrastructure sites national standard—developed a Enforcement Manual to guide PRP management. It has been downloaded 60 times City of ‘Reducing Litter Education / Developers, Illegally dumped Whittlesea, Edgars Creek, Illegal dumping has reduced by Whittlesea and Dumped Engagement contractors, wider rubbish Merri Creek, Darebin 54% Rubbish’ Enforcement community Creek and Plenty River Yarra Ranges ‘Communities for Education / Local communities Litter entering local Yarra River (Warburton), 130 litres of plastic bands, rings Shire Council Platypus— litter- Engagement waterways and Monbulk Creek (Belgrave), and hair ties collected by students free towns’ Infrastructure affecting platypuses Watts River, Graceburn across Yarra Ranges Council, Creek (Healesville) totalling 33,399 bands Beautiful ‘Love Where You Education / Local community Litter left on the street Hotspots around the 58% reduction in the number of Brimbank / Live’ drain project Engagement and local business which flows into Sunshine CBD, Kororoit litter items found in hotspot sites Friends of association waterways Creek and Sunshine Kororoit Creek Heights Primary School City of Greater ‘Greater Education / Both smokers and Cigarette Butts 8 Hotspot sites in the 71% reduction in the number of Dandenong Dandenong Engagement non-smokers in the Dandenong region cigarette butts Cigarette Butt Infrastructure Dandenong Litter Prevention community Campaign’ Friends of ‘Wader Beach for Engagement No specific target Tidal litter ending up on Williamstown Litter survey data added from Williamstown Birds not Litter’ group – trying to Wader Beach Wader Beach and Greenwich Bay Wetlands find the source of to the Australia-wide Tangaroa plastic litter on Blue litter database—used to raise Wader Beach awareness within government and assist in policy Hume City Smart bins, Stop Education / School children, Takeaway precinct 3 hotspot sites in Hume: Litter decreased by 61% Council Litter! Engagement local traders and litter, school litter and Hothlyn Drive Infrastructure wider community café litter (Craigieburn), Mahoneys Road (Campbellfield),

Prepared for MWRRG 25 Litter Hotspots Program Final Evaluation—Final Report

Organisation Project Best practice Sector or target Litter type Region Key project outcome components group Melrose Drive (Tullamarine) Maroondah City Dorset Recreation Education / Users of the Dorset Litter from reserve Dorset Recreation Significant decrease in god litter Council Reserve impact Engagement Recreation Reserve, users Reserve, Maroondah around the reserve on Tarralla Creek Infrastructure in particular dog Enforcement walkers Nillumbik Shire ‘Ramp up Litter Education / Primary and Litter from skatepark Eltham Skate Park, Eltham 60% increase in platypus Council Prevention— Engagement secondary aged users that affects awareness and 27% reduction in Eltham Skate Infrastructure school-children platypus in surrounding litter Park’ using the skate park waterways Plastic Bag Free ‘Plastic Bag Free Education / Local community of Plastic bag users of Warburton Plastic bag use estimated to have Warburton Warburton’ litter Engagement Warburton (local Warburton community reduced by 15-20% prevention traders and and traders who program schools) provide plastic bags Wyndham City ‘Hook, Line and Education / Local fishermen, Litter entering Werribee River and Hotspot sites have seen a 63% Council Litter’ Engagement wider community waterways and fishing Werribee South foreshore, average reduction in litter Infrastructure debris Wyndham City Whittlesea City ‘Prevent Litter— Education / Commercial and Litter falling off trailers Whittlesea 20% increase in trailers covering Council Cover Your Load’ Engagement residential users of from use of the their loads Infrastructure local landfill and dumpsites recycling facilities

Prepared for MWRRG 26 Litter Hotspots Program Final Evaluation—Final Report

Outcomes that grants have contributed to beyond the life of projects

Importantly, the majority of organisations funded through the Program have continued to deliver outcomes in litter management beyond the funding period, sometimes as a direct result of the Litter Hotspots Program.

Through interviews, eight (40%) project representatives indicated that their organisation is continuing to implement litter prevention initiatives either following on from or similar to their funded project, and 13 (65%) indicated that their organisation is implementing additional litter management interventions separate from the Litter Hotspots Program.

Two Litter Hotspots funded projects won Keep Victoria Beautiful Sustainable Cities awards. Greater City of Dandenong’s ‘Cigarette Butt Litter Prevention Campaign’ received an award for litter prevention and the Port Phillip EcoCentre received the Clean Beach/Waterway—Community Project award for their ‘Turn off the Tap—Catchment to Bay Litter Prevention and Monitoring’ project.

The funded ‘Caring For Our Bays’ project has also been nominated and selected as a finalist in two awards—the Landcare Victoria Coastcare Category awards and the Victorian Regional Achievement and Community Awards.

Another example of continuing Program outcomes is provided in Box 2 below.

Box 2. ‘Plastic Bag Free Warburton’: outcomes beyond the life of the Litter Hotspots Program.

The ‘Plastic Bag Free Warburton Litter Prevention Project’ is set to continue delivering outcomes beyond the life of the Litter Hotspots Program. Community organisation Plastic Bag Free Warburton plan to partner with neighbouring towns to assist them in developing their own localised approach to be plastic bag-free. Within their own town, they are also set on continuing their ‘Sew Warburton’ events, where they sew reusable shopping bags and leave them in strategic shopping hubs around the town. They are also planning to have another community event to continue to raise awareness of single-use plastics in both their community and the wider vicinity. The community group has recently been given a working space in a local sustainability hub which will be an invaluable resource for their future work.

External factors

A range of external factors were observed to have impacted on the achievement of project and Program level outcomes, as identified through interviews. Providing context for evaluation results around the achievement of outcomes at the project level, these include:

 Internal staffing and resourcing issues within funded organisations affecting the delivery of projects, identified by seven interviewees. For example, some funded local governments reported difficulties engaging council departments that are not normally involved in litter management to contribute to their projects.  Industry specific factors affecting individual projects, identified by three interviewees.

Prepared for MWRRG 27 Litter Hotspots Program Final Evaluation—Final Report

 Other external factors identified by funded project representatives include: changes over the life of contracts, difficulty in knowing whether litter at a particular site was just moved or actually prevented, and wider government restrictions and compliance concerns.

Unexpected outcomes

A range of unexpected outcomes were identified as resulting from the Program—these were largely positive rather than negative, with interviewees stating that a surprising outcome was the unexpectedly positive results achieved through their funded projects.

 Seven interviewees observed that ongoing impacts achieved after their funded project was a positive unexpected outcome arising from the Program, for example:

“The [follow-up initiative] was never planned. They decided after the council project was finished that they would carry on—that was unexpected.” (Funded project representative, interviewee)

 Five interviewees identified that their assumptions about litter habits were challenged through the Program, or that the extent of the litter problem at hotspot sites was larger than expected. 4.2.3 Consultation, communications and engagement

Program level consultation, communications and engagement

At the Program level, consultation, communications and engagement were effective throughout the Litter Hotspots Program. The Program successfully established an ongoing network in the litter prevention space, and broadened and diversified the range of community, council and business organisations partnering on litter prevention initiatives.

Through this evaluation, aspects of MWRRG’s approach to consultation, communications and engagement were noted as being particularly effective and contributing to the value of the Program:

 Encouraging collaboration, networking and knowledge sharing between funded organisations and other stakeholders.  Providing general support to funded projects and offering clear, consistent and timely communications throughout grant administration process—including applications, contract commencement, implementation, and reporting phases of the grant cycle.  Internal communications, leading to improved linkages between departments within MWRRG.  External communications with funded projects, community stakeholders and portfolio partners (including Melbourne Water, DELWP, EPA and SV, for example).

Of the 27 interviewees, including staff, stakeholders and project representatives, who responded to this question, 100% indicated that working relationships between the MWRRG, funded organisations and other stakeholders were effective, for example:

“Well, I think it was entirely effective. The communication was always timely, well-pitched and accurate. The staff from the MWRRG are energetic and personable, as well as very knowledgeable.” (Stakeholder, interviewee)

Prepared for MWRRG 28 Litter Hotspots Program Final Evaluation—Final Report

The CLEAN network was a crucial component of effectively achieving outcomes relating to increasing capacity for best practice litter management, and strengthened the network of stakeholders working in the litter management space.

Box 3. Reflections on effectiveness of the CLEAN network.

The aim of the CLEAN network events was to provide an opportunity for grant recipients to gain or enhance skills in litter management. The events also provided an opportunity for grant recipients to share project lessons learned and developments with both each other and MWRRG. In general, the feedback around the CLEAN network events was positive:

“I went to the events they were running—networking meetings and events—which have been very beneficial, especially hearing the case studies and making new connections and contacts.” (Funded project representative, interviewee)

Some individuals who had extensive experience in litter management felt they did not learn anything new at the CLEAN events. In the future, the events could be pitched at two different levels—those new to the project management space and those who have extensive skills in this space but wish to upskill in a few key areas. We note that this would require more investment in terms of time and finances for MWRRG.

Further reflections on the CLEAN network and suggestions for improvement are discussed in more detail in Section 7.2.3.

Organisations funded through the Litter Hotspots Program were well supported by the MWRRG throughout the delivery of their projects. Through interviews, 16 grant recipients (80%) reported that they felt very supported by MWRRG, with all interviewees reporting that they felt at least somewhat supported (see Figure 5). This was also strongly reflected in the feedback provided through interviews.

Prepared for MWRRG 29 Litter Hotspots Program Final Evaluation—Final Report

90%

80% 70% 60% 50% 40% 30% 20%

10% Proportionintervieweesofproject 0% Not at all supported Somewhat Moderately Very supported supported supported

Figure 5. Extent to which funded organisations felt supported throughout the delivery of their Litter Hotspots Program project (n=20)

4.2.4 Best practice litter prevention

A key characteristic of the Litter Hotspots Program was its alignment and commitment to improving the implementation of litter prevention in Victoria, based on VLAA’s best practice model. The three key elements of the best practice model are education, infrastructure and enforcement, underpinned by other components including research, collaboration, communication, incentives, and monitoring and evaluation (see Figure 6). The key elements of the best practice model used in each of the 25 funded projects is presented in Table 5 above (page 23), demonstrating that all Litter Hotspots projects employed at least two of the three elements of education, infrastructure and enforcement.

Prepared for MWRRG 30 Litter Hotspots Program Final Evaluation—Final Report

Figure 6. Victorian Litter Action Alliance best practice model for litter prevention (VLAA 2017).

Insights from this evaluation confirm that, in general, litter prevention initiatives that employ all three key elements are most effective (see Box 4 on the following page for a brief case study). Through interviews, grant recipients reflected on their implementation of aspects of the best practice model, for example:

“That was very much the model we always work towards—the three tiers of infrastructure, education and enforcement […] We are using this approach through this whole grant we made sure that the three areas were being considered and equally involved. There’s no point focusing on one without the other two.” (Funded project representative, interviewee)

One grant recipient stated that their project could have been more successful with a stronger focus on education and enforcement in addition to infrastructure, highlighting the value of using all three components. Because of the diversity of funded interventions, the best practice model was not always appropriate or relevant to all projects through the Litter Hotspots Program, and some funded organisations adapted the best practice model to their own projects, for example:

“We didn’t actually use [the model], as we worked out that the litter toolkit was better suited to councils and not what we were doing. We developed a best practice management model for citizen science rather than council work. We used a pre-existing data base and before this project started most groups were only cleaning up and not doing data collection, and so they decided to come on board with us…” (Funded project representative, interviewee)

It was noted by various grant recipients that of the three elements, enforcement can often present the most challenges—particularly in terms of funding, staff resourcing and the efficacy of enforcement measures and approaches.

Prepared for MWRRG 31 Litter Hotspots Program Final Evaluation—Final Report

Box 4. Best practice litter prevention: Greater City of Dandenong.

Greater City of Dandenong’s ‘Cigarette Butt Litter Prevention Campaign’ was identified as an example of implementing the best practice model for litter prevention, including infrastructure, education and enforcement. The council educated the community about cigarette butt litter in targeted hotspots, significantly increasing community awareness through implementing a face to face education program, and the creation of a versatile anti-butt litter brand. Results from this project show that employing all three aspects of the best practice model delivers the best results— infrastructure, community education and follow up with enforcement.

Prepared for MWRRG 32 Litter Hotspots Program Final Evaluation—Final Report

5 Program delivery 5.1 Overview This section responds to the following evaluation questions:

 Has the program been delivered within its scope, budget, within the expected timeframe, and in line with appropriate governance and risk management practices?  Has the department demonstrated efficiency and economy in relation to the delivery of the program?

Evaluation results presented here demonstrate that:

 The Litter Hotspots Program was generally delivered within scope, on budget and on time.  The Program was delivered with relative efficiency and economy.  The MWRRG demonstrated appropriate governance and risk management practices in delivering the Litter Hotspots Program.  Overall, the MWRRG is seen to have effectively managed the Litter Hotspots Program between 2014 and 2017.  Interviewees reflected on the relative effectiveness of Litter Hotspots Program administration compared to other similar government funded grants programs. 5.2 Scope, budget, timeframes From its outset, the scope of the Program was to identify litter hotspots and develop local partnerships with community, businesses and local government to reduce litter—by supporting infrastructure, education and awareness, and enforcement projects through a grants and capacity building program.

Program objectives within this scope were developed (as described in Section 1.2 above). The key Program components for delivering these objectives included administering two grant funding rounds (including small grants of less than $15,000, and multi-year grants of up to $200,000), and facilitating capacity building, networking and information sharing among Program partners, recipients and stakeholders.

These Program components have been successfully implemented. The scope of the Program remained materially unchanged between 2014 and 2017—key components have been delivered and no scope changes were reported. As such, the Program has been delivered within scope.

Overall, program delivery and final reporting to DELWP are expected to be completed on time by the scheduled 30 September 2017 timeframe. However, delays in project delivery throughout the Program have resulted in some timelines being revised throughout the three years. Overall, Program components and funded project milestones are on track for timely completion.

Grants funding was fully allocated across the funding rounds, and the Program budget is expected to be expended in full by 30 September 2017 as planned. Funds from the 2016-17 financial year have been carried forward to 2017-18 to enable final milestone payments for multi-year grant recipients and evaluation payment before 30 September 2017. As shown in Table 6 and Table 7:

 Total Program funding allocation from the Sustainability Fund was $1,900,000.

Prepared for MWRRG 33 Litter Hotspots Program Final Evaluation—Final Report

 Total budget expenditure to June 2017 is $1,760,973, with a total of $139,026 carried over to the 2017-18 financial year ($38,975 for operational expenses, $93,187 for outstanding project milestone payments, and $6,864 for MWRRG’s Program management).

Table 6. Litter Hotspots Program funding allocation from the Sustainability Fund.1

Program components 2014-15 2015-16 2016-17 Total

MWRRG Program management $110,000 $130,000 $130,000 $370,000

Operational Program budget $50,000 $50,000 $52,000 $152,000

Small grants $78,000 $57,000 $65,000 $200,000

Large grants $250,000 $425,000 $503,000 $1,178,000

Total (ex GST) $488,000 $662,000 $750,000 $1,900,000

Table 7. Litter Hotspots Program budget expenditure of Sustainability Fund allocation.2

Program components Total budget Total expenditure Funds carried over allocation to June 2017 to 2017-18

MWRRG Program management $359,421 $352,561 $6,864

Operational Program budget $162,858 $123,883 $38,975

Small and large grants $1,377,720 $1,284,533 $93,187

Total (ex GST) $1,900,000 $1,760,977 $139,026

The allocation of funds were reviewed and revised annually prior to the expenditure of grants and agreements in order to meet program needs.

An additional $332,753 was contributed by funded organisations and project delivery partners, as well as $1,138,870 of in-kind contributions leveraged from funded organisations and project delivery partners. This is discussed in further detail in Section 5.4 below on economy and efficiency. We also note, referring to Table 6 and Table 7 above, that less funding was spent on MWRRG’s Program management than was initially budgeted—allowing more funding to contribute to Program outcomes through funded projects.

1 Not inclusive of MWRRG or partner contributions. 2 Based on figures provided by MWRRG.

Prepared for MWRRG 34 Litter Hotspots Program Final Evaluation—Final Report

The obvious cost drivers for the Program are the funds provided to council and community organisations through small and large grants. A substantive investment was also made in providing capacity building, networking and knowledge sharing support through the operational budget.

The amount allocated to administration (MWRRG Program management—equivalent to 19% of the total Sustainability Fund contribution) is reasonable, considering the high level of support provided to funded organisations by MWRRG staff throughout the Program and the additional value that this provided (see Sections 4.2.3 and 7.2.1 for discussion). This is comparable to other grant programs we have evaluated. No benchmarks for program administration costs were available for comparison. 5.3 Governance and risk management MWRRG used appropriate governance and risk management practices in delivering the Litter Hotspots Program.

Program employed a tiered governance approach to facilitate engagement and capacity building at the community and local government, MWRRG and portfolio levels. Each governance group contributed to regular meetings, reporting and informed Program delivery. Key governance levels are described in Figure 7 below.

Waste & Resource Recovery Oversight Minister DELWP Project Control Board (DELWP, MWRRG…)

Management

MWRRG

Capacity VLAA, CLEAN Litter Forum Litter (Reps from each Project Org. Building Portfolio & & Metro Litter Network) Agencies

Hotspot Project Team Hotspot Local Planning Project Team & Hotspot Project Hotspot Implementation Partners Project Team

Figure 7. Litter Hotspots Program governance structure (MWRRG 2016).

Key components of Program governance processes included:

 stakeholder mapping  risk analysis framework and risk management processes  documentation and assessment of key issues affecting delivery  implementation schedule for delivering key Program actions and addressing identified issues

Prepared for MWRRG 35 Litter Hotspots Program Final Evaluation—Final Report

 communications strategy  monitoring and evaluation processes  budget management and reporting processes  financial expenditure and fund tracking.

Through yearly iterations of Litter Hotspots Program project plans (2014-15, 2015-16 and 2016-17), governance processes were updated and adapted in response to Program developments, external changes and contextual factors. Quarterly progress reports were submitted to DELWP to meet Sustainability Fund requirements. Specifically, grant administration processes included a sound grants selection process, involving clear criteria and rigorous guidelines.

MWRRG effectively managed Program level risk and supported the management of project level risk. The ‘Yarra River Litter Removal’ project is a key example of a funded project that successfully managed a high level of risk. The project leveraged management of this risk, by using it as a capacity- building tool. The project was run by the 3rd Croydon Scouts Group, and involved young people removing litter in the Yarra River via canoes. 3rd Croydon Scouts Group ran internal trainings on risk assessment and mitigation to develop a risk management framework. It was a great opportunity for them to upskill on risk management, and they have developed a useful tool that they will use in future projects.

Overall, MWRRG is seen to have effectively delivered the Litter Hotspots Program between 2014 and 2017. Through interviews with 20 grant recipients undertaken as part of this evaluation, all reported that they were either very satisfied (80%) or satisfied (20%) with MWRRG’s management of the Program (see Figure 8).

90%

80%

70%

60%

50%

40%

30%

20%

Proportionintervieweesofproject 10%

0% Not at all satisfied Dissatisfied Satisfied Very satisfied

Figure 8. Satisfaction with MWRRG’s management of the Litter Hotspots Program (n=20).

Prepared for MWRRG 36 Litter Hotspots Program Final Evaluation—Final Report

5.4 Efficiency and economy in delivery The Litter Hotspots Program was delivered with relative efficiency and economy.

The Program leveraged cash ($488,659) and significant in-kind ($1,138,870) contributions from funded organisations and delivery partners, substantively increasing the value of the Program relative to the Sustainability Fund investment ($1,900,000).

Unprompted, three interviewees reflected on the relative effectiveness of Litter Hotspots Program administration compared to other similar government funded grants programs.

A key component of Program delivery included providing grants varying in size and time period, testing which funding amounts and timeframes were most effective and efficient from the perspective of both MWRRG and the funded organisations.

Experiences from the Program have led to insights transferrable to other similar government grant funding programs, notably:

 From the perspective of both MWRRG and funded councils and community organisations, a 12-month grant is considered too short and a three-year grant is possibly too long, depending on the structure and administration of funded projects.  18-month grants were considered to provide optimal efficiency and effectiveness.  The opportunity for 3-year grants funding was well received by councils, community groups and wider litter management stakeholders.  While 3-year grants provide certainty in funding, which is important for both councils and community groups, the requirements for project planning create too much rigidity and the need for contract variations to adapt the project to meet emerging needs and changing contexts/external factors leads to unnecessary administrative effort (for both MWRRG and the funded organisation). By their nature, three-year projects require a certain amount of flexibility which currently does not correlate with Victorian Government standards for grants administration.  12-month grants present risks in terms of delays in project delivery. This timeframe can limit the outcomes achieved due to limits on the time available for delivering project activities, and potentially have a disproportionate level of administration compared to the value of the grant.

In meeting the stated objectives of the Litter Hotspots Program (as presented in Section 1.2), the Program was delivered in the following ways to ensure the most effective and efficient use of resources:

 using a catchment-wide, partnership approach, encouraging lasting outcomes around the health of both the Yarra River and Port Phillip Bay  funding targeted projects, including diversity among the key audiences and litter types of interventions  capacity building activities for funded organisations facilitated by MWRRG alongside the administration and management of the grants program  cash and in-kind contributions leveraged

Prepared for MWRRG 37 Litter Hotspots Program Final Evaluation—Final Report

 active networking among funded organisations and wider litter management stakeholders in Victoria  funding a combination of community organisations and local councils with varying levels of experience and capability in litter prevention  a strong focus on measuring outcomes and demonstrating the impact of the funded initiatives.

Prepared for MWRRG 38 Litter Hotspots Program Final Evaluation—Final Report

6 Justification and value 6.1 Overview This section responds to the following evaluation question:

 What is the evidence of a continued need for the program at this time and role for government in delivering this program?

Findings presented in this section demonstrate that:

 Overall, the Litter Hotspots Program provided value, evidenced by the outcomes achieved, and contributed to meeting a demonstrable need.  Funding for the Litter Hotspots Program has not been renewed and the Program will be discontinued. However, there is still a need for Victorian Government intervention in litter prevention.  Since the commencement of the Program, there have been some key contextual changes influencing litter management in Victoria, including: o Establishment of the Litter Innovation Fund—a $700,000 fund administered by SV that provides grants of up to $20,000 to organisations that provide innovative project proposals to reduce litter or dumping. o The Victorian Litter Strategy lapsed in 2014, and litter was incorporated into the Victorian Waste Education Strategy released in 2016. Despite litter now forming part of the Victorian Government’s wider waste education policy, it is perceived by some stakeholders that litter has declined as a priority—this reflects a gap in understanding among stakeholders about litter funding opportunities and the strengthening of statewide approaches to litter management. o The remit for litter management in Victoria has been broadened to include more of a focus on illegal dumping and micro-plastics in the River and Bay, although this is perceived by some stakeholders to have shifted the focus away from the broader problem of litter behaviour within the community.  Generally, the Litter Hotspots Program was well aligned with Victorian Government priorities, policies and strategies (including those of MWRRG, DELWP and SV). During the course of the Program, alignment was strengthened throughout the portfolio with the release of the Victorian Waste Education Strategy, the Victorian Litter Plan and the Metropolitan Waste and Resource Recovery Implementation Plan. 6.2 Program value and demonstrable need Overall, the Litter Hotspots Program provided value, evidenced by the outcomes achieved (as described in Section 0), and contributed to meeting a demonstrable need. This was clearly defined in Program planning and governance processes and was reflected in Program objectives and outcomes achieved.

While the Program has come to a close, the fact that the Victorian Government has supported the Litter Innovation Fund and other litter management initiatives indicates that there is still a need for government intervention in litter prevention. Lessons learned from the Litter Hotspots Program

Prepared for MWRRG 39 Litter Hotspots Program Final Evaluation—Final Report can provide useful insight to inform these initiatives, and MWRRG can continue to support litter management in collaboration with portfolio partners, local government, industry and community organisations. This is discussed in further detail in Section 7).

In some cases, some project representatives indicated that without funding received through the Program they would not have otherwise been able to deliver their litter prevention initiative— whereas, others reflected that the Program provided an additional incentive to deliver their project.

“[The Program] has made a huge difference to our local community in that we are able to offer significantly more to the local community in terms of what we offer to the [participants]. This would not have been possible without the assistance of the Program.” (Funded project representative, interviewee)

When asked through interviews, 22 (79%) grant recipients, MWRRG staff and external stakeholders stated that they believe there is a great deal of value in the Victorian Government continuing to fund a similar program in future.

25

20

15 Projects Staff 10 Stakeholders Total Numberofinterviewees 5

0 No value Some value Moderate A great deal of value value

Figure 9. Extent to which interviewees believe that there is value in the Victorian Government funding a similar program in future (n=28).

Through interviews, a selection of stakeholders, MWRRG staff and project representatives identified that in future there would be value in scaling up successful projects and taking a more catchment-wide or regional approach, not just council or local community specific. For example, investing in existing successful projects and making them available to all councils.

Unprompted, interviewees reflected on the value of future funding is in taking the key lessons learned from the Litter Hotspots Program and applying these to future programs. For example, case studies that have been developed through the Litter Hotspots Program can be used to inform future project delivery.

Prepared for MWRRG 40 Litter Hotspots Program Final Evaluation—Final Report

6.3 Wider context In terms of the Programs’ broader political and policy context, the Program sits under the Victorian Litter Plan (VLP). The VLP is a $1 million project that aims to increase the capacity of litter stakeholders to address litter issues, announced in October 2015. The VLP funded Round 3 of the Program, and also funds Victoria’s Litter Report Card, as well as the Victorian Litter Innovation Fund and other interventions to address Victoria’s priority litter issues.

The Report Card identified Victoria’s five main litter issues as:

 litter in coastal areas and waterways  illegal dumping of waste  roadside litter  cigarette butt litter  plastic and microplastic litter. The Litter Hotspots Program funded projects under these five key priority areas (see Table 5). Funding for the Litter Hotspots Program has not been renewed and the Program will be discontinued.

Since the commencement of the Program, there have been some key contextual changes influencing litter management in metropolitan Melbourne, including:

 Establishment of the Litter Innovation Fund—a $700,000 fund administered by SV that provides grants of up to $20,000 to organisations that provide innovative project proposals to reduce litter or dumping.  The Victorian Litter Strategy lapsed in 2014, and litter was incorporated into the Victorian Waste Education Strategy released in 2016. Despite litter now forming part of the Victorian Government’s wider waste education policy, it is perceived by some stakeholders that litter has declined as a priority—this reflects a gap in understanding among stakeholders about litter funding opportunities and the strengthening of statewide approaches to litter management.  The remit for litter management in Victoria has been broadened to include more of a focus on illegal dumping and micro-plastics in the River and Bay, although this is perceived by some stakeholders to have shifted the focus away from the broader problem of litter behaviour within the community. 6.4 Alignment to government priorities, policies and strategies Generally, the Litter Hotspots Program was well aligned with Victorian Government priorities, policies and strategies (including those of MWRRG, DELWP and SV), although alignment across the portfolio was identified as an area for improvement.

Prepared for MWRRG 41 Litter Hotspots Program Final Evaluation—Final Report

The Program was demonstrated to clearly align with the following key government priorities, policies and strategies throughout its delivery through annual iterations of the Litter Hotspots Program Project Plan:

 Metropolitan Waste and Resource Recovery Implementation Plan 2016 (MWRRG)3  Victorian Litter Plan 2016 (SV)4  Litter Innovation Fund 2016-17 (SV)5  Victoria’s Litter Report Card 2016 (SV)6  Victorian Waste Education Strategy 2015-2020 (SV)7  Draft 2017-2027 Port Phillip Bay Environmental Management Plan 2016 (DELWP)8  Environment Protection Act 1970 (Victorian Government)9  Stormwater Strategy 2013-18 (Melbourne Water)10  Best Practice Guide for Managing Litter and Illegal Dumping at Clothing Bins 2013 (SV)11  local government waste and litter reduction strategies.

3 4 5 6 7 8 < https://www.coastsandmarine.vic.gov.au/coastal-programs/port-phillip-bay> 9 10 11

Prepared for MWRRG 42 Litter Hotspots Program Final Evaluation—Final Report

7 Looking to the future 7.1 Overview This section responds to the following evaluation questions:

 What are the lessons for similar programs?  If ongoing funding was provided, what level of efficiencies could be realised?  What would be the impact of ceasing the program for example, service impact, community, jobs in both the public sector and the wider community) and what strategies have been identified to minimise negative impacts? What alternative approaches or strategies could provide a fall-back position?

As the Litter Hotspots Program comes to a close, discussion in this section focuses on the lessons learned and insights that can be used to inform future grant funding programs and litter interventions. Findings demonstrate that:

 The key identified success factor in the Litter Hotspots Program was the MWRRG’s management and administration of the Program.  A key success factor was the partnership component of projects, but also the collaborative component of the Program itself.  The two key challenges with the Program identified by interviewees were issues with reporting requirements and the length of grant rounds. The challenge of managing community or volunteer-run organisations with lower delivery capacity or experience with government grants and litter management, as well as managing councils and organisation with higher capacity for grants management and litter projects was also highlighted.  The two key barriers to success at the individual project level through the Litter Hotspots Program were around staff and volunteer management within funded organisations, and difficulties engaging internal staff for project buy-in as well as the target audience for interventions.  Four key suggestions for Program level improvements emerged through this evaluation: o simplifying application and reporting processes o supporting momentum and ongoing funding for litter prevention interventions o providing grants over 18-month timeframes as opposed to 12 months or three years, and continuing to develop the catchment-wide approach to litter management.  Key ongoing challenges for litter management in Victoria identified through this evaluation include: o balancing support for new entrants in the litter prevention space, while pushing those already engaged to continue improving best practice o challenges with gaining consistent data collection o balancing the need for innovation and supporting wider implementation of proven approaches o MWRRG’s ongoing capacity to resource and fund litter management o Continuing to support momentum in litter prevention and maintaining engagement with the diversity of stakeholders implicated in litter management.

Prepared for MWRRG 43 Litter Hotspots Program Final Evaluation—Final Report

7.2 Lessons learned 7.2.1 Success factors

Program level

The key identified success factor in the Litter Hotspots Program was the MWRRG’s management and administration of the Program, with 24 interviewees (86%) stating that the support, communications and oversight provided by MWRRG worked particularly well.

Contributing to this, the role of the Program Coordinator was consistently raised as a key factor of Program effectiveness, for example:

“[The Program Coordinator] encouraged us to think about things creatively when we came across obstacles. He made us see that things weren’t necessarily obstacles, but rather needed us to change our focus.” (Funded project representative, interviewee)

Notably, when asked to identify barriers to success, more than half of interviewees (14) stated that they had no issues with the way that MWRRG managed the Program and could not identify any challenges with program delivery.

The other main success factor identified through interviews was the partnership approach and collaborative aspect of the Program. Twelve interviewees reflected on this, for example:

“There were opportunities for collaboration and sharing of information and that was really important and saved time and ensured that things that were successful in other projects were able to be applied to our projects—we were able to collaborate with other successful project applicants.” (Funded project representative, interviewee)

Other key success factors at the Program level identified by interviewees included:

 The capability building aspect of the Program, particularly through the CLEAN network, as discussed in Section 4.2.3.  The flexibility of MWRRG in their management and administration of the Program, adapting to changes and emerging contexts that the Program and individual projects were delivered within.  The catchment-wide approach to Program delivery, including the alignment of projects across the catchment and supporting momentum in the litter prevention space through the structure of funding rounds, for example:

“We are strong advocates of a catchment-wide approach to any coastal and waterway issues. It is all one big ecosystem and all connected […] We cannot work in isolation from community, other groups or government on litter issues. [The Litter Hotspots Program] helped organisations get out of their ‘private patch’ and collaborate.” (Funded project representative, interviewee)

Project level

As well as at the Program level, the partnerships and collaborative approach was identified as a key success factor of individual funded projects. This encompasses both internal collaboration within organisations and external partnerships with other organisations, for example:

Prepared for MWRRG 44 Litter Hotspots Program Final Evaluation—Final Report

“The real success for us was the collaboration between the sustainability team and the waste services team and the on-ground pride crew […] and the environmental health officers [within Council]. Councils can be quite divided up, and this was an opportunity for collaboration.” (Funded project representative, interviewee)

Partnerships were successfully used in many of the projects and offered resources that contributed to the value of the projects—be it financial, human resources, technical, or networking opportunities.

Other identified success factors at the project level though the Litter Hotspots Program include the opportunity to implement the best practice model for litter prevention and build on past lessons learned through projects, and the lasting outcomes and flow-on impacts for litter management achieved through individual projects.

Through this evaluation, interviewees were asked to reflect on their lessons learnt about makes litter interventions most effective, these particularly included:

 taking a collaborative, catchment-wide approach  effective engagement with the community  maintaining momentum in intervention approaches and engagement with the community and project partners  capacity building and knowledge sharing with other projects through a community of practice  implementing and developing the best practice model, utilising a range of approaches and activities  targeting litter prevention at the source.

Others also included:

 establishing buy-in with delivery organisations and partners, particularly stakeholders at senior levels  effective project planning and strong governance mechanisms  targeting and tailoring interventions (for example, using culturally appropriate communications)  investing in research  using positive messaging and consistent communications. 7.2.2 Barriers to success

Program level

The two key challenges with the Program identified by interviewees were issues with reporting requirements and the structure of grant rounds. The challenge of managing community or volunteer-run organisations with lower capacity for grants and litter management as well as managing councils with higher capacity was also highlighted.

Issues with grants administration, particularly around reporting processes, were raised by eight funded project interviewees. In particular, two funded project representatives noted that MWRRG’s

Prepared for MWRRG 45 Litter Hotspots Program Final Evaluation—Final Report management of reporting deadlines and processes negatively impacted on their organisation, but this was impacted by grant recipients’ organisational capacity to manage grant administration.

Other interviewees noted that application and reporting processes were assisted greatly by the support they received from MWRRG staff, and noted that reporting is a necessary component of receiving government funding.

A key lesson learned from the Litter Hotspots program is that 18-months is an optimal period for grant funding. Five interviewees identified barriers regarding the length of funding periods and the structure of grant rounds. As discussed in Section 5.4, while the opportunity for varying grant amounts and longer funding periods was well-received by councils and community organisations, there were challenges with both 12-month and three-year projects:

“I think the last model of 18-month projects was probably the best—it gave projects the opportunity to work across all the seasons, and align with organisational funding cycles. But the one year program was too short […] 18 months was right.” (Stakeholder, interviewee)

When asked what didn’t work so well about the way the Program was delivered, interviewees identified the following additional barriers to success:

 issues with the consistency of data collection and the use of data collection tools  the structure of milestone payments to grant recipients  challenges with managing the varying levels of capacity for best practice litter management among community organisations that are often volunteer-run and more highly resourced council grant recipients.

Project level

The two key barriers to success at the individual project level through the Litter Hotspots Program were around staff and volunteer management within funded organisations, and difficulties engaging internal staff for project buy-in as well as the target audience for interventions.

Nine funded project representatives reported volunteer and staff management issues as barriers to the successful delivery of their projects, often leading to delays. This included problems associated with a high reliance on volunteer involvement in some projects, and the impacts of staff turn-over with funded organisations.

Eight interviewees identified challenges securing buy-in from internal staff (both on-ground and senior level) and engaging the target audience for their litter intervention that had to be overcome to enable successful project delivery.

When asked about what didn’t work so well about their funded projects, interviewees also identified the following challenges:

 difficulties with data collection methods to demonstrate the impact of interventions (particularly in relation to measuring behaviour change components of litter interventions)  managing seasonal factors, both social and environmental  complexities associated with working in bureaucratic systems.

Prepared for MWRRG 46 Litter Hotspots Program Final Evaluation—Final Report

7.2.3 Suggested improvements

Program level

Four key suggestions for Program level improvements emerged through this evaluation— improving application and reporting processes, supporting momentum and ongoing funding for litter prevention interventions, providing grants over 18-month timeframes as opposed to 12 months or three years, and continuing to develop the catchment-wide approach to litter management.

 Eight interviewees suggested that reporting and applications processes could be improved, for example, reducing the number of different forms required.  Seven interviewees suggested the need for more or ongoing funding, to enable continued momentum and up-scale successful projects.  Following on from discussion in Sections 5.4 and 7.2.2, interviewees suggested providing grants longer than one year, but not three-years—that 18 months is optimal.  Interviewees suggested that the catchment-wide approach to the Program was effective (as discussed in Section 7.2.1), but that this approach could be strengthened in future.  Specifically, interviewees also reflected on ways that the CLEAN network could be improved—for example that making the CLEAN network events available to stream online would make them more accessible to those with conflicting commitments or geographical constraints.

Through interviews, stakeholders reflected on changes to the CLEAN network over time, noting that it no longer only involves councils and is far more diverse, encompassing a wide range of community groups as well. This means there is potentially now a greater variation in the level of capacity for best practice litter management among network members—this is discussed in more detail in Section 7.2.4 on ongoing challenges.

Other suggestions for improvement at the Program level included:

 implementing a more consistent, overarching media campaign  improved data collection methods, including providing additional support to funded organisations  encouraging more catchment-wide projects that include joint funding applications between multiple organisations.

Project level

Through interviews, representatives from funded organisations offered suggestions for improvement at the project level, including:

 increasing capacity for effective project planning, and consistent implementation of project plans  further increasing the level of collaboration in project delivery  improved internal staff management, as well as management of external contractors and delivery partners.

Prepared for MWRRG 47 Litter Hotspots Program Final Evaluation—Final Report

7.2.4 Ongoing challenges

Key ongoing challenges for litter management in metropolitan Melbourne identified through this evaluation include:

 Balancing support for new entrants in the litter prevention space, while pushing those already engaged to continue improving best practice—this is a key challenge for MWRRG’s ongoing work in supporting litter management through capacity building, networking and knowledge sharing.  Challenges with consistent data collection—has been identified as an emerging issue by MWRRG. MWRRG and SV are continuing to advocate for and develop data collection tools and online data storage systems for delivery partners, councils and community organisations to use.  Balancing the need for innovation and supporting wider implementation of proven approaches—a key challenge for the Victorian Government more widely is in managing the tension between providing grant funding for innovation and funding projects that are tested and proven to be effective.  MWRRG having ongoing capacity to resource litter management—Litter Hotspots Program funding is being discontinued and MWRRG were unsuccessful in their application to DELWP and the Port Phillip Bay Fund to obtain funding to continue a similar program. In this context, MWRRG has identified that it will continue to support networking, knowledge sharing and the development of best practice litter management and has allocated a small operation budget to this effect.  Continuing to support momentum in litter prevention and maintaining engagement with the diversity of stakeholders implicated in litter management, particularly in the absence of ongoing funding.

“At the core, litter impacts so many people and requires so many different responses. It brings people together, but makes it difficult because it requires so many people.” (Stakeholder, interviewee)

Prepared for MWRRG 48 Litter Hotspots Program Final Evaluation—Final Report

Appendix 1: Methodology in detail Inception meeting An inception meeting was held on 30 May 2017. This allowed the FPC team to connect with the MWRRG team and Project Manager to:

 clarify the objectives of the evaluation, including what ‘success’ looks like and what the results of the evaluation would be used for (in addition to the requirements of the DTF Guidelines)  agree on reporting, timelines and project management processes  discuss previous Litter Hotspots Program evaluations, including the approach taken to analysing project-level data  review and discussion of the proposed methodology  identify existing documentation, information and data sources  source contact details for participants and stakeholders  discuss risks associated with the evaluation  agree on the format of the deliverables.

Following the inception meeting the evaluation plan and methodology was developed and provided to MWRRG for sign-off. Document review and gap analysis We undertook a thorough document review and gap analysis of all existing information sources and evaluation data. This was a crucial stage in the project and informed subsequent stages of the evaluation.

The following documents were reviewed and analysed:

 Litter Hotspots Program Plan, Best Practice Litter Prevention framework including objectives, key performance indicators and Investment Logic Map  Three previous Litter Hotspots Program evaluation reports  Litter Hotspot Project Plans and project reporting data from the 25 funded projects  MWRRG quarterly milestone reporting and project management data  Case studies, films and project collateral  Project funding agreements  CLEAN Litter Network documents and event plans  peer reviewed and grey literature.

Through the document review we undertook an alignment exercise to determine the extent to which existing data was adequate to answer the evaluation questions, and identified the key areas where additional data collection was needed. Primary data collection The exact methods and purpose of primary data collection activities (and supporting stakeholder engagement planning) were further informed by the gap analysis and ongoing consultation with MWRRG.

Prepared for MWRRG 49 Litter Hotspots Program Final Evaluation—Final Report

We have undertaken 28 interviews with MWRRG delivery staff (3), key stakeholders (5) and a selection of project officers from the 25 funded projects (20). Interviews have enabled us to collect data on areas not covered in any existing data sources and based on areas identified in the gap analysis. This provided an opportunity for key partners, stakeholders and project officers to provide any additional reports, documents or data that may be relevant to the evaluation—as well as providing them with an opportunity to offer insights into emerging trends, examples of best practice, regionally specific and issues based information. Data analysis After primary data collection was completed, we analysed all data sources relevant to the evaluation. We used both qualitative and quantitative techniques to summarise and synthesise evidence against the evaluation questions. Specifically, we used thematic techniques to analyse qualitative data—grouping, summarising and quantifying themes that emerge from open-ended questions. Quantitative data was analysed using descriptive and inferential statistics as appropriate. Reporting and presentation After data analysis was completed, we prepared a draft evaluation report, which was provided to MWRRG on 8 September 2017.

On 22 September 2017 the FPC team met with MWRRG to discuss the draft report. Based on feedback from MWRRG at this meeting, we revised the draft report and provided a final report to MWRRG (this document).

We have prepared a Power Point presentation and supporting materials to complement the evaluation report and guide a presentation session with MWRRG and key partners. We will present a ‘walk through’ of the final evaluation findings and recommendations to build a shared understanding of Program achievements, insights and lessons learned among delivery staff and stakeholders.

Prepared for MWRRG 50 Litter Hotspots Program Final Evaluation—Final Report

Appendix 2: Rubric Table 8. Rating rubric for evidence of outcomes achieved through the Litter Hotspots Program.

Rating Descriptor of achievement of outcomes / evidence for change

Achieved or exceeded / evidence of Strong quantitative evidence of change across key significant change indicators

Positive feedback from all stakeholder groups (very little negative feedback)

Mostly achieved / evidence of Quantitative evidence of change across key indicators change Mostly positive feedback from all stakeholder groups (some negative feedback)

Progress towards achievement / Some quantitative evidence of change evidence of some change A combination of positive and negative feedback from stakeholder groups

Little or no achievement / little or Quantitative evidence that no change has been no evidence of change achieved

Mostly negative feedback from all stakeholder groups

Prepared for MWRRG 51