Final Report Farming for Sustainable Soils Project

Prepared for North Central Catchment Management Authority

Farming for Sustainable Soils Project Final Report

Key contacts

North Central Catchment Management Authority 03 5448 7124 628-634 Midland Highway Huntly VIC 3551

First Person Consulting 03 9600 1778 Suite 3, Level 2, 190 Queen Street Melbourne VIC 3000 ABN 98 605 466 797 www.fpconsulting.com.au

Document details

Title: Farming for Sustainable Soils Project Final Report

Authors: Rebecca Denniss (First Person Consulting), Dan Healy (First Person Consulting), and Phil Dyson (Farming for Sustainable Soils Project Manager, North Central Catchment Management Authority)

Version: Final_UPDATED

Revision date: 16 November 2018 (original version of Final Report revised on 29 June 2018)

Client: North Central Catchment Management Authority

Prepared for North Central Catchment Management Authority i Farming for Sustainable Soils Project Final Report

Contents Tables ...... iii Figures ...... iii Acronyms ...... iv 1 Executive summary ...... 1 2 Introduction ...... 3 2.1 Overview ...... 3 2.2 Scope and approach ...... 3 3 Farming for Sustainable Soils Project ...... 5 3.1 Overview ...... 5 3.2 Project background ...... 5 3.3 Project inputs, activities and outputs ...... 8 4 Findings and evidence ...... 12 4.1 Overview ...... 12 4.1.1 Evidence base ...... 12 4.2 Effectiveness ...... 13 4.2.1 Outputs and outcomes ...... 13 4.3 Appropriateness ...... 20 4.4 Efficiency ...... 25 4.5 Impact ...... 30 4.6 Future ...... 32 4.6.1 Future opportunities ...... 32 4.6.2 Strengths and limitations ...... 33 4.6.3 Suggested improvements and ongoing challenges...... 36 5 Conclusion and recommendations ...... 39 5.1 Final evaluation of the Farming for Sustainable Soils Project ...... 39 5.1.1 Conclusions ...... 39 5.1.2 Recommendations ...... 43 5.1.3 Future activities for the asset ...... 43 Appendix 1—Methodology ...... 45 Overview ...... 45 Methodology in detail ...... 45 Inception meeting ...... 45

Prepared for North Central Catchment Management Authority ii Farming for Sustainable Soils Project Final Report

Project plan ...... 45 Document review ...... 48 Case study development ...... 48 Conference support ...... 49 Interviews and surveys ...... 49 Analysis and reporting ...... 49 Limitations...... 50

Tables Table 1. Summary of data collection components...... 4 Table 2. Survey data summary...... 4 Table 3. FSS Project budget and expenditure overview: Australian Government funding and community contributions...... 8 Table 4. FSS Project expenditure breakdown...... 8 Table 5. Years of FSS Group operation in Phase 2...... 9 Table 6. Evidence of achievement of FSS Project planned inputs and activities...... 10 Table 7. Extent of achievement of FSS Project outputs...... 11 Table 8. FSS Project target outcome achievement...... 14 Table 9. Snapshot of FSS Project partnerships...... 18 Table 10. Cost-effectiveness estimates...... 26 Table 11. Additional cost-effectiveness estimates...... 27 Table 12. Distribution of grant funding to FSS Groups...... 29 Table 13. Annual survey 2013-2017: farmer perceptions of progress in sustainable soil management knowledge and practice...... 33 Table 14. Strengths and limitations of the FSS Project...... 34 Table 15. Findings against evaluation questions...... 40 Table 16. Evaluation framework (adapted from FSS Project MERI Plan, NLP Monitoring and Reporting Plan and information provided by North Central CMA)...... 46 Table 17. Summary of data collection components...... 49

Figures Figure 1. FSS Project logic model (adapted based on the logic model presented in the FSS Project Midterm Review)...... 7 Figure 2. Final evaluation survey: extent of improvement in confidence (n=57) and knowledge and skills (n=56) resulting from the FSS Project...... 16

Prepared for North Central Catchment Management Authority iii Farming for Sustainable Soils Project Final Report

Figure 3. Annual survey 2013-2017: extent of contribution to understanding of soil health (n=122) and extent of motivation to undertake additional actions (n=136)...... 17 Figure 4. Overall participant satisfaction with the FSS Project from final evaluation survey (n=54) and annual survey (n=137)...... 21 Figure 5. Final evaluation survey: extent of improvements in soil quality and productivity resulting from FSS Project (n=49)...... 22 Figure 6. RMCG: discernible differences between treatments used in field trials (yes, no, maybe) (n=45) and success of field trails (high, medium, low) (n=44)...... 23 Figure 7. Distribution of soil trials conducted by FSS Groups (RMCG 2018)...... 23 Figure 8. Intent for management practice change after (FSS event feedback surveys, n=75) and reported intention to make additional changes or adopt new soil management practices in future as a result of the FSS Project (final evaluation survey, n=57)...... 31

Acronyms CfoC Caring for our Country

FPC First Person Consulting

FSS Project or the Project Farming for Sustainable Soils Project

MERI monitoring, evaluation, reporting and improvement

North Central CMA or the CMA North Central Catchment Management Authority

NLP National Landcare Program

Front cover photo credits: Patrick Gilmour, Adam Twigg, Rebecca Denniss, Adam Twigg (from left to right).

Prepared for North Central Catchment Management Authority iv Farming for Sustainable Soils Project Final Report

1 Executive summary The Farming for Sustainable Soils (FSS) Project is a community-based initiative that supports sustainable soil management. The North Central Catchment Management Authority (CMA) has delivered the FSS Project throughout the region since 2009, and Phase 2 of the Project ends in June 2018 after five years of Australian Government funding. Prepared in collaboration between the CMA and First Person Consulting, this is a summary of the Final Report on Phase 2 of the FSS Project.

Conclusions The FSS Project has been a successful investment in improving sustainable land management practices in north central Victoria. The Project demonstrates the effectiveness of community-based approaches to programs in the natural resource management sector. The FSS model is an effective model for soil health initiatives going forward, with the three key components of testing and assessments, knowledge sharing and capacity building, and field trials and demonstrations.

Through the FSS Project, local communities have been empowered to improve the health of their soil assets, with a range of outcomes achieved in relation to knowledge, skills and confidence, improved sustainable soil management practices, agricultural productivity, soil health and community resilience. Many of the longer-term impacts of the Project are yet to be realised, and the FSS Project will have ongoing value in years to come.

Recommendations 1. There is a clear need and opportunity for additional work in supporting farmers to improve their soil health going forward. Soil health should continue to be a priority for land managers in the north central catchment, supported by government, non-government and private sector stakeholders. 2. The FSS Project is a successful model for sustainable soil management practice change. The North Central CMA should continue to use this community-based approach in delivering sustainable agriculture programs and services, as well as promoting use of the model more widely. 3. The outcomes of the community-based approach should continue to be monitored over time, to evaluate the longer-term strengths of this model compared to other approaches to sustainable agriculture programs. In particular, this should focus on measuring any landscape scale changes occurring as a result of the FSS Project. 4. Collection and analysis of biophysical data should be prioritised in future soil health projects to more effectively monitor and evaluate outcomes, appropriateness and cost- effectiveness.

Future activities for soil health assets in the catchment The CMA is currently developing a Soil Health Action Plan and finalising FSS Project delivery. Building on the achievements of the FSS Project, the CMA should: continue to maintain the strong relationships developed with local communities through the Project; offer ongoing support to farmers in the catchment to ensure that potential longer-term benefits of the Project are achieved; continue applying for external funding to provide sustainable land management support; apply the

Prepared for North Central Catchment Management Authority 1 Farming for Sustainable Soils Project Final Report lessons from the FSS Project to the design and delivery of other programs and services; and improve the consistency of monitoring and evaluation to better understand the extent of achievements, outcomes, lessons learned, and wider impacts of investments in farming communities.

Prepared for North Central Catchment Management Authority 2 Farming for Sustainable Soils Project Final Report

2 Introduction 2.1 Overview The North Central Catchment Management Authority (the CMA or North Central CMA) has delivered the Farming for Sustainable Soils Project (FSS Project or the Project) since 2009, supported by the Australian Government. Phase 2 of the FSS Project ends in June 2018 after five years of funding through the National Landcare Program (NLP) and the former Caring for our Country (CfoC) Program.

The FSS Project is a community-based initiative that supports sustainable soil management. The North Central CMA works with community leaders and groups of local landholders across the catchment to encourage the adoption of sustainable farming practices that protect and enhance soil health—with additional benefits of improved agricultural productivity and strengthened social connectedness of farming communities. Through supporting soil testing and assessments, knowledge sharing and capacity building, and field trials and demonstrations, the FSS Project’s vision is that:

“North Central Victoria has secured the health and productivity of soils, improved ecosystem services that flow from healthy soils, and built resilience to climate change through increased soil health.”

Prepared in collaboration between the CMA and First Person Consulting (FPC), this is the final report on Phase 2 of the FSS Project—involving an evaluation of FSS Project delivery, demonstration of outcomes achieved and ongoing value of this investment in farming communities. 2.2 Scope and approach The scope of this evaluation and reporting consultancy was for FPC to work closely with the North Central CMA and key Project stakeholders to:

• respond to evaluation questions, and internal and external reporting requirements with a strong evidence base • demonstrate the value of the FSS Project—for farmers, the natural resource base and the community • capture key findings, lessons learned and recommendations for improving delivery of similar programs and services in future.

This approach involved the following key components:

• developing a series of 13 case studies to demonstrate the outcomes and achievements of the Project • presenting a conference session and providing support at the FSS Conference in March 2018 • collecting and synthesising a range of data to report on the FSS Project within CMA guidelines and consistent with NLP MERIT reporting processes.

This involved analysing existing monitoring and evaluation data, previous reports and Project management documentation. FPC liaised with North Central CMA staff, Community Facilitators, key stakeholders, participating landholders and community members, other consultants and experts involved in Project delivery, and conducted additional interviews and surveys to contribute to the

Prepared for North Central Catchment Management Authority 3 Farming for Sustainable Soils Project Final Report final report. A review workshop was also held in June 2018, at which the findings, conclusions and recommendations in this final report were discussed with core Project delivery staff and stakeholders.

Table 1 provides a summary of each data collection method and participants. Table 2 provides a summary of respondents to the survey conducted through this evaluation, as well as the annual survey data that was provided by the CMA.

Table 1. Summary of data collection components. Data collection method Participants Discussions with FSS Project stakeholders (including expert 4 presenters and consultants) Semi-structured phone interviews with FSS Community Facilitators 2

Face-to-face interviews with FSS Community Facilitators 4 Face-to-face interviews with FSS Project participants 5 Semi-structured phone interviews with North Central CMA staff 2 Semi-structured phone interviews with FSS Project participants 8 Final evaluation survey conducted over the phone with FSS Project 57 participants FSS Conference data collection 50 FSS Project review workshop 16

Table 2. Survey data summary. FSS Group Annual survey Final evaluation survey respondents (2013-2017) respondents (2018) Charlton 12 5 Lockington 50 5 Paradise 8 6 Pyramid Hill 31 12 Smeaton 0 6 Timor West 11 7 Glenloth East 43 16 Wycheproof Total 155 57

A detailed description of the methodology is included in Appendix 1, including data collection and stakeholder engagement methods, as well as limitations and assumptions.

Prepared for North Central Catchment Management Authority 4 Farming for Sustainable Soils Project Final Report

3 Farming for Sustainable Soils Project 3.1 Overview This section outlines the context of the FSS Project and its core components (Section 3.2). It presents the key inputs, activities and outputs delivered through Phase 2 of the Project, and an assessment of the extent to which these have been achieved throughout the funding period. 3.2 Project background The FSS Project’s focus on soil health recognises the vital importance of soils for agricultural production, environmental health, and the communities that depend upon them. The FSS Project is one of two projects with a focus on soil health delivered by the North Central CMA. The FSS Project is delivered in collaboration with local farming communities and industry stakeholders across the catchment, and complements the services provided by the Regional Landcare Facilitator. Improving the health and productivity of the soil—as a key natural asset—is a core priority for the region and the CMA is currently in the process of developing a Regional Soil Health Action Plan to deliver soil health goals over the next two decades.

The current Phase 2 of the FSS Project builds on the success of Phase 1—engaging with more farming communities across the catchment to continue building knowledge, skills and capacity in relation to soil health and increasing the adoption of locally appropriate sustainable farming practices. The model involves developing local FSS Groups, which are led by Community Facilitators and supported by the CMA.

The establishment of an FSS Group involves the following five-step process:

• Public meeting to establish the concept—The relevant community within a priority area is invited to a meeting to discuss the opportunity to participate as an FSS Group. The Project is described in detail and farmers are invited to consider their participation Where there is 100% agreement to participate, the planning process proceeds. • Local part-time FSS Facilitator—A local FSS Facilitator is appointed to support the Group, to assist with the development of a Local Area Soil Protection Plan, and to work in partnership with the North Central CMA to deliver the Project. • Public meeting to begin building the Local Area Soil Protection Plan—A public meeting is held to gather information from farmers to contribute to developing a Local Area Soil Protection Plan. Farmers speak about their enterprise and the influences that have shaped the way they farm over the past decade, identifying their most challenging soils and soil management issues. • Building the Plan—The Local Area Soil Protection Plan is developed as a blueprint for community involvement in the FSS Project over the following three years, and a further public meeting is convened to gain community feedback. • Annual Activity Schedule—An Annual Activity Schedule is developed by each FSS Group. This outlines each of the activities the Group will engage in over the coming year (inclusive of growing season) consistent with a budget allocated by the North Central CMA. Funds are then allocated to the group subject to governance considerations by the North Central Standing Grants Committee.

Prepared for North Central Catchment Management Authority 5 Farming for Sustainable Soils Project Final Report

The delivery of a Local Area Soil Protection Plan involves the following core activities:

• Soil assessments—Farmers sample the soils of their local area and submit them for chemical analyses. Soil pits are excavated and an assessment of local soil health is made by a soil scientist. This allows farmers to understand both soil chemistry and soil structure, identifying the constraints and opportunities that varying soil types present for improving agricultural production and soil health. • Capacity building and knowledge sharing—This involves bringing FSS participants together with a range of soil experts to learn new skills and approaches to soil health that they can use on their own properties, allowing them to share and discuss soil management practices and experiences with others in the farming community. • Field trials and demonstrations—This gives landholders the opportunity to trial new soil management techniques on their properties with the support of their FSS Group, allowing outcomes and lessons learned to be monitored and shared with others in the community.

The logic model for the FSS Project clearly outlines the inputs, activities and outputs of the Project and how these are linked to intermediate outcomes, end of Project outcomes and longer-term outcomes that will continue beyond the funding period (see Figure 1 on the following page).

A more detailed description of the FSS Project philosophy and delivery model is provided in the Midterm Review undertaken by RMCG.1 Examples of Local Area Soil Protection Plans and Annual Activity Schedules are also available on the North Central CMA website.2

1 RMCG (2017), ‘Midterm Review—Farming for Sustainable Soils Project’, http://www.nccma.vic.gov.au/resources/publications/fss-mid-term-review-march-2017 2 North Central CMA (2018), ‘Farming for Sustainable Soils Resources’, http://www.nccma.vic.gov.au/projects/agriculture#node-116

Prepared for North Central Catchment Management Authority 6 Farming for Sustainable Soils Project Final Report

Figure 1. FSS Project logic model (adapted based on the logic model presented in the FSS Project Midterm Review).

Prepared for North Central Catchment Management Authority 7 Farming for Sustainable Soils Project Final Report

3.3 Project inputs, activities and outputs Phase 2 of the FSS Project is funded by $2.77 million Australian Government grant funding provided to the North Central CMA between 2013/2014 and 2017/2018. This includes both NLP funding (2014/2015 to 2017/2018) and funding through the former CfoC Program (2013/2014 and in part 2014/2015). Table 3 outlines total FSS Project funding expenditure against the planned budget for each financial year. All grant funds were fully expended by 30 June 2018 as per the funding agreement between the CMA and the Australian Government. The FSS Project has leveraged a total of $401,790 in additional value through community contributions, as reported by the CMA. Various industry and community organisations have partnered with the CMA through the FSS Project and contributed resources in supporting the delivery of Local Area Soil Plans. Table 4 shows a detailed expenditure breakdown for the FSS Project between 2013/2014 and 2017/2018.

Table 3. FSS Project budget and expenditure overview: Australian Government funding and community contributions.

Financial year Program Total Australian Expenditure Carry forward Estimated funding Government community source budget3 contribution 2013/2014 CfoC $581,445 $552,365 $29,080 $100,100 CfoC $348,600 $189,859 $158,741 2014/2015 $83,150 NLP $232,606 $391,347 $4,572 2015/2016 NLP $531,205 $517,665 $13,510 $107,200 2016/2017 NLP $525,000 $477,037 $47,963 $67,980 2017/2018 NLP $559,000 $649,583* $0* $43,360 Total $2,777,856 $2,777,856* $401,790 * Projected expenditure based on budget.

Table 4. FSS Project expenditure breakdown. Year Program Total CMA General Community Facilitator Levies funding Budget wages expenses grants payments etc.4 source 2013/2014 CfoC $581,445 $186,117 $61,648 $197,000 $38,000 $69,600 CfoC $348,600 $109,367 $14,726 $0 $31,666 $34,100 2014/2015 NLP $232,606 $101,764 $48,950 $190,000 $15,833 $34,800 2015/2016 NLP $531,205 $204,351 $32,094 $170,000 $47,500 $63,720 2016/2017 NLP $525,000 $235,820 $13,092 $135,000 $38,000 $55,125 2017/2018 NLP $559,000 $345,933* $138,412* $90,000 $31,500 $43,738* Total $2,777,856 $1,183,352 $308,922 $782,000 $202,499 $301,083 Percentage 100% 43% 11% 28% 7% 11% * Projected expenditure based on budget.

3 Includes combined NLP and CfoC funding. 4 Includes CMA levies, contributions to communications, Indigenous support and GIS.

Prepared for North Central Catchment Management Authority 8 Farming for Sustainable Soils Project Final Report

The focus of Phase 2 was to complete activities planned by two existing FSS Groups (Charlton and Lockington) and to establish six new Groups (Wycheproof, Paradise, Smeaton, Pyramid Hill, Timor West and Glenloth East). Groups participated in adaptive learning programs on the adoption of sustainable land management practices relevant to their local area—with a particular focus on soil structure, hydrologic performance, ground cover and soil organic carbon. Table 5 shows the FFS Groups supported in Phase 2.

Table 5. Years of FSS Group operation in Phase 2.

FSS Group 2013/2014 2014/2015 2015/2016 2016/2017 2017/2018 Lockington (from Phase 1) Charlton (from Phase 1) Wycheproof Paradise Smeaton Pyramid Hill Timor West Glenloth East

Across Phase 1 and Phase 2, the FSS Project supported a total of fourteen rural communities in the north central region of Victoria, extending from the northern plains southwards into the mid slopes region and as far as the headwaters of the Great Dividing Range.

As shown in FSS Project logic model, the activities and outputs listed below have been designed to lead to the immediate outcomes of increased knowledge and skills, improved networks, and the trialling of soil improvement practices. It was expected that by the end of the Project there would be broader networking and sharing information and experiences, adoption of soil improvement practices, and sustained motivation for communities to take responsibility for the health of their local soils.

All FSS Project inputs, activities and outputs have been delivered and completed, and a range of key outputs and activity targets have been exceeded. Table 6 on the following page presents planned FSS Project inputs and activities, with evidence of achievement. Table 7 then presents planned FSS Project outputs, with evidence of achievement against key targets.

Prepared for North Central Catchment Management Authority 9 Farming for Sustainable Soils Project Final Report

Table 6. Evidence of achievement of FSS Project planned inputs and activities.5

Project components Planned Final FSS Project achievements Inputs Australian Government $2,777,856 between 2013/2014 and funding (NLP and CfoC) 2017/2018. Community contributions Estimated $401,790 from a range of industry and community partners. Foundational activities FSS Reference Group for FSS Reference Group established, comprising governance and two members of the North Central CMA Board implementation of MERI Plan and two members of the North Central formed Catchment Consultative Committee. Quarterly FSS Reference Best practice governance has been Group meetings implemented. Meetings held on a needs basis, usually two or three times per year. Development of MERI Plan Completed. Development of Project Plan Completed. Six new Groups identified and Six new FSS Groups formed, in addition to the formed two existing Groups. Each FSS Group had around 30 active participants, with an estimated total of 240 participants. Six new local Community New Community Facilitators employed across Facilitators employed six regions (Wycheproof, Paradise, Smeaton, Pyramid Hill, Timor West and Glenloth East). Intervention activities FSS Group planning initiation 12 workshops held (two per Group for workshops initiation and planning). A third workshop was also held with each Group after Local Area Soils Plans had been drafted. Prepare Local Area Soil Plans Six Local Area Soils Plans submitted (one per Group). Prepare Activity Schedules All Annual Activity Schedules submitted by (grants process) and approval each Group. by North Central CMA Standing Grants Committee FSS Group baseline soil 347 samples collected, laboratory tested and assessments interpreted6 FSS Group knowledge 57 knowledge sharing, and capacity building building field based events events held. and workshops FSS Group trials and 47 group trials and demonstrations conducted. demonstrations FSS Monitoring, evaluation MERI Plan implemented. and reporting Midterm Review completed by RMCG in 2017 and Final Evaluation completed by FPC (this report).

5 Inputs and activities data provided by the North Central CMA and sourced from FSS Project dashboard through MERIT. 6 As at March 2017, as reported in the FSS Project Midterm Review.

Prepared for North Central Catchment Management Authority 10 Farming for Sustainable Soils Project Final Report

Table 7. Extent of achievement of FSS Project outputs.7 Output type Output description Output target description Target Achievement Participants By 30 June 2018, 225 land Number of volunteers 225 2408 managers will engage in activities participating in project or be provided information to activities develop their skills in sustainable Number of Indigenous 0 0 farming practices. participants at project events Total number of new 0 1,212 participants (attending project events or activities for the first time) Events By 30 June 2018, 225 members of Total number of 41 57 farming communities will have community participation participated in FSS Group activities and engagement events that will build their skills, run knowledge, experience and capacity in terms of achieving soil health and sustainable land management in their local area. Training By 30 June 2018, 225 land Total number of people 225 6439 managers will engage in activities completing formal or be provide information to training courses develop their skills in sustainable farming practices. Management By June 2018 the four FSS current Total number of farming 12 153 practice FSS groups will have established entities adopting change 12 trials/demonstrations as an sustainable practice intermediate step in the adoption change of sustainable management Area of land (hectares) 120 33610 practices. changed to sustainable practices Area of land (hectares) on 500 1,92411 which improved management practices have been implemented

7 Targets and achievement data sourced from FSS Project dashboard through MERIT. Achievement data is accurate at time of writing and does not include any additional data from Stage 7 of Project delivery. 8 Data provided by the North Central CMA reports a total of 240 unique participating landholders in local FSS Groups between 2013 and 2018. A figure of 2,990 is reported in MERIT, however this includes duplicate participants across multiple years of the Project. 9 This refers to the total number of land managers who attended FSS capacity building events who may or may not have been active FSS Group participants. FSS supported events were regularly attended by farmers, landholders and community members who were not formal members of FSS Groups. 10 This refers to the total hectares of the 47 trial sites supported by the FSS Project. 11 This refers to the total hectares of paddocks or farms on which trials were conducted.

Prepared for North Central Catchment Management Authority 11 Farming for Sustainable Soils Project Final Report

4 Findings and evidence 4.1 Overview This section presents the findings of this evaluation and the evidence base to support these findings under five criteria:

• Effectiveness (Section 4.2) • Appropriateness (Section 4.3) • Efficiency (Section 4.4) • Impact (Section 4.5) • Future (Section 4.6) 4.1.1 Evidence base

A description of the scope and approach for this evaluation is presented in Section 2.2 above, and the full methodology is outlined in Appendix 1. In addition to the key components of the evidence base for this evaluation, the following should also be considered:

• Case studies—A series of 13 case studies were developed to demonstrate the value and outcomes of various components of the FSS Project and to provide farming communities with accessible information about sustainable soil management practices. These case studies, as well as three previously commissioned case studies, should be considered additional evidence in support of the evaluation findings and are available on the North Central CMA website.12 • Trials of potential sustainable practices—RMCG worked with the North Central CMA to collate and analyse results on the outcomes of 47 field trials funded and conducted through the FSS Project between 2013 and 2018. This report (referred to extensively in Section 4.3) covers the activities of seven FSS Groups in Phase 1, assessing the efficacy of soil management trials, identifying key learnings and areas for further exploration. It concludes that trials and demonstrations “were a practical vehicle for farmers testing ideas and trying new practices […] designed to allow farmers to gain experience with new farming systems, crops and technologies, to begin to validate proposed options that could improve soil condition and build community confidence in their application”.13 • External research—An independent study commissioned by the CMA in 2016 provides further insight on the effectiveness of the FSS model, largely drawing on data from farming communities in Phase 1 of the Project. A survey of 800 landholders shows that participants in the FSS Project rate their knowledge of soil health higher than non-participants, and that participants are more likely to implement management practices linked to improved soil health outcomes.14 While this is not directly relevant to Phase 2 of the Project, it should be considered complementary evidence of the model’s efficacy.

12 FSS Project case studies are available at http://www.nccma.vic.gov.au/projects/agriculture#node-116 13 RMCG (2018) 'Farming for Sustainable Soils— On-farm Trials Report’ (not publicly available), p. 28. 14 Curtis, A & Mendham, E (2016) ‘Participation in Soil Health Groups: Does it make a difference? A final report to the North Central Catchment Management Authority’,

Prepared for North Central Catchment Management Authority 12 Farming for Sustainable Soils Project Final Report

A range of additional reports and Project resources should be read in association with this Final Report. These are presented through MERIT reporting and are also available on the North Central CMA website:15

• Midterm Review—Farming for Sustainable Soils (RMCG, March 2017) • Local Area Soil Protection Plan for Timor West • Local Area Soil Protection Plan for Glenloth East / Wycheproof • Annual Activity Schedule for Timor West • Annual Activity Schedule for Glenloth East / Wycheproof • Review of the Timor West FSS Group • Review of the Glenloth East / Wycheproof FSS Group

4.2 Effectiveness 4.2.1 Outputs and outcomes

As demonstrated in Section 3.3 above, all planned outputs and activities have been delivered. The FSS Project has achieved its intermediate and end of Project outcomes, as well as contributing to achievement of longer-term outcomes:

• Intermediate outcomes (achieved) o increased knowledge of soils and skills in sustainable soils management amongst participants o improved networks of interested farmers active in soil management in the region o soil improvement practices are being trialled by participants. • End of Project outcomes (achieved) o farmers are networking and sharing experiences and information, both within and outside their groups o soil improvement practices are being adopted by participants and the broader farming community o more motivated communities taking responsibility for the health of local soils. • Longer-term outcomes (contributed to achievement) o increased confidence in recommended practices and ongoing trialling and adoption by all farmers in the region o a productive agricultural food sector o stronger social resilience o maintenance, protection and / or improvement to ecosystem services (soil, water and vegetation components).

Institute for Land, Water and Society, Charles Sturt University https://www.csu.edu.au/__data/assets/pdf_file/0007/2730193/Report_96_Soils_Report.pdf 15 North Central CMA (2018), ‘Farming for Sustainable Soils Resources’, http://www.nccma.vic.gov.au/projects/agriculture#node-116

Prepared for North Central Catchment Management Authority 13 Farming for Sustainable Soils Project Final Report

All four key target outcomes for the FSS Project have either been achieved or exceeded. Table 8 provides findings the extent of achievement of each target outcome, and more detailed evidence is presented below in relation to each target outcome.

Table 8. FSS Project target outcome achievement.16

Target outcome Achievement and evidence Knowledge, skills By 2018, to have at least 225 Exceeded. and practice change land managers involved in a Up to June 2018, a total of 240 farmers, in sustainable land schedule of FSS activities that landholders and community members have management will increase their skills and been involved as active participants in local FSS knowledge of sustainable land Groups, and a total of 643 farmers, landholders management. and community members have attended FSS events or workshops. Partnerships By 30 June 2018 FSS groups will Exceeded. form six new partnerships for Through the Project, it is estimated that FSS additional resources from Groups across the catchment have formed a industry and other total of 35 new partnerships with industry and organisations and help in other organisations to support the delivery of supporting the delivery of Local Local Area Soil Protection Plans, far exceeding Area Soil Protection Plans. the target of six.17 Community leaders By 30 June 2018 FSS will Achieved. increase the knowledge of Seven FSS Community Facilitators have been seven community FSS members recruited, trained and supported by the North sufficiently for them to act as Central CMA to engage their local farming FSS advocates in supporting communities and deliver the FSS Project ‘from other farming communities. the ground up’ (including existing Community Facilitators from Charlton and Lockington). Wider reach By June 2018, 675 farmers Achieved. outside the FSS groups will have It is estimated that 720 farmers, landholders, an increased awareness of the stakeholders and community members outside sustainable practices adopted the FSS Groups structure have been made aware by the groups. of the FSS Project and messages about soil health and sustainable land management.18

Knowledge, skills and practice change in sustainable land management There is clear evidence demonstrating that farming communities are increasing their knowledge, skills and awareness of sustainable soil and land management through their participation in the FSS Project.

It is estimated that between 2013 and 2018, a total of 347 soil health assessments, 57 capacity building workshops or activities, and 47 soil management field trails have been delivered by farming communities across the catchment through the FSS Project.

16 Target outcomes as listed in the FSS Project’s MERIT Portal through the NLP. 17 This is based on an estimate that each of the seven FSS Groups partnered with five organisations. 18 This is based on an estimate that for each of the 240 active participants in FSS Groups an additional three people were made aware of the FSS Project and messages about soil health and sustainable land management.

Prepared for North Central Catchment Management Authority 14 Farming for Sustainable Soils Project Final Report

The collection of chemical and physical information was essential in the assessment of soil health and soil condition by each of the FSS Groups. This information afforded a ‘benchmark’ that supported the development of a community knowledge base allowing for the formulation of informed approaches to overcoming soil constraints. Throughout the Project, each of the FSS Groups collected soil samples for laboratory analyses, and further established their understanding of local soils through the excavation of soil pits—supported and overseen by soil scientists from the region.

After assembling soil data FSS Groups conducted workshops in which experts soil scientists and agronomists provided an interpretation of their meaning. Laboratory assessments included parameters such as organic carbon, salinity, acidity, cation exchange, and nutrient status. Physical assessments included observations of soil structure, root development, subsoil constraints and responses to changes in paddock management (for example, subsoil manure injection).

Independent of the NLP, the North Central CMA constructed a generic corporate library for its datasets, and specifically tailored it to accommodate the FSS soils data. While the database has not yet been populated, this data is held within the North Central CMA document management system, and by each of the FSS Groups. FSS data will be entered into the corporate library, as part of a project identified through the Soil Health Action Plan for North Central Victoria. This project aims to ensure soils information is well managed, shared and easily accessible for the appropriate need.

The outcomes and experiences of FSS Groups undertaking soil assessments through the Project are explored in more detail in the following case studies developed by FPC:

• Finding out what’s going on underground: the value of soil pits—"Soil testing and assessments undertaken through the Farming for Sustainable Soils Project”19 • Discovering soil health opportunities and limitations: calling in the experts—"Independent expertise tailored to the needs of farming communities in the Farming for Sustainable Soils Project”20 • The value of soil care: changing approaches to soil management—"Insights from Farming for Sustainable Soils Project participants in the Wycheproof, Glenloth East and Charlton areas of north central Victoria”21

Through a survey undertaken during the concluding stages of the Project FPC found the following:

• 89% (51) of participating farmers stated that their confidence has increased in their capacity or ability to adopt new soil management practices since being involved in the FSS Project • 96% (54) stated that their knowledge and skills in relation to soil health have improved since participating in the FSS Project • around a third of respondents reported considerable improvements in both of these areas (see Figure 2 on the following page).

19 http://www.nccma.vic.gov.au/resources/publications/finding-out-whats-going-underground 20 http://www.nccma.vic.gov.au/resources/publications/discovering-soil-health-opportunities-and-limitations 21 http://www.nccma.vic.gov.au/resources/publications/value-soil-care

Prepared for North Central Catchment Management Authority 15 Farming for Sustainable Soils Project Final Report

45% 42% 39% 40% 35% 32% 30% 28% 25% 25% 19% 20%

15% 11% 10%

Proportion respondents of Proportion 4% 5% 0% Not at all improved Somewhat Moderately Considerably improved improved improved

Confidence in capacity or ability to adopt new soil management practices Knowledge and skills in relation to soil health

Figure 2. Final evaluation survey: extent of improvement in confidence (n=57) and knowledge and skills (n=56) resulting from the FSS Project. There is also clear evidence that farmers are adopting new sustainable land management practices as a consequence of participating in the FSS Project.

Just over three-quarters of surveyed farmers (77%, 44) reported that they have made changes on their property resulting from their involvement in the FSS Project. Among these respondents, 46% (19) said they would not have been able to make these practice changes without the Project. Forty- four per cent (18) stated that they were going to make these changes anyway but that the Project encouraged them to do it sooner. One limitation of the surveys was that they were restricted to interviews with participating farmers. Regional on-ground survey was not conducted to assess actual practice change.

Annual survey responses between 2013 and 2017 show that for the vast majority of participants, the FSS Project has increased their understanding of soil health beyond their own farm (98%, 119), and motivated them to undertake more trials, implement new approaches and learn more about soil heath (98%, 133) (see Figure 3 on the following page).

An example of the knowledge and skills gained by landholders, the range of soil management approaches trialled, and the positive outcomes experienced is evident in the following quote from a participating farmer:

“Regular soil testing, just generally more knowledge on chemicals and weather […] The subsoil manuring trial is still working up there. That was five or six years ago, and the chicken manure is still working. We put in ten different products […] gypsum, cow manure and then some commercial fertilisers and some liquid ones as well. We dug soil pits to see what the roots had done, and then after about three years we did another one to see what had happened. [It was] unbelievable how much better the soil was [after the FSS trial]. Brassicas were growing up to my knees!” (participating farmer)

Prepared for North Central Catchment Management Authority 16 Farming for Sustainable Soils Project Final Report

50% 44% 45% 43% 40% 34% 35% 31% 30% 24% 25% 20% 20% 15% 10% Proportion respondents of Proportion 5% 2% 2% 0% Not at all A little Somewhat Definitely

FSS Project contributed to participants' wider understanding of soil health

FSS Project motivated participants to undetake more trials, implement new approaches or learn more about soils Figure 3. Annual survey 2013-2017: extent of contribution to understanding of soil health (n=122) and extent of motivation to undertake additional actions (n=136). The majority of respondents noted that they had used their soil assessments in some way. Annual surveys throughout the FSS Project show that after undertaking soil assessments on their properties, 50% (73) of participants went to on seek additional expert assistance in analysing the results of their soil tests. Further, 50% (67) of participants reported that they used their soil test results and changed their soil management practices as a result, 37% (49) used their results but did not need to change their soil management practices. Only 13% (17) did not use their results.

In addition, directly following on from their involvement in the FSS Project, 67% (38) of the participants surveyed through this evaluation have further soil management practice changes either planned or in progress. These changes include both continuing the practices that farmers trialled through FSS Project, as well as implementing additional soil management practices they learned about through the Project—for example, ongoing soil monitoring, changing additive and fertiliser practices, changing cropping rotations, introducing direct drilling and deep ripping, and various other methods of increasing organic matter and improving soil structure.

“We're running a controlled traffic system now. [We] did work with soil scientists on deep ripping and changing the way we're sowing […] also now using soil moisture probes.” (participating farmer)

“[We] fenced off some trees on the property and we have gone all direct drilling now. […] We also don't burn stubble anymore. [We’re] also trying to keep the chemicals out of the soil as much as possible.” (participating farmer)

Partnerships It is estimated that 35 partnerships with industry and other organisations were developed through the FSS Project, which contributed to the delivery of Local Area Soil Plans. Key partnerships have provided additional value through in-kind contributions as well as knowledge sharing, capacity building and practice change among farming communities in the catchment.

Prepared for North Central Catchment Management Authority 17 Farming for Sustainable Soils Project Final Report

All of the FSS Groups partnered with a range of organisations in the delivery of the FSS Project, often resulting in the establishment of ongoing partnerships. Table 9 presents a snapshot of key partnerships developed throughout the FSS Project between 2013 and 2018.

Table 9. Snapshot of FSS Project partnerships.

Organisation Entity type Involvement and support provided FSS Group through partnership partners Birchip Cropping Group Primary Support for cropping trials. Pyramid Hill industry group Southeast Soil and Water Commercial Assistance with soil sampling and soil field Wycheproof (Christian Bannan, soil entity days. Glenloth East scientist) Pyramid Hill Timor West Australian Grain and Commercial Support for cropping trials. Timor West Forage Seeds entity Dellavedova Fertilisers Commercial Support for cropping trials. Timor West entity Agriculture Victoria State Supports the provision of specific All (Department of Economic Government technical advice on soils and agronomy to Development, Jobs, the FSS Groups, and runs a Transport and Resources) complementary Soil Health Program Elders (Rural Services) Commercial Support for remote monitoring and Paradise entity reporting of soil moisture. Timor West Support for trials. Glenloth East Perennial Pasture Commercial Support for perennial pasture trials. Paradise Systems entity Landmark St Arnaud Commercial Support for remote monitoring and Paradise entity reporting of soil moisture. Land-Mate (Ararat Prison) State Support in establishing fences and trees. Paradise Government Donald High School School Support for tree establishment. Paradise Millers Agricultural Commercial Assistance with formation of the Pyramid Pyramid Hill Supplies Pyramid Hill entity Hill FSS and the establishment of remote soil moisture monitoring RMCG Consulting Commercial Assistance with Local Area Soil Protection All entity Plans and growing season Annual Activity Schedules. Various consulting services and expert advice on soil management. Landmark Wycheproof Commercial Support for remote monitoring and Wycheproof entity reporting of soil moisture. Fait Fertilisers Commercial Support with trial sites. Wycheproof entity Agrivision (Tom Lord, Commercial Agronomic support and advice. Wycheproof agronomy consultant) entity

Prepared for North Central Catchment Management Authority 18 Farming for Sustainable Soils Project Final Report

Community leaders Community Facilitators have been a crucial part of the community-based schedule of activities delivered through the FSS Project between 2013 and 2018. In the areas of Charlton, Lockington, Paradise, Smeaton, Timor West, Glenloth East, Wycheproof and Pyramid Hill, seven Community Facilitators have acted as expert peer mentors since 2013. Throughout the Project they have been given the skills, knowledge and capacity to influence landholders, farmers and community members across the catchment to improve soil health and sustainable land management practices.

Community Facilitators were the interface between the North Central CMA and their local farming communities in delivering the Project. Community Facilitators and the CMA Project Team worked together in establishing FSS Groups and engaging landholders in their local area, preparing Local Area Soil Protection Plans and Annual Activity Schedules, co-ordinating soil assessments and testing, capacity building workshops, expert presenters and demonstrations, and designing field trials, as well as Project administration, monitoring and reporting, and advocating for the FSS Project more widely.

Community Facilitators were essential in the community-based approach of the FSS Project and have been described as critical to the success (or otherwise) of their Groups. In most cases they are landowners or famers in their local communities with firsthand knowledge and experience of the soil health challenges faced by their FSS Group members. Having local Community Facilitators was seen to build trust within the FSS process and encouraged a journey of peer-to-peer learning among Group members.

There were also additional positive outcomes for community connectedness fostered by the FSS Groups, led by the Community Facilitators. This is discussed in more detail throughout following sections of this report. Case studies developed as part of this evaluation also provide insight on the value of Community Facilitators in the FSS Project.22

Wider reach The FSS Project has exposed the wider community to messages about soil health and stories of improved sustainable land management practices. This has been enabled through FSS Group participants, Community Facilitators, North Central CMA staff, formal FSS Project partners and other key stakeholders.

The FSS Project Team estimates that around 720 additional farmers, landholders and community members have been exposed to messages about soil health and sustainable land management practices. This is estimate is based on a range of factors:

• Through annual survey data between 2013 and 2017, 55% (72) of participating farmers reported that they have shared information, knowledge or skills learned from the FSS Project with other farmers not involved in the Project. • North Central CMA staff and FSS Community Facilitators have actively promoted the FSS Project and sustainable soil management through Landcare groups and networks, including

22 ‘Facilitating change in soil health: The experience of being a Community Facilitator in the Farming for Sustainable Soils Project’, North Central CMA http://www.nccma.vic.gov.au/sites/default/files/publications/case_study_02_facilitators.pdf

Prepared for North Central Catchment Management Authority 19 Farming for Sustainable Soils Project Final Report

the Loddon Plains Landcare Network, Sutton Grange Landcare, Mid-Loddon Landcare Network, Northern United Forestry Group and the North Central Catchment Management Regional Roundtable Community Consultation. In the most recent six-monthly NLP reporting period alone, the FSS Project reached a total of 115 non-FSS participants through these groups and their events. • Records from engagement events and capacity building activities supported by the FSS Project show consistent attendance by landholders and community members who are not registered members of an FSS Group. For example, more than half of the attendees at the FSS Conference in March 2018 were non-FSS Project participants. • A range of resources (including case studies, soil health information guides, maps, videos, etc.) are publicly available on the North Central CMA website and have been routinely accessed by the wider community. • Evidence from Community Facilitators, participating farmers and North Central staff interviewed through this evaluation suggests that the FSS Project has had a wider impact within farming communities, with anecdotal reports of increasing interest in sustainable soil management throughout the catchment.

“Everything we did was open to the wider public and through some of the agronomy firms and the North Central CMA and the Landcare Network, we have been able to spread the message about the things we’ve been doing with the Project to a wider audience.” (FSS Community Facilitator)

“The Groups keep picking up new members, even at this late stage of the funding. Particularly young farmers […] one young guy comes and then the next time he brings another one—they might not officially join but they come along to things and get themselves on email lists.” (CMA staff member)

4.3 Appropriateness Overall, activities were appropriate for achieving the intended FSS Project outcomes. As discussed in detail in Section 4.2 above on effectiveness, for the majority of participants, improvements in soil health skills, knowledge and experience gained through participating in the FSS Project led to an increased adoption of sustainable land management practices.

Overall, the community-based model of the FSS Project is seen to meet the needs of farmers. Feedback from participating farmers, Community Facilitators, CMA staff and other Project stakeholders was generally very positive, suggesting that the community-based approach was a key factor of success for the Project. Overall, the model is seen to be effective in engaging farming communities and tailoring sustainable soil management activities to the needs of participating landholders, with additional outcomes for community connectedness.

“[Our] confidence has increased by being part of the whole Project. Some of the stuff we already know, some is [new]. The networking side of it with other people is really useful, both with local community and with experts.” (participating farmer)

Prepared for North Central Catchment Management Authority 20 Farming for Sustainable Soils Project Final Report

“It is quite rare in my professional experience to get programs that actually ask the landholders what they want to do […] 90% of the success of the FSS Project has been because of that community-driven approach.” (CMA staff member)

The vast majority of participating farmers, landholders and community members were satisfied with their involvement in FSS Project activities and reported that activities helped them improve their approach to soil health.

• From annual survey results between 2013 and 2017, 99% of participating farmers reported that the FSS Project assisted them to achieve their sustainable soil management goals, with 26% (38) reporting a significant amount, 53% (78) reporting a moderate amount, and 20% (29) reporting a little. And 90% of participating farmers reported that they were either moderately or very satisfied with the FSS Project overall. • Of farmers surveyed through this evaluation, 87% (46) agreed that activities in the FSS Project were either moderately or considerably helpful in influencing their approach to soil management. The majority (85%) of participants also reported that they were either moderately or very satisfied with the FSS Project overall (see Figure 4).

46% 46% 50% 44% 45% 39% 40% 35% 30% 25% 20% 13% 15% 9% 10%

5% 2% 1% Proportion respondents of Proportion 0% Not at all satisfied Somewhat satisfied Moderately Very satisfied satisfied

Evaluation survey (2018) Annual survey (2013-2017)

Figure 4. Overall participant satisfaction with the FSS Project from final evaluation survey (n=54) and annual survey (n=137). Activities in the FSS Project have contributed to improved soil quality and agricultural productivity in north central Victoria. Through the survey undertaken through this evaluation, 67% (33) of participating landholders have seen some improvement in their soil health as a result of the FSS Project, and further 29% (14) have stated that it is too early to tell (see Figure 5 on the following page). In addition, 54% (26) have seen some improvement in the productivity of their farming business resulting from the FSS Project, and 38% (18) stated that it’s too early to tell (see Figure 5).

“With soil structure and trying to build organic matter in our soil, deep ripping made a big difference. I'm a bit more aware of soil health issues with the soil pits too. We had a much better yield from crops where we had done the deep ripping trials.” (participating farmer)

Prepared for North Central Catchment Management Authority 21 Farming for Sustainable Soils Project Final Report

40% 38%

35% 31% 31% 29% 30% 24% 25% 20% 13% 12% 15% 10% 10% 8% 4%

Proportion respondents of Proportion 5% 0% No improvement Some Moderate Considerable It's too early to tell improvement improvement improvement

Soil quality Productivity

Figure 5. Final evaluation survey: extent of improvements in soil quality and productivity resulting from FSS Project (n=49).

Funded field trials and demonstrations were a core component of the FSS model that were found to both provide value through their outcomes and results, but also provided value to farmers as a learning activity and engagement tool. Relevant to the appropriateness of the FSS Project, RMCG states: “This type of on-farm demonstration helps bring clarity to some of the uncertainties farmers have about the effectiveness of practices in improving soil condition and their adoptability, including cost”.23

Results from FSS field trials collated and analysed by RMCG show that discernible differences were observed between the treatments in around half of the trials (see Figure 6 on the following page). While there is limited quantifiable evidence to support this, visual differences were observed, and this provided value to farmers in discussing key learnings and likely outcomes if trials were upscaled on their properties. Figure 6 also shows that the majority of trials confirmed good practice, with 14% leading to new knowledge. Around one third of trials had low success or were affected by poor growing seasons in 2014/2015.24

Participating farmers reported that just as much benefit can come out of the learnings of failed trials than successful ones:

“The best thing about some of these trials was that we could see if things would work. […] It’s probably more what I didn’t do, not so much as what I did do. There were things I was going to do then everyone did it and it didn’t work, so I decided not to waste my time and money on doing them.” (participating farmer)

23 RMCG (2018) 'Farming for Sustainable Soils—On-farm Trials Report’ (not publicly available), p. 19. 24 RMCG (2018) 'Farming for Sustainable Soils—On-farm Trials Report’ (not publicly available), p. 19.

Prepared for North Central Catchment Management Authority 22 Farming for Sustainable Soils Project Final Report

50% 50% 50% 42% 44% 40% 40% 36%

30% 30%

20% 14% 20% 14% 10% 10%

0% 0% Yes No Maybe High (new Medium Low knowledge) (confirmed good (abandoned/failed practice) due to season)

Figure 6. RMCG: discernible differences between treatments used in field trials (yes, no, maybe) (n=45) and success of field trails (high, medium, low) (n=44).

RMCG’s On-farm Trials Report covers the activities of seven FSS Groups in the region, extending from the northern plains southwards into the mid slopes region as far as the Great Dividing Range at the top of the Avon Richardson catchment in the south west (see Figure 7).

Information was collected from the 47 trials. Almost half were focused on amelioration of soil compaction and hardpans to improve soil condition and reduce sub soil-constraints on production. All Groups conducted trials looking at direct physical methods of improving soil structure and most also conducted cropland management trials, including demonstrating alternative plants to increase soil organic carbon, tillage and seeding techniques and the efficacy of different fertilisers. Several Groups looked at pastures and different soil measurement and monitoring techniques to improve decision making.

Soil structure was the biggest issue that farmers addressed through the trials. There is a growing understanding among farmers that getting their system right and growing plants that will increase soil organic carbon will be the key to improving soil condition. Figure 7. Distribution of soil trials conducted by FSS Groups (RMCG 2018).

Prepared for North Central Catchment Management Authority 23 Farming for Sustainable Soils Project Final Report

Most of the direct physical techniques trials were found to be unaffordable for current dryland farming systems in the north central region. The trials and demonstrations were in the main, not replicated scientific trials but were a practical vehicle for farmers testing ideas and trying new practices in the paddock. They were designed to allow them to gain experience with new practices, crops and technologies, to begin to validate proposed options that could improve soil condition and to build community confidence in their application.

In many instances the approach saw failures because of inherent difficulties with extreme climatic variance (droughts and floods), however, farmers accepted this as part of otherwise normal seasonal variability. Cropping trials at Wycheproof, for example, were subject to poor yields because they were conducted in drought, while subsoil manuring and trash incorporation worked well at Timor West following the wet spring of 2016. The experience with climate variability underlined issues with assessment of organic carbon content and balance of regional soils—reinforcing observations from historical measurements that suggest substantive changes only occur where practice change occurs over longer timeframes.

Informing future soil health initiatives, key messages and further research questions have been identified for each of the main trial types and sub-types in RMCG’s On-farm Trials Report.25

On-farm trials are explored in more detail in the following case studies developed by FPC:

• Inspiring practice change: the value of conducting soil health trials—"Trialling new soil management techniques through the Farming for Sustainable Soils Project”26 • Regenerating soil through additives and ameliorants—"Trialling inputs to improve soil health through the Farming for Sustainable Soils Project”27 • The value of soil care: changing approaches to soil management—"Insights from Farming for Sustainable Soils Project participants in the Wycheproof, Glenloth East and Charlton areas of north central Victoria”28

Monitoring and reporting of field trials has been identified as an area for improvement (see Section 4.6.2 on strengths and limitations). However, there are inherent challenges with soil management— particularly in measuring soil health improvements within short timeframes.

While the activities in the FSS Project were seen to be adequate and effective in delivering soil health outcomes, the Project was limited by time and resource constraints and the long-term nature of soil management. Through interviews and surveys conducted through this evaluation, participating farmers, Community Facilitators, CMA staff and other Project stakeholders reflected on the challenges of achieving soil health outcomes within a short period of time—considering additional factors like seasonal conditions and financial pressures. Many farmers described the practice changes they have made and the ongoing improvements they will continue to see as a

25 RMCG (2018) 'Farming for Sustainable Soils—On-farm Trials Report’ (not publicly available). 26 http://www.nccma.vic.gov.au/resources/publications/inspiring-practice-change 27 http://www.nccma.vic.gov.au/resources/publications/regenerating-soil-through-additives-and-ameliorants 28 http://www.nccma.vic.gov.au/resources/publications/value-soil-care

Prepared for North Central Catchment Management Authority 24 Farming for Sustainable Soils Project Final Report result of the FSS Project, and others described how the knowledge they have learned through the FSS Project will inform future management decisions:

“Improvements are slow and take time, they depend on seasonal conditions, and that. It's too early to say but if it was raining more regularly we'd see a considerable improvement, but there just hasn't been much rain.” (participating farmer)

This will also be discussed in Section 4.5 on the overall impact of the Project.

There are some other key challenges in meeting the needs of farmers through devolved grants programs such as the FSS Project. Through surveys and interviews, stakeholders raised some insights on appropriateness of activities, for example:

• It is difficult to pitch all activities at the right level when there is diversity in the needs and experience of farmers—some participants were relatively advanced in their knowledge and practice of sustainable land management, and others felt they need to “get into the basics”. • Farmers consistently reflected that, while the Project provided overall benefit to them, there were some instances where activities were not relevant or provided little value. The soil management context of north central Victoria varies greatly, as does the context of local areas within the catchment, farming properties and even individual paddocks—ensuring the relevance of sustainable land management capacity building and practice change activities is particularly challenging in relation to soil health. However, a key strength of the FSS Project reported by participating farmers is the ability for local communities to pursue soil health topics most relevant to them.

The strengths and limitations of the FSS Project model, lessons learned and suggestions for improvement are discussed in more detail in Section 4.6 on future implications. 4.4 Efficiency The FSS Project was delivered within its scope, budget and expected timeframes. As reported in Section 3.3, all Project inputs, activities and outputs have been delivered and completed, and a range of key outputs and activity targets have been exceeded. All grant funds were fully expended by 30 June 2018 as per the funding agreement between the CMA and the Australian Government, and the Project has leveraged a total of $401,790 in additional value through community contributions. Delivery of the FSS Project was supported by:

• FSS Project Team—including the Project Manager and various support staff within the CMA • risk management structures—evidenced through the NLP MERIT Portal • financial management protocols—in accordance with NLP requirements and CMA guidelines • governance structures—including the FSS Project Reference Group and the North Central CMA Standing Grants Committee • MERI—including internal CMA review processes and reflection sessions with key Project delivery staff, an evaluation plan developed through MERIT, an external Mid-term Review (RMCG) and external Final Evaluation (FPC) • external consultancies where necessary—for example, independent advice on the development of Local Area Soils Plans.

Prepared for North Central Catchment Management Authority 25 Farming for Sustainable Soils Project Final Report

The Project appears to have been well-managed by the FSS Project Team and the CMA. Few concerns were raised throughout this evaluation, and stakeholders and delivery staff report that any issues arising throughout the Project were most often appropriately managed. For example, some problems were encountered with the Smeaton FSS Group, including: some division in the community between conventional farmers and those embracing more sustainable farming practices, capacity issues with the Smeaton FSS Community Facilitator, some field trials failing to produce useful outcomes, and lack of ongoing engagement by local landholders in the Group. These issues were managed by the FSS Project Manager in consultation with CMA senior management and the Standing Grants Committee, with funding for a potential fourth year of the Smeaton FSS Group reallocated within the Project budget.

It appears that the FSS Project was a cost-effective investment in engaging local farming communities to improve the health of their soils and agricultural productivity—particularly considering the depth of engagement of participants in the Project across multiple years. However, it is not possible to directly compare the cost-effectiveness of the FSS Project to other similar programs or approaches due to lack of publicly available data allowing for an accurate comparison. Table 10 shows three cost-effectiveness estimates for the FSS Project, including: the estimated cost per total hectares with direct practice change funded by the FSS Project ($1,444), cost per total FSS capacity building event attendee ($4,320), and cost per total active FSS Group participant ($11,574). It should be recognised that these cost-effectiveness measures are limited, and do not comprehensively reflect the value (financial and non-financial value) of investment in the FSS Project.

Table 10. Cost-effectiveness estimates. Cost-effectiveness Cost-effectiveness measure Calculation estimate Cost per total hectares with direct practice $2,777,856 / 1,924 hectares $1,444 per hectare change funded by the FSS Project Cost per total FSS capacity building event $2,777,856 / 643 attendees $4,320 per event attendee29 attendee Cost per total active FSS Group participant $2,777,856 / 240 participants $11,574 per FSS Group participant Recognising the limitations of this approach, some additional cost-effectiveness measures have been calculated as a comparison. The average size of properties managed by landholders participating in FSS Groups is 1,436 hectares—with properties ranging from 24 to 6,500 hectares, as reported through annual surveys conducted by the CMA. The total number of active participants in FSS Groups throughout the Project is 240 and the total amount of Australian Government grant funding is $2,777,856, as reported above. Based on this data, and changes reported through the survey undertaken through this evaluation, additional cost-effectiveness estimates were calculated (see Table 11 on the following page).

29 This refers to the total number of land managers who attended FSS capacity building events who may or may not have been active FSS Group participants. FSS supported events were regularly attended by farmers, landholders and community members who were not formal members of FSS Groups.

Prepared for North Central Catchment Management Authority 26 Farming for Sustainable Soils Project Final Report

Table 11. Additional cost-effectiveness estimates. Proportion of Estimated hectares Estimated Reported change as a FSS Group managed by proportion cost per result of participating Calculation participants of participants hectare in the FSS Project30 (240 total)31 (344,640l)32 managed Changes in soil 185 (77%) $265,443 (77%) $2,777,856 / $265,443 $10.46 management practices Increased confidence 214 (89%) $306,811 (89%) $2,777,856 / $306,811 $9.05 in soil management Improved knowledge and skills in relation to 230 (96%) $330,942 (96%) $2,777,856 / $330,942 $8.39 soil management Average $9.30

The community-based approach used in the FSS Project is an effective model for achieving intended outcomes. This is supported by the range of evidence presented through this evaluation— for example, the high levels of participation and ongoing engagement in the FSS Project (with two out of four outcomes targets exceeded, see Section 4.2), positive support from participating farmers, Community Facilitators, CMA staff and other Project stakeholders, and only one key suggestion for improving the efficiency of Project delivery.

Over the past five years, through the FSS Project, 240 farmers from eight farming communities have sought to understand the condition of their soils, build their knowledge base through workshops with expert scientists and trial promising land management practices that build soil health and soil condition. Their activities have extended through experimentation with cover crops, deep ripping and gypsum application, injection of animal manures, adoption of pulse crops, sustaining ground cover, break-of-slope tree planting to intercept groundwater in salinity control, shifting to liquid fertilisers, and many more. Each farming community participates in a process that transitions through soil assessment, knowledge building through access to expert scientists and trialling alternative approaches on local properties.

The FSS Project structure with three core components and an underpinning in community ownership is likely to be an effective model for future soil health and other agricultural extension programs. From the 1990s, NRM and agricultural extension in Australia moved from a one-on-one focus to community-based approaches—but community-based NRM approaches are not always effective, particularly when narrowly scoped, under-resourced and not tailored to local contexts.33 Evidence presented throughout this evaluation has demonstrated positive outcomes of the FSS Project’s community-based model. Importantly, the FSS model includes a strong focus on peer-to-

30 Based on results of the survey undertaken through this evaluation. 31 Total of 240 active participants in FSS Groups between 2013 and 2018, as reported by the CMA. 32 Total hectares of land managed by participating farmers based on average property size of 1,436 hectares reported through annual surveys between 2013 and 2017. 33 Curtis, A, Ross, H, Marshall, GR, Baldwin, C, Cavaye, J, Freeman, C, Carr, A & Syme, GJ (2014) ‘The great experiment with devolved NRM governance: lessons from community engagement in Australia and New Zealand since the 1980s’, Australasian Journal of Environmental Management, Vol. 21, No. 2, pp. 175-199.

Prepared for North Central Catchment Management Authority 27 Farming for Sustainable Soils Project Final Report peer learning and is supported by grant funding to local communities—two aspects identified as factors that strengthen the effectiveness of community-based NRM program.34

“Participation as a member of a group and / or network is the basic building block of community- based NRM” and maintaining engagement and increasing new membership in community-based NRM has been strongly linked to the achievement of increased on-ground outcomes.35 Feedback from participating farmers, Community Facilitators, CMA staff and other Project stakeholders suggest that the community-based approach was a key factor that has led to the successful achievement of intended outcomes:

“The key lesson for me is the consolidation of that community-based approach. I can’t see any other way that we can achieve the kind of changes we’re looking for without having that kind of approach.” (CMA staff member)

Community-based natural resource management (NRM) is seen as “a cost-effective platform for rural development that extends beyond NRM to provide an important part of the social capital in rural areas”.36 This is evident through outcomes of the FSS Project. For example, farmers in the Glenloth East and Wycheproof FSS Groups experienced hardship due to drought, and the opportunity for them to connect through the FSS Project has improved community resilience:

“You’ve got a social benefit—the first two years we ran we had two terrible droughts and getting farmers together to talk was a huge thing. I found I had more members turning up in those dry years and I think a lot of it came back to the social side, knowing that they’re not the only ones involved and going through hard times.” (FSS Community Facilitator)

The community-based model of the FSS Project is explored in more detail in the following case studies developed by FPC:

• Enriching soils, enriching communities: the value of a community-led approach— “Collectively working to improve soil health”37 • Facilitating change in soil health—“The experience of being a Community Facilitator in the Farming for Sustainable Soils Project”38

While, overall, resources were used efficiently, an opportunity to improve the impact of financial investment through the FSS Project was identified by Project delivery staff.

Community grants provided to FSS Groups were spread over three or four years, with funding distributed relatively evenly across years, but generally a higher amount was provided in the first year (see Table 12 on the following page). From a Project management perspective, this funding

34 Curtis et al (2014). 35 Curtis, A & Sample, R (2010), ‘CBNRM in Victoria: Contributing to dialogue, learning and action’, Institute for Land, Water and Society, Charles Sturt University http://athene.riv.csu.edu.au/~acurtis/reports/CBNRM_Victoria_Final_Report_July.pdf 36 Curtis et al (2014). 37 http://www.nccma.vic.gov.au/resources/publications/enriching-soils-enriching-communities-value- community-led-approach 38 http://www.nccma.vic.gov.au/resources/publications/facilitating-change-soil-health

Prepared for North Central Catchment Management Authority 28 Farming for Sustainable Soils Project Final Report structure allowed for substantial investment in the establishment of each FSS Group in the first year—including funding for developing Local Area Soils Plans to be delivered in subsequent years, starting with soil assessments and establishing field trials, supported by capacity building.

While this approach was appropriate for grant management and acquittal of funds invested in farming communities, key Project delivery staff reported that this funding structure presented some challenges. In interviews undertaken through this evaluation, various stakeholders suggested that the first year of funding could have been dedicated to building community Groups and undertaking extensive testing and planning (gathering base line data for use in developing the appropriate trials and potential practices to include in Local Area Soil Plans, with smaller amounts of funding), which in turn could have informed the planning of more tailored activities in later years (with more substantial funding). It was suggested by some stakeholders that this alternative funding structure could allow more time to understand the appropriate financial scale of soil health interventions in the early stages (such as, through soil pits and soil testing), leading to more targeted and larger investment in on-ground works (such as, field trials and demonstrations, based on the results of soil assessments).

Table 12. Distribution of grant funding to FSS Groups. FSS Group 2013/2014 2014/2015 2015/2016 2016/2017 2017/2018 Total Lockington $35,000 $35,000 Charlton $35,000 $35,000 Wycheproof $50,000 $35,000 $35,000 $120,000 Paradise $50,000 $35,000 $30,000 $115,000 Smeaton $50,000 $30,000 $30,000 $110,00 Pyramid Hill $50,000 $35,000 $35,000 $25,000 $145,000 Timor West $40,000 $40,000 $35,000 $30,000 $145,00 Glenloth East $35,000 $35,000 $70,000 Total $170,000 $210,000 $170,000 $135,000 $90,000 $775,000

The level of ongoing funding required to support the continuation of community groups like those established through the FSS Project is often raised as an issue in relation to the overall effectiveness of community-based NRM programs.39 There is evidence to suggest that the FSS Project will continue delivering value and outcomes beyond the life of the funding period due in part to the depth of engagement and formation of local networks—as discussed in detail in Section 4.5 below on overall impact. While it is unclear what level of investment would be required to continue supporting farming communities engaged through the FSS Project, there is evidence to suggest that there would be value in continuing to support ongoing improvement of sustainable soil management knowledge, skills and practices in some way.

Section 4.6 presents the future needs and opportunities identified for farming communities to continue improving the health of their soils beyond the FSS Project, and Section 5 includes a series of

39 Curtis et al (2014).

Prepared for North Central Catchment Management Authority 29 Farming for Sustainable Soils Project Final Report recommendations for how best the CMA can continue supporting farming communities in the catchment and build on the outcomes achieved through the FSS Project in future. 4.5 Impact Based on a synthesis of all evaluation data available, the overall impact of the FSS Project is the empowerment of farming communities to continue improving sustainable soil management going forward. The FSS Project has demonstrated the effectiveness of the community-based model for delivering ongoing value for soil health, capacity for sustainable soil management, community connectedness and agricultural productivity.

“At the end of the day, farming communities are responsible for the health and productivity of their soils and environmental integrity […] We work with communities and try to empower them through the Project.” (CMA staff member)

Evidence presented in Section 4.2 (Effectiveness) and Section 4.3 (Appropriateness) above demonstrates that key changes were directly produced by FSS Project interventions:

• Management practices—the majority of participating farmers have made changes in their soil management practices resulting from their involvement in the FSS Project and for the vast majority of participating farmers, the FSS Project has led to improved confidence, knowledge and skills about sustainable soil management and motivated them to undertake more trials, implement new approaches and learn more about soil. • Effectiveness of delivery—the vast majority of participating farmers, landholders and community members were satisfied with the FSS Project and reported that Project activities helped them improve their approach to sustainable soil management. • Changing asset condition—the majority of participating landholders have seen some improvement in the condition of their soil as a result of the FSS Project, and over half have seen some improvement in the productivity of their farming business.

There are likely to be ongoing impacts of the FSS Project that are yet to be realised—particularly changes in the condition of soil assets. With the community-based model of the FSS Project including three key components—soil testing and assessments, knowledge sharing and capacity building, and field trials and demonstrations—the Project empowered farming communities in north central Victoria to deliver outcomes within the funding period, but to also set communities on a path to continue achieving longer-term outcomes.

There is limited evidence to demonstrate landscape scale impacts of the FSS Project on the condition of soil assets in the catchment to date. However, there is evidence to suggest that if practices adopted through the Project are maintained, soil health in north central Victoria will improve through time, and that FSS participants will continue making sustainable soil management practice changes in future. For example, the majority of attendees at FSS events (69%, 52) report that they intend to make management practice changes in future—and 67% (38) of participating landholders surveyed through this evaluation have additional changes and new soil management practices either planned (35%, 20) or in progress (32%, 18) as a result of the Project (see Figure 8 on the following page).

Prepared for North Central Catchment Management Authority 30 Farming for Sustainable Soils Project Final Report

80% 69% 70%

60%

50%

40% 35% 32% 30% 20% 21% 20% 12% 10% 5% 5%

0% No / unlikely Maybe / uncertain Yes, I have things Yes, I have things in planned / likely progress / definitely

FSS event feedback surveys (2016-2017) Final evaluation survey (2018)

Figure 8. Intent for management practice change after (FSS event feedback surveys, n=75) and reported intention to make additional changes or adopt new soil management practices in future as a result of the FSS Project (final evaluation survey, n=57). The nature of soil management means that improvements in soil health can often take up to ten years to be fully realised after management changes have been implemented and maintained. In addition, financial constraints, seasonal variability and other challenges faced by farmers mean that management decisions can be delayed, even after farmers have been equipped with the knowledge and skills to implement changes. Participating farmers reflected on this through interviews and surveys:

“With the weather station, it’s been interesting just to monitor soil temperatures and soil moisture, we've only had it for two years […] We won't make any management decisions though for a while yet until we look at the data from the station.” (participating farmer)

“It’s still a bit early to tell. When you apply gypsum and lime, you need three-to-five years before you see overall effect. But I've seen an improvement over the last two years and expect to see more to come. With productivity, some of the yields farmers have seen in the region [after the FSS Project] with limited rain have been far greater than we would expect.” (participating farmer)

There are also other examples of how the FSS Project has continued to deliver ongoing value in relation to key Project outcomes, such as:

• The value of partnerships—the Pyramid Hill FSS Group worked on a cropping trial in partnership with the Birchip Cropping Group, which led to the establishment of a new pulse cropping trial with the Grains Research and Development Corporation providing additional value for farmers in the region. • Strengthened community networks—both the Timor West and Lockington FSS Groups evolved from existing Landcare Networks in their local communities, and after the FSS Project, community involvement in Landcare has increased and strengthened in these areas.

Prepared for North Central Catchment Management Authority 31 Farming for Sustainable Soils Project Final Report

• Peer-to-peer learning—after funding for the Wycheproof FSS Group finished in 2016, members of the Wycheproof Group continued their involvement by joining Glenloth East and mentoring less experienced members of the Glenloth East Group.

“[One of the best things about the FSS Project was] talking to other people and realising that it’s good to take a risk and try different things […] don't be scared to do that, no-one is going to bag you if it doesn't work […] and if it goes wrong, then other people can learn from that, and if it goes right equally people want to learn from that.” (participating farmer) 4.6 Future Outcomes, achievements and lessons learned from the FSS Project demonstrate that there are clear opportunities for the North Central CMA to continue its work empowering farming communities to improve their sustainable land management practices, including for soil health.

Analysing the strengths and limitations of the FSS Project have also led to a range of lessons learned that can be used to inform future programs and services delivered by the North Central CMA—for soil health programs and more widely. 4.6.1 Future opportunities

There is a clear opportunity for the North Central CMA to continue working with local farming communities to improve sustainable land management practices, particularly in relation to soil health.

Overall, landholders recognise that they have made progress in their knowledge and practice of sustainable soil management, but that there is a clear need for continuous improvement to build on the outcomes achieved through the FSS Project.

At the FSS Conference in March 2018, the CMA received overwhelming feedback from attending farmers, community members, landholders and other stakeholders that soil health is a key priority for the north central catchment going forward.

Annual survey responses collected between 2013 and 2017 show the following insights:

• 90% (97) of survey respondents stated that they would find it useful to have more information or assistance in interpreting soil test results for their properties—suggesting there is an opportunity to continue working with farmers in the catchment to understand and apply soil testing. • The majority of farmers (86%, 130) reported that they feel they have either a moderate (48%) or significant (38%) amount more to learn about sustainable soil management going forward. • In relation to farmers’ perception of awareness of sustainable soil management in their districts, the majority of respondents (89%, 138) suggest that either some (37%) or most (52%) farmers in their district are aware of sustainable soils practices (such as reduced cultivation, permanent groundcover, building organic matter, etc.), and the majority of respondents (88%, 136) suggest that some (52%) or most (36%) farmers are striving to implement those sustainable soils practices.

Prepared for North Central Catchment Management Authority 32 Farming for Sustainable Soils Project Final Report

In addition, Table 13 demonstrates that the majority of landholders rate themselves in the middle of the scale in relation to sustainable soil management practices.

Table 13. Annual survey 2013-2017: farmer perceptions of progress in sustainable soil management knowledge and practice.

Survey question Summarised responses How sustainable do you feel your Not at all Okay for the Okay for the Okay for the Good forever current soil management practices sustainable next few next decade next 20 (actively are generally? (n=151) 2% years 31% years restoring soil) 35% 17% 15% Where would you rate your Very poor Poor Moderate Considerable Exceptional knowledge on how to achieve knowledge knowledge knowledge knowledge knowledge sustainable soil management at the 2% 10% 72% 15% 1% moment? (n=156) How far are you along the road to Just starting Getting Okay, but Pretty good, I'm there achieving sustainable soils on your 13% somewhere more work nearly there nothing more to do to do farm? (n=159) 20% 8% 59% 1% How often are you likely to go Every crop Most crops Once every Rarely Never against your understanding of or season or seasons few crops or 37% 1% sustainable practices? (n=147) 2% 9% seasons 50%

4.6.2 Strengths and limitations

The community-based approach—providing flexibility for community groups to tailor the soil health activities to their local needs within the boundaries of the NLP and CMA guidelines—was consistently identified as the key strength of the FSS Project by participating farmers, landholders and community members, CMA staff, Community Facilitators and other Project stakeholders.

“They let farmers be farmers and let farmers do things the way they want to. Flexibility of delivery was the best thing about the Project.” (FSS Community Facilitator)

Across all the data reviewed through this final evaluation, feedback on the FSS Project was predominately very positive about the Project’s approach, outcomes and delivery. However, some critical feedback and limitations of the Project have been identified.

Table 14 on the following page presents the key strengths of the FSS Project identified through this evaluation. Alongside the key strengths, it also presents the limitations of the Project that present opportunities for improvement in future.

The challenges in delivering an initiative like the FSS Project are summarised by a CMA staff member:

“It’s a compromise between what we have to do with the funding and what we can achieve with the farming community Groups.” (CMA staff member)

Prepared for North Central Catchment Management Authority 33 Farming for Sustainable Soils Project Final Report

Table 14. Strengths and limitations of the FSS Project.

Key strengths of the FSS Project Opportunities to strengthen the Project Community-based approach Project structure • The community-based approach was • Some Project stakeholders suggested that consistently cited as a crucial success factor the structure and timing of the funding of the FSS Project. It provided FSS Groups provided to FSS Groups could be revised— with flexibility within the boundaries of NLP particularly the structure of providing a and CMA guidelines to achieve soil health, large portion of funding early in the three- community resilience and agricultural or four-year funding cycle for each Group productivity outcomes. (as discussed in Section 4.4 above). • The localised approach allowed for tailored • In some cases, stakeholders reported that activities and maximised farmer Annual Activity Schedules could have been participation (particularly among better aligned with growing seasons and landholders not previously engaged in CMA funding cycles. It was felt by some that FSS projects). Groups had a limited ability to adapt their Practical, applied and localised investment through planned activities within each year, and the Project that the timing of Project planning and funding cycles may have impacted on • Groups had the opportunity to choose farmer participation. expert presenters, activities and topics of interest. The Project provided Groups with Tailoring activities and content to meet the needs expert advice and on-ground of all farmers demonstrations that would not have been • It was challenging for Community possible for individual farmers due to the Facilitators and FSS Groups to meet the investment required. varying needs and contexts of all farmers in • Investment into local technology was their local communities. Sometimes critical (for example, moisture probes and activities and content were not pitched at weather stations funded through the the right level for all participants. Project now provide localised data to • While this was a source of frustration for inform land management decisions). some participants throughout some stages • Supporting practical and applied activities of the Project, overall, participating farmers (including field trials and demonstrations) reported that they were satisfied with the was appropriate for farmers, as a Project and activities helped them achieve Community Facilitator noted: “farmers their soil health goals. learn best by doing”. Upscaling trialled soil management techniques Opportunity for landholders to understand the • A limitation of the FSS Project involves the condition of their soils and learn from experts challenge of how to move from the trial • The soil testing and assessment component phase into application of soil health of the FSS Project allowed farmers to techniques at the whole farm scale. For understand the condition of their soils and example, it was reported by some FSS the challenges and opportunities that their Groups that subsoil manuring is effective soils present—this was seen to be but too expensive for most farmers to use particularly valuable to participants. consistently across their properties, and • Support from experts to further interpret the RMCG FSS On-farm Trials Report found soil testing and assessments led farmers to that some direct physical methods of implement various soil management improving soil structure are not economical solutions on their properties and informed for dryland broadacre application in the

Prepared for North Central Catchment Management Authority 34 Farming for Sustainable Soils Project Final Report

Key strengths of the FSS Project Opportunities to strengthen the Project development of field trials of various region.40 This is a key challenge in the techniques. journey to improving soil health that • Communities would not have had access to farming communities in the catchment soil testing and assessments (like soil pits) continue to address following on from the from experts and consultants without the FSS Project. FSS Project, and often participants have Limited evidence of landscape scale changes had ongoing interaction with these experts • There is limited evidence to demonstrate or engaged additional expert advice landscape scale impacts of the FSS Project following the FSS Project to assist with their on the condition of soil assets in the soil management. catchment to date. However, there is Three complementary components of the Project evidence to suggest that soil health will • Project stakeholders noted that the three continue to improve in future as a result of FSS components—soil testing and the Project (as discussed in Section 4.5 on assessments, knowledge sharing and overall impact). capacity building, and field trials and Limited reporting on trials and demonstrations demonstrations—complemented each • While funded field trials and other well, contributing to the success of demonstrations were found to provide the Project. value through their outcomes and results, Peer to peer learning and community as well as providing value to farmers as a connectedness established through the Project learning activity and engagement tool (see • Peer-to-peer learning and knowledge Section 4.3), reporting was limited. sharing through community FSS Groups is • Collection of baseline data and ongoing cited as a key strength because it builds monitoring of results by FSS Groups confidence in soil management techniques throughout trials and demonstrations could among farmers. have been more consistent across the • The community-based model is also seen to Project. contribute to community wellbeing and Governance structures stronger community connectedness— • There is an opportunity for internal working together gives farmers solidarity. governance processes (and associated Community Facilitators planning, approvals and reporting • The approach of employing a local structures) to be improved by the CMA in Community Facilitator to go out to farmers the management of devolved grants and ask what they want to know (rather programs—ensuring that these processes than being told what to do by the CMA) do not constrain on-ground delivery. For was seen as a key strength, building trust example, the timing and frequency of and credibility in the Project and motivating Standing Grants Committee meetings could farmers to continue improving soil health. be revised to better meet the needs of • Individual Community Facilitators have community-based program management. often been identified as crucial to the • There are inherent challenges working to success of the FSS Project. achieve long-term soil management • The opportunity for Facilitators to meet outcomes within short-term organisational collectively was also a strength of the structured and external funding cycles. Project, allowing for sharing and discussion Some Project stakeholders suggested that about things that were working well or not local community-based soil management

40 RMCG (2018) 'Farming for Sustainable Soils—On-farm Trials Report’ (not publicly available).

Prepared for North Central Catchment Management Authority 35 Farming for Sustainable Soils Project Final Report

Key strengths of the FSS Project Opportunities to strengthen the Project so well, and to create synergies between programs are more appropriate in four-or Groups in their content, activities and five-year cycles rather than three-year expert presenters. cycles. Role of the North Central CMA Expectations and guidance • The role of the North Central CMA in • In some cases, Community Facilitators may overseeing the FSS Project was a key have benefitted from additional guidance strength, through providing overarching or support from the CMA in relation to the management of the Project and planning, reporting and information empowering local communities. management components of their roles. • While the Project was community-based, For example, more formal training and the North Central CMA provided extensive clearer communication about some of the support throughout the Project, including: expectations of Community Facilitators building the capacity of Community may have assisted them in their roles. Facilitators, managing consultants, Monitoring, evaluation and reporting communications, financial management, • In some instances, there was a lack of NLP progress reporting, and managing consistency in monitoring and evaluation internal relationships with the CMA throughout the Project—for example, with Standing Grants Committee, the FSS Project the tracking of some key outputs and Reference Group and CMA executive activities. management. • There were limitations in the use of biophysical information in the assessment of practice change, and the impact of practice change. It was beyond the resources of the Project to conduct regional on-ground surveys, and it was generally recognised that meaningful assessments of changes in soil condition could not be made within available timeframes. The North Central CMA has, however, developed a protocol for longer term assessments. • Having a clearer framework for establishing baseline data at the Group and individual level may have assisted in more effectively demonstrating the impact of the Project over time. • Ongoing reporting requirements for Community Facilitators throughout the Project could have been improved to streamline annual reporting.

4.6.3 Suggested improvements and ongoing challenges

The following key suggestions for improvement have been raised consistently by participating farmers, Community Facilitators, CMA staff and other Project stakeholders engaged through this

Prepared for North Central Catchment Management Authority 36 Farming for Sustainable Soils Project Final Report evaluation. Some of these suggestions can be applied more generally to other NRM programs and services beyond the FSS Project.

• Aligning support and service delivery to fit in with seasonal growing cycles. Continue aiming to schedule events and activities at times and structured in ways that will maximise farmer attendance—for example, scheduling events in January-March and July-October, holding half-day events in the early morning or evening. • Continuously improving the ways that results from field trials and demonstrations are made available, explained and interpreted within communities and used by landholders in their local contexts. Building on the achievements of funded field trials and demonstrations, this may involve additional community meetings (co-ordinated by Community Facilitators) to reflect on the successes and challenges of different soil health interventions. This may feed into community-based decision-making for future investments in soil health and would help to increase landholders’ confidence to make practice changes, using the shared learnings to allow them to try things on their own properties. • Continue working with farming communities within the constraints faced by landholders. Farmers face challenges such as lack of time, seasonal conditions, financial pressures and varying capacity for management changes—continuing to adapt to these constraints is important in empowering communities to be responsible for sustainable land management in their local areas. • Continue to strengthen embedded structures for ongoing knowledge sharing established within the Project—between farmers, within Groups, across Groups and Community Facilitators, from past to current or new Groups.

Identified throughout the Project was the key challenge of how to keep farmers engaged in their local FSS Groups after the initial soil testing and assessment activities in the first year. Some key factors that contributed to successful and ongoing engagement of farmers throughout the FSS Project include:

• Community Facilitators working closely with their communities to ensure that activities and information continue to be relevant to their Group members • where possible, using the results from soil testing and field trials to inform the ongoing development of Group activities, based on the availability of results • involving agronomists within the FSS Group meant that farmers kept engaged… new things emerging, always talking to farmers, keep it relevant • Setting up trials that have synergy with what farmers are already doing – these ones worked best and kept people most interested • Identifying the ‘movers and shakers’—including consultants, agronomists, producer groups, one or two key farmers.

Most of the limitations of the Project raised by farmers reflect inherent challenges with delivering sustainable agriculture programs more widely. There are several ongoing challenges faced by the North Central CMA and other regional NRM organisations raised through feedback from farmers in interviews and surveys, notably:

Prepared for North Central Catchment Management Authority 37 Farming for Sustainable Soils Project Final Report

• Seasonal conditions—particularly rainfall variability and longer-term changes associated with climate change impacting on the ability of farmers to make practice changes. • Financial pressures limiting the investment available for farmers to fund infrastructure and inputs associated with practice changes. • Variability in soil conditions within regions, properties and within individual paddocks, limiting the adaptability and scalability of soil health techniques across the landscape. There are 18 different geomorphic settings in the catchment, adding to the challenge of how best to support landholders achieve more sustainable farming practices. • Poor soil condition and structure, resulting from 150 years of cultivation practices that have degraded soil health. • Uncertainty in the availability of funding for community-based grants, and associated challenges with external funding cycles. Continuity in investment in sustainable land management is important to maximise potential economic, social and environmental value—particularly when most soil health interventions take upwards of five years for their outcomes to be realised. • Uncertainty in ensuring that management practice changes are maintained by landholders, and that landholders will continuously use the knowledge and skills gained to further improve management practices. • The challenge of how to best support farmers in the FSS Project beyond the funding period, considering the most effective ways to maintain farmer engagement in sustainable soil management. For example, the Group-to-Group mentoring between the Wycheproof and Glenloth East FSS Groups may be replicable, and some farmers have indicated that additional short-term participation in FSS Groups would be helpful.

“It seems like [it was] a lot of effort, but I don't know if it has led to change. The reality is that weather impacts so heavily on what people can do.” (participating farmer)

“Would have been good if it ran for a little bit longer. Some of these projects take a while to get going, and then you're just getting some good trials in and the project finishes. Another year would have cemented the core findings and what we were doing a bit more.” (participating farmer)

Some of these key challenges—particularly the question of ongoing dependence on external funding for community groups and the challenge of continuing to motivate farmers in self-directed sustainable practice change—are supported by the literature.41 These have been addressed in the following sections on recommendations and future activities for soil assets in the catchment (Section 5.1.2).

41 For example, Curtis et al (2014).

Prepared for North Central Catchment Management Authority 38 Farming for Sustainable Soils Project Final Report

5 Conclusion and recommendations 5.1 Final evaluation of the Farming for Sustainable Soils Project This section presents the overarching findings and implications of the Final Evaluation of the FSS Project delivered by the North Central CMA between 2013 and 2018. 5.1.1 Conclusions

In conclusion, the FSS Project has been a successful investment in improving sustainable land management practices in the north central catchment of Victoria.

The FSS Project demonstrates the effectiveness of community-based approaches to agricultural extension programs in the NRM sector. The FSS model is an effective model for soil health initiatives going forward, with the three key components of testing and assessments, knowledge sharing and capacity building, and field trials and demonstrations.

Through the FSS Project, local communities have been empowered to improve the health of their soil assets, with a range of key outcomes achieved in relation to knowledge, skills and confidence, improved sustainable soil management practices, agricultural productivity, soil health and community resilience. Many of the longer-term impacts of the Project are yet to be realised, and the FSS Project will have ongoing value in years to come.

Table 15 on the following page outlines the findings in response to the evaluation questions.

Prepared for North Central Catchment Management Authority 39 Farming for Sustainable Soils Project Final Report

Table 15. Findings against evaluation questions. Evaluation question Sub-questions Key findings and examples Effectiveness To what extent have To what extent have farmers All planned outputs and activities have been delivered. The FSS Project has achieved its intermediate and intended Project adopted sustainable end of Project outcomes, as well as contributing to achievement of longer-term outcomes. All four key outcomes and outputs management practices as a target outcomes for the FSS Project have either been achieved or exceeded. been achieved? consequence of their No unintended outcomes were reported throughout the evaluation. However, the Project has resulted in Were there any involvement in the Project? some unexpectedly positive results. For example, several target outcomes being substantially exceeded. unexpected or To what extent is farmer There is clear evidence demonstrating that farming communities are increasing their knowledge, skills and unintended outcomes? participation in local area soil awareness of sustainable soil and land management through their participation in the FSS Project—with protection groups improving 89% of participating farmers reporting that their confidence has increased in their capacity or ability to adopt their knowledge and skills in new soil management practices since being involved in the FSS Project, and 96% reported improved sustainable agriculture? knowledge and skills in relation to soil health. There is also clear evidence that farmers are adopting new sustainable land management practices as a consequence of participating in the FSS Project—with 77% of farmers reporting having made changes on their property resulting from their involvement in the FSS Project, and, for 98% of participants, the FSS Project has motivated them to undertake more trials, implement new approaches and learn more about soil heath. Appropriateness To what extent were To what extent are the Overall, activities were appropriate for achieving the intended FSS Project outcomes, and the community- Project activities practices adopted by based model of the FSS Project is seen to meet the needs of farmers. appropriate? participating farmers For the majority of participants, improvements in soil health skills, knowledge and experience gained adequate and effective in through participating in the FSS Project led to an increased adoption of sustainable land management delivering soil health? practices. To what extent are The vast majority of participating farmers, landholders and community members were satisfied with their improvements in skills, involvement in FSS Project activities and reported that activities helped them improve their approach to soil knowledge and experience in health—with 99% of participating farmers reporting that the FSS Project assisted them to achieve their the attainment of social sustainable soil management goals, and 90% reporting that they were either moderately or very satisfied with health leading to improved the FSS Project overall. adoption of sustainable land The FSS Project has contributed to improved soil quality and agricultural productivity in north central management practices by Victoria—with 67% of participating landholders having seen some improvement in their soil health as a result participants in local groups? of the FSS Project, and 54% having seen some improvement in the productivity of their farming business.

Prepared for North Central Catchment Management Authority 40 Farming for Sustainable Soils Project Final Report

Does the community-based While the activities in the FSS Project were seen to be adequate and effective in delivering soil health model meet the needs of outcomes, the Project was limited by time and resource constraints and the long-term nature of soil farmers? management. Efficiency To what extent was To what extent is the program The FSS Project was delivered within its scope, budget and expected timeframes. All Project inputs, activities the Project delivered cost effective in engaging and outputs have been delivered and completed. All grant funds will be fully expended in line with NLP within its scope, local farming communities in requirements, and the Project leveraged additional value through community contributions. budget and expected soil health programs that It appears that the FSS Project was a cost-effective investment in engaging local farming communities to timeframe? secure productivity and improve the health of their soils and agricultural productivity—particularly considering the depth of ecosystem services? engagement of participants in the Project across multiple years. However, it is not possible to directly How could resources be used compare the cost-effectiveness of the FSS Project to other similar programs or approaches due to lack of more productively and publicly available data allowing for an accurate comparison. efficiently? There is evidence to suggest that the FSS Project will continue delivering value and outcomes beyond the life of the funding period due in part to the depth of engagement and formation of local networks through the Project. Impact What has been the In what ways and to what The overall impact of the FSS Project is the empowerment of farming communities to continue improving overall impact and extent has the Project sustainable soil management going forward. ongoing value of the contributed to changing asset The FSS Project has demonstrated the effectiveness of the community-based model for delivering ongoing Project? condition, management value for soil health, capacity for sustainable soil management, community connectedness and agricultural practices, and / or productivity. Key changes were directly produced by FSS Project interventions—for example, 77% of farmers effectiveness of delivery? surveyed through this evaluation have made sustainable soil management practice changes following their To what extent were the involvement, and 46% of these farmers said they would not have been able to make these changes without changes directly or indirectly the Project. produced by the Project There are likely to be ongoing impacts of the FSS Project that are yet to be realised, with communities now interventions? set on a path to continue achieving longer-term outcomes. For example, the majority of attendees at FSS What are the ongoing impacts events (69%) report that they intend to make management practice changes in future—and 67% of of the Project? participating landholders surveyed through this evaluation have additional changes and new soil management practices either planned (35%) or in progress (32%) as a result of the FSS Project.

Prepared for North Central Catchment Management Authority 41 Farming for Sustainable Soils Project Final Report

Future What lessons learned What are the strengths and There is a clear opportunity for the North Central CMA to continue working with local farming communities can be used to inform limitations of the Project? to improve sustainable land management practices, particularly in relation to soil health. Overall, future programs and How would stakeholders, landholders recognise that they have made progress in their knowledge and practice of sustainable soil services delivered by participants and Project management, but that there is a need for continuous improvement to build on the outcomes achieved NCCMA? delivery staff do things through the FSS Project. differently in future? The community-based approach—providing flexibility for community groups to tailor the soil health activities to their local needs within the boundaries of the NLP and CMA guidelines—was consistently identified as the key strength of the FSS Project by participating farmers, landholders and community members, CMA staff, Community Facilitators and other Project stakeholders. Across all the data reviewed through this final evaluation, feedback on the FSS Project was predominately very positive about the Project’s approach, outcomes and delivery. Some critical feedback and limitations of the Project have been identified. However, most of the limitations of the Project raised by stakeholders reflect inherent challenges with delivering agricultural extension programs more widely—particularly seasonal conditions, financial pressures, variability in soil characteristics, challenges with external funding cycles and uncertainty in ensuring that management practice changes are actually maintained by landholders, and that landholders will continuously use the knowledge and skills gained to further improve management practices. The key priority for the North Central CMA going forward is likely to be the challenge of how best to support farmers in the FSS Project beyond the funding period, considering the most effective ways to maintain farmer engagement in sustainable soil management.

Prepared for North Central Catchment Management Authority 42 Farming for Sustainable Soils Project Final Report

5.1.2 Recommendations

Four overarching recommendations have emerged from the key findings and evidence synthesised through this evaluation:

1. There is a clear need and opportunity for additional work in supporting farmers to improve their soil health going forward. Soil health should continue to be a priority for land managers in the north central catchment, supported by government, non-government and private sector stakeholders. 2. The FSS Project is a successful model for sustainable soil management practice change. The North Central CMA should continue to use this community-based approach in delivering sustainable agriculture programs and services, as well as promoting use of the model more widely. 3. The outcomes of the community-based approach should continue to be monitored over time, to evaluate the longer-term strengths of this model compared to other approaches to sustainable agriculture programs. In particular, this should focus on measuring any landscape scale changes occurring as a result of the FSS Project. 4. Collection and analysis of biophysical data should be prioritised in future soil health projects to more effectively monitor and evaluate outcomes, appropriateness and cost- effectiveness. 5.1.3 Future activities for the asset

These overarching recommendations lead into a series of more detailed recommendations for activities to continue improving soil assets throughout the catchment and reach the overarching FSS Project vision and longer-term outcomes.

The FSS Project’s vision is that: “North Central Victoria has secured the health and productivity of soils, improved ecosystem services that flow from healthy soils, and built resilience to climate change through increased soil health”. Longer-term outcomes intended to achieve this vision include: increased confidence in recommended practices and ongoing trialling and adoption by all farmers in the region; a productive agricultural food sector; stronger social resilience; and maintenance, protection and/or improvement of ecosystem services.

The North Central CMA is currently developing a Soil Health Action Plan42 and finalising delivery of the FSS Project. The RMCG FSS field trails report also provides a clear summary of key learnings from sustainable soil management approaches trialled through the Project and identifies areas for further exploration to continue improving soil health in the catchment.43 Building on this work and the achievements of the FSS Project, it is suggested that the North Central CMA should:

• Continue to maintain the strong relationships developed with local communities through the FSS Project, fostering continued stewardship and community ownership of sustainable soil management. This may include:

42 North Central CMA (2018) ‘Soil Health Action Plan’, http://www.nccma.vic.gov.au/projects/agriculture#node-1780 43 RMCG (2018) 'Farming for Sustainable Soils—On-farm Trials Report’ (not publicly available).

Prepared for North Central Catchment Management Authority 43 Farming for Sustainable Soils Project Final Report

o continuing to provide support, resources and advice to FSS Groups, allowing them to be self-sustaining beyond the FSS Project funding period o establishing forums for structured peer-to-peer (or Group-to-Group) knowledge sharing and capacity building among landholders going forward (for example, similar to the mentoring relationship between Wycheproof and Glenloth East FSS Groups). • Offer ongoing support to farmers in the catchment to ensure that the potential longer- term benefits of the Project are achieved. This may include: o providing additional advice to farmers on interpreting soil assessment results and understanding associated opportunities, limitations and potential soil management practice changes o continuing to work closely with other organisations delivering complementary sustainable soil and land management support to landholders in the catchment (particularly Agriculture Victoria) and align with other CMA programs where possible (including the Regional Landcare Facilitator) o continuing to support farmers to conduct their own soil management trials, with the CMA providing assistance on monitoring approaches to measure outcomes o focusing on methods and management techniques that can be more easily and viably upscaled to whole paddocks or farms, and/or working with farmers to trial these kinds of techniques themselves o sharing trial results, case studies and sustainable soil management resources to maintain interest and involvement among those who have had contact through the Project and promote the benefits of sustainable soil management to those not reached directly through the Project. • Continue applying for external funding (i.e., through the Commonwealth NLP or the Victorian Government) to provide programs and services to farmers, landholders and community members in the catchment to improve sustainable land management practices. This may include continuing to focus on: o working with farmers to overcome key barriers for improving their sustainable farming practices (i.e., seasonal variability, historically poor soil structure, financial constraints, ongoing impacts of climate change, etc.) o establishing ongoing partnerships between landholders, landholder groups, local industry groups and commercial entities, building on partnerships developed through the FSS Project. • Apply the lessons from the FSS Project to its design and delivery of other agricultural extension programs and services, continuing to work towards the most effective ways of engaging with landholders to inspire practice change. • Improve the consistency of monitoring and evaluation to better understand the extent of achievements, outcomes, lessons learned, and wider impacts of soil health programs and services delivered by the CMA. This may include following up with FSS Group participants in future years to evaluate and capture any longer-term sustainable soil management practice changes or social, economic or environmental outcomes resulting from the FSS Project.

Prepared for North Central Catchment Management Authority 44 Farming for Sustainable Soils Project Final Report

Appendix 1—Methodology Overview The key components of FPC’s methodology for delivering the final evaluation of the FSS Project include:

• Inception meeting • Project plan • Document review • Case study development • Conference support • Interviews and surveys • Analysis and reporting

Each of these components are described in more detail in the sections below, as well as an overview of key limitations of the consultancy. Methodology in detail Inception meeting

An inception meeting was held at the beginning of the project between the FPC team and the North Central CMA team in December 2017. The following items were discussed at this meeting:

• the CMA’s objectives for the project, including what ‘success’ looks like • risks associated with the project and how they will be managed • review of FPC’s approach to the consultancy • reporting, deliverables, timelines and project management processes, including key milestones and payment schedules • access to information and resources, including relevant data and documentation. Project plan

Following discussion at the inception meeting and building on the approach outlined in the initial proposal, FPC developed a project plan—including an agreed approach to delivering the consultancy by mid-May 2018 and a high-level evaluation framework (see Table 16 on the following page)

Prepared for North Central Catchment Management Authority 45 Farming for Sustainable Soils Project Final Report

Table 16. Evaluation framework (adapted from FSS Project MERI Plan, NLP Monitoring and Reporting Plan and information provided by North Central CMA). Criteria Key evaluation question Sub-questions Indicators and things to consider Data sources Effectiveness To what extent have intended To what extent have farmers • Annual survey of extent of • Case studies Program outcomes and outputs adopted sustainable management practice change among soil • Interviews been achieved? practices as a consequence of their groups and the extent of • Existing Project data involvement in the program? adoption of sustainable Were there any unexpected or • Survey practices unintended outcomes? To what extent is farmer participation in local area soil protection groups improving their knowledge and skills in sustainable agriculture? Appropriateness To what extent were Program To what extent are the practices • Document the extent of • Project management activities appropriate? adopted by participating farmers adoption of sustainable documents adequate and effective in delivering practices among FSS Groups • Existing program data soil health? through annual surveys • Interviews completed early in the To what extent are improvements in • Case studies growing season Annual survey skills, knowledge and experience in of group members the attainment of social health leading to improved adoption of sustainable land management practices by participants in local groups? Does the community-based model meet the needs of farmers? Efficiency To what extent was the To what extent is the program cost • Record the cost of the • Project management Program delivered within its effective in engaging local farming delivering the Project documents scope, budget and expected communities in soil health programs (including in-kind) relative to • Existing program data timeframe? that secure productivity and the number of farmers • Interviews ecosystem services? engaged and the extent of • Survey adoption of sustainable How could resources be used more practices over the life of the productively and efficiently? FSS Project. Assessment to be completed through the annual

Prepared for North Central Catchment Management Authority 46 Farming for Sustainable Soils Project Final Report

survey each year mid-growing season. Impact What has been the overall In what ways and to what extent has • Survey of farmers to ascertain • Survey impact and ongoing value of the the Program contributed to changing satisfaction gained from • Case studies Program? asset condition, management involvement in soil protection • Interviews practices, and / or effectiveness of programs and the extent that • Existing Project data delivery? they have become motivated to explore and adopt To what extent were the changes sustainable practices. Annual directly or indirectly produced by the survey completed mid-year, Program interventions? using template. What are the ongoing impacts of the Program? Future What lessons learned can be What are the strengths and • Identified strengths, • Project management used to inform future programs limitations of the Program? weaknesses and documents and services delivered by improvements • How would stakeholders, Existing Project data NCCMA? • Lessons learned and participants and Program delivery • Interviews suggestions for improvement staff do things differently in future? • Survey

Prepared for North Central Catchment Management Authority 47 Farming for Sustainable Soils Project Final Report

Document review

FPC undertook a document review of all existing information and data sources provided in the early stage of the consultancy. This process included:

• reviewing existing documents and information provided by the CMA to help identify appropriate case studies • initial analysis of relevant Project documentation and existing data sources to determine how the most appropriate evidence could be used to respond to key evaluation questions • a gap analysis to identify gaps in existing data to be addressed through additional data collection—through case studies, discussions with the parallel consultancy (RMCG), the FSS Conference and additional interviews and surveys • working closely with North Central CMA staff to ensure transfer of relevant program documentation, data sources (including access to the MERIT Portal), and relevant delivery staff, stakeholders and Project partners. Case study development

FPC developed a total of 13 case studies to illustrate the involvement of farming communities in the FSS Project and demonstrate the outcomes and impact of the Project in response to the key evaluation questions. These included eight short case studies and five more in-depth case studies.

Development of case studies involved the following:

• semi-structured interviews with CMA staff, Community Facilitators, participating landholders and other Project stakeholders to collect insights, reflections and information about the FSS Project achievements and outcomes (including both face-to-face interviews at site visits and phone interviews) • working with a videographer to film face-to-face interviews at site visits, to contribute to complementary video case studies • photography (a combination of FPC photography at site visits, and images provided by case study participants and Project delivery staff) • integrating the data collected through interviews with analysis of other existing documents and data sources, synthesising all relevant information into clear, concise and insightful case studies • a combination of quotes, photos, summarised information and text analysis, presenting stories of change resulting from the FSS Project • incorporating feedback from the CMA and utilising graphic design expertise to finalised publishable versions of the case studies, which are now available on the North Central CMA website.

FPC worked closely with CMA staff to select the most appropriate case study participants and topics, based on their experience delivering the FSS Project and existing relationships with landholders, stakeholders, community members and delivery partners.

Four of the five more in-depth case studies are complemented by videos, which are also available on the North Central CMA website.

Prepared for North Central Catchment Management Authority 48 Farming for Sustainable Soils Project Final Report

Conference support

FPC provided support to North Central CMA staff in delivering components of the FSS Project Conference held on 8-9 March 2018 in Bendigo. In consultation with CMA staff, FPC delivered a reflection and evaluation session at the conference. This involved a short presentation about the final evaluation of the FSS Project, a series of guided discussion about feedback on the FSS Conference, and data collection on the outcomes of the Project more widely, as well as key priorities for sustainable soil management going forward. Interviews and surveys

FPC also conducted series of phone interviews and surveys to help meet evaluation objectives and requirements. Table 17 provides a summary of each data collection method conducted by FPC through this consultancy and the number of participants.

Table 17. Summary of data collection components. Data collection method Participants Discussions with FSS Project stakeholders (including expert 4 presenters and consultants) Semi-structured phone interviews with FSS Community Facilitators 2

Face-to-face interviews with FSS Community Facilitators 4 Face-to-face interviews with FSS Project participants 5 Semi-structured phone interviews with North Central CMA staff 2 Semi-structured phone interviews with FSS Project participants 8 Final evaluation survey conducted over the phone with FSS Project 57 participants FSS Conference data collection 50 FSS review workshop 16

Interviews (both face-to-face and phone interviews) were semi-structured and included a combination of both open-ended and closed questions. Face-to-face interviews were digitally recorded and transcribed for analysis, and phone interviews were transcribed by the interviewer at the time of the interview.

The survey was conducted over the phone with FSS participants and responses were manually entered into online platform SurveyMonkey.

Interview and survey questions were developed based on the Project Plan and in consultation with the FSS Project Manager. Analysis and reporting

After data collection was completed, all existing data was analysed as well as the data collected throughout this consultancy in response to evaluation questions. Data analysis involved both

Prepared for North Central Catchment Management Authority 49 Farming for Sustainable Soils Project Final Report qualitative and quantitative techniques as appropriate and was aligned with the evaluation framework.

FPC developed an evaluation report (this report), synthesising all evaluation findings emerging from data analysis. This report has been structured based on guidance provided by North Central CMA and NLP reporting requirements and has been informed by review and feedback in consultation with the CMA. A review workshop was also held in June 2018, at which the findings, conclusions and recommendations in this final report were discussed with core Project delivery staff and stakeholders. Limitations

The following limitations of this consultancy should be considered:

• As is standard with any similar data collection processes, the landholders who participated in interviews and surveys may have a self-selection bias (for example, they may have been more engaged in the Project than others or may have been more dissatisfied with a particular component). For the survey conducted, this was addressed by actively phoning landholders, rather than relying on passive recruitment through an online survey. • In a similar way, survey and interview results also rely on self-reported changes in behaviour and understanding by participating landholders. FPC has attempted to triangulate these results using several methods (for example, combining evidence from site visits, reports from previous consultancies and insights from Community Facilitators, CMA staff and experts and consultants who assisted in Project delivery) to ensure that findings are based on evidence from multiple sources. • Limited biophysical data was available, and any observations of soil condition improvement rely largely on landholders’ reported perceptions.

Prepared for North Central Catchment Management Authority 50