Thematic Evaluation of the Territorial Employment Pacts

Final Report to Directorate General Regional Policy

OCTOBER 2002

by ECOTEC Research & Consulting Limited

13B Avenue de Tervuren B-1040 Brussels BELGIUM

Tel: +32 (02) 743 89 49 Fax: +32 (02) 732 71 11 Thematic Evaluation of the Territorial Employment Pacts

Final Report to Directorate General Regional Policy

October 2002

ECOTEC Research and Consulting Limited

13b Avenue de Tervuren Priestley House B-1040 Brussels 28-34 Albert Street Belgium Birmingham B4 7UD Tel: +32 (0)2 743 8949 United Kingdom Fax: +32 (0)2 743 7111 Tel: +44 (0)121 616 3600 Fax: +44 (0)121 616 3699 Web: www.ecotec.com E-mail: [email protected] Modesto Lafuente 63 – 6a E-28003 Madrid Spain Tel: +34 91 535 0640 Fax: +34 91 533 3663

6-8 Marshalsea Road London SE1 1HL United Kingdom Tel: +44 (0)20 7089 5550 Fax: +44 (0)20 7089 5559

31-32 Park Row Leeds LS1 5JD United Kingdom Tel: +44 (0)113 244 9845 Fax: +44 (0)113 244 9844 Evaluation of the Territorial Employment Pacts – Final Report –October 2002

TABLE OF CONTENTS

EXECUTIVE SUMMARY……………………………………………….……………i INTRODUCTION...... 1 CHAPTER 1: SETTING THE SCENE: BASELINES...... 2 1.1 The Territorial Employment Pacts Programme ...... 2 1.2 The basic TEP concept...... 3 1.3 The Selection Process...... 5 1.4 The 89 Territorial Employment Pacts...... 7 1.5 The basic attributes of the selected TEPs...... 8 1.6 The differential availability of resources ...... 9 1.7 Administration of the TEP Programme...... 10 CHAPTER 2: ANALYTICAL FRAMEWORK...... 12 2.1 The Focus of the Evaluation...... 12 2.2 Key evaluation questions ...... 13 2.3 The evaluation methodology ...... 14 2.4 Dealing with baseline variety ...... 14 2.5 The Evaluation framework ...... 15 2.6 The Stages of the Evaluation...... 16 CHAPTER 3: PACTS IN PRACTICE...... 19 3.1 Pact attributes...... 19 3.1.1 Bottom up Partnerships ...... 20 3.1.2 Formal, broad-based partnership...... 22 3.1.3 Integrated Strategy ...... 24 3.1.4 An innovative approach ...... 24 3.1.5 Attribute Summary...... 25 3.2 Management quality ...... 26 3.2.1 Aspects of management ...... 26 3.2.2 Individual qualities...... 27 3.2.3 Structures...... 27 3.2.4 Planning...... 28 3.2.5 Management summary...... 29 3.3 Pact Achievements...... 30 3.3.1 Achievements overall...... 30 3.3.2 Labour market achievement in general ...... 31 3.3.3 Employment creation ...... 32 3.3.4 Partnership activity...... 34 3.3.5 Funding and Territorial development...... 37 3.3.6 Mainstreaming and replicability...... 39 3.3.7 Summary of Pact achievements ...... 39 3.3.8 The association between attributes, management quality, and achievements ...... 40 CHAPTER 4: CONTEXTUAL FACTORS WHICH MAY EXPLAIN PERFORMANCE ...... 42 4.1 Introduction ...... 42 4.2 Contextual factors not readily testable...... 43 4.2.1 Developments in the macro-economy...... 43 4.2.2 Wider policy framework within which the Pact operates ...... 43 4.2.3 External economies of scale and available infrastructure ...... 43 Evaluation of the Territorial Employment Pacts – Final Report –October 2002

4.3 Governance context: Member State variations in the opportunities for TEPs ...... 44 4.3.1 Types of context...... 44 4.3.2 Governance context analysis...... 46 4.4 Governance Level...... 53 4.5 The importance of vertical links...... 56 4.6 Differences in governance context within Member States...... 57 4.7 Pact performance and ‘family’ context...... 58 4.7.1 Family clusters ...... 58 4.7.2 Human geography ...... 60 4.7.3 Pacts and the influence of geography: summary...... 64 4.7.4 Type of labour market and model of intervention...... 65 4.8 The role of the European Commission...... 65 4.9 Summary of influential factors...... 68 CHAPTER 5: TERRITORIAL EMPLOYMENT PACT EFFECTIVENESS AS A POLICY INSTRUMENT...... 70 5.1 Added value ...... 70 5.1.1 Partnership...... 70 5.1.2 Policies and programmes ...... 72 5.1.3 Innovative Actions and Initiatives...... 72 5.2 Cost effectiveness...... 74 5.2.1 Programme effectiveness ...... 74 5.2.2 Scale ...... 77 5.3 Catalytic effect...... 79 5.3.1 Leverage over additional resources...... 79 5.4 Dissemination, Mainstreaming and Replication ...... 83 5.4.1 Internally focused...... 84 5.4.2 The development of Pact networks ...... 84 5.4.3 Local profile ...... 85 5.4.4 Externally focused...... 86 CHAPTER 6: CONCLUSIONS AND RECOMMENDATIONS ...... 89 6.1 Pacts as a Policy Instrument for Local Employment and Development ...... 89 6.2 Pacts from a National Perspective ...... 92 6.3 Pacts from a Territorial Perspective ...... 94 6.4 Summary of conclusions regarding Pacts as a policy instrument in action ...... 96 6.5 The Territorial Pacts Programme as a Pilot Action ...... 98 Evaluation of the Territorial Employment Pacts – Final Report –October 2002

Executive Summary

This final report presents the findings of the evaluation of the Territorial Employment Pacts (TEPs) as a policy instrument. The report synthesises data gathered during all stages of the evaluation, using the general analysis of all 89 individual TEPs and the detailed analysis of the 30 TEPs that were examined as Case Studies.

SETTING THE SCENE

TEPs were part of a trend characteristic of the mid-1990s towards policy initiatives based on the idea of multi-stakeholder partnerships at a local level, designed to tackle unemployment and to promote job creation. The Pact Programme was formally launched in Dublin in December 1996, under the Irish Presidency. The promoting Directorate was DG Regional Policy (taking responsibility for Pacts in Objective 1 areas) with the DGs for Employment (Objective 3) and Agriculture (Objective 5) sharing the Pacts and taking some under their administrative umbrellas.

The TEPs were provided with technical assistance by the European Commission – up to a maximum of €300 000 – but were expected to lever in additional resources from, for instance, the European Union Structural Funds. In practice the level of direct resourcing was sufficient for little more than the establishment and support of Pact partnerships and small sums for experimentation. It gave scant opportunity for the TEPs to engage in projects funded under their own auspices from which direct outcomes could be measured. It is important to understand that this had an important and highly differential bearing on what the TEPs could hope to achieve and is critical to the findings of this evaluation.

Pacts were inserted into a wide variety of contexts. The TEP territories covered populations from 16 000 to over three million, and varied in size from Pacts covering single municipalities to whole regions. The number of partners varied from five to 105. Pacts confronted both ‘slack’ and ‘tight’ labour-market conditions. Pacts also differed in the extent to which they were able to lever in additional resources.

Furthermore Member States were responsible for selecting the Pacts in their countries and different Directorates General adopted somewhat different approaches in the administration of the TEP Programme. Therefore, the outcome of the selection and management process was a Pact population of wide variety, administered differently.

ANALYTICAL FRAMEWORK

The evaluation takes a thematic approach to the TEPs – that is, it draws out common themes and issues. The aim was to factor out the baseline variety of the Pacts and the different opportunity structures they faced across the 15 EU Member states, with a view to understanding how Pacts performed within their individual circumstances.

The evaluation was carried out by: • reviewing background documentation and monitoring data gathered by the Commission and Technical Assistance Office, including Pact applications, interim and final reports, and responses to previous data gathering exercises • interviewing key respondents in the Commission

i ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

• gathering views from national respondents such as Pact promoters and national experts • undertaking 30 case studies distributed across the 15 Member States and representative of the diversity of Pact circumstances

The basic methodology involved • describing the Pact programme and its variety • assessing the extent to which the Pacts supported demonstrated the basic concept upon which the programme was based • assessing the capacity and quality of Pact management • identifying the achievements which could be attributed to Pacts with certainty • examining the relationship between these achievements and the extent to which Pacts were what was intended and were well managed • on the basis of Pact characteristics, achievements, and management quality, assigning overall scores to Pacts, enabling them to be ranked and grouped for subsequent analysis • examining the influence that a variety of contextual factors (such as governmental context, scale, and location) had upon the observed performance of Pacts • assessing the cost effectiveness of the programme, its catalytic effect, sustainability and influence on the mainstream • drawing conclusions from the analysis as to the effectiveness of the programme, and making recommendations for future actions

EVALUATION OF PACT PERFORMANCE

This part of the report examines the basic questions of whether the Commission acquired Territorial Employment Pacts possessing attributes consistent with the intentions of the programme (bottom up, broad based, integrated strategy, innovative); how well the Pacts were organised and managed; and what individual Pacts achieved in relation to the objectives of the programme. This then enables the evaluation to examine possible reasons for the observed differences.

Pact Attributes: TEPs were all intended to embody four key characteristics: a genuinely “bottom-up” partnership; a broad-based partnership; possess an integrated strategy, and be deemed innovative. The evaluation investigated the degree to which the Pacts attained these attributes. Overall, in terms of meeting the requirements of the Programme, the evaluation found that sufficient ‘genuine’ Pacts were created to be able to consider the performance of the Programme as a policy instrument, in the knowledge that the sort of entities envisaged by the Commission were indeed created.

Management quality: An examination of the management of Pacts shows more weaknesses and variability. There seems to have been managerial and process weaknesses in a substantial subset of Pacts, which will have had a bearing on their likelihood of achieving positive results. Good management was often associated with the competence of a key individual in the management team of the Pact. On the other hand, the weakness of some TEPs was linked to the inexperience of the management team. Pacts benefited from continuity in the management team. There was a noticeable association between those Pacts which best represented the ‘ideal’ model, and those which were best managed.

ii ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

The type and ambition of goals and objectives established by Pacts shows considerable variability. This arises from two factors – the lack of clear specification by the Commission as to the criteria by which the achievements of Pacts might be judged; combined with the substantial diversity of contexts, scales and partnership types experienced by individual Pacts. In this context the appropriateness of objectives set can be seen as stemming from the experience of the management set up. Taken altogether, the evidence provided shows the importance of good and stable management for the work of Pacts.

Pact achievements: There is an enormous amount of variety in the achievements and results that emanate from the Pacts. While some stress their achievements in concrete terms of creating employment, the claimed achievements of most Pacts are rather less tangible and are in practice more consistent with the purpose of the programme. Little consistent and quantified information is however available across Pacts as a whole, and few Pacts recorded substantial data concerning their own activities. Qualitative and subjective information from the Pacts, combined with descriptive documentation and the opinions of national observers, gives a reasonable understanding of the areas where Pacts felt themselves to be successful or not. Pact achievements are thus considered in terms of labour market impact and employment creation, partnership activity, mobilising funding and ensuring mainstreaming/ replicability.

When examining labour market achievement in general, 39 Pacts demonstrated full achievement of their objectives in relation to the programmes’ employment related purpose, whilst only 34 partially demonstrated this, and 14 did not achieve their objectives. Whilst they do not capture the richness and diversity of Pact activities, these figures give a good summary indication of the perceived achievements of Pacts. Direct creation of employment has taken place in some Pacts but as a whole is of relatively little importance as an achievement of the programme. Where it has taken place it offers some models of collaborative approaches to new project development and operation, but in these cases the main benefit comes from the process of stimulating the idea and putting it into practice in the first place, rather than the job creation figures themselves.

Pacts’ views of their achievements are dominated by the creation and nurturing of partnerships and associated co-operative activity, and the benefits arising of improved trust, co-ordination, understanding, influence and common purpose. This sort of achievement is fully consistent with the purposes of the programme, and Pacts have been able to give examples of the positive wider effects arising. These wider effects are diverse and include more consensual and effective resource deployments, stimulation and putting into operation of project ideas, realignment of policy, better articulation of supply and demand.

69 Pacts demonstrated successful increases of co-ordination of and co-operation amongst partners, a high figure which reflects the emphasis of the programmes’ intentions. In areas with a multitude of pre-existing partnerships and other agencies involved in employment policies at local level, Pacts had little to offer in terms of institution building. In areas that lacked a tradition of partnership-oriented co- operation in the field of labour-market interventions, the creation of the Pacts had the

iii ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 potential to contribute substantially to the development of partnership capacity and new ways of addressing problems in the local labour markets.

Regarding the leverage of other funding streams, 74 Pacts levered additional public sector funds and 61 EU funds, whilst 56 levered private sector resources. This attainment of influence over funds in addition to the limited TA available to Pacts is important since in the field of employment development there is little evidence the progress can be made in the absence of at least some support funding.

Mainstreaming of the Pacts has been relatively limited and the wider lessons of Pacts have not been disseminated extensively. This is partly due to the diffuse nature of the achievements themselves, and the locality specific circumstances which shaped their evolution, from which it is difficult to generalise. It is also due in some cases to a lack of understanding by many Pacts that dissemination was a priority activity, and lack of resources directly available within the Pact programme.

Overall, the achievements of Pacts were very varied, of differing quality and relevance to the purpose of the Programme as a whole. Evidence of their influence on the labour market is scarce due to the indirect nature of much of their work. Nevertheless, taking all the information available together it is possible to distinguish between those which can be considered on balance as successful examples of the Programme’s intention in practice, and those which were less successful. This then provides a means for the factors which influence relative success to be examined.

CONTEXTUAL FACTORS WHICH MAY EXPLAIN PERFORMANCE

Having examined the question of exactly what was funded by the Commission; how well it was managed; and what can be demonstrated to have achieved, the evaluation goes on to discuss a range of factors which have, to a greater or lesser degree, influenced the types of work undertaken by Pacts and the value of their achievements. These influential factors examined included: • the governance context facing the pacts, • the governance level at which Pacts were inserted and • other factors such as the geo-economic context; the labour market conditions; and approach adopted (i.e. reinsertion of target groups, or generalised labour market action)

There are however some additional factors whose reliable analysis is impossible in the remit of this evaluation, such as: developments in the macro-economy; the wider policy framework within which the Pacts operate; external economies of scale and infrastructure.

In the instance of governance context, the evaluation examined Pacts grouped by “crowded platform”, “receptive space”, and “available space”, defined as follows: • Crowded Platform emerges where there has been a long history of governmental intervention in the operation of labour markets and in the territorial development process at all levels, and partnership working is common and often overlapping. (Denmark, France, Netherlands, Sweden, UK) • Receptive space exists where there is already in place an accepted and legally determined role for devolved governance powers at the local level, but this is

iv ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

not systematically combined with extensive, multi stakeholder partnership working. (Austria, Belgium, Finland, Germany, Ireland, Italy, Spain) • Available space reflects conditions where there has been only a limited history of local-level activity in relation to the labour market, and where labour policy has traditionally been the preserve of national (or devolved regional or provincial) government agencies. (Greece, Luxembourg, Portugal)

In summary, the evaluation found that the clearest influence on Pact performance comes from the combination of Member State attitude and governance context situation. Crowded policy arenas made for a more difficult environment for Pacts to work in. Those which succeeded in these conditions were those which either linked in with a government initiative heading in the same direction, or confined themselves to a focus on limited local actions where value could be added. Pacts found it easier to find a role where there was a legal framework and some tradition of local employment action and partnership working, but not a congested environment of overlapping remits and bodies. Much could be achieved in empty policy spaces, but only with constructive support from a superior authority

Very large Pacts found it difficult to achieve focus and could be undermined by excessive rivalry between constituent partners. They succeeded best with limited but highly focused strategic roles, but needed strong government support for this to happen. Very small Pacts relied on strong vertical links with higher authorities if they were to achieve any strategic results. Otherwise they tended to revert to direct project work, which might be more or less successful, but did not advance the Pact concept very much.

Overall, Pacts were influenced by a complex of factors, some within their control, and some beyond it. The interplay of these was complex, but even allowing for this, it must be concluded that the Pact approach can achieve success in most environments. The wider value of the success achieved is more variable however. A well-motivated team can find useful ways to spend the money invested in them on activities that have beneficial labour market outcomes. But the value of the Pact model, based on partnership formation and nurturing, strategic focus, and co-ordination of activities, will be best achieved with a combination of:

Ø Receptive space in the policy and practice environment Ø A positive attitude from superior authorities Ø The availability of sufficient bodies able and willing to be co-ordinated, and who’s joint working is genuinely additional to existing structures Ø Good internal management Ø The strategic vision to identify opportunities for new approaches, and the application to put them into practice.

TERRITORIAL EMPLOYMENT PACT EFFECTIVENESS AS A POLICY INSTRUMENT

This chapter of the report examines the extent to which the TEP Programme as a policy intervention: • added value to the policy and implementation structures in the localities in which Pacts operated

v ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

• delivered results cost effectively • was a catalyst for new types of co-operation and action, and • generated results and lessons which would either continue in the Pact territories concerned beyond the time of Commission support (sustainability), or have relevance for other actors attempting similar types of intervention (dissemination) or formulating policy responses (mainstreaming)

1.0 Added value

The added value accruing from the TEPs can be seen in three ways: • partnership development and working: the encouragement of partnership working had most potential to add value in ‘free space’ countries, and those ‘receptive space’ countries where co-ordination at the level where the Pact operated was not usual. • co-ordination and rationalisation of policies and programmes: The Pact programme added real value where it introduced new ways of working, new ideas and techniques, stemming from its vision of how employment and local development might be better organised and delivered. • implementation of specific labour-market actions: Added value from the TEPs Programme can be observed in the implementation of innovative actions; and in the ‘sponsorship’ of other labour-market initiatives by providing them with ‘seed’ money, advice and technical assistance.

Cost effectiveness

The net contribution of the European Commission of up to €300 000 to each Pact is a modest amount considering their scale and ambition. This input on behalf of the EU ‘purchased’ 89 Pacts, which involved a significant number of what can be characterised as ‘new’ partnerships, as well as a collection of achievements related to employment and territorial development outcomes that can be to a greater or lesser degree attributed to these Pacts.

• Where Pacts found little room for manoeuvre, or lack of support from key authorities; or were poorly managed, they achieved little, and the investment made can be considered a drag on the programme; • Where Pacts chose to focus on employment action which could have been funded under other mainstream EU or national programmes, there occurred substantial dead-weight at the programme level. Where Pacts added value, in the ways outlined above, the cost effectiveness of the programme is rather higher. Despite the limited funds available to each Pact, there were many instances of significant change occurring as a result of their work.

There is an argument to be made that a greater level of funding might have achieved proportionately more. Also, the issue of timescale is important as many Pacts would have benefited from a longer timeframe, especially in situations where the Pact idea was new to enable all the necessary stages to be completed and the Pacts to be embedded and sustained.

Catalytic effect

vi ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

The catalytic effect is assessed in this part of the report, by the ability of the Pacts to achieve influence over a wider pool of resources, over and above the amount of Technical Assistance that Pacts received from the European Commission. This ability has been mixed and three ways in which influence was obtained can be identified: • responsibility for deployment of other external funding streams being given to Pacts (‘programme management’), with a particular emphasis on EU programmes • the Pact partnership agreeing amongst themselves to use some or all of their ‘own’ resources to take forward common activities (‘pooling’) • the work of the Pact achieving influence over the resource deployment decisions of other stakeholders inside or outside the partnership (‘influence’)

Dissemination, sustainability and mainstreaming

Across the board, a mixed picture emerges and three broad models of dissemination have been identified: • ‘internally focused’ dissemination concentrated on the distribution and processing of information throughout the Pact partnership itself • ‘local profile’ dissemination, attempting to gain recognition for the Pact widely in the local area • ‘externally focused’ dissemination aimed at communicating the lessons of the Pact beyond the territory itself, to audiences including superior authorities which can, at least in part, be related to mainstreaming.

The effect of TEPs upon the mainstream has been somewhat mixed. Few Pacts have been mainstreamed into the Structural Funds Programme for 2000-2006. The Pact methodology has had a more lasting effect. Some Member States have made it possible to set up Pact-like bodies, even though the TEPs themselves have not been systematically mainstreamed.

CONCLUSIONS

The pilot action and its evaluation enabled us to make some relatively clear judgements on the value of Territorial Employment Pacts as a policy instrument. By way of a final summary the evaluation offers the following statements with a good degree of certainty;

vii ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

• Other things being equal, the evaluation indicates that the most likely spatial scale at which the right sort of conditions may be expected to prevail is the sub-regional level and in the case of an employment focused activity this is most effectively associated with the travel-to-work area, the bassin d’emploi or the continuous urban labour market. • However, these conditions in relation to Pacts represent an average situation around which there can exist a very considerable range of other possibilities where successful interventions can be made and where success can be achieved in employment and local development. • This variety in the possible local partnership forms and spatial scales observable for territorial pacts is a product of the fact that they are embedded differentially across member states and within regions according to prevailing circumstances. • Pacts and similar local partnership structures are emerging across what can be described as a highly variable European opportunity surface where their reception ranges from enthusiastic acceptance through relative indifference to a degree of resistance and this serves to explain the variety of ways they are embedded.

Among the many issues for Pacts revealed by the pilot five particular ones are worthy of emphasis: o The need for space to act – a foothold in local structures is essential o The need for reality checks on ambition – correctly addressing problems where they have leverage is critical o The need for size and scope to fit the mission – a balance is vital where the costs of partnership are commensurate with the added value to be achieved in delivering real outcomes o The need for requisite capacity and competency – leadership and management is challenging for pacts and local partnerships and a key variable in success o The need for focus – pacts and local partnerships need to have clear and deliverable objectives

For territorial policy, the evaluation indicates that the application of the pact principles can have generic value and are worthy of promotion in themselves, but: o The mechanisms for them to be successfully applied are not going to be available everywhere o A prior stage of indeterminate length and resource demands will be essential to create the necessary threshold conditions and for capacity to be built in those places where the starting position is low o For some states and regions the pact principles are already strongly embedded in domestic policy and the additional value of a Commission initiative is likely to be limited except where it gives the opportunity for further experiment and innovation o Across the board the unequivocal value of the pilot action was the scope it offered for innovation and experiment regardless of the circumstances in which the Pacts found themselves and in this respect it has to be considered as being both a valuable exercise and good value for money

Finally, the evaluation presents conclusions about the Territorial Employment Pacts Pilot Action itself. Recommendations for similar initiatives in the future are offered,

viii ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 learning from certain weaknesses identified in the materialisation of the TEP Pilot Action: • Even where the objective of a pilot action is to capture as wide a range of possible examples as possible, there is a need to be careful about controlling for the variety in the set. Part of this controlling process has to come from a greater clarity of pre-definition and pre-selection. Initiatives of this kind require considerable pre- planning, a clear explanation to the relevant stakeholders, and an acceptance that results are likely to be partial and highly varied. • A clear view of the likely outcomes to be expected is required from an early point, with associated explanation as why such results are sought, and clear specifications regarding the uses to which EU funds can be put. Monitoring, benchmarking and evaluation tools capable of identifying and validating results should also be provided to promoters. Where flexibility is required this should be built in to the indicators put forward for project promoters to work with. • Where (as in the case of Pacts) the success or failure of the action bears heavily on the partnership bodies that are to carry it out, there needs to be some guidance on what these sorts of bodies might look like. Post-selection from an open call at least gives the opportunity for judgements to be made. • There needs to be a realistic calculation of the minimum level of resource (in terms of timescale and funds) needed to give each project a fair chance of convincing its constituency that the exercise is worthwhile and to carry out the necessary tasks. • The bureaucratic process needs to be wary of being slow to sanction spend at the beginning but quick to demand commitment, spend and accountability towards the end of the programme as confidence and goodwill of the project sponsors may suffer as a result.

ix ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

INTRODUCTION

This is the final report of the evaluation of the Territorial Employment Pacts, carried out between November 2001 and July 2002. It presents the findings of the evaluation of the Pacts programme as a policy instrument, bringing together results and analysis from all stages of the evaluation process.

The report is structured as follows:

Section one provides an overview of the background to the TEP Programme, and gives essential information on the 89 Pacts funded. Section two presents the methodological framework of the evaluation. Section three summarises the outputs and results achieved by Pacts, leading on to section four which considers various factors which influenced observed Pact performance. Section five examines the added value, cost effectiveness and catalytic effect of the Pacts while section six contains conclusions and recommendations for future action.

1 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

CHAPTER 1: SETTING THE SCENE: BASELINES

1.1 The Territorial Employment Pacts Programme

◊ Antecedents

Territorial Pacts owe their origins to a general shift of policy interest toward the exploration of local and bottom-up approaches to employment creation that took place during the middle 1990’s. Today, Territorial Employment Pacts (TEPs) or similar approaches based on multi-stakeholder partnerships at local level form part of the standard policy portfolio in many European nations. Less than a decade or so ago, however, these sorts of approaches were virtually unheard of. The Pacts were launched as just one initiative among many during a creative and fast moving period of policy restructuring to confront unemployment and social exclusion.

Ireland can lay some claim to being among the first to introduce this sort of local partnership approach to tackle unemployment and job creation. As early as 1991, for example, during the negotiation of the second agreement between government and the social partners, the trade unions proposed that the response to long-term unemployment should be area based. The first Local Partnerships were quickly established, funded mainly by the Irish exchequer with a contribution from the ESF. In 1994 the Irish and EU strands of policy were brought together within the Operational Programme for Local Urban and Regional Development.

In parallel, the Patti Territoriali also emerged in Italy as a means of brokering local political and economic conflicts and bringing the economic and social partners together locally to find solutions to unemployment. The law that promoted them defined them as "agreements, promoted from local agencies, social partners, or from other relevant public or private bodies relative to a program of participation in the promotion of the local development" (trans. from Law. 662/96, art. 2, codicil 203, 1996).

At about the same time (1995) the Local Development and Employment Initiative (LDEI) sponsored by the Commission’s Cellule de Prospective was beginning to have a powerful influence on European thinking through its promotion of area based approaches to the creation of new sources of employment and attacking intractable unemployment.

◊ Commission engagement

The flow of ideas about a local dimension to employment and development policy was, therefore, gaining strength during the latter half of the 1990’s. The different policy directorates– agriculture, employment and regional – were themselves introducing policies with a strong local and partnership approach into their portfolios during the 1994 – 1999 programming period for the Structural Funds. Many emerged as Community Initiatives and projects under Articles 6 and 10 to give scope for further experimentation. For example, a conference in Rome under the auspices of DG Employment was already defining the shape of Territorial Pacts (following

2 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 closely on the Italian model) a year before their formal launch. In case of agriculture and rural development, the LEADER initiative was being strongly promoted as a local area policy framework. Similarly, concerns about the need to respond to the persistence of pockets of long-term unemployment within regions were being increasingly emphasised within DG Regional Policy following a strong lead from the Commissioner.

The precise process by which the Territorial Employment Pacts Programme per se emerged from this complex and fast-moving background to become adopted as a mainstream Commission initiative is a subject of some debate. Nevertheless Pacts were formally launched in Dublin in December 1996, appropriately under the Irish Presidency. The promoting Directorate was DG Regional Policy with the DGs for Employment and Agriculture sharing the Pacts and taking some under their administrative umbrellas.

◊ Financial arrangements

Unusually, the Territorial Pacts Programme was seen from the outset as being funded through Technical Assistance funds. The timing of the launch in relation to the programming periods for Structural Funds (close to the end of the cycle) gave little scope for financial support from this source. The proposition was that a small amount of technical assistance would be sufficient to introduce and “prove” the value of the TEP concept to Member State authorities. This would offer the opportunity for the authorities to apply the principles of TEPs to the later stages of the current Programme and then to build future financial support for them into the process of drawing up the Operational Programmes and SPDs for the 2000-2006 period. The “deflator” (additional resources becoming available for the 1994-1999 Programmes as a result of currency movements and therefore unassigned to priorities) was regarded as a potential current source of funding for the Pacts. Aside from this, however, the resources available to install and develop Pacts de novo were extremely small by the standards of any normal policy initiative. The funds available per Pact were €200,000, rising after the launch to € 300,000 for those able to claim a second tranche. This level of resourcing offered support for little more than to facilitate the establishment of Pacts and offer small sums for experimentation. It gave scant opportunity for the TEPs to engage in projects funded under their own auspices from which direct outcomes could be measured. It is vital to understand that this had an important and highly differential bearing on what the TEPs could hope to achieve and is critical to the findings of this evaluation.

1.2 The basic TEP concept

◊ The employment context

The insertion of the word “employment” into the title of the Pacts Programme is also critical to an understanding of the way the outcomes emerged. Pacts were clearly positioned against the problem of the day for Europe - persistent and significant long- term unemployment. Pacts were a product of their time. Over time, however, the context against which they are operating has tended to change and their value is becoming more generally identified with a role as a catalyst for creative ideas of all

3 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 kinds in the process of local development. It is also the case that the places chosen to experiment with TEPs experienced employment problems differently and that these too changed over time. Some suffered long-standing endemic job shortages and general long-term underemployment or unemployment. Others had suffered a recent “employment shock” (Berlin-Neukölln (DE), for example) or from the effects of industrial restructuring. Yet others experienced not general unemployment but the exclusion of certain groups even where the labour market was buoyant (Tottenham (UK), for example). Pacts were inserted into a wide variety of employment contexts and their ability to influence outcomes was itself highly variable – a dimension we have also tried to include in this evaluation.

◊ Bottom-up approach

The Pacts were designated as ‘bottom-up’ partnerships. The idea was to encourage initiatives and schemes to address unemployment that originated at a ‘grass-roots’ level. There was, however, no particular expectation that Pacts would interact directly with local communities. A genuinely “bottom-up” partnership was conceived to be one engaging actors or intermediaries to represent the interests of a clear local constituency and able to draw upon a pool of local knowledge. These actors should be in a position to design and implement projects and initiatives that reflect the particular configuration of labour-market needs in the area. This local strategy would be recognised, understood, valued and applied by the stakeholders in the interests of the beneficiaries. What we have found, of course, is that this sort of prescription offers the widest possible range of interpretation and clearly what some would call “bottom-up” would not be recognised as such by others. Once again, the outcome for the evaluation is variety.

◊ Formal, Broad Based Partnership

Pacts were required to construct a formal, broad-based partnership. In a Pact representing the core concept, the partnership would be expected to bring together all actors with the potential to make an impact on the labour market in the area – and in the most innovative Pacts this would include some unexpected or unusual organisations. The partnership would also plan, agree and deliver the action plan. The actions agreed by the Pact would need to be translated into concrete, co-ordinated actions. The variety of forms that this can take is also wide. In some cases it is possible to identify TEPs that simply “re-badge” existing state dominated or multi- agency partnerships with little more than a nod to the Pact ideal. In others, there is the genuine de novo establishment of a new body with a range of functions new to the local level. The results of the evaluation reveal the breadth of variety that exists across the 89 Pacts between these two extremes.

◊ Co-ordination / Integration

The construction of an integrated strategy was a primary requirement. The interpretation of this can, however, vary in two highly significant ways. According to the first, it is an approach that seeks to tackle unemployment through combining elements related to the demand for, and supply of, labour. There should be a clear attempt to match supply and demand through the delivery of actions that

4 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 demonstrably bring them together. This kind of integrated strategy, for instance, would not only seek to equip the unemployed with new skills, but also ensure that the skills that they are taught reflect genuine needs in the local economy. The second interpretation concerns the co-ordination with existing programmes, initiatives and actions in the field of labour-market interventions. This would often involve finding and filling a ‘gap’ for the TEP to add value within the existing range of provision.

◊ Innovation

The requirement for a Pact to be innovative suggests that it must carry out actions and initiatives that are in some way different to mainstream provision or that clearly offer added value. Some Pacts might be able to pilot actions and test out new ways of tackling labour-market problems. Given the small resources at their disposal and their weak leverage on mainstream programmes at the outset, it is this aspect of the activities of the Pacts that turns out to be very interesting. Even with small amounts of money it was possible for some Pacts to be highly creative.

◊ Appropriate spatial scales

The original requirement for TEPs was that they should be set up at a level “above the municipality and below the region”. This instruction was a source of some difficulty when it came to the process of selecting TEPs for inclusion in the Programme. As we shall show, some Member States either ignored the requirement or managed to define it in a way that was more appropriate for them, which did not always prove to be conducive to success (Nord-Rhein-Westfalen (DE), for example, where the Pact was set up at a scale which proved too large). Across the board, the choice of the appropriate spatial scale emerged in practice to be another source of extreme variability for the evaluation to handle. As we shall go on to show, the scale at which they were in fact inserted had a great deal to do with the existing governance structure of the Member State and early prescription was subsequently overtaken by more practical considerations in the selection process.

1.3 The Selection Process

It is a particular feature of the TEP Programme that Member States had a central role in the selection process. Member States were invited to propose Pact territories to the European Commission. At the European level, although other innovative actions, such as the contemporary pilot action for long-term unemployed, normally launched calls for proposals that were put through a filtering process by the Commission, this was not the case for the TEPs. Since TEP funding was limited, as we have seen, to a relatively small amount of Technical Assistance (TA), access to the mainstream Structural Fund programmes was expected to be critical for them. Where there might be a need to reallocate funding between programmes (and not just those of European origin), it was felt that Member States should have the right to select their TEPs. Another suggested rationale behind the decision to let Member States propose TEPs was an attempt to involve Member States from the beginning, with the final objective being to mainstream the innovative actions TEPs were expected to pilot.

5 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

During the interviews carried out with EU officials in the context of the evaluation, there was a view that selection of TEPs by Member States themselves had in some cases negative implications for their quality and characteristics. The level of interest among Member States was seen as generally low and their views on the “innovative” character of the initiative varied significantly. The impact of this was that some of the proposed TEPs were perceived to be not well designed or to represent the purpose for which the programme had been designed.

All of this has considerable importance for this evaluation and the outcomes that it demonstrates. The question that arises is whether the selection procedure adopted led to supporting the creation of a group of Pacts that provides the concept of Territorial Employment Pacts with an appropriate vehicle to demonstrate its real merit as a policy instrument.

The selection procedures within Member States level fell broadly into two main categories: (1) Launch of calls for proposals and selection by a national authority responsible for the TEPs, (2) Identification of potential territories by national authority and subsequent development of proposal/action plans.

(1) In a number of countries a government department in the policy area relevant to the TEPs launched an open call for proposals. In most cases this was the Ministry of Labour. Choices were then made on the basis of a series of pre-defined criteria. For example, in Denmark, 5 TEPs out of 20 applications were selected on the basis of the coherence between their expected initiatives and funds to be used, partnership structure and links to national and local activities on the labour market.

In Sweden, bids were chosen to ensure that varying types of regional labour markets and governance were covered. In Finland, TEPs were selected by the Ministry of Labour from among 26 of the 78 applications for the national local partnership pilot programme. Selection criteria included scope of partnership and commitment of partners, innovation with regard to tackling unemployment, regional representation and attempt to create permanent jobs and prevent long-term unemployment.

In the UK, there was an initial decision not to implement TEPs on the basis that they offered nothing new. However, a change of government led to a late call for proposals. This was not the subject of an announcement in the Official Journal. Instead, the programme was assigned to the DTI, which took initial responsibility for it in the UK. Responsibility later transferred to the Department for the Environment, Transport and the Regions (DETR). Bids were to be submitted via the Government Offices for the Regions and little information appears to have been offered to inform bidders about the overall context, purpose and details of the Pact Programme. Typically, a single partner developed bids – usually the local authority – after a brief consultation with existing partners. Each Government Office submitted a single preferred bid to the DETR and the DETR then approved a shortlist of ten bids, which it submitted to the European Commission.

6 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

(2) In another group of countries, the government departments responsible for the TEP selection identified territorial areas where they felt the local labour market would benefit from TEP intervention. This was the case in Belgium-Flanders for example, where the European Employment Unit considered that a broad partnership would help tackle unemployment problems faced by factory closure in two areas. In Greece too, an inter-ministerial committee set up for this purpose selected seven areas. Each area subsequently formed a local partnership.

In France, an inventory of pre existing partnership experience was implemented during the first quarter of 1997 and a document collecting and describing fifty eight experiences was prepared by the DATAR, the authority in charge of the TEP implementation. This inventory showed a broad variety of existing partnerships and DATAR tested the interest of some of them to set up a TEP. After an iterative process ten projects were selected (from regions of varying sizes, with different types of pre- existing partnerships and levels of unemployment) and put forward and were subsequently approved.

Another interesting issue in respect of the selection process is to determine how the total number of TEPs finally chosen by the European Commission (89) was arrived at. Our understanding from discussions with EU officials is that this figure resulted from the European Commission’s post-selection of the “best” TEPs from the proposals from the Member States. We understand that no figure had been set up a priori though all Member States had to be covered.

The nature of the selection process and, in particular, both its effective devolution to Member States and the variability with which the goals of the TEP Programme appear to have been transmitted to/received by those States leaves us with a population for the evaluation that is much less coherent than would normally be the case. The variability is extreme and we have had to devise a methodological approach that helps us to deal with this.

1.4 The 89 Territorial Employment Pacts

In total, then, 89 participating areas were selected throughout the EU for partnerships to be set up. In each area, a Pact coordinator was appointed with the task of promoting and developing an integrated action plan for employment.

The Action Plan process

An Action Plan was called for in each case. This was to be a document outlining the needs of the territory that the Pact was to address, the strategic priorities and the specific actions through which these could be met. The Action Plan was to be drafted at the start of the Pact and would set out objectives and targets for the Pact to meet. The Action Plan was to be a strategic document that the partners would commit to, and it would contain plans for mobilising funding from a variety of sources (Structural Funds, public and private funds) in order to implement the measures on which the partners would have agreed. The Action Plans were submitted to the Commission for approval, at the start of the programme. Each Action Plan had to be approved by the Commission for the TEP TA funding to be made available to each Pact. In some cases, a period of intensive correspondence between the Pact and the

7 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

Commission preceded approval. The Action Plans were intended to contain targets and indicators against which the achievement of their objectives would be measured. The targets set however, did not prove appropriate or realistic in all cases.

1.5 The basic attributes of the selected TEPs

The selected TEPs by size of territory, population and number of partners

The scale at which the Pacts were intended to operate varied significantly. Using different measures of scale, the following ranges emerge:

The population covered ranged from 16,000 inhabitants (Dudelange, LUX) to over 3 million people (as in the cases of the West Midlands (UK) and Nord-Rhein-Westfalen (DE) Pacts). Similarly, the size of the territory covered ranged from Pacts tightly bounded on one small municipality to Pacts covering whole regions will have the advantage of generally low physical movement costs but at the same time low access to opportunity for levering additional resources. Larger scale will present wider opportunity but the additional costs of distance and organisation.

Table 1 Number of Inhabitants in the Number of Pacts % of Pacts in this territory covered by the Pact category 1-100,000 26 30% 100,000-300,000 34 38% 300,000-700,000 16 18% 700,000 and above 13 14% Source: EI2000 database, Pact Action Plans

The scale of the partnership, as measured by the number of partners in it, ranged from 5 to 105, implying partnerships that were tightly configured for focused delivery while others were large verging on the amorphous. In about a third of the Pacts, partnerships tended to include between 11 and 25 stakeholders. Certain trends can be noted in the Pacts of the same country as presented in the table below which shows into which category the majority of Pacts from each country fall.

Table 2 Number of partners Number of % of Pacts Majority of Pacts from this Pacts country in this group Between 2-10 18 20% Austria, Netherlands Between 11-25 33 37% UK, Sweden, Ireland, Greece, Spain, Denmark, Germany, France Between 26-50 26 29% Italy, Finland 51 and above 8 9% Portugal Source: EI2000 database, Pact Action Plans

8 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

The selected TEPs by labour market conditions at the start of the Programme

The examination of Pact starting points (see chart below) showed that there were slack labour market conditions in all areas where TEPs operated in Germany, Spain, Greece and Finland. Austria and Denmark were the only countries (apart from Luxembourg, which only had one TEP) where the majority of TEPs were in areas with tight labour market conditions.

Chart 2 Segmentation of Pacts by Labour Market conditions by country Labour Market Conditions

12 10 8 Slack 6 Tight 4 2 0 A D Dk F UK B I Ire L Nl S E El P Su

Note: Slack labour market conditions are defined here is indicating relatively high unemployment / low vacancy rates in the territory, compared to those pertaining in the country as a whole (or general region in the case of smaller countries in larger economic spheres) Tight labour market conditions defined as indicating low unemployment and high vacancy rates, often associate with shortages of skilled workers, relative to the country / region

1.6 The differential availability of resources

Each Pact was entitled to EU funding for technical assistance, up to a total of €300 000 intended to cover the development of the Action Plan and dissemination measures. The Action plans were submitted and approved by the Commission, between March and December 1998, and at the same time were submitted to the Monitoring Committees for the relevant Structural fund programmes to assess whether any funding could be secured from mainstream Structural Fund resources. The rationale was that the TEP Programme made small amounts available for Technical Assistance, to enable Pacts to plan co-ordinated actions and secure funding to implement such actions from the mainstream structural funds and other sources. In this context, the timing of the introduction of the TEPs was intended to coincide with the mid-term review of assistance for Objective 1,3,4 and 5b Programmes and the beginning of the second programming period (1997-99) for Objective 2. In reality, some Pacts were established too late to benefit from the re-programming process.

The above meant that Pacts had differing scope to influence the use of structural funds monies, which in turn implied that the amount of resources that the Pacts could play a role in channelling was widely variable. At the one extreme are some of the UK Pacts that had no more than the TEP Programme technical assistance monies at their disposal. At the other end of the spectrum are some of the East German Pacts like Zeitz. This was responsible for contributing to the decisions about directing a total of €312 million (from EU, private and national public sources) to appropriate initiatives.

9 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

The issue of the funds that Pacts had a role in appropriating is complex, depends on the delivery model chosen (co-ordination, funding of actions etc) and will be explored further in chapter 5.

The selection process that we described above served to present a population for the evaluation that sees the 89 Pacts setting out from widely different starting points. Trying to abstract from this degree of variety in the level and circumstances of the territory at which the TEPs were inserted is the subject for Section 4 of this report. It is, however, also the case that the variety in the outcomes of the TEPs can be said to be in some measure a function of the different ways in which they were handled by the Directorates General (DGs) with responsibility for managing them.

1.7 Administration of the TEP Programme

As we have indicated, Pacts were assigned to different DGs of the European Commission, usually according to the status of region where the Pact belonged. DG Agriculture managed those in Objective 5b regions. DG Regional Policy administered those in Objective 1 or 2 regions and Pacts from other territories came under the responsibility of DG Employment (under Objective 3). Each Pact was assigned to an official within the responsible DG, the desk officer (rapporteur). The role and tasks of the desk officer appear to have been quite loosely defined. The EU- level interviews and national fieldwork carried out under this evaluation indicate that this role appears to have been interpreted differently by individual desk officers. On the whole, Pacts experienced superficial contacts with their desk officers on administrative matters with only a small number of Pacts enjoying close interchange and follow up by their desk officer.

A subcontractor was engaged to set up a Technical Assistance Office (TAO) at European level (hereafter the TAO is also referred to as EI2000). The role of the Technical Assistance Office was to monitor progress of the TEPs, offer advice and guidance to Pacts through a network of national experts (an expert was assigned to each of the Pacts in every country). EI2000 was also responsible for organising dissemination activities for the Programme as a whole as well as events bringing together the Pacts for the exchange of experience.

Moreover, the TEP Programme was conceived as a pilot initiative to be implemented in line with the standard model for Community Pilot Schemes1. Pilot Schemes make funds available to projects aiming to experiment with new approaches and allow some flexibility throughout the life of projects to revise objectives and the actions required to meet them. In the event however, it appears that different Directorates General adopted different approaches in the administration of the TEP Programme with DG Agriculture attempting to follow rigid procedures in terms of how the EU funds provided for Technical Assistance were spent, contrary to the flexibility intended by designing the TEP Programme as a Pilot Action.

Overall, then, the outcome of the selection and management process was a population of wide variety, administered differentially. This variety was itself mapped onto

1 Second Interim Progress Report on the TEPs, Commission working paper, November 1999

10 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 further variety in terms of the territorial and governance contexts into which the Pacts were inserted.

11 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

CHAPTER 2: ANALYTICAL FRAMEWORK

2.1 The Focus of the Evaluation

The Terms of Reference (see Annex 1) were clear both as to the objectives and required content of the evaluation. The evaluation was to take a thematic approach to TEPs as structural policy instruments – in other words, to draw out common themes and issues rather than concentrating on the outcomes of individual TEPs; also, the focus was on the instrument itself, as a means of taking forward employment policy objectives, rather than simply focusing on the impact of the initiative in part or whole. The main purpose of the evaluation was not to be able to state how many jobs were created by TEPs for example, but whether the TEPs were a valuable instrument for structural intervention at sub-regional level.

The evaluation therefore considers the pertinence of the TEPs as a funding instrument. It is a strategic evaluation, seeking to assess the value of the policy instrument as a whole rather than focusing on the direct or indirect results of TEPs individually or in aggregate.

The variety of the Pacts that we have already demonstrated indicates that there is unlikely to be a “one-size-fits-all” recommendation for best practice - given the variety of local circumstances into which they were to be embedded. For this reason, the methodological approach allowed for some general measurement of overall outcomes but rooted this in an a priori judgement that the lessons will be contingent to the background circumstances.

The evaluation method has sought to address the following key principles:

◊ The need to achieve robust results from a highly variable baseline population at the outset: The TEPs were set up in a variety of territories with different needs with a range of economic and development problems. Pacts faced a variety of contingent circumstances in the local economy and conditions in the labour market that it made it easier or more difficult for them to make a visible contribution.

◊ The need to recognise the differential opportunities open to TEPs between Member States (and sometimes across regions within the same Member State). Pacts were to be inserted in a variety of national, regional and local government structures across the 15 Member States. At one extreme Pacts would explore the development of a new tier of local governance where little already existed. At the other extreme, they had to make the case for their existence against a well- founded pre-existing local government structure and pre-existing partnerships to a greater or lesser degree. A variety of intermediate conditions existed between these extremes.

◊ The need to identify what worked for individual TEPs and what did not: The methodology followed in this evaluation has been concerned with conceptualising ways to filter out the variation resulting from the variety of baseline situations

12 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

related to spatial-economic conditions and institutional/governance structures. The factors described above will have had a powerful impact on the operations of the Pacts and it is the lessons about how they were able to deal with these differing contextual circumstances that the evaluation focuses on.

2.2 Key evaluation questions

The evaluation therefore sought to provide answers to the following questions:

• what sorts of partnership approaches were applied in what sorts of contingent circumstances? • to what extent could these be described as genuinely “bottom-up”? • what measurable added value did the Pact partnerships contribute - to regional and local governance? - as a vehicle for employment and skills generation? • what working methods were adopted in building the Pact partnerships under differing circumstances and how successful were they? • what lessons the Pacts offer about key success factors both in general and under particular conditions? • what key lessons for overall policy development can be derived from the Pacts Programme?

The Terms of Reference were specific in the key evaluation questions that were the subject of this evaluation. These are presented in the table below, together with the report sections in which they will be addressed:

Table 4 Evaluation Question to be addressed, as expressed in the TOR Addressed in (1) the relevance of the approach Chapters 3 (establishing what approach has been followed in drawing up and 4 and implementing the Pacts, on the basis of typologies reflecting the diversity of: (a) approaches and aims; (b) the activities and measures selected; (c) intermediary bodies and final beneficiaries receiving assistance; (d) results; (e) conception, monitoring and implementation; (2) the cohesion between the objectives and the instruments Chapters 3 chosen and 4 (establishing whether the instruments chosen are consistent with and appropriate to the objectives pursued); (3) the catalytic effect and impact on mainstream assistance Chapter 5 packages (identifying the instrument’s catalytic effect, its impact on innovation in mainstream Community structural assistance in the 1994-99 period and on the consolidation of this innovation

5 Regulation (EC) No 1260/1999, OJ L161, 26.6.1999.

13 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

in the 2000-06 programming period for Objectives 1, 2 and 35); (4) the cost and effectiveness as compared with other forms of Chapter 5 assistance (measuring the cost and effectiveness of this type of method compared with mainstream Community structural assistance with regard to job creation or other objectives); (5) added value Chapter 5 (determining the added value of the approach as regards, in particular, working methods, action taken and the effects of disseminating good practice (at regional, national and national level); (6) the lessons to be learnt Chapter 6 (drawing conclusions about possible changes to innovative measures and regional development programmes at the mid- term stage (2003) and future regional policy assistance in the context of enlargement).

2.3 The evaluation methodology

Designing an appropriate methodology to recover robust results from a wide variety of circumstances has been the particular requirement of an evaluation method for the project. This section presents the rationale behind the chosen design. The aim was to factor out the baseline variety of the Pacts and the different opportunity structures they faced across the 15 EU Member states with a view to understanding how Pacts performed within their individual circumstances. The intention was to look for associations in the dataset both in terms of groups into which the Pacts may fall and the way different groups of Pacts handled the tasks confronting them. This was expected to enable the evaluation to assess the way Pacts embedded themselves into given local circumstances and how this could lead on to successful job creation and development.

2.4 Dealing with baseline variety

The 89 Pacts are spread across a geographical range from the island of Réunion in the Indian Ocean to the edge of Lapland and in size from the cities of London and Berlin to the Dudelange municipality in Luxembourg. In terms of labour market conditions, they range from metropolitan centres where jobs simply cannot be filled but certain groups of people are unable to find work to sites of industrial collapse where jobs are at a premium for almost everybody.

As a basic principle of the methodological approach, therefore, we have determined that it is essential to control for these sorts of basic variations by a process of stratification or segmentation. We have thus assigned the 89 Pacts into a limited number of groups or families that face at least broadly similar geographical, labour market and development conditions. These represent the given geographic-economic circumstances that each Pact sponsor has to face. It is simply inappropriate to run a methodology about success or failure in attempting local solutions by ignoring the differences that confront Berlin (DE) or Pays de Valois (FR).

14 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

2.5 The Evaluation framework

In devising an appropriate framework to evaluate the performance of the TEPs and taking note of the method set out in Working Paper 8 of the MEANS series, we attempted to establish a common terminology. Figure 1 sets out the basic evaluation model. For the purposes of this evaluation we believe that we can offer some insight of how to measure efficiency by looking at inputs in relation to outputs and how to measure effectiveness by examining the objectives and results. The issue of assessing relevance through needs and impact cannot be specified at the level of the regional or local economy, the available data simply do not support it, but can be dealt with in a limited way for the immediate action space of the Pact itself.

Figure 1: The MEANS Evaluation Process

Relevance Needs Utility Impacts

Effectiveness Results

Inputs Efficiency Outputs

Inputs – Outputs

In essence, the input from the Commission to achieve the outcomes it hoped to achieve for the TEPs was € 200,000-300,000 per Pact for technical assistance actions. This was added to in some cases at the national level by Structural Funds resources either from the deflator or from re-steering funds from the mid-term review of operational programmes. The total cost of the programme was of the order of € 25 millions.

From the output side, what was achieved was the creation of 89 Territorial Employment Pacts each with a set of attributes deemed beneficial for policy. Some of these attributes were pre-specified (bottom up, broad-based partnership, innovation, integrated strategy). Others are attributes added by the Pacts themselves over and above the basic parameters in order to carry out their objectives. From the perspective of the evaluation what we will seek to add in subsequent sections of the report is greater evidence for what it was that the inputs purchased in terms of what we have labelled “Pact attributes” (broad based, bottom up partnership, pursuing an integrated, innovative strategy). The real value for money of the programme can be determined in the wider context by the extent to which those sets of attributes proved to be beneficial for employment and development in the areas of operation of the 89 Pacts.

15 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

Objectives – Results

While the objectives for the Commission in launching the programme as an experimental venture can be assumed to have been to learn how such an initiative can contribute to policy development, it was not always clear to the Pacts themselves what the overall aim was for them as individual organisations. From time to time there was pressure to count the numbers of new jobs that Pacts might have had a hand in creating, but the sheer difficulty of accurate counting when Pacts were simply catalysts for others who created the jobs made this impractical. The end results that the Pacts themselves sought to achieve might well have been jobs and local development but these have to be seen as being at some remove from what the TEPs themselves would want to be measured on. In measuring the effectiveness of Pacts it is, therefore, appropriate to examine how far they achieved the objectives they set for themselves. We shall, therefore, be using the label achievements for this dimension of the study so as to be clear that these are distinct from the “results” that no doubt have accrued somewhere in the wider local economy but which are impossible to measure. In delivering these achievements the ways the Pacts were managed is an important variable and one from which we can learn a great deal. The quality and effectiveness of management is also a measure on which the best TEPs can be distinguished from the rest and we shall be in a position to report some findings on this measure.

Needs – Impacts

As we suggested earlier the programme-wide evaluation of the impact of TEPs is not within our reasonable reach. Each individual Pact, through its Action Plan, was required to “assess” the needs of its local area and to attempt to devise a set of actions that would make some meaningful (useful, relevant) impact on things within its span of control. Some of the most valuable learning from the programme is embedded in interpreting the way Pacts under different governance contexts and in different economic and geographical circumstances (family groups) understood what needed to be done (objectives) and organised themselves to tackle it (achievements).

2.6 The Stages of the Evaluation

The key points about the method are that it:

• Assembles data available about all 89 Pacts as far as this is possible, from documentary sources (action plans to final reports) • Uses country and specific Pact perspectives to supplement this, primarily from the original Pact experts and Commission desk officers • Combines this to provide an overall rating of each Pact against three key areas: Pact attributes; Pact management and partnership capacity; and identifiable achievements; and then relates these to possibly influential factors • Supplements this generalised analysis with case studies of 30 Pacts representing different conditions, enabling key issues to be explored in more depth • Uses stakeholder interviews at EU and National level to provide contextual information

16 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

This range of information is then used to examine the effectiveness of the Pact process as a policy instrument potentially capable of creating activity able to impact on employment and territorial development.

The evaluation has been structured around five stages of work. These are briefly outlined below:

Stage of the Evaluation Information sources Evaluation Analytical used Output Framework described in The Segmentation Stage The Pacts were segmented into a Pact Action Plans and Inception Section 2.4.1 number of ‘families’, based on shared external perspectives (e.g. Report, characteristics concerning the context TAO data and interviews First Interim within which the Pacts operate (spatial with the TAO national Report context and local labour market and experts) economic conditions). This enabled the evaluators to explore relevant groupings that might both help explain how well Pacts performed in different conditions and offer insights into how policies might be designed for family types.

In parallel to the above, the first phase of the evaluation involved EU-level 12 interviews were interviews which provided an conducted with overview of the TEP Pilot Action as a Commission officials whole from the viewpoint of its involved in the creation sponsors. and delivery of the Pact programme and a representative of the TAO. The Assessment Stage Subsequently and throughout the Judgements about the Second 4.2.2 evaluation, an assessment was made of success or failure of a Interim Report all Pacts in relation to their progress Pact were based on against core Programme objectives. evidence from the An indicative procedure for measuring following information the performance of the Pacts was sources: undertaken with the purpose of seeing (1) available Pact Interim what variables influence the success or and Final Reports; failure of TEPs in general and not to (2) the ECOTEC National ‘rate’ or ‘assess’ the individual TEPs. Fieldwork Reports, A set of 15 uniform criteria were especially for the case determined and Pacts were awarded a study Pacts ‘score’ for each of the criteria. (3) TAO documentation (4) telephone interviews with the TAO National Experts carried out by the core evaluation team. The Case Study Stage A network of national evaluators The team of national Country 2.4.3 carried out national fieldwork in the correspondents carried out Fieldwork 15 EU Member States. The team desk research and Reports (these investigated how the TEP Programme interviews at national reports can be operated and was perceived on the level and at the level of found in

17 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

Stage of the Evaluation Information sources Evaluation Analytical used Output Framework described in whole in each country and looked at the selected Pacts. A total Annex 3). 30 case studies in depth to examine the of 185 respondents key features, actions and outcomes of participated in the Pacts, and to consider possible causal national fieldwork, factors underlying the observed including Pact co- relationship between structure/context ordinators, a variety of and outcomes. Pact partners and respondents at national level. A detailed list of respondents is found in Annex 2. Assessment of performance by segments This phase involved further analysis of This stage involved Second segments (families and governance undertaking new analysis Interim Report types) against outcomes, relating the assessment of Pact outcomes to the ‘families’ identified in the first stage and the opportunities open to Pacts in differing national governance systems. The focus is on testing hypotheses to provide evidence of the possible relationships between features and outcomes. Final analysis and reporting This stage brings together the results Using material from all Final Report of all the stages of the evaluation to previous stages of the provide a full picture of the Pact evaluation Programme as a whole combined with in depth reporting on specific examples and lessons coming out of the evaluation.

18 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

CHAPTER 3: PACTS IN PRACTICE

This chapter answers the basic questions: • did the Commission acquire Territorial Employment Pacts possessing attributes consistent with the intentions of the programme (bottom up, broad based, integrated strategy, innovative)? • how well were Pacts organised and managed? • overall what did individual Pacts achieve in relation to the objectives of the programme?

The answers to these questions are then drawn together to arrive at a summary analysis of the overall success of individual Pacts, which will enable us to examine in later chapters possible reasons for the observed differences.

3.1 Pact attributes

The entities expected to be created under the TEPs programme were specified in relation to a number of attributes, two relating to the nature of the partnership to be mobilised (bottom up, broad based), one to the broad thrust of their intentions (implementation of an integrated strategy) and one to the need for experimentation (adopting an innovative approach). Beyond this their nature was left to the requirements of the local situation and preferences of the organisers, so for example there was considerable variation in the ways in which partnerships were formed, the degree of formality underpinning their relationship, the number and roles of staff (if any) employed by, or seconded to, the Pact organisation.

In arriving at a consistent view as to whether the intended type of Pacts were acquired it has therefore been necessary to concentrate on the four essential attributes. Our starting point here, are the results of the ‘scoring’ exercise described in chapter 2 (the Assessment Stage of the evaluation). Some 30 Pacts embodied all four attributes to a high level, and a further 25 scored higher than 75%, indicating at least partial achievement of all four. Table 3.1 shows the numbers of Pacts recorded as displaying the necessary attributes:

Table 3.1: Distribution of Pact Attributes

Attribute Fully Partially Not demonstrated No information demonstrated* demonstrated Bottom up approach 51 26 11 0 Broad based partnership 51 19 11 7 Integrated strategy 52 n/a 19 17 Innovative approach 66 n/a 17 5

* Note the distinction between fully and partially demonstrated only applies to bottom up approach and broad based partnership

These results show therefore that the great majority of Pacts (62%) conformed to the basic attribute structure required of them by the Programme. A bottom up approach is shown here to be the most common core attribute, with only 11 not demonstrating this in their work and 77 at least partially providing evidence. This is closely followed by

19 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 a broad based partnership, with 70 Pacts providing evidence to demonstrate at least partial achievement of this requirement. An innovative approach in the context of the Pact come next, with 66 Pacts providing evidence, whilst only 52 were able to demonstrate creation of an integrated strategy. This last figure should be seen in the context of no information from 17 Pacts, but also 19 being unable when questioned to provide evidence that this was achieved.

The following sections describe the attributes and examples of them in practice in more detail.

3.1.1 Bottom up Partnerships The Pacts were supposed to be ‘bottom-up’ partnerships. The idea was to encourage initiatives and schemes to address unemployment that originated at a ‘grass-roots’ level. This was based on the view that local understanding and untapped local creativity could provide solutions more appropriate for the local situation, and more likely to be successful. Whether a TEP was bottom up or not was largely determined by the composition of its partnership, the interests represented by members, and the extent to which they themselves possessed quasi-democratic legitimacy from their constituencies. There was no particular expectation that Pacts would interact directly with local communities. A genuinely bottom-up partnership would be one engaging actors which genuinely represented the interests of a clear local constituency. It would be able to draw upon a wealth of local knowledge and would be in a position to design and implement projects and initiatives that reflected the particular configuration of labour-market needs in the area. This local strategy would be recognised, understood, valued and applied by the stakeholders. Moreover, the involvement of the local actors would have to be ‘real’ and sustained – that is, the partners’ presence would be more than merely token.

Fifty seven per cent of the Pacts successfully managed to construct a bottom-up partnership. A further 30% of Pacts only partially achieved a bottom up partnership, while a further 12% were not at all successful in ensuring a bottom-up approach.

Two broad models of active development of a bottom up approach can be identified. The first were Pacts which involved local partners in the development of the action plan.

Examples: Involvement of Partners in developing the Action Plan The Berlin-Neukölln Pact (DE) ensured that it was bottom up by systematically involving all of the local partners in the elaboration of the TEP’s action plan. The six main priorities around which the action plan was initially built were elaborated in ‘brainstorming’ sessions involving the local partners, facilitated by an independent company briefed to ensure full participation and consensus. A diversity of local partners were involved, including local SMEs (rather than representative organisations), large companies based in the area, a housing association, and the district administration. The actors were then assigned specific responsibilities for development and delivery of the individual activities in the plan.

In some TEPs, the definition of the action plan followed a bottom-up approach by building upon existing local resources and initiatives. In the Voralberg Pact (AUS), the definition of the action plan built upon a "social partners summit meeting" held in December 1996, which had already defined a strategy for combating long-term unemployment in the locality. The

20 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 public labour market service in Vorarlberg (AMS Vorarlberg), with the consent of the partners, used this as the basis for the successful TEP application. During the implementation phase of the Pact, all initiatives and projects were developed by and agreed with all partners of the Pact partnership.

The second model involved Pacts which sought to involve partners in specific local actions, often based on the perceived benefits or expertise they could bring, or their interest in a specific theme or strand of activities.

Examples: Involvement of Partners in specific actions through setting up ad hoc sub-groups The Comité de Bassin d’emploi, synonymous with the Pact in Albertville (FR), adopted an approach by which it created a specific task group for each issue it identified as needing action. These brought in a diverse range of local actors, some becoming involved in a number of groups, others relating to just one. One group dealt with proximity services, and involved 40 institutions in this sector. The involvement of grass-roots organisations and local actors was related to specific actions and projects and not to the development of the action plan. “It was a compromise between the necessity to define a comprehensive Action Plan, which could only be implemented at the Comité de Bassin level, and effective projects, which required significant local involvement.” (Pact organiser). When successful this approach brought in a much wider and more diverse range of interest groups than could practically have been involved in creating an overall strategic plan, although in this case at the expense of providing little opportunity for debate as to the overall direction of the Pact.

A more integrated approach using elements of bottom up involvement in both strategic planning and direct action can be seen in the Bruxelles-Capitale Pact (BE). An agreed limit was set on the number and range of partners to be included in the formal partnership, in order to ensure an efficient and rapid decision-making processes. This was considered to be of great importance in avoiding paralysis, since it was also decided that all decisions should be based on consensus. Partners in this group represented a limited set of public, private and third sector stakeholders, mostly bodies operating at the scale of the Pact territory as a whole, but representing de minimus a diversity of interests. Any deficit in bottom up legitimacy was tackled by bringing together stakeholders from outside the formal partnership, including individuals representing target groups, to discuss specific projects. The Pact was based around six axes (or central themes), for each of which a consultative ‘thematic group’ was convened. These discussed possible projects and followed them up. All possible stakeholders were invited to the thematic groups to ensure a broader participation than was provided by the Pact membership itself. Through collaboration with the target groups - such as SMEs, shopkeepers, and representatives of immigrant communities - a “down-to-earth” approach was provided. These thematic groups involved such things as listening to the past experiences of streetworkers or to the opinions of individual employers. High levels of commitment from local employers and the population were reported by the Pact and attributed to this approach.

Similar approaches were adopted by the Halle-Vilvoorde (BE) and the Norte Alentejo (PT) Pacts.

Elsewhere, a more limited and less effective approach to the bottom up approach was observed. Pacts in the UK for example were reported as largely driven forward by Local Authorities even though they were formally overseen by partnerships operating at the local or sub-regional level. They were the key drivers, setting the agenda and defining the parameters within which the Pact was expected to operate. In these cases locally conceived ideas were encouraged and developed into actions, but within the context of a disconnected quasi consultative process, and a concentration of

21 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 management and responsibility in the Local Authority. Where the community sector was involved, this was mostly through representative organisations rather than through actual practitioners. Key target groups (and to a lesser extent small businesses) were rarely involved in the management and were involved only indirectly, through project activity. The implication of this is that, although the activities were conceived at a local level, the ‘local level’ was in practice in the UK largely synonymous with the ‘local authorities’.

To summarise, there was predictable variability in the extent to which Pacts could lay claim to being genuinely bottom up. In the main most at least made attempts to respond constructively to this requirement, and it is clear that two models of involvement emerged, one which sought to involve the grass roots perspective in strategic action planning, the other which injected this perspective into more operational areas. Whilst a few combined both approaches, most did not. The likelihood of the one or other was strongly associated with the size of the partnership, as discussed in the next section.

3.1.2 Formal, broad-based partnership There are several aspects to the idea of a formal, broad-based partnership. In a Pact representing the core concept, the partnership would bring together all actors in the area with the potential to make an impact on the labour market, and would be expected to include, as a minimum, employers, local authorities and organised labour. In the more innovative Pacts this included some unexpected or unusual organisations. For example, the Calatino Sud Simeto Pact (IT) included the National Confederation of Farmers and the Association of Olive Producers in its membership. The Albertville Pact (FR) included local education institutions, unusual in the French context. A successful partnership would also plan, agree and deliver the action plan, and would contain few if any passive members not fully committed to the success of the TEP.

Fifty seven per cent of TEPs (51) successfully managed to construct an ‘appropriate’ partnership, involving all the expected stakeholder groups, while for a further 21% of Pacts (19) the achievement of this was at least partial – therefore nearly three quarters of Pacts can be defined as successfully achieving some or all of this requirement of the model. Unsurprisingly there is a strong correlation between these and the group scoring highly on grass roots involvement – 37 scored top marks on both, 22 top marks on one and partial scores on the other. An appropriately broad based partnership would by definition need to include representation of the bottom up point of view (although it would also need to include other stakeholders too).

There was little connection of success in achievement of this criterion with Member State, most countries containing both good and less good examples. Those with existing traditions of partnership most often used existing networks and ‘grafted’ the Pact onto their existing structures (for example Ireland and the UK), whilst those with little partnership tradition such as Portugal created new ones.

22 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

Good examples of broad based partnerships include:

• Ceuta (ES), where the TEP mobilised ‘a genuinely broad-based partnership which functioned with agility and consensus and was the place of discussion for all decisions’2. Its membership included: 1. Local government: Regional Department for Economy and Employment; 2. Government bodies: Regional Directorate of INEM (National Employment Institute; Regional Directorate of Employment); 3. Private sector: Business Confederation of Ceuta; Chamber of Commerce and Industry of Ceuta; 4. Trade Unions: UGT; CC.OO; 5. Other: Regional Federation of Neighbourhood Associations.

• Bruxelles-Capitale Pact (BE), which mobilised all the social partners and a wide range of institutional representatives. The Pact involved 150 people in reflection, decision-making and actions, and acted as a forum to bring together representatives of the regional authorities and the project promoters.

Useful evidence of the importance of private sector involvement to effective partnership was found. For several Pacts (e.g. Berlin-Neukölln (DE) and St Herblain (FR)), the involvement of the private sector was held to be a significant advantage in their effectiveness in articulating labour supply and demand. Conversely, for several Pacts the absence of private-sector involvement was recognised as a weakness. For example:

• The Albertville Pact (FR) had a problem with the “low level” of private sector participation, with no involvement from significant medium-sized businesses. Their interests were not felt to coincide with those of a geographically defined territory. They did “not want to take responsibility for a process for which they do not feel responsible.” and as a result the Pact had little ability to innovate in articulating supply and demand.3

• The Hérault Pact (FR) had a similar problem. The Pact, being a political institution, had mainly administrative partners: the region, the State, State agencies, and the Chambers of Commerce – there were no private sector enterprises and only limited involvement of social economy partners. Its large scale was also cited as an inhibiting factor on the breadth of its base since it only felt able to involve a limited number of partners, whilst a much larger number could have been involved.

In sum, a majority of Pacts achieved an appropriate partnership membership, in a variety of circumstances. Many reported this to have had a positive impact on their ability to create the sort of integrated strategies discussed next.

2 Case study report 3 Project quote from case study report

23 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

3.1.3 Integrated Strategy Of the 89 Pacts, nearly 60% (52) were assessed as having created integrated strategies consistent with the objectives of the programme. For 20% of Pacts this attribute was not achieved, while for a further 19% (17) there was not enough evidence to state whether they had developed an integrated strategy or not.

Integrated strategies can be modelled in one of two ways. The first approach tackled unemployment through actions that demonstrably matched labour supply and demand. These sought to equip the unemployed with new skills, but also to ensure that the skills taught reflected genuine needs in the local economy. In the best examples this went further and involved additional measures to facilitate the process. An example of this, albeit on a small scale, was the work of the Tottenham (UK) Pact which matched skills’ training for unemployed people in a deprived area to employment opportunities at an airport. A key element of this was working with public transport providers to make transport to the airport available and accessible.

The Iisalmi Pact (SF) contained projects to promote training and skills in a way tailored to match labour supply and demand. In 2002, the regional vocational training institute was reported to have succeeded in achieving a ‘complete’ match between the two elements, by providing work placement training geared to meeting the specific skill needs of large industrial companies in the region.

However, not all Pacts were so successful in achieving an integrated strategy. For example the South Tyneside Pact (UK) attempted to carry out activities linked to both labour supply and demand, but the degree of integration is reported as weak. Most project work operated independently, largely as a result of being delivered by individual partners rather than by the partnership together. Some partners described the Pact as just a ‘list of projects’.

The second model of an integrated strategy involved filling a ‘gap’ in the existing range of provision, where a Pact’s actions would seek to complement existing initiatives. The main focus in these cases was to address unmet needs and fill gaps in existing local policies, meaning some strategies were skewed either to labour supply- or demand-oriented activities. This was a particular feature of Pacts in situations where similar provision or partnerships were already in existence. By taking a more narrow focus, some TEPs found that they were able to "fill a gap" in the existing range of provision, without placing the Pact in direct competition with existing initiatives. For instance for the Viborg Pact (DK), the strategy was considered to be integrated since the initiatives that it contained - which focussed on marginalised groups in the labour market – were seen as a complement to already existing policies and programmes.

3.1.4 An innovative approach For a Pact to be viewed as innovative, it must be seen to have carried out actions and initiatives that are in some way different to mainstream provision. For instance, many Pacts sought to pilot actions and test out new ways of tackling labour-market problems. Of the 89 Pacts, nearly three-quarters (66 TEPs) can be considered to have been innovative in some way.

24 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

This figure does however include Pacts for which the simple process of working in partnership was innovative. For the purposes of assessing whether the entity supported by the Programme was configured as a Pact as intended to be, innovation of this nature has been included. In a broader sense this does not cover more thoroughgoing innovation, as represented by examples such as the Hérault Pact (FR), whose main benefit was its stimulation of innovative projects. For example, the Pact helped to set up ‘Pole Employment – Services’ which created jobs in the service sector by bringing together the producers of services (such as NGOs and companies) and consumers. It also established itself as an ‘investment bank’ for innovative project proposals, providing both financial and non-financial support, as well as access to the département through its membership of the partnership.

In the UK, the TEP Programme was seen ‘as an R&D function in the classic sense, where new ideas are born and project tested’4. However, their capacity to turn innovative ideas into practice was limited by the lack of funding available to deliver projects. New projects were either very small (if funded from the TEP Programme funding) or restricted in their scope for innovation (if funded from mainstream programmes, with other priorities).

Some Pacts demonstrated a more strategic innovative potential by developing new opportunities for policy responses. This approach appears to have been more realistic for some than undertaking direct project work, given the low budgets at their disposal. For example the Wien Pact (AUS) was innovative in drafting and implementing tailor-made measures for specific target groups. A special concern of this TEP was focussed on the integration and upward mobility of women, and explicitly targeted female migrants – measures absent from policy and practice until that point.

Another example of TEP innovation concerns the opportunities the Pact approach offers for transference of experience and dissemination of best practice from other Member States. The Berlin-Neukölln Pact (DE) explicitly made this one of the main themes of its strategy. The Pact’s ‘campaign No. 2’ aimed to make use of ideas coming from other European regions and apply them for the first time to a German context. For instance, this ‘campaign’ led to the setting up of an action called ‘Neue GründerZEIT’, where young people between 18 and 35 were encouraged to set up their own businesses. This scheme explicitly followed a model originally developed in the UK.

3.1.5 Attribute Summary We have dwelt on the issue of attributes at some length since arriving at a view as to whether the basic underlying concept was achieved in practice is crucial to our ability to report on the effectiveness of the Pacts programme as a policy instrument. The conclusions of this examination are that:

• some 62% (55) of Pacts were either fully consistent with the model, or were close to it.

4 overall evaluation of the British TEPs by the EI2000 National Expert

25 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

• a bottom up approach, and a broad based partnership, were strongly associated one with another, and were also both the most frequently achieved characteristic • integration of strategy was attempted and to a degree achieved by most, but this meant different things in different places and was influenced by the prevailing policy and practice framework in the territory • most attempted to be innovative, but what counts as innovation varied considerably, and to some extent there were limited expectations beyond the novel experience of partnership working in countries with little tradition of this approach • for the purposes of the evaluation we can proceed on the basis that sufficient ‘genuine’ Pacts were created us to be able to move on to consider the performance of the Programme as a policy instrument, in the knowledge that the sort of entities envisaged were indeed created.

Having established this, we now turn to consider the extent to which Pacts were competently and consistently managed.

3.2 Management quality

3.2.1 Aspects of management The ‘quality’ of Pact management is likely to have had a bearing on the performance of Pacts irrespective of other contextual factors. A Pact established in the most favourable of circumstances may still perform poorly if its management and organisation was not up to the job; conversely, skilled and committed management might succeed in delivering good results even if the Pact had other external problems. In determining the overall success of Pacts therefore it is important to consider management quality, firstly in its own right, and secondly in relation to its impact upon Pact achievements. This analysis is based on an examination of the Pacts themselves, in particular the roles carried out by those specifically assigned to the task of managing and co-ordinating the Pact, such as the Pact Co-ordinator appointed by most Pacts. Four primary factors have been used to assess management quality across the generality of Pacts: • the appropriateness of management arrangements and delivery for the Pact’s objectives and structure • the degree of focus and clarity achieved by the Pact in its mission and objectives • the stability of key personnel and their roles within the Pact • whether the Pact successfully applied for the second tranche of funding

Table 3.2 shows the number of Pacts demonstrating these management qualities:

Table 3.2: Distribution of management qualities Management quality Demonstrated Not demonstrated No information Appropriate management 41 25 22 Focus and clarity 53 23 11 Management stability 27 21 40 Second tranche of TA 72 14 3

26 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

Judgements have been reached concerning each of these factors for each Pact, which have been fed into the overall scoring process. In addition, the analysis takes into account the more detailed information available from the case studies and other qualitative interviews.

In comparison to the attribute scores, management of Pacts by these figures shows more weaknesses and variability. Most Pacts, some 60%, had objectives clearly stated, and appropriate to the intentions of the programme, but for a quarter the information provided showed there to be lack of clarity or other problems in the project planning. Less than half were able to demonstrate fully appropriate management structures for the scale and intentions of the Pact, in most cases arising from lack of resources, whilst only 27 had demonstrable stability in management (although no judgement could be made for 40). Drawing down the second tranche of funding was more successful however, with 72 having done this, with most of those which did not, being DG Agriculture Pacts and Greek Pacts which were not given the opportunity to do so.

Even allowing for the lack of information for many Pacts, these figures indicate there are likely to have been managerial and process weaknesses in a substantial sub set of Pacts, which will have had a bearing on their likelihood of achieving positive results. The rest of this sub section discusses some of the reported management issues for Pacts, and their consequences.

3.2.2 Individual qualities Good management is often associated with the competence of a key individual in the management team of the Pact: ‘…the success (of the Pact) depends…on the personality of the co-ordinator of the partnership, a veritable conductor who carries out the monitoring and furthers the progress of the Pact’.5

The Bruxelles -capitale (BE) Pact was said to have particularly benefited from the presence of a good Pact Co-ordinator in complex partnership environment.

On the other hand, the weakness of some TEPs was linked to the inexperience of the management team. For example the management of the Pays de Valois Pact (FR) was said to have lacked technical expertise; the absence of more knowledge of EU Structural Funds was judged to be a weakness for the Ussel Pact (FR); Nord Est Napoli (IT) saw ‘a total lack of preparation of the territory – in terms of the professionalism of the human resources’6.

3.2.3 Structures For some of the other Pacts that encountered managerial weakness, the problems were of a more structural nature. Castilla y Leon (ES) developed a management structure that was ‘bureaucratic’, leading to a degree of ‘slowness’ that had ‘negative

5 Pact response to EI2000 questionnaire 6 Ibid

27 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 repercussions on the realisation of the actions’7. The EI2000 National Expert for Greece was extremely critical of the management of most of the Greek Pacts. Management then requires not simply technical ability, but also a degree of experience and capacity concerning the particular objectives and modus operandi of Pact like structures.

Pacts benefited from continuity in the management team. Many Pacts identified instability in the management team as a disadvantage. Instability of management personnel, particularly for the key position of Pact Co-ordinator, was a major weakness for some. 24% of the Pacts (21) were judged to have suffered from instability in the management team. For example the Hainault Pact (FR) experienced three changes in the technical co-ordinator. This weighed heavily on the evolution of the Pact and created problems both for its future development and for its financial and administrative monitoring.

For some Pacts, weakness in management is attributable to the lack of resources. This could be any combination of task overload and/or insufficient manpower. For example lack of human resources was said to have prevented Ita Lappi (SF) from influencing work at a grass-roots level. The geographical area covered by the Pact was very large relative to the limitations of staff and economic resources. The Albertville Pact (FR) reported that the two people with full-time jobs (Pact Co- ordinator and one secretary) was not sufficient to carry out all the Pact’s activities, particularly when it involved co-ordinating different groups brought together in a single project. The high level of expectations on some Pacts was not consistent with the relatively low level of funding available and management capacity this was able to buy.

Overall Pacts benefited from the presence of staff who were technically experienced in partnership working, strategic planning, and the mobilisation of diverse funding resources. The existence of a strong, committed and active partnership could help to overcome, or prevent the emergence of, difficulties in this area, although the usefulness of this depended on the relationship between the management team and the other parties. Herault (FR) is a good example of a well supported Pact, with a supportive base in the Departmental administration, and clearly committed partners.

Management efficiency was also about an ability to adapt strategies to the context the Pact was facing. The Iisalmi Pact (SF) was confronted with an area whose problems reflect those experienced by similar areas located on Europe’s periphery: falling numbers of businesses as well as the migration of young people to cities and to other countries. Despite an inability to impact on the overall unemployment rate, it demonstrably achieved a number of positive and innovative results, attributed in the case study report to “high management efficiency in extremely unfavourable circumstances”.

3.2.4 Planning The type and ambition of goals and objectives established by Pacts shows considerable variability. This arises from two factors – the lack of clear specification

7 Ibid

28 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 by the Commission as to the criteria by which the achievements of Pacts might be judged; combined with the substantial diversity of contexts, scales and partnership types experienced by individual Pacts. In this context the appropriateness of objectives set can be seen as stemming from the experience of the management set up. With external factors expected to have an important influence on the labour market in any given locality, a degree of realism might be expected in the goals established by Pacts. There would be little point in Pacts setting themselves a target of reducing the rate of unemployment in their area, or of creating substantial numbers of jobs, when they did not have the resource, time or leverage to do so. For example the Zeitz Pact (DE) set itself the over- ambitious goal of reducing the rate of unemployment in the area by half in five years. This was not achieved– which attracted some adverse publicity in the media – but this should not be considered a complete failure, as the problem stemmed from setting unrealistic objectives. No Pact could hope by itself to have such a dramatic effect on the labour market in its area – but it could help to facilitate developments that might contribute to the achievement of such an objective.

3.2.5 Management summary Taken altogether, the evidence provided shows the importance of good and stable management for the work of Pacts. Taking the second tranche of funding out of the equation, given that its receipt (when it was available) seems to have been largely automatic, fewer than half of Pacts were able to show all the management qualities which might be seen as required for successful operation. There is good correlation between the scores achieved and qualitative information from wider sources. We are confident therefore that the relative management scores ascribed to Pacts accurately portrays their overall management quality.

We have so far established that a majority of Pacts were close enough to the overall model to be considered examples of the intended organisational form sought by the Commission. We have also concluded that the quality of management was an important factor in Pacts’ abilities to work effectively, and that around half had management weaknesses that will have impacted on their work. If plotted one against the other as shown in Figure 3.1 it can also be shown that there is a degree of association between these factors, i.e. those weak in management quality are more likely to score low on Pact attributes. The correlation is by not means complete, however it is strong enough for us to conclude that this is a reasonable conclusion to draw.

29 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

Figure 3.1: Association of management and attributes

R2 = 0.3346 100.00% 90.00%

80.00% 70.00%

60.00% 50.00% 40.00% Management 30.00% 20.00%

10.00% 0.00% 0.00% 20.00% 40.00% 60.00% 80.00% 100.00 % Attributes

3.3 Pact Achievements

3.3.1 Achievements overall We will now turn to consideration of the achievements of Pacts, and discuss what can be said with confidence about the results of their work. The achievements of Pacts can be seen in a number of different ways. It is important to emphasise that the Pacts were not expected to deliver and record results in a way that conformed to any specific, prescribed format. By their very nature they were expected to take action in a wide variety of different ways, and were not charged with the task of creating specific, output focused ‘projects’, which are much more the normal currency of EU supported interventions. Therefore, there is an enormous amount of variety in the achievements and results that emanate from the Pacts. While some of the Pacts stress their achievements in concrete terms of creating employment, the claimed achievements of other Pacts are rather less tangible even if potentially no less valuable.

30 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

The scoring process rated Pacts in relation to six key areas, and table 3.4 shows the numbers demonstrating evidenced achievement in relation to these.

Table 3.4: Distribution of Pact achievements

Area of achievement Fully Partially Not No information demonstrated* demonstrated demonstrated Pact achieved self defined 39 34 14 1 objectives towards employment development Pact achieved demonstrably 69 n/a 10 9 improved co-ordination and co-operation Pact levered8 additional EU 61 n/a 14 12 resources Pact levered additional Public 74 n/a 7 7 funds Pact levered additional Private 56 n/a 21 11 funds Pact learning mainstreamed or 30 37 17 4 replicated

3.3.2 Labour market achievement in general

Whilst they do not capture the richness and diversity of Pact activities, these figures give a good summary indication of the achievements of Pacts across a range of areas, as perceived by Pacts themselves. 39 demonstrated full achievement of their objectives in relation to the programmes’ employment related purpose, whilst 34 partially demonstrated this, and 14 did not achieve their objectives. The diversity of objectives and activities will be described below, and it is difficult to draw clear conclusions as to the areas which were more or less successful. Evidence for the achievement of objectives comes from the variety sources available, but is in many cases based on qualitative data and the triangulated opinions of observers rather than the provision of hard data. Even those Pacts which supplied numerical information, for example in relation to job creation, usually qualified this by saying it did not provide an accurate picture of the value of their achievements. The most commonly cited achievements tended to be in relation to the mobilisation of actors, increased engagement and co-ordination of key stakeholders, and successful encouragement of employment related planning and actions amongst partners. The definition of what success means in these circumstances is problematic, and can often only really be seen in the description of project activity. Nevertheless, some more specific achievements can be described.

8 Levered in this case means either attained control over existing resources for employment / development purposes, or secured additional resources for the territory

31 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

3.3.3 Employment creation The TEP Programme was of course at root designed to promote employment creation. Given the nature of the programme’s objectives and the obligations put upon Pacts it would clearly not be expected that individual Pacts, nor the programme as a whole, would directly create significant numbers of jobs. Its employment impact would rather be achieved through the co-ordination of activities, stimulation of idea generation, dissemination of planning information etc. Most Pacts understood this well, and in the main emphasised their indirect achievements. Several Pacts also highlighted the difficulty of isolating their effects in generating employment from other relevant developments in the area, such as the state of the macro-economy. There is a degree of confusion as to the extent to which the Commission expected and required Pacts to deliver substantive direct employment outputs, such as job numbers. Whilst this is not implicit in the documents establishing the programme and only touched on lightly in the application materials, some Pacts asserted that they felt under pressure to deliver such direct outputs and were asked to do so by Commission desk officers.

Direct The Pacts which sought to emphasise their achievements in terms of job creation tended to be those that carried out specific activities with their own sources of funding (in all cases involving funds ‘levered’ into the Pact’s sphere of influence – no Pacts solely used their Pact funds for direct project work). This is seen very clearly for the St Herblain Pact (FR). The Case Study reports the number of jobs claimed as being directly created as a result of the Pact, along with the specific sub-project responsible, as follows: ◊ Development of a project approach that sought to optimise the creation of new SMEs: 133 jobs (instead of 40 expected) ◊ Making available new jobs: 138 jobs (instead of 70 expected) ◊ Organisation of new types of labour contract: 11 jobs (instead of 40 expected) ◊ Development of qualifications and employability: 22 jobs

The Pact also reported that the rate of unemployment in the area was reduced by approximately two percentage points during the course of the Pact’s life, but pointed out the difficulty of distinguishing the effect of the Pact from the general improvement in the employment situation resulting from France’s improving macro- economic climate. Similarly, whilst the Berlin-Neukölln Pact (DE) was able to create 2000 new jobs and places for vocational training, it acknowledged that it was unable to affect the overall rate of unemployment in the area. The Pact was hindered by the continuing weakness of the macro-economy in the former German Democratic Republic.

The Dudelange Pact (LUX) cited as its most important achievement the creation of a Service Emploi, which created a new local process for the individual guidance of job seekers and sought to match up the skills of jobseekers with the skills needs of employers. This can be considered a direct influence on job creation since the Service was formed and managed by the Pact, and interacted directly with unemployed beneficiaries. The Service Emploi helped 425 people between 1997 and 1999. The Pact made the point however that it felt it inappropriate to measure the success or failure of the Pact in crude numerical terms (and felt it was under pressure from the

32 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

Commission so to do). Given low local unemployment its emphasis was not on creating large numbers of jobs but rather on integrating the relatively small numbers of socially excluded people into the labour market.

The Imathia Pact (EL) created 99 jobs out of 123 it had planned, these being in the provision of social services to the elderly, disabled people and childcare. This Pact also ran training courses for unemployed people designed specifically to provide skills demanded in the local labour market, and claimed a 100% success rate in securing work for the 60 people trained by the project. In this example the Pact showed awareness of the wider implications of its work – the training was designed as a demonstration of the an alternative approach to the usual short term method, criticised and unpopular in the locality. The project’s emphasis on training geared to local labour market needs was regarded as highly innovative, although has yet to be mainstreamed. It is instructive to note the emphasis placed by the project on recording and reporting numerical job outcomes when its primary objective was attitude change – a reflection of the absence of clear indicators for indirect achievements.

Indirect Whilst these, and other, Pacts were able to provide direct evidence of employment impact, the last example emphasises the importance of considering their wider impacts. Pacts were not created to give individual beneficiaries employment – the billions of Euros passed through the Structural Funds, let alone national programmes, dwarf any impact individual Pacts or the programme as a whole might have. The question is more one of the extent to which the activities of Pacts have succeeded in demonstrating the effectiveness of different approaches to employment at a territorial scale, which have the potential for replicability elsewhere. It might be argued that providing job numbers or other numerical data is important to the credibility of models claimed as having wider relevance, and evaluations of other employment programmes have often struggled to give an accurate quantitative picture of the sort of results which might be expected from the implementation of new models in different circumstances. In the case of the Pacts however the value of the job creation data is fundamentally flawed for two reasons. Firstly, none of the Pacts, even in the context of in depth case studies designed to examine the dynamic of inputs and outputs, was able to detail the true ‘cost benefit’ of the projects in operation, nor provide a workable formula for their implementation in other circumstances. Project promoters are used to the demands of European and national programmes for basic throughput data, and the Pacts providing such information had not sought to go beyond the usual reporting requirements. The available data is therefore of passing interest, but offers little for comparative analysis.

Secondly, the Pacts which directly ran local projects were undertaking activities not strictly compatible with the purpose of the programme in the first place. TEPs were not intended to run local projects, they were supposed to operate at a higher or different level, co-ordinating, planning, informing, disseminating. We would hesitate to criticise individual Pacts too strongly for focusing on project delivery. Many poorly understood the purpose and intentions of the programme, and Member States were responsible for their selection in the first place. In some cases Pacts ended up running local project activity for want of anything more constructive to do, in the face for example of a congested local context which saw little value in the creation of a new partnership, Tottenham (UK) for example. In other cases, such as Dublin (IRL) the

33 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

Pact had a role in the generation of project ideas, which it then helped put into place and operate. Idea generation and stimulation of new activity does fall within the Pact remit, even if in this case it also led on to direct involvement in delivery. Despite these explanatory factors however we are obliged to conclude that the evidence presented concerning the statistics of employment creation offers little for evaluation of the TEP programme as a policy instrument apart from highlighting some of the places where it went astray.

The Hérault Pact (FR) discussed its indirect effect on the labour market in relation to its role as a ‘bank for projects’ related to unemployment problems, which attracted many clients in search of technical assistance and funding. The Pact funded fifty projects out of four hundred ideas presented, and initiated intermediary steps for a further 150. Clearly amongst a portfolio of this size there are likely to have been varying degrees of success and impact on the labour market, which the Pact will have to some extent influenced. However, its role in this could only be estimated, the fact that it brought one in eight new ideas into existence and assisted the development of a further three in eight is the limit to what can be directly attributed.

An example of indirect employment effects through business support is provided by the Agro Nocerino Sarnese Pact (It), which sought to stimulate the creation of new businesses in the area. Financial assistance provided by the Pact contributed to the stimulation of “entrepreneurial activities” by co-financing structural enlargement and financing the creation of new companies (profit and non-profit making). With the help of ERDF funding that was levered in: • 4 handcraft shops were enlarged; • 7 manufacturing units were refurbished, 23 were enlarged and 15 were created; • 1 non-profit making enterprise was enlarged and 4 were created; and • 2 cooperatives were enlarged and 1 created.

The Pact has met the objectives it aimed to achieve. It also contributed to the creation of AgroInvest in 1999, at the suggestion of the Pact Meeting Board. AgroInvest is a society for urban regeneration, whose activities foresee, amongst other things, the creation of two new Industrial Zones.

3.3.4 Partnership activity Sixty nine Pacts demonstrated successful increases in co-ordination of and co- operation amongst partners, a high figure which reflects the emphasis of the programme’s intentions, and the relative ease of making progress in this area if stakeholders can be persuaded to come together. As shown in Table 3.5, by the end of the period 50 Pacts were effectively new partnerships delivering new functions, and 16 old partnerships carrying out new functions – which can crudely be classified as additionality. At the beginning of the TEP Programme, only 31% of Pacts (28) were judged to involve new partnerships performing new functions. This is indicative of the dynamic nature of the TEP Programme. Many of the TEPs started out, for example, as old, established partnerships performing old functions. However, by the end of the Programme, these had changed both in their membership (becoming effectively new partnerships by taking in new partners), and in the functions that they carried out, and is consistent with the objectives of the programme as a whole

34 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

. At the end of the Programme, only a small proportion of Pacts were judged to be old partnerships performing an old function (12, or 13%) In areas with many existing local-level employment initiatives and partnerships, these Pacts had to invent new ways to add value, or simply ended up as ‘re-branding’ exercises of existing initiatives or partnerships. This implies a deadweight effect caused by giving money to undertake activities which were already being done. A further eight were new partnerships carrying out old functions – effectively displacement of existing activity, and of questionable added value.

Table 3.5: Partnership functions

At end of period, Pact was New partnership delivering new functions 50 Old partnership delivering new functions 16 New partnership delivering old functions 8 Old partnership delivering old functions 12

In the case of the St Herblain Pact (FR), the idea of partnership working was not new. However, the creation of the Pact presented an opportunity to “solidify and extend” the existing partnership to encompassed a wider range of businesses and new social economy organisations, in addition to the existing local authorities dominated membership. The Technical Assistance was used specifically to consolidate the process of partnership building. The Pact had a similar effect on the partnerships that pre existed in Hérault (FR). These were not broad based and in the main embraced public-sector institutions. The Pact was able to develop wider understanding of the value of extended partnership for the effectiveness of actions designed to combat unemployment.

Clearly the partnership tradition and entities existing in the locality would be significant factor influencing the scope that Pacts to add value in terms to partnership capacity. In areas with a multitude of pre-existing partnerships and other agencies involved in employment policies at local level, Pacts had little to offer in terms of institution building. The Tottenham Pact (UK) for example was established alongside a very broad overarching partnership in place.

In areas that lacked a tradition of partnership-oriented co-operation in the field of labour-market interventions, the creation of the Pacts had the potential to contribute substantially to the development of partnership capacity and new ways of addressing problems in the local labour markets. The Vale do Sousa Pact (PT), is an example of this through creation of a new structure to perform a co-ordination function that was not until then happening.

In some areas where the existing institutional structure was already in development, Pacts were a timely addition to a general wave of recognition of the value of Pact-type initiatives and devolution, and succeeded in building upon this trend. For example, the Limerick Pact (IRL) was grafted onto the existing Paul Partnership, which was subsequently expanded and consolidated.

35 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

This brings us to the issue of improved co-ordination and co-operation amongst partnerships. 69 Pacts (77%) reported clear evidence of improvements in the co- ordination of organisations and partnerships, whilst 10 (11%) indicated that this had not occurred (information is lacking for the rest). Diverse evidence for the advantages accruing from increased co-operation between local stakeholders have is available from a wide range of Pacts, for example:

• The Bremen Pact (DE) led to increased trust among the various partners, facilitating joint activities and co-ordinated planning. The Pact cited this as a concrete achievement in its own right.

• The Halle/Vilvoorde Pact (BE) likewise created ‘a permanent and balanced dialogue between the partners’.

In some instances, the TEP assumed the role of co-ordinating a broad range of labour-market related initiatives and actions. For example:

• The Tongeren (BE) area benefited from the Pact’s ‘reinforcement of the links between the range of local actors’ and was the place where negotiations linked to the attempt to secure Objective 2 funds took place.

• The Hamburg Pact (DE) also assumed a broader co-ordinating role. For a city-state like Hamburg, the TEP approach was considered a good way to centre all the various labour-market activities around one focal point.

• This kind of role is apparent even more clearly for the Valles Occidental Pact (ES).‘…a key element of success was the concentration of objectives and the convergence of genuine co-operative actions. A process of cooperation - within the framework of the Pact - emerged in three senses. Vertically, through the establishment of compromise linking the content (of the Pact) with organisations at a superior level. Horizontally, between diverse agents – through the realisation of joint activities between different sorts of agents that do not normally collaborate with each other. Horizontally, between diverse territories that do not normally collaborate with each other.’9

• Two of the Italian Pacts, Catania Zona Sud (IT) and Calatino Sud Simeto (IT), also saw benefit in the TEPs’ promotion of partnership and co-operation. For Catania Zona Sud, the Pact methodology has proved useful in that ‘the strengths are the institution of a new method, concertation, for the planning and management of interventions’. Meanwhile, for Calatino Sud Simeto, ‘there prevails at present a new confidence in the possibility to steer micro- territorial development of creative interventions/projects that genuinely add value.’10

• The Vorarlberg Pact (AUS) has facilitated increased co-operation between a wide range of actors involved in labour-market interventions in the Land. In

9 Case study report 10 EI2000 questionnaire

36 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

January 1998, the Pact’s monitoring committee decided on the establishment of a new consultative body for the Pact - the so-called "platform of employment projects" (Plattform der Arbeitsprojekte). The members of this body were representatives of all independent labour market projects or initiatives carried out in the Land of Vorarlberg. The platform is managed and co-ordinated by the Pact administration, which ensures that projects are closely linked to the activities of the Pact.

The co-operation and co-ordination engendered by the many Pacts is for them a legitimate achievement. Even in the absence of job creation benefits that can be directly attributable to the Pact, the indirect benefits in terms of enhancing institutional capacity are potentially of no less significant. This distinction drawn between direct and indirect benefits is clearly made by the Peine (Niedersachsen) Pact11 (DE). The Pact wanted to emphasise that the real achievement of the Pact was to clarify that the so-called ‘software’ of labour-market interventions – such as communication, co-operation and discussion – is just as important as the ‘hardware’, such as job creation.

In Spain “… the Pact (model) has settled the basis for the definition of a new territorial model for intervention that is essentially decentralised, integrated and participatory. This should be accompanied by the establishment of a real vertical cooperation among territories as well as by a higher commitment from public and private sources. In any case, the pacts have evidenced the necessity to increase competencies and financial capacities at local level.” 12

Extension of partnership models to include other sectors was also cited as signficant. The Ceuta Pact (ES) demonstrated that the private sector has an important role to play. “there is a certain development of the idea that the public administration is not the only actor that has the responsibility to solve the problems of a particular territory.”13

3.3.5 Funding and Territorial development Seventy four Pacts levered additional public sector funds and 61 EU funds, whilst 56 levered private sector resources. This attainment of influence over funds in addition to the limited funding directly available to Pacts is important since in the field of employment development there is little evidence the progress can be made in the absence of at least some support funding. The high figure for the public sector reflects its primary role in most Pacts, as well as its importance as they key actor in relation to territorial development initiatives. The private sector figure is higher than might have been expected and demonstrates an unexpected ability on the part of Pacts to mobilise the financial resources as well as good will of the private sector. Full understanding of the financial picture would however necessitate analysis of detailed data which is not available.

11 response to question 92 of the EI2000 questionnaire 12 quote from Spanish case study 13 Pact quote

37 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

The Norte Alentejo Pact (PT) focused on increasing the impact of structural policies on the labour market in the area, seeking to mobilising public and private actors in the territory. The Pact directly addressed the lack of its own direct funds by seeking to influence the deployment of others:14

‘Since there are no direct funds (beyond the TEP TA) for the realisation of its activities, the Norte Alentejo TEP presents itself above all as a forum for decision- making – of decision-making to ensure the application in the best possible way of the financial means of the different partners that make up the forum. This forum undoubtedly improves the overall effectiveness of the handling of the financial resources of the different partners’.

If a Pact wanted to carry out its own actions with its own resources, it would obviously help to have a broad partnership financially committed to the Pact. Moreover, if a Pact was given an entrenched and formally approved position within the existing system of governance, then it would be far more difficult for superior authorities – such as regional and national tiers of government – to avoid close co- operation with, and support of, the Pact. This would make the Pact more likely to be a recipient of financial support. However, the securing of financial resources to carry out specific interventions is not a prerequisite for a successful TEP. The case of Norte Alentejo illustrates another possibility: the ability of a TEP to act as a co-ordinating body that brings strategic overview to the range of different actions and initiatives that characterise the field of direct labour-market intervention, and influences the spending decisions of others.

In general, the German Pacts proved to be successful at levering in resources from external sources in order to finance their activities. The Berlin-Neukölln Pact (DE) was particularly successful at mobilising funds from private industry, while the Chemnitz Pact (DE) mobilised large sums stemming from the Federal Government.

One of the ultimate objectives of the TEP Programme was to contribute to the ‘territorial development of the area in which the Pact was situated’. Throughout the programme there is a lack of hard evidence as to what this meant in practice, since it is a diffuse concept and could conceivably cover a wide range of social and economic issues. The scale of Pact operation is also too small for available indicators of territorial development to be of use in assessing their impacts. The section above on partnership capacity provides a number of examples of where the activity and co- ordination work of Pacts and their partnerships led to qualitative change in the local area, and further evidence of development, albeit unquantified, includes the following:

• the Catania Zona Sud Pact (IT) covered the South side of the city, targeting a population of 300,000 and its Pact actions in co-ordinating re-development has made the area for investments, resulting in ‘many’ private enterprises in tourism and services moving there. This is regarded locally as a major accomplishment, with a significant effect on the general economic development of the territories in which they are located.

14 answers to EI2000 questionnaire

38 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

3.3.6 Mainstreaming and replicability Thirty Pacts demonstrated achievement of a substantial degree wider influence from their work, through mechanisms including: • continuation of the Pact under alternative funding sources • adoption of the Pact method by other actors • dissemination of elements of its method to receptive audiences

An example of a Pact succeeding in establishing continuity for its role comes from Italy: the basic structure of Agro Nocerino Sarnese Pact (IT), the ‘S.I.L.’, ‘has so far survived the end of the Community Pact and is still active on the territory encouraging local initiatives. On 24 May 1999, the Pact has been entrusted by the now 12 Communes partners with the task of ensuring a help desk function (‘Sportello Unico’) to assist local initiatives in the region.’15

A further 37 only partially achieved any wider influence, and 17 did not achieve this at all. Given the programme was intended as an experiment to demonstrate the value of the co-ordinated approach these results are at best middling, and call into question the degree of embeddedness of Pacts in their national contexts. There is a frequent tendency for European programmes to be seen as separate from the national, and for funded activity to be seen as an isolated phenomenon. Whilst the Pacts delivered higher results in their creation and maintenance of partnerships, these figures show less continuity after the EU funding ended, but little compensating adoption of the method elsewhere.

3.3.7 Summary of Pact achievements The survey of the achievements evidenced by Pacts highlights a number of general points about the programme: • solid, reliable, quantified and unambiguous evidence for the achievements of Pacts is scarce. Little consistent information is available across Pacts as a whole, and few recorded substantial data concerning their own activities • when asked, for example in case studies, to provide clear evidence, most Pacts were able to provide qualitative and subjective information, as well as indicating sources of alternative perspectives. This, combined with descriptive documentation and the opinions of national observers, does give a reasonable understanding of the areas where Pacts felt themselves to be successful or not, and patterns do emerge • Pacts’ views of their achievements are dominated by the creation and nurturing of partnerships and associated co-operative activity, and the benefits arising of improved trust, co-ordination, understanding, influence and common purpose. This sort of achievement is fully consistent with the purposes of the programme, and Pacts have been able to give examples of the positive wider effects arising.

15 Quote from Pact

39 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

• these wider effects are diverse and include more consensual and effective resource deployments, stimulation and operationalisation of project ideas, realignment of policy, better articulation of supply and demand • direct creation of employment has taken place in some Pacts but taken as a whole of is relatively little importance as an achievement of the programme as a whole. Where it has taken place it offers some models of collaborative approaches to new project development and operation, but in these cases the main benefit comes from the process of stimulating the idea and putting it into practice in the first place, rather than the job creation figures themselves • the gaining of influence on resource distribution has taken place, and has brought a more strategic approach the fore, bringing wider benefits in terms of less overlap and greater clarity • the wider lessons of Pacts have not been disseminated widely. This is partly due to the diffuse nature of the achievements themselves, and the locality specific circumstances which shaped their evolution, from which it is difficult to generalise. It is also due in some cases to a lack of understanding by many Pacts that this was a significant activity, and lack of resources directly available within the Pact programme to devote to dissemination.

3.3.8 The association between attributes, management quality, and achievements In addition to describing the recognisable achievements of Pacts in their own right, this chapter also seeks to establish a shorthand method for summarising their overall success. Using the six indicators described above, it is possible to arrive at an overall achievement score, in parallel with the attribute and management scores described earlier.

Figure 3.2: Relationship between attributes and management, and results

R2 = 0.4585 100.0% 90.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Mean (attributes and management) 0.00% 20.00% 40.00% 60.00% 80.00% 100.00 % Results

40 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

Figure 3.2 plots the aggregate attribute/management score for each pact against its aggregate results score. This shows a strong association between these elements – in other words, Pacts scoring highly on both management and attributes are substantially more likely to score well on aggregate results as well. Whilst there are some which differ from this pattern, the overall correlation is strong. From this we conclude that combining all three elements (attributes - structure, management - process, achievements - results) into one single figure provides a good overall summary of the relative overall success of individual Pacts. Those scoring highly will be most ‘Pact like’, be well managed, and have achieved clearly identifiable results. Using the combined score has the added advantage of using more individual data items, thereby helping to smooth out rogue results arising from lack of individual pieces of evidence. Pacts with insufficient data availability (defined as having information available for ten or fewer questions) are excluded from analysis using this overall score.

41 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

CHAPTER 4: CONTEXTUAL FACTORS WHICH MAY EXPLAIN PERFORMANCE

4.1 Introduction

Having examined the question of exactly what was funded by the Commission; how well it was managed; and what can be demonstrated to have achieved, we now turn to consideration of the factors which may have a bearing on the variable levels of success achieved. In undertaking this exercise some caution is necessary. We have sought to demonstrate in the previous chapter that the overall ‘score’ achieved by Pacts is a useful means of summarising their relative successes, and that higher scoring Pacts can be considered better examples of Pact characteristics, effective management, and programme relevant achievement than those scoring lower. However, we have been compelled to exclude a small number of Pacts from this analysis due to lack of data, and for some others there are weaknesses in the quality of information upon which our rating judgements have been based.

It is also the case that the scoring exercise was unable to fully take into account the extent to which the achievements of Pacts were truly additional, and not simply displacement of activity which might otherwise have, or would have more appropriately have, been undertaken by other actors. 16 Finally, the scores should not be taken as an indication of absolute achievement, more as a means of judging their successes relative to one another. It is this relative position, particularly when related to possibly influential factors, that provides the main benefit of this approach for the analysis of possible contributing factors to observed outcomes. Our analysis is not of course limited to the examination of relative scores. It brings in information from interviews and Pact case studies to illuminate the observed variations in performance, and to identify the mechanisms by which influential factors have affected Pacts or been overcome by them.

Our approach to influential factors involves testing a number of hypotheses. These can be stated as follows, and will be sequentially tested: • the ‘governance context’, that is, the framework of political structures and accountability in general, and policies and practices towards local employment development in particular, will have had the most significant influence over Pacts’ abilities to be successful. • the geo-political level will also have been influential - regional level and above too distant from local understanding; the very local level insufficiently engaged with the key levers of influence needed for effective employment action. • the size of population covered by Pact work will have had an important bearing. Pacts serving very large populations (say 1m+) are likely to have been unwieldy and lacking in focus. Pacts covering small populations (say 200k-) will have been lacking in critical mass, and been too far from the strategic level, to be fully effective.

16 The issue of added value is discussed in more depth in Chapter 5

42 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

• Pacts in the same ‘family’ (used to cluster Pacts into similar types of area, labour market and main approach adopted) will tend to have similar experiences and success scores, and families will vary one from another • The elements which were used to construct the ‘families’ may in their own right have had a bearing on observed outcomes – the geo-economic context; type of labour market conditions; and primary approach defined as required in the area (specific reinsertion of target groups, or generalised labour market action)

There are however some additional factors we have not been able to test. The nature of the programme itself and the prescribed actions of Pacts, as well as the availability of data, renders reliable analysis of these issues impossible. They are mentioned here so as to draw attention to their potential impact, and to indicate the desirability of their examination in any future evaluation.

4.2 Contextual factors not readily testable

4.2.1 Developments in the macro-economy

If a Pact is situated in an area that is suffering as a result of an economic downturn, it will be difficult to imagine circumstances in which it could single-handedly reverse the adverse developments in the labour market that would be present. That is not to say that the Pact might not help to alleviate these conditions, but it would not be enough to reverse the profound effects of the economic cycle on the labour market. For instance, the Berlin-Neukölln Pact (DE) did manage to create 2000 new jobs and places for vocational training, but could not reduce the overall rate of unemployment in the area. In this case the fact that the overall rate of unemployment did not fall had far more to do with the macro-economic climate in which the Pact operated than with any failure on the part of the Pact.

4.2.2 Wider policy framework within which the Pact operates Policies determined by higher levels of government – or by other institutions – will often have a profound effect on the local economy. Policy decisions in this area can include anything from national planning guidance that influences economic development decisions in an area, to national policies that set wage and non-wage labour costs, to the inability of a Pact area to influence the prevailing interest rate (which, as of 1 January 1999 for the majority of those Member States that had joined Economic and Monetary Union, was set by the European Central Bank).

4.2.3 External economies of scale and available infrastructure The attractiveness or unattractiveness of an area will often be linked to the existence or absence of external economies of scale. If an area already contains a critical mass of some companies that are in the same or complementary industries or sectors, the area will possess an appeal that geographically isolated or peripheral areas will have difficulty in matching. Also, employment creation in an area will often be influenced by the area’s infrastructure and the ‘business friendliness’ of prevailing conditions, ranging from image and housing to investment tax breaks.

43 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

These factors cannot be tested statistically, the Pact’s possible influence over them, even where it exists, being too small for detection. However, we have sought to take them into account in our examination of the hypotheses, particularly where they may offer some explanation for observed differences.

4.3 Governance context: Member State variations in the opportunities for TEPs

4.3.1 Types of context The central hypothesis for this evaluation was that the ‘governance context’ would be a key determinant in the success of Pacts. This refers to the effects of the system of governance into which the Pact has been inserted. This takes into account the space for insertion within national/regional policy frameworks, the ‘goodness/suitability’ of the fit of the TEP to these existing policy frameworks, and the attitude of the existing layers of government to the TEP.

Member states have been divided into three broad groupings in relation to the policy framework and governance context prevailing during the lifetime of the Pacts. This is shown in Table 4.1

Table 4.1: Governance Contexts

Governance Type Member State Context A Full spectrum – Denmark crowded platform France Netherlands Sweden United Kingdom B Available policy Austria framework and Belgium receptive space Germany Finland Ireland Italy Spain C Narrow spectrum – Greece available space Luxembourg Portugal

It should be noted that the application of policy did to some extent change during the course of the TEP programme, and there is room for debate as to the exact assignment of countries to groups. However, in our view these groups of countries do show broad similarities which meant the Pacts were confronted with essentially similar policy and practice environments within each group, which were distinct from those in the other groups. These distinctions are summarised below:

44 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

A) Full-Spectrum Local Governance Context – the “Crowded Platform”

This kind of government context emerges where there has been a long history of governmental intervention in the operation of labour markets and in the territorial development process at all levels. Moves toward multi-level governance systems have taken on board the principle of open partnership. A feature of this kind of government context is a kaleidoscope of agencies, partnerships and local groups, often interacting with each other in different fora and for different purposes. There is likely to be a multiplicity of strategies, often with broad analyses and plans expressed at national and regional level used as a framework for ‘nested’ sub-regional and local action plans, each level contributing to the level above. There will often be targets and indicators associated with the overall strategy. A degree of duplication and at times rivalry between agencies with overlapping remits is to be expected, along with expressions of ‘initiative fatigue’ and ‘overplanning’ on the part of local delivery agents. Stakeholders from different areas of government as well as the private and third sectors will tend to be used to involvement in action which seeks to include their particular point of view in planning and delivery. On many occasions however they will not be specifically resourced to take part in joint activity.

A feature of this kind of governance context is an established tradition of local level partnerships and a variety of agencies, partnerships and local actors occupying the space the TEPs were designed to inhabit. Operating on this terrain are structures like the following:

• Local offices/branches of the national employment and training services, national agencies for the promotion of entrepreneurship and the development of SMEs, administratively directed from the provincial or regional level or from sub-regional level of central government administration; • Local Authorities with a role in the “activation” of the local unemployed or those excluded from the labour market generally. • Local NGOs, social partner bodies or Third System Organisations (TSOs); • Partnerships and consortia at a variety of spatial/territorial levels that bring some or all of the above organisations together to create multi-stakeholder organisations to address job insertion, social inclusion, the focus on target groups, and the wider brief of local development and employment.

B) Established Governance Context for Local Territorial Action – “Receptive Spaces”

This type of governance context exists where there is already in place an accepted and legally determined role for devolved governance powers at the local level, but this is not systematically combined with extensive, multi stakeholder partnership working. Partnerships may exist but will tend to be limited in scope or number, and there is unlikely to be substantial overlap of remits. Devolved forms of partnership are not simply a lower tier in an extended hierarchy of central administration but have defined powers in relation to employment and development. It is appropriate to include in this group cases where the delegation of powers to local partnership organisations is partial but significant.

45 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

C) Narrow-Spectrum Local Governance Context – “Relatively Free Space for Pact Operation”

This kind of governance context reflects conditions where there has been only a limited history of local-level activity in relation to the labour market, and where labour policy has traditionally been the preserve of national (or devolved regional or provincial) government agencies. There often exists a deficit of devolution of power at the local level, and an absence of pre-existing organisations and structures that could play host to a Pact as a project. Stakeholders from outside the public administration will have little direct experience of involvement in joint activities for employment and local development.

In these countries there has been a limited history of governmental and quasi- governmental intervention in the operation of labour markets and in the territorial development process except through the operations of the central state and its agencies or through semi-autonomous regional bodies. There is a deficit at the strictly local level with no strong devolution to this level. Overall, there is a narrow spectrum of levels of governance in general and few levels in the territorial hierarchy of governance. There is little evidence of a move to multi-level governance systems and a motivation to take on board both the principle of open partnership and the “bottom- up” approach.

Operating on such ‘virgin’ territory are the following kinds of structures: • Local offices/branches of the national/autonomous regional employment and training services, national development agencies etc.; • Local Authorities with little or no remit to intervene in the process of economic development or regeneration; • Little (or at best patchy) development of local NGOs, or Third System Organisations (TSOs); • Little more than embryonic development of local partnership or stakeholder bodies dedicated to taking a more strategic view of employment local development.

It cannot be expected that Pact will perform the same kind of tasks and assume the same basic form in these radically different governance contexts. Instead, the TEPs would need to embed themselves differently in different circumstances. It can therefore be argued that the performance of Pacts would largely be determined by the extent to which they managed to take a form and assume a role appropriate to the governance context. It would also important for a Pact to receive the support of all relevant tiers of government, as well as other non-governmental actors, such as the private sector.

4.3.2 Governance context analysis

Against this background, our hypotheses were that: § the crowded policy platforms would stifle the ability of Pacts to achieve their ends § Member States with some policy activity at the sub-regional level but without a serious tradition of locally co-ordinated, multi stakeholder employment action would provide the most fertile ground for the Pact idea to achieve results

46 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

§ countries with a complete absence of local action might offer promise but would be likely to need more time for the Pact approach to work than was available in this programme, and may encounter resistance or hostility from established power structures

A brief summary of the ‘space for insertion’ in Member States and what these meant for Pacts, derived from national level interviews with government officials and national experts, is provided in Table 4.2

Table 4.2: Space for insertion of Pacts: Comments by country COUNTRY COMMENTS ON SPACE FOR INSERTION A: Crowded Platforms France There were already numerous territorial initiatives in place; often the Pacts risked duplicating existing activities. Denmark TEPs were able to make little impact on employment policies due to a “crowded platform” and the negative attitude of labour ministry. Netherlands A large number of partnerships and initiatives constituted a crowded platform where Dutch Pacts had to be inserted Sweden Many regional and employment initiatives were already covering mostly the same areas as the Pacts. UK There was a highly crowded platform, which included several high-profile government area-based initiatives, e.g. New Deal. B: Receptive space Austria Pacts were able to find the niche from which to successfully mobilise local forces and emerge as innovative platforms. Belgium At the time of the start of the TEP Programme, there was little previous tradition of partnership working on employment at the local level. Finland The TEPs brought value added to local, regional and national employment policies and strategies even though the role of local actors in the labour market had begun to be recognised in the early 1990s. Germany Whether there was space for the Pacts to operate depended on the level at which they were inserted (platform was relatively crowded at Land level). However, in general, there was more space for Pacts to embed themselves in the former East Germany. Ireland Ireland had possessed an established framework for area-based approaches since the early 1990s. This pre-existing experience was an advantage for Ireland as it meant that no time/effort was spent in setting up the necessary administrative framework. Italy There was space for local-level employment initiatives. The existence of the Italian government’s Territorial Pacts based on narrower partnerships offered a head start for creating wider TEP partnerships. Luxembourg The TEP methodology found space to be embedded and Pact- style employment initiatives are now being launched in other municipalities.

47 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

COUNTRY COMMENTS ON SPACE FOR INSERTION Spain The experience regarding partnership and group working brought added value and is contributing to structural change in all economic and social partners. C: Free space Greece Overall, there was a potential opportunity and space for the Pacts to operate. Labour market interventions were dictated by central government and there was virtually no previous experience of partnership working at the local level. Portugal Lack of decentralisation meant there was space for local level initiatives and partnerships to be set up

Analysis of Pact success scores by Member State and governance context shows some striking patterns, which go some way to confirming our hypotheses. These are shown in Figure 4.1. Those Member States with an existing policy platform but ‘receptive space’ for its implementation, those in the intermediate position, show the most consistently high mean scores. All these Member States Pacts have mean scores higher than countries in any other category, with the sole exception of the Netherlands, and there are few scoring less than 60%. With the exception of Belgium and Spain, almost all Pacts in these Member States are clustered between around 70% and 95%17.

Figure 4.1: Country scores by governance context

TEP Overall performance by Member State and Governance Context

A B C 100.00% 90.00% 80.00% 70.00% 60.00% 50.00% 40.00%

Performance (%) 30.00% 20.00% 10.00% 0.00% NL F S UK DK IRL I A SF D E B P L EL Member State

A = Crowded: NL, FR, SE, UK, DK

17 It should however be noted that the Italian results are only based on four Italian Pacts, since reliable information on which to base scores was not available to the evaluation team. Two French Pacts were also excluded for the same reason.

48 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

B = Receptive: IRL, IT, AUS, SF, DE, ES, BE C = Free: PT, LUX, EL

In the crowded platform countries a different picture emerges. With the exception of the Netherlands, these Member States scored consistently lower than those in group B, with around half of Pacts in the UK and Denmark returning the poorest scores of any country except Greece. There were however some exceptions – apart from Denmark, all these MS did have successful Pacts, with scores into the 90%+ range. The range of scores from high to low in the UK and Denmark is particularly broad.

Turning to the vacant space countries, there is a strong contrast between the two Member States. With one exception in the 70% range, all the Greek Pacts scored poorly, some disastrously. In Portugal by contrast two of the three Pacts scored very highly, and were on a par with the top scores in other countries. What then do these results tell us concerning the hypotheses?

Established Governance Context for Local Territorial Action– “Receptive Spaces”

With regard to the receptive space countries, there is good evidence that this policy environment can be the most consistently conducive to success by Pacts. The classic cases are the Italian model of the Patti Territoriali and the Irish Local Partnerships. The Italian Pacts are recognised in the context of a law that underpins their role. The Irish Pacts, by virtue of their association with the Local Partnerships, began with a clearly defined status and had the considerable benefit of being promoted directly by the office of the Prime Minister.

For the purposes of the evaluation, however, a difficulty arises in establishing the discrete contribution of the Pacts themselves when they are part of the established national structure. This is very different from what was described earlier as the “re- badging” of existing partnerships in, say, the UK case. The principles of Pact action were clearly understood and accepted in the Italian and Irish cases whilst the same could not be said for many of the local authority dominated Pacts in the UK. In these cases the case for Pact-type intervention is already accepted. The ease with which a Pact can be established in such a context can be seen in the Dundalk and Drogheda Pact (IRL), where two existing area-based partnerships combined to establish a Pact as a response to unemployment in their areas.

A distinctive feature of this kind of governance context was the acceptance of a clear role for the role of the local partnership in employment and development issues, combined with a defined division of central and local powers in this area of policy. A further feature was the influence that local partnerships had at the centre, exemplified in the way in which the Irish local partnerships during the first few years of their existence reported directly to the office of the Prime Minister. Pacts in this kind of governance context provided evidence of the avoidance of the high transaction costs involved in making a case for themselves (as compared with Pacts facing either a crowded context or a free space), and were able to move quickly to the instrumental role of finding and implementing new ways of rendering latent jobs in the local context solvent (to use the terminology of the Commission’s LDEI programme).

49 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

We would argue that Pacts in Germany, Austria, Spain, Belgium, Luxembourg and Finland might be assigned to a variant of the above governance context. This differs from conditions in Ireland and Italy in that the delegation of powers to the local partnership organisations was as yet partial, with some powers that might be devolved to the local level still “reserved” for the central or regional authorities – many of these countries have federal structures, with powerful regional authorities charged with substantial devolved powers.

In Belgium, for example, the division of economic and labour-market responsibilities between local, regional and national government can be described in the following way, which shows the policy areas reserved for certain tiers of government, along with areas of overlap.

- Employment national and regional - Skills and training regional - Entrepreneurship national and regional - Local economic development: regional

The Bruxelles-capitale Pact (BE) provides a good example of a division of responsibility between different tiers of government. The labour market policy context was characterised by a regional employment policy, with an orientation being provided by national policy and EU guidelines. Partnerships were in place before the creation of the Pact - for instance, between employers and the ORBEM (Brussels public employment service). However, there was no large forum bringing together all partners involved in the Brussels labour market. The creation of the social and economic council – where social partners and the government meet - marked an attempt to start a dialogue between social and economic partners, but it was not fully operational when the TEP was set up. This gap was partly filled by the Pact.

The Crowded platform

With regard to the crowded platform countries, the picture is clearly more varied. The UK, Sweden and Denmark provide some good examples of where Pacts ran into problems in finding a role in the prevailing congested environment. Here, Pacts were unable to create a role for themselves in the face of established partnerships or agencies already fulfilling employment related roles, and there are clear examples of Pacts facing competition and hindrance. These include several UK Pacts – West Cornwall (UK), Coatbridge and Easterhouse (UK), and East Midlands (UK) all cited difficulties in establishing a coherent role for themselves, which led to ineffective action and an inability to mobilise additional resources to provide leverage.

Crucial in finding a meaningful role for Pacts in these crowded circumstances was the attitude of the political authorities to the TEP. For example, the Pacts in La Réunion (FR) and Albertville (FR) both had a difficult relationship with the relevant regional authorities, which were the dominant force in the area with regard to economic policy. By contrast, the Hérault Pact (FR) benefited from the active support of the département in its activities. This Pact’s executive management was integrated into the management of the département. Without the support of the relevant political

50 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 authorities, situation in a crowded policy platform create ‘the danger of becoming just another initiative’ seeking recognition and a role.18

In support of this contention, there is evidence that the erratic performance of the Danish Pacts might be partly attributable to the attitude of the central government – in this case, the Labour Ministry – towards the Pact programme. Pacts were of the view that the Labour Ministry already considered the idea of working in partnership to be well-established in the Danish setting19, regarding the activities of the TEPs to be superfluous. The Viborg Pact (DK), for instance, preferred to communicate directly with the Commission, thereby by-passing the Labour Ministry entirely.

Some other Pacts did however manage to define a more positive role for themselves. For the most part they followed the route of trying either to help re-engineer existing structures, or to adopt a part of the structure and re-brand it to make a TEP contribution. This is demonstrated by the Albertville Pact (FR), which used the opportunity to consolidate and enlarge an existing partnership. Pacts facing a crowded platform have either sought to add a degree of additional integration or co- ordination to existing groupings of partners, or to improve the innovative ability of these groups or re-branded organisations, as is illustrated by the following examples:

• The Tottenham and Haringey Pact (UK) – the highest scoring UK Pact - was inserted into a context containing an existing partnership (the Tottenham Partnership). The Pact’s distinctive contribution was the ‘risk’ money that it provided, which supported innovation that could not otherwise be undertaken. This Pact actively sought to find a ‘gap’ in the existing partnership structure to avoid duplication with the activities of existing programmes, initiatives and institutions. As a model however this example should be seen in light of its relatively small scale of operation. The Tottenham area is more local than sub regional, and the Pact sought to concentrate its limited funding on a small number of ‘project’ activities generated locally. These were successful and genuinely innovative in the local context, bringing labour demand and supply together in an effective way. However, they are more an example of the successful search for a useful role to play, in this case the running of projects which could legitimately have been supported under Objective 3 or EMPLOYMENT, than they are of strategic co-ordination and added value at the partnership level.

• The Limburg Pact (NL), on the other hand, established effective co-ordination of regional level actions designed to create new employment. The Pact operated in an area that had a population of approximately 1.3 million people. It was also confronted by a crowded platform. In such circumstances, it was logical for the Pact to concentrate its efforts on the task of co-ordination. The Pact emphasised particular objectives of the TEP Programme, notably utilisation of all possible resources to promote and develop an overall regional employment strategy that is broadly recognised and supported by all those involved in the labour market; and full exploitation of the advantages of integration and co-ordination of policy measures and resources on a regional scale. Given its regional scale, this Pact chose to concentrate its resources on co-ordination, and did not dilute its resources

18 Quote from Pact management 19 Case study

51 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

in a search for conventional project activity. Limburg was the highest scoring Pact in NL20. The Netherlands delivered the best scoring Pacts of any Crowded Platform country, attributable to their establishing clear roles for themselves in a way which clearly differed from the existing framework. Their arrival coincided with a Government initiative to increase the integration of labour market policy at regional level, and it is notable that most Dutch Pacts operated at the larger, regional scale, consistent with this policy priority. Despite the congestion of initiatives, a centrally determined emphasis on the need to increase co-ordination and eliminate overlap clearly benefited these Pacts.

Narrow-Spectrum Governance Context – Relatively “Free Space” for Pact Operation”

Since this context is limited to only two countries, Greece and Portugal, which display distinctly different experiences, we will discuss them separately before drawing some general conclusions.

The Greek Pacts found themselves in a governance context characterised by relatively ‘free space’ for a TEP-like intervention. However, the absence of the pre-existing organisations and structures that made insertion difficult in the context of a “crowded platform” did not provide an easy route for the Pacts. Almost all the Greek Pacts reported negative (or at least unsupportive) attitudes to partnership as a valid governance form for intervention in the operation of the labour market and development process. The traditional processes of ‘dialogue’ between the social partners, often in practice expressed as conflict, was a further hindrance since there was little tradition of collaborative action between the two sides of industry to draw upon. Key stakeholders were reported as being unwilling to cede their perceived ‘power’ by opening up decision-making processes.

Portugal offers a contrast. The ‘marketplace’ was described as every bit as open for new local partnerships as Greece, but some moves towards the adoption of “bottom- up” initiatives had begun at different levels. At the time of Pact operation these were in experimental mode and did not then represent the emergence of complete legally underpinned local level of governance. They tended to be sporadic and a result of regional or local experimentation, and in this context the Pacts programme was a timely opportunity for further development of this idea. This was particularly the case where local government bodies were sympathetic:

The Norte Alentejo Pact (PT), for example, benefited from the support of the regional authorities, even though the commitment of the national government is reported as having been less than wholehearted. The Portugese Pacts have generally been successful in engaging local actors in “bottom-up” activities and in creating effective partnerships. In the cases of Marinha Grande (PT) and Norte Alentejo (PT) Pacts, positive development has been linked by the Pacts to the fact that both were areas with a strong local identity, making it easier to mobilise actors to pursue a common strategy. Subsequently an increased role for locally-driven approaches in tackling

20 The two strongest performing French Pacts were St Herblain (FR) and Hérault (FR) – both of which largely concentrated on carrying out a strategy of co-ordination.

52 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 unemployment was included in the Portuguese National Employment Plan, with specific reference to TEPs. This document expressly underlines the role of the TEPs in elaborating a strategy and integrating the work of the institutional, public and private actors.

The mere existence of space in the system of governance is by no means a sufficient condition to ensure the success of a Pact. If the governance context does provide a free space, there is at least the potential for the development of a new kind of labour- market intervention through partnership building. However, the contrasting experiences of Greece and Portugal show that the absence of supportive layers of government – be they at a regional or national level – meant that Pacts had trouble establishing themselves and making a valuable contribution to labour-market policy making in the area. In this governance context, then, it will be the attitude of the political authorities that plays the decisive role in determining whether or not a Pact approach is successful.

4.4 Governance Level

The level at which Pacts operated, that is, their position in the hierarchy of governmental responsibilities, was also considered as a possible factor to explain variable performance.

The evaluation established the positioning in regional and sub-regional structures based on information from the Action Plans. Table 4.3 suggests that the level of insertion varied considerably between countries: Demark, Austria and Portugal favoured insertion of TEPs at the regional level. Greece and the Netherlands inserted all TEPs at provincial level, while France used this level for all but one. Germany, Finland, Italy and the UK were biased towards insertion at the municipal level.

Table 4.3 Level at which the Pacts Number of Pacts % of Pacts were inserted Regional 17 19% Sub-regional (provincial) 32 36% Sub-regional (local) 40 45%

Figure 4.2 Level of insertion by country

Level of Insertion

12 10

8 Regional 6 Provincial 4 Municipal 2 0 I D UK Su E B Ire S A F L P Dk El Nl

53 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

Figure 4.3 shows Pact scores divided between three levels – regional, ‘provincial’, and local. It was necessary to make some subjective judgements in assigning Pacts to these levels, particularly at the boundary between provincial and local. In the main, local is defined as at the lower levels of authority, whilst provincial will operate at a level which brings together a number of lower levels and will include at least some wider strategic functions and some co-ordination role. Regional is defined either as the EU conventional region, or such a large sub set of such a region that, in other countries or circumstances, it might well be classified as a region in its own right. The population size of regions can vary considerably – from over three million in North Rhine Westphalia (DE) and Oresund (DK) to only 11,000 in Stromstad (SE).

Figure 4.3: Pact success scores by governance level

TEP Overall Performance by spatial coverage

100.00% 90.00% 80.00% 70.00% 60.00% 50.00% 40.00%

Performance (%) Performance 30.00% 20.00% 10.00% 0.00% Regional Provincial Local Spatial Coverage

The majority of Pacts operated at the provincial level, which is consistent with the overall model expected by the Commission. The effect of Member State selection of Pacts can however be seen in the existence of substantial sub sets of Pacts operating at regional and purely local levels. In terms of scores, the highest scores were attributed to the local level, with the regional level lower and provincial lowest of all, although the difference between the means was not great. Given the spread of Pacts across the scoring range in all three groups little of significance can be read into these results in terms of the likelihood of success. Examination of the experiences of Pacts at the high and low points however indicates that different strategies are required for success depending on the level. In summary, these can be categorised as:

54 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

Regional The more successful Pacts focused on large strategic issues; mobilising the significant regional players who were able to influence or co-ordinate lower level bodies. They did not tend to get involved in direct projects creation even less their operation, nor in detailed local level actions. The Dutch Pacts, particularly Limburg (NL), provide particularly good examples. The less successful Pacts were overwhelmed by the scale of the task, and found too many organisations to co-ordinate. The were unable to mobilise the forces necessary to effect changes at this scale, and in some suffered from the competitive instincts of lower level bodies who were reluctant to pool their resources and influence. With half hearted co-operation from key stakeholders these Pacts lacked focus and influence. North Rhine Westphalia (DE) is a good example of this experience. Provincial This level threw up a wider variety of models. The more successful were found to prioritise bringing organisations together above the local level, and focused on the strategic priorities for the sub region in ways which had not been attempted before. These tended to be characterised by the successful involvement of a wide range of stakeholders, in particular substantive contributions from the private and third sectors. In situations where this was a novel approach notable achievements were cited – a good example of this is Iisalmi (SF). The other prevailing model at this level involved more direct project work. When focused on stimulating local project ideas and seeking to deliver the resources and support necessary for their delivery by other actors this could lead to significant achievements and innovation – the Dublin (IRL) Pact is a good example here. Those which became more directly involved in the management and direct operation of local projects were rather less successful, their resource level being too small for much achievement beyond the running of small scale local activity. The processes involved in managing such project programmes also distracted attention from any wider strategic activity. An example of this experience is provided by the East Midlands Pact (UK) Local The Pacts scoring well here tended to be those which concentrated on more direct project work, particularly those which generated new ideas and facilitated the funding and support process for their realisation. Pacts all received the same amount of direct funding, €300,000 maximum, irrespective of their location and scale of territory. This could achieve little in the way of direct results in a large area, but if deployed in more local circumstances on few activities would be more concentrated. The Tottenham Pact (UK) is a good example of this approach. These Pacts will have scored relatively well as a result of having achieved their own employment related objectives. In general however we would not consider these to be the best examples of the Pact idea in action, since their focus did not tend to reach the strategic and co-ordinating level sought by the Pact ‘philosophy’. There is considerable likelihood of deadweight with such direct project operating Pacts. The less successful local Pacts were characterised by an inability to influence or mobilise the

55 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

more powerful actors needed for effective action to be forthcoming. Operating from a local base, if superior authorities were unsympathetic, and if non governmental stakeholders were uninterested, these Pacts were left with few alternative options. Whilst some went down the project running route, others were left with somewhat meaningless discussions and plans with on one available to take them forward. This was a particular feature of the St Helens Pact (UK) some Greek Pacts.

4.5 The importance of vertical links

The experience of Pacts at all levels indicates the importance of vertical, and in particular upwards, links to their ability to achieve results. The Pact process and philosophy emphasised horizontal (drive for broad-based partnerships) and downwards links (bottom-up approach), and as was shown earlier, most Pacts understood this imperative and at least made efforts to ensure they were forged or reinforced. In many cases this was successfully achieved. It can be argued however that insufficient emphasis in the programme, and subsequently by Pacts, was placed on creating upwards vertical links, especially with authorities at higher levels (namely, regional and local).

In ‘crowded platforms’ vertical links were needed to differentiate Pacts from other similar initiatives, and to demonstrate to key stakeholders that the Pact was worth supporting. The Netherlands example of Pacts being connected to new government policy towards co-ordination of employment strategy shows how superior authority support gave the regional Pacts vital political capital, whilst in France this support was far more variable. Some Pacts, like Hérault (FR), received support from the département level; La Réunion (FR) and Albertville (FR), however, experienced a difficult relationship with the regional authorities. Conversely, in the UK, Pacts made little attempt to forge strong links with regional and even less national authorities, focusing more on ‘re-branding’ existing partnerships as Pacts and encouraging wider participation. The UK Pacts relative lack of success is at least in part attributable to their lack of engagement with the authorities setting priorities for sub regional development.

In territories with ‘free space’, the vertical connections established between the Portuguese sub-regional Pacts and the regional authorities was central to their ability to pursuade other stakeholders to become involved. The success of these Pacts was also influential in new central government thinking on local employment development, shown by its inclusion in the National Action Plan for Employment. It is unlikely that this level of support would have taken place without the support for the approach by the regional authorities, which provided an important communication up from the Pacts and into the centre. In Greece by contrast almost none of the Pacts established positive relationships with higher authorities, although this is at least as much the result of lack of interest on the part of the latter as it is a failure to try by the Pacts.

In ‘receptive spaces’, the definition itself implies the existence of a nascent policy framework sympathetic to the Pact type approach. Selection of Pacts by the Member

56 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

State authorities was in many cases done on the basis of their suitability for the embedding of local development policy, so the Pact would be formed with its vertical links already in place – the Irish Pacts are good examples of this.

In Spain, the forging of a new form of connection between the level and higher authorities was recognised as a central achievement: ‘Basically, the pact methodology involves the definition of a new territorial model based on a higher participation of the local level. This process requires several changes and a consensus among actors of different political orientations and administrative levels on very sensitive areas. In this sense, the pact contributes indirectly to raise the reflection about the necessity to develop a new model of territorial intervention.”21

However, this does not mean all Pacts already had, or successfully maintained, all the vertical links needed for successful operation. A particular weakness was the use of vertical links as channels for dissemination and mainstreaming. Whilst many Pacts were able to use their connections to higher authorities to legitimise and strengthen their operational position, rather fewer transmitted information back up the line with the aim of embedding the approach in wider programmes. Examples of success in this respect include the Austrian Pacts, whilst others such as the Amberg-Sulzbach Pact (DE) show a more limited focus on delivering their direct work rather than communicating the lessons.

4.6 Differences in governance context within Member States

The way in which governance context is discussed in this evaluation – that is, with each Member State providing the same governance context to each Pact – is something of a simplification of the full situation. Certain Member States have ‘asymmetrical’ elements that furnish equivalent levels of government in different regions with different competencies. For instance, in Spain ‘autonomous regions’, such as Catalonia, have had more powers devolved to them than other regions, such as Andalusia. In Germany, on the other hand, the administrative structure of some Länder differs from that which prevails in others. In the city-states of Hamburg and Bremen, for example, the Land governments are the city authorities; the authorities possess the effective powers of two tiers of government in one. German towns situated in other Länder, therefore, have a Land government positioned above them.

These examples illustrate the fact that different governance contexts exist simultaneously within the same Member State. The particular governance context faced by the Pact depended on the particular configuration of powers and institutions of government within the locality into which it has been inserted. It has been beyond the scope of this evaluation to pursue this line of investigation in greater depth, but it is nevertheless important to draw attention to this issue as an important determinant of a particular Pact’s governance context.

21 quote from Spanish national report

57 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

4.7 Pact performance and ‘family’ context

4.7.1 Family clusters In an attempt to cluster Pacts into groups sharing similar features, three core characteristics (spatial context, labour market conditions, and the dominant development problem) were used to divide them into ‘families’. This segmented the 89 Pacts into nine well-filled family groups and one smaller one for islands and enclaves. The three criteria used represent a simple model through which to examine the variability of Pact performance, and was also used to construct the sample from for selection of case studies. This ensured detailed information was available about Pacts across different situations.

The table below outlines the basic system used to segment the 89 Pacts into families.

Table 4.4 Descriptor 1: Spatial Descriptor 2: Labour Descriptor 3: Dominant context market conditions Problem to be Addressed 1. Rural Peripheral 1. Tight but access 1. General Reconversion problems 2. Urban/Industrial 2. Slack 2. Insertion for Excluded groups 3. Semi Rural 4. Metropolitan 5. Island or Enclave

This method allows development of a further hypothesis for testing, that:

• Pacts in similar spatial and labour market conditions, delivering similar operational responses, will tend to achieve similar degrees of success

Figure 4.4 presents the mean scores and range of scoring divided between the different family groups – the key underneath describes each family, and the distribution of individual Pacts between these is given in annex 3.

In practice this division of Pacts has not proved to demonstrate systematic differences between Pact performance as a result of family group. The highest mean score was achieved by those in urban/industrial areas with slack labour markets which focused on group insertion, with a figure in the high 80s; conversely the lowest mean at around 60% came from those in semi rural areas with tight labour markets, focusing on group inclusion. All other family groups delivered mean scores between around 70 and 80%.

58 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

Figure 4.4: Pact scores by family group

TEP Overall Performance by Family Group

120.00%

100.00%

80.00%

60.00%

40.00% Performance (%) Performance

20.00%

0.00% 222 311 321 422 221 412 511 121 322 312 Family Group (Sorted by mean score)

Key Descending order of mean score 222 Urban industrial, slack labour market, insertion of excluded groups 311 Semi rural, tight labour market, general re-conversion 321 Semi rural, slack labour market, general re-conversion 422 Metropolitan, slack labour market, insertion of excluded groups 221 Urban industrial, slack labour market, general reconversion 412 Metropolitan, tight labour market, insertion of excluded groups 521 Island or enclave, slack labour market, general reconversion 121 Rural peripheral, slack labour market, general reconversion 322 Semi rural, slack labour market, insertion for excluded groups 312 Semi rural, tight labour market, insertion for excluded groups

The clustering exercise demonstrates therefore that the primary reasons for variability in Pact successes do not appear to lie in the influence of these factors taken together, i.e. in the particular contexts represented by the clusters. Is there then any distinct influence discernible from the individual elements? Figure 4.4 looks as the individual elements sequentially – human geography; type of labour market; and primary model of intervention.

59 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

4.7.2 Human geography

Figure 4.5: Scores by family elements

TEP Overall Peformance by family elements

100.00%

90.00%

80.00%

70.00%

60.00%

50.00%

40.00%

Performance (%) 30.00%

20.00%

10.00%

0.00% 1** 3** 2** 4** 5** *1* *2* **1 **2 Family Group

Key Geography 1** Rural peripheral 3** Semi rural 2** Urban industrial 4** Metropolitan 5** Island or enclave Labour market *1* Tight labour market *2* Slack labour market Model **1 General reconversion **2 Group insertion

Rural areas The mean scores show a modest degree of variation between different geographical contexts, with rural-peripheral areas returning the lowest mean, and urban industrial the highest. All types of area include both high and low scoring Pacts. The poorer performance of more rural peripheral Pacts may be attributable to the difficulties

60 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 associated with their management by DG Agriculture, with few such Pacts able to claim their second tranche of funding.

However there is also evidence to support the view that these sparsely populated areas may be less appropriate targets for Pact like intervention due to lack of critical mass – they cover small populations and there are relatively few bodies available for co- ordination or joint action. Pacts in a context that is overwhelmingly rural and remote – such as those of Itä Lappi (SF) and Iisalmen seuta (SF) – operate over a far larger area than others, with a low density of population. In such circumstances, a Pact covers a vast area, but the area represents a relatively homogeneous one, with common characteristics and relatively little variability in, for example, labour market conditions and economic influences. The implications of this for how a partnership works in practice and for the whole idea of being “bottom-up” are quite different under these conditions. The portfolio of actions for a TEP in such an area is more distinctly different than areas with denser populations and more complex economies. There is something of a parallel with the issues affecting islands and enclaves, which whilst not covering such large areas (by definition) tended in the Pact programme to be areas with relatively small scale populations and peripheral economies. Pacts that operated in such conditions in the main sought to help and stimulate entrepreneurship in the territory to tackle the flight of businesses and depopulation. The integration of socially excluded groups into the labour market was seen as less of a major problem than in many urban areas, while the numbers of unemployed workers in need of retraining – such as those formerly active in the forestry industry in northern Finland - was higher. Owing to the geographical remoteness of the area, the idea of a ‘bottom- up’ partnership based on the presence of a wide range of actors active in the field of labour-market policy that meet regularly was different. Pacts in peripheral rural areas had fewer partners, and co-ordination between them partners was logistically more problematic.

The smallest scale rural Pacts were found in Finland, covering populations of less than 25,000 inhabitants. Medium scale included the Matese (IT) and Jamtland (SE) Pacts covering populations of around 100,000 inhabitants, while larger Pacts included North Wales (UK) and Norte Alentejo (PT) with populations close to half a million people. The partnerships in most of these Pacts in rural areas were relatively contained as they included less than 15 partners. The examination of the Pact Action Plans showed that 90% of these Pacts included stimulating entrepreneurship in their portfolio of actions, while 60% stated actions to tackle rural underemployment as the object of activities. As far as target groups are concerned 40% of the ‘rural’ Pacts expressly targeted rural hidden employment and spatially localised unemployment, with a third of them targeting low-skilled workers. The more successful of these Pacts were characterised by: • commitment to joint action by relatively small partnerships, involving the key local partners with labour market influence • successful stimulation of entrepreneurship, often involving close involvement in devising and delivering programmes • good understanding of the need to address supply and demand side issues simultaneously, for example the retraining of forestry workers in skills in demand within a reasonable distance

61 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

The less successful by contrast were: • too diffuse in their membership, with too few members meeting infrequently • unable to find any purchase on fundamental economic and labour market issues

Semi rural and urban industrial The urban industrial group provides an instructive contrast. Although including a small number of very low scoring Pacts, this group was heavily clustered at the upper end of the scale. A very similar although slightly less definite pattern applies to the semi rural group. In these cases the reasons for good performance may be attributed to effective application of the core Pact idea - sub-regional co-ordination of existing local bodies. These areas tend to be characterised by medium to high population densities, distributed amongst medium sized, distinct settlements. Each of these will usually have its own identity and associated local administration, as well as other locally focused bodies such as chambers of commerce and third sector networks. The successful Pacts in these circumstances tended to be those which managed to mobilise the different actors across larger territories, stimulating new contacts and new actions.

For many of the Pacts, the original prescriptions for how TEPs were to be established saw them best fitted to operating at a level that coincides with free-standing towns or that are located in semi-rural municipalities. The spatial scale “above the municipality but below the region” virtually predisposed them to be based on this geographical context. Local partnerships also work well at this scale, with easy access and the prospect of regular interaction, but without the intensity of organisational relationships found in metropolitan areas. It would be expected a priori that Pacts in what we have labelled the semi-rural family found it easiest both to adopt the prescribed form and be able to deliver the main objective of the programme. For example, the Kronjyland Pact (DK) saw its scale (with 120,000 inhabitants) as an advantage. Pacts seemingly operate well in coherent areas that represent a travel-to- work area.

There are 29 pacts in the two families in semi-rural territories facing slack labour markets. The action plans are very diverse as the areas face such different needs requiring different approaches. Some patterns can be noted in terms of the target groups of these pacts, however. Young people are the predominant concern in half of the Pacts in semi-rural areas. This is followed by women, low-skilled workers and unemployed due to the decline of the traditional industrial base in the area. These were groups targeted by approximately 40% of the Pacts in semi-rural areas whether they faced tight or slack labour market conditions.

While the semi-rural cases might well have been those that fitted best to the Pact ideal, the selection process did throw up some serious anomalies that took the economic and geographical context well outside the range prescribed for TEPs. For example, a number of Pacts covered very large populations and what could only be described as whole regions. The Nordrhein-Westfalen Pact (DE) encompassed three areas in the Ruhr region that collectively took in approximately three million inhabitants. Unsurprisingly, the Pact ran into problems - with the emergence of three distinctly different approaches and sets of priorities that could not be reconciled with one another. It was noted in the EI2000 Final Report of the National Experts that the Nordrhein-Westfalen Pact lacked a coherent sense of identity. Nord Brabant (NL)

62 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 covered 2,3 million people while Öresund (DK) and Cadiz (ES) also had populations that were considered too large. In fact, Öresund covered an area with approximately 3.5 million people. In some cases, like the East Midlands 3 cities Pact (UK) and the West Attika and Pireaus Pact (EL), problems emanated from the fact that the localities involved faced needs that were too divergent and even had competitive attitudes towards each other. This led to difficulties in following coherent strategies, with the West Attika and Pireaus Pact ending up forming two separate partnerships, action plans and being implemented as two separate Pacts.

Where these Pacts were active in Objective Two areas, and the added value of a TEP in terms of stimulating a new partnership-oriented form of labour-market intervention was potentially limited. The more successful of these Pacts found a role that did not duplicate activities carried out elsewhere and by other institutions and bodies, for example in St Herblain (FR).

Metropolitan Metropolitan areas show a considerable range of scores. About half the group scored very well whilst about one third were very poorly rated. In the main, large metropolitan areas (in the Pacts programme mainly capital cities or very major regional centres) were tight labour markets with significant labour shortages, albeit with pockets of unemployment of varying size and significance.

Four of the ten Pacts in Metropolitan areas covered a population of 300,000 while another three Pacts covered close to 1 million inhabitants. In terms of the size of their partnerships, six of the ten Metropolitan Pacts were based on medium-sized partnerships of between 11 and 25 core partners. Their portfolios of actions were quite diverse and no strong patterns can be noted, except for an emphasis on measures to increase employability and tackling the unemployment of young people, which was included in the Plans of half of the Metropolitan Pacts. The main target groups that were the focus of Metropolitan Pacts with tight labour markets were young people and long term unemployed (5 out of 8 Pacts targeted them). This was followed by other disadvantaged groups such as people from ethnic minorities, low skilled workers and less often groups such as travellers or women returners to the labour market. Interestingly, young people were also the target of the Action Plans of 3 of the 4 Pacts in Metropolitan areas with slack labour markets. In areas like this with tight labour markets, there will inevitably be less emphasis on stimulating entrepreneurship. Rather, the challenge will be to bring about a better match between the demand for, and supply of, labour by integrating excluded groups into the labour market.

Large cities have been the subject of many initiatives and programmes, both national and European, in recent years, and also tend to have relatively dense, often competing, political and bureaucratic structures. The experience of Pacts shows that the successful approaches in these circumstances found ways to bring different stakeholders together, most often in ways which brought labour supply and demand into better alignment. Particularly highly rated were approaches which addressed support actions to smooth the path from lack of employability towards suitability for the types of jobs on offer, and tackled barriers in the way – the Bruxelles Capitale (BE) and Dublin (IRL) Pacts are good examples of this.

63 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

In contrast, the poorly scoring metropolitan Pacts tended to be hampered by their inability to find a satisfactory role within the crowded political and policy environment. This was particularly the case in the UK, where partnership working and co-ordinated action has been the main policy instrument for several years. Metropolitan areas tend to be recognised as unified labour markets and economies. In member States with a partnership working orthodoxy, a lot will already be taking place, and the Pact approach struggles to find a role.

4.7.3 Pacts and the influence of geography: summary

Table 4.5 brings together the various points made above concerning the diversity of interaction between Pacts and their geographical context. It is clear that, whilst local conditions provide a context against which Pacts needed to react, there was no specific set of circumstances which would ensure Pacts were successful or unsuccessful in different types of places. Instead, Pacts tended to focus on similar types of activities, and were drawn towards partnership structures of similar sizes, dependent on their ability to influence the macro economy:

Table 4.5: Interconnection of contexts Spatial Portfolio of actions Ability to affect Size of setting undertaken territory’s macro- partnership economy Rural Stimulate entrepreneurship; Highly constrained by Small measures to tackle rural difficult economic climate under-employment (e.g. with largely external retrain unemployed skilled causes. Working against and unskilled workers) declining population size and outmigration Metropolitan Integrate excluded groups Restricted but greater Medium to / urban into the labour market; potential than in remote large industrial retrain unemployed in areas rural areas – particularly in with slack labour markets areas with tight labour markets; in territories facing crowded policy platforms, the ability of the Pact to make an impact particularly conditioned by its ability to adapt to existing governance context and structures Semi-rural Enormous diversity, but Generally good potential – Medium some emphasis on reaching if the Pact successfully out to young people, low- adapts to the governance skilled workers, and context unemployed in need of retraining

64 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

4.7.4 Type of labour market and model of intervention

Figure 4.4 above also contrasted the scores of Pacts in tight and slack labour markets, and the dominant model of intervention. The definition of tight or slack was arrived at in relation to the relative state of the labour market in the Member State. For a labour market to be classed as tight the predominant condition needed to be the significant availability of jobs and report by employers of skills shortages and recruitment difficulties. This does not preclude the existence of unemployment, although in these conditions this will tend to have been concentrated in particular micro localities and/or amongst population sub-groups, often lacking particular skills or other facets of employability. The dominant model of intervention was divided between those Pacts with a specific focus on the re-insertion of defined groups into the labour market; and those with a more strategic focus on the labour market and local economy more widely.

According to the figures, there was little discernible difference in success scores between Pacts when divided up in this way. The mean scores are almost identical, and both groups include Pacts rated very highly. The main area of difference comes with the more poorly performing Pacts in relation to labour market conditions, which are somewhat more concentrated in slack labour market areas. The lack of a clear difference between successful and unsuccessful Pacts in relation to these issues largely explains the failure of the family clusters to offer any insights into systematic influence of context on achievements.

The analysis of family groups therefore allows us to arrive at the following conclusions:

• using these three factors to cluster Pacts into family groups is a convenient way of describing those with similar characteristics, but there is little association between family group and likelihood of successful outcomes. • when separated into individual factors, type of geographical area has the most noticeable impact, with Pacts at the extremes (rural-peripheral, island or enclave, metropolitan) returning mean scores lower than those in the more intermediate semi-rural and urban industrial locations. • however there are examples of high and low scoring Pacts in all geographical areas. The best are characterised by an appropriate adaptation to local conditions, the poorest by a failure to define a complimentary role for themselves. The governance context and the Pact’s accommodation with it remains the most important conditioning factor • different types of labour market and of intervention model do not in themselves have a significant impact on Pact successes

Before concluding this chapter we consider one final influential issue – that of the relationship between Pacts and the European Commission itself.

4.8 The role of the European Commission

An important factor that has conditioned the performance of the Pacts is related to the role of the European Commission. The Commission created the programme in the

65 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 first place, and managed and co-ordinated it throughout its life. Although primary decisions as to which Pacts should be supported were made by Member States, the Commission maintained the continuing links with Pacts, provided their funding, and received information from them. More details of the relationship between Pacts and the Commission was given in the introduction, however for the purposes of this analysis we can consider this issue in the following way:

• in addition to all other influential factors, it can be hypothesised that the nature and content of the relationship between Pacts and the Commission is likely to have an impact on the Pacts’ ability to be effective. Additionally, since administration of the programme was distributed between three DGs, there may be systematic differences in these relationships

The Commission provided a relatively small amount of technical assistance to Pacts, and gave responsibility for direct relations with them to a network of desk officers distributed between the DGs responsible for regional policy, employment, and agriculture. All these officials were responsible for volumes of other work elsewhere, and we must acknowledge that in comparison with the amount of attention required by, for example, management of mainstream Structural Funds, relatively little could be devoted to Pacts. Even allowing for this however Pacts’ impressions were of considerable variability in the seriousness with which they were regarded by Commission officials, the messages they were given, and the support received.

Some positive comments were made concerning Commission support for finalisation of Pact Action Plans, particularly with regard to practical measures, and the bringing of greater clarity to the programme’s intentions. Pacts were required to agree their action plan formally with the Commission. More common however were less positive comments, particularly concerning the extent of contact, and a perceived emphasis on quantitative outputs. Some Pacts felt they needed more guidance from the Commission as to what they were supposed to do, and also reported delays in getting questions answered or basic bureaucratic procedures completed. Others felt the Action Plan stage put pressure on them to establish inappropriately quantified output targets, particularly with regard to employment creation. Overall, reporting between Pacts and the Commission was weak. Whilst Pacts were expected to undertake their own evaluations and co-operate with any organised by the Commission, they were not required to ring fence any funds for this, nor were they given any specific advice or instructions as to what sort of information to collect in what form for monitoring and evaluation purposes. As we have seen, this has led to a general weakness in the data available for evaluation, which lacks consistency, detail and depth.

The Dudelange Pact (LUX) for example expressed concern at the apparent inability of its Commission contacts to understand the nature of the work it was trying to undertake, which focused upon integration of highly disadvantaged people into a tight labour market with low unemployment. It felt under pressure to establish and report on job creation indicators which it regarded as inappropriate. Indeed, one of the lessons from the Pact was there should be clarity about the partnership’s mission compared to the European Commission’s strategy from the start. Otherwise, there might be conflicts between the view of the European Commission (here: need for job creation) and the view of the Pact partnership (emphasis on support to job seekers and for selective job creation via economic diversification). In this case the Pact’s success

66 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 could not be measured by the creation of jobs in purely numerical terms since it sought to integrate small numbers of people suffering from social exclusion back into the labour market.

Pacts’ relationships with DG Agriculture are felt by all sides to have been more problematic than those with the other DGs. At its most basic this involved the inability of any DG Agriculture Pacts to claim their second tranche of funds, meaning they missed out on one third of the resources their colleagues had available to them. There were other more generalised expressions of dissatisfaction with this relationship, which seems in the main to have arisen from a lack of serious attention being given by this part of the Commission to the programme, reflecting a view that it had little to offer in a policy context where existing programmes such as the Leader Community Initiative were already tackling broadly similar issues. Pacts reporting to DG Agriculture were particularly concerned at a perceived unwillingness on its part to countenance deviations, however sensible, from stated objectives. Pacts felt this to be at odds with the experimental and innovative nature of the programme

Clearly, in the case of the DG Agriculture projects, Pacts suffered explicitly from lack of additional resources to which they should have had claim. This will have restricted the volume and nature of work they were able to undertake. Beyond this it is more difficult to identify and specific impact on Pact achievements arising from their direct interaction with the Commission. Indeed, the lack of clear advice and policing of monitoring and evaluation itself makes reaching such judgements more problematic.

Where it can be argued that the Commission’s actions had an effect was more at the level of overall understanding of what the programme was intended to achieve. Its conception was as a type of intervention radically different from traditional project based approaches, with the emphasis on partnership working and strategic co- ordination rather than direct project work. As has been detailed at some length above, many Pacts do not appear to have fully understood this distinction, or if they did, were left with little option but to run projects if they were to find anything useful to do, given the unfavourable policy contexts in which they found themselves. Mixed or absent messages from the Commission as to what it was funding Pacts to achieve, and how they should assess whether they had achieved it or not, undermined the coherence of the programme and likelihood of good quality learning not only being created in the first place, but also understood and disseminated. As a pilot project, there was merit in defining things loosely at the outset to allow the diversity to develop and to see what worked where. That said, it was clearly not helpful to issue mixed messages in relation to quantified targets and outputs.

A number of conclusions can be drawn:

• In the first instance, the intended mission of the Pacts needed to be adequately understood across the Commission services. Subsequently, the European Commission should have more clearly communicated what precisely and realistically could have been expected from the Pacts. The Commission needed to appreciate that Pacts operated in different contexts, which would lead to different types of priorities.

67 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

• the Pacts would have benefited from greater stability in personnel at the Commission - several Pacts made the point that their desk officer at the Commission did not really know the Pact, or even what the programme was really supposed to be about. This is to some extent understandable, considering that the funds involved in the TEP intervention are very small compared to the funds tied to the portfolios of Commission desk officers. It nevertheless led to disappointments on the part of Pacts.

4.9 Summary of influential factors

The analysis provided in this chapter leads to a number of conclusions, some of the most significant of which can be summarised as follows:

• Pacts could do little in the face of macro economic factors beyond their control, and these could have far greater impact on the labour market and employment situation than anything the Pact might do. Their work should always be seen against this background

• the clearest influence on Pact performance comes from the combination of Member State attitude and governance context situation. Crowded policy arenas made for a more difficult environment for Pacts to work in. Those which succeeded in these conditions, were those either linked in with a government initiative heading in the same direction, or confined to a focus on limited local actions where value could be added. Pacts found it easier to find a role where there was a legal framework and some tradition of local employment action and partnership working, but not a congested environment of overlapping remits and bodies. Much could be achieved in empty policy spaces, but only with constructive support from a superior authority

• very large Pacts found it difficult to achieve focus and could be undermined by excessive rivalry between constituent partners. They succeeded best with limited but highly focused strategic roles, but needed strong government support for this to happen. Very small Pacts relied on strong vertical links with higher authorities if they were to achieve any strategic results. Otherwise then tended to revert to direct project work which might be more or less successful, but did not advance the Pact concept very much

• Pacts in peripheral areas had more difficulty finding constructive roles for themselves than those in more densely populated (both people and institutions) places, due to the difficulty of finding critical mass. They were particularly prone to helplessness in the face of large scale macro economic forces

• the nature of the labour market, and the broad model of intervention followed, did not in themselves impact on Pact’s likelihood of success

• overall, Pacts were influenced by a complex of factors, some within their control, some beyond it. The interplay of these was complex, but even allowing for this it must be concluded that the Pact approach can achieve success in most

68 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

environments. The wider value of the success achieved is more variable however. A well motivated team can find useful ways to spend the money invested on activities which have beneficial labour market outcomes. But the value of the Pact model, based on partnership formation and nurturing, strategic focus, and co- ordination of activities, will be best achieved with a combination of:

Ø receptive space in the policy and practice environment Ø a positive attitude from superior authorities Ø the availability of sufficient bodies able and willing to be co-ordinated, and whose joint working is genuinely additional to existing structures Ø good internal management Ø the strategic vision to identify opportunities for new approaches, and the application to put them into practice

The analysis has so far concentrated on the nature of the Pacts, their achievements, and the factors which have influenced these results. In the next chapter we will turn to consider the Pact programme in a more holistic way, in relation to key programme expectations such as added value.

69 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

CHAPTER 5: TERRITORIAL EMPLOYMENT PACT EFFECTIVENESS AS A POLICY INSTRUMENT

The previous sections have presented an overall description of the scope of the TEP Programme; summarised what can be said with reasonable certainty as to the achievements of Pacts; and discussed a range of factors which have, to a greater or lesser degree, influenced the types of work undertaken by Pacts and the value of their achievements. The following material changes focus and considers the TEP Programme as a policy intervention, which was intended to operate in such as way as to: • add value to the policy and implementation structures in the localities in which they operated • deliver their results cost effectively, in that the investment made was justified by the Pact’s achievements • be a catalyst for new types of co-operation and action, primarily by bringing existing resources and organisations into new forms of relationship and stimulating new forms of activity • generate results and lessons which would on the one hand continue in the territory concerned beyond the time of Commission support (sustainability), and on the other have relevance for other actors attempting similar types of intervention (dissemination) or formulating policy responses (mainstreaming)

5.1 Added value

The added value accruing from the TEPs can be seen in a number of different ways. These can be grouped under three broad headings: 1. partnership development and working 2. co-ordination and rationalisation of policies and programmes 3. implementation of specific labour-market actions

5.1.1 Partnership The programme as a whole was organised around the idea that co-operative partnership working would add value to the conception, implementation and delivery of employment focused activity. The contribution of the programme here can be divided into two. First, there was an opportunity for the chosen sites to experiment with partnership models appropriate to their own circumstances. This adds value to the policy process by providing guidance on the dimensions of a general working methodology for partnership under different sorts of circumstances. Second, the experience of these partnerships can offer important lessons for policy about how the space to innovate might be constrained under the different sorts of governance system that operate across Member States.

Following on from the analysis in Chapter 4, it is apparent that the encouragement of partnership working had most potential to add value in ‘free space’ countries, and those ‘receptive space’ countries where co-ordination at the level where the Pact

70 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 operated was not usual. In these circumstances added value came from the resourcing of activity to build and nurture partnership structures of various kinds, and establish a method of working not previously known or prioritised. The likelihood of positive accomplishment in these circumstances was largely dependent on the quality of the relationship of the Pact managers with key stakeholders above, below, and alongside their level of working. Where these were supportive and / or willing to become directly involved (for example the Portuguese Pacts in general), it is clear that partnership working had the potential to enable information to flow between partners in new ways; joint discussion and decision making to take place with better information, more debate, and more ownership of the process. In these circumstances the input of the Pact was by definition additional to that possible under existing structures, since they were introducing a novel form of action. This could be described as added capacity building.

As discussed in Chapter 4, it was not inevitable that value would be added in these circumstances. Many Pacts were less than successful in creating and effectively using partnerships, with the main impediment being the quality of their relationships with others, with additional problems for some arising from lack of a clear consensus on what sort of issues might be tackled (Greek Pacts in general). Whilst the formation of partnerships might be achieved, it was not necessarily the case that this would lead to positive labour market or development activity if, for example, the partnership could not agree on priorities, or if its objectives were unable to progress through lack of support elsewhere.

In the ‘crowded platform’ and some receptive space countries, the added value of the programme was more likely to arise where the additional resource provided the opportunity to work with existing structures in providing the space to innovate. We have discussed the barriers likely to arise from situations where the Pact had to fit in with existing partnerships and initiatives, and shown how the successful Pacts were those which found a positive role to play or niche in which to operate. We can identify added value here in those Pacts which used the limited funds available to introduce a new element into this situation, for example by rationalising the membership of existing partnerships; enabling existing ones to create complementary roles for themselves; or researching complex issues (Dublin (IRL)). Conversely, there were other situations where, whilst the Pact may have found an apparently useful role, for example in formulating and managing local projects, the added value is more difficult to identify. In these cases the Pact was effectively substituting for work that might just as well have been delivered through other programmes. In these cases little value was added beyond the increase in the volume of activity taking place – more absolute output, but only of things that could have been done by others (Tottenham (UK)).

A combination of finding an improved working method for partnership that could, of itself, serve to create more space for action can be seen where the opportunity was taken to widen and strengthen existing partnerships. Where this succeeded in bringing a new set of partner types to the table, for example by adding private sector or social economy representatives where they were not previously present there is clear added value. (Bruxelles Capitale (BE)). In other cases the added value was less clear in that the partnerships simply expanded by bringing in more members, but of the same type as already present. The Pact might have provided the ‘psychological’

71 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 spur to expansion, or the funds to enable larger meetings and activities, but true added value here would need to be sought in the novelty of activity undertaken rather than the method of partnership working itself.

5.1.2 Policies and programmes The work of Pacts in this context was largely concerned with either the co-ordination of the activities of others (including partnership members) alongside creative activities in relation to the labour market. For the former, the added value of this particular programme can be seen through the opportunity provided for Pact partners to take a more strategic and consensual overview of their collective employment related work. The encouragement of partnerships to inform themselves better; or to bring their individual activities into better alignment; or seek to avoid duplication and gaps in coverage, would all potentially add value. As with other issues however the question must be asked here as to whether this was a genuine addition, in that it simply could not have taken place without the stimulus of Pact thinking, or whether it came largely from the availability of a little more funding which supported obvious actions. The distinction here may be subtle but is important. The Pact programme added real value where it introduced new ways of working, new ideas and techniques, stemming from its vision of how employment and local development might be better organised and delivered. Where the funds it made available were simply used to undertake activity that was already well understood but needed extra resources the added value is much less. We have already seen this in relation to the funding of local projects. It applies just as much where all that happened was an increase in partnership membership or more joint working, where just giving more funds from any source could just as well have achieved the same result.

Examples of ‘real’ added value identifiable from the evaluation in the case of policies and programmes included: • introduction of ‘territorial’ thinking, encouraging partners to focus on the issues confronting particular places seen as a whole, and involving key stakeholders (Albertville (FR)) • rationalisation and co-ordination of policy and activity, where complex frameworks became simpler and more efficient (French Pacts in general) • extension of existing notions to a new level, for example by taking existing moves to devolution further ‘downstream’ (Vallès Occidental Pact (ES), Limburg (NL)

5.1.3 Innovative Actions and Initiatives Added value from the TEPs Programme can also be observed in specific labour- market initiatives and actions. It tended to take the form of the implementation of innovative actions; and in the ‘sponsorship’ of other labour-market initiatives. The programme as a whole was dedicated to the encouragement of innovation, and as discussed above there is evidence to show how Pacts felt themselves to be acting innovatively and in what contexts. Some of the more innovative direct labour market actions were seen in some of the least promising governance contexts such as the UK, where there was little scope for other work. Some Pacts turned to the invention and delivery of labour market actions to give themselves a constructive role. Some added

72 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 value here came from genuinely new approaches (the Tottenham (UK) example of supply and demand matching being a good example), but it is hard to argue that this was really within the strategic objectives of the programme. There is also an argument that these sorts of actions were in danger of representing deadweight.

Other examples of added value from direct delivery include the filling of gaps in mainstream provision, which was the prime concern of many of the Pacts situated in the crowded policy platform situation. The Viborg Pact (DK) for example focussed its actions on marginalised groups, believing that existing policies and programmes did not address the particular problems of these groups in an adequate way. In this case, the Pact was able to provide services or carry out actions that complemented rather than duplicated existing mainstream provision. The Calatino Pact (IT) sought to integrate the activities of the Pact with other related actions and programmes, such as Leader II, PRUSST, Urban II, and the One-stop Business Centre. The idea of linking the actions of the Pacts to other initiatives to avoid duplication is a point made by several of the TEPs. The added value from this sort of approach lies in its ability to demonstrate the viability of actions based on analysis of the current situation and engineering responses that fill gaps and avoid duplication. The extent to which this is a valuable output for the programme’s purposes lies in their demonstration effect, and in the possibility of stimulating these sorts of actions through new forms of partnership.

Another way linked to specific activities in which the Pacts have been able to provide added value concerns their ability to act as a ‘sponsor’ for other labour-market initiatives. In Sweden, for example, some Pacts provided ‘seed’ money for pilot and feasibility studies before projects were put into practice. The South Tyneside Pact (UK) similarly provided small-scale financial support for providers to help them in both the development and delivery of projects. According to the Case Study, this ultimately made the Pact more of an ‘influencer’ of projects rather than a direct deliverer. The St Herblain Pact (FR) also acted as a facilitator by assisting the activities of the plate-forme d’initiative locale. This body – which was created by a network of partners at a national level - encouraged new projects by supporting them with technical assistance and funding. The St Herblain Pact helped the plate-forme to mobilise local partners for its specific projects. Added value here comes from the Pact’s use and demonstration of an approach to local employment development not previously seen – strategic, informed, co-ordinated and in some cases influencing the deployment of significant funds. Mainstream programmes rarely work in this way, with projects at local level often being a large and confusing mosaic of funding sources. Promoters have frequently been in overt or covert competition with each other, and there has been little attempt to influence funding decisions across programmes and calls for application with a strategic overview of what is really needed. The moves towards such overviews taken by, for example, the EQUAL community initiative and Objective 3 co-financing, are a reflection of the recognised need for co-ordination. The TEPs programme funded a number of projects which adopted this approach successfully, and this is a clear area where added value from indirect influence over labour market action can be seen.

In the above we have emphasised several times the importance of the demonstration effect, the fact that for a programme like TEPs it is not the work in its own right, but its ability to demonstrate the effectiveness of alternative approaches that matters most.

73 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

Therefore the added value of, for example, identifying and filling gaps in provision, lies in whether the way this was done was new or interesting, and whether the lessons of this were identified, learned, and transmitted to others. This issue is considered in more detail in section 5.4 below.

5.2 Cost effectiveness

5.2.1 Programme effectiveness

The input of the EU into the Programme consisted of a first tranche of € 200,000 to each of the 89 Pacts, and a further € 100,000 to the majority of Pacts (with the exception of the Greek Pacts, some UK Pacts and some of the Pacts managed by DG AGRI). In addition to these inputs, the EU paid for a central Technical Assistance Office (EI2000), whose role was to facilitate Pacts in the formulation of their Action Plans as well as offering advice and helping monitor their progress during the life of the Programme. Finally, EU funds were devoted to the production of dissemination materials on the TEP Programme (brochures and publications) as well as networking events and seminars. The total commitment therefore comes to around €25m, a small figure compared to mainstream programmes, although not out of line with many other Commission experimental initiatives such as Third System and Employment. The exception to this is of course the Human Resource Initiatives which have committed hundreds of millions of Euros since the mid 1990s.

This input of funds from the EU ‘purchased’ 89 experimental Pacts. These, as we have seen, involved a significant number of what can be characterised as ‘new’ partnerships, as well as a collection of achievements related to employment and territorial development outcomes that can be to a greater or lesser degree attributed to these Pacts. This report has gone into detail as to the achievements which can be ascribed to the Pacts individually and as a whole, and demonstrated the huge variety of experiences of Pacts and different ways in which they conceptualised the results of their work.

Since the issue of cost effectiveness can only be assessed through estimation of the counterfactual, and no reliable quantification is possible, it is difficult to estimate the cost effectiveness that Pact outputs represent. However, considering the 89 Pacts as a whole, we would estimate that some 25% of Pacts not only achieved substantive local results consistent with the objectives of the programme, but also understood how and why these achievements had been reached, and furthermore effectively communicated to other parties such that the lessons were to some useful degree taken up and acted upon. These Pacts could be considered to be 100% effective

For the remainder, we estimate that some 45% produced useful results in one or two of the three conditions outlined above, but not the full set of circumstances necessary to be considered what the Commission had in mind at the outset of the programme. These could be considered perhaps 50% effective. The remaining 30% produced little of value for the purposes of the programme. They can be considered as ineffective.

74 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

Relating this to the programme spend enables us to provide a view of the amount of funding which was effective, as follows:

Effectiveness of spend Calculation of value Value spend productively obtained Fully effective Pacts = 25% @ 100% €6.25m 25% effectiveness * €25m programme spend Partially effective Pacts = 45% @ 50% €5.63m 45% effectiveness * €25m programme spend Ineffective Pacts (in terms 30% @ 0% effectivenss €0 of programme objectives) = 30% Total effective spend €11.88m Proportion of spend 47.5% effective

Relating this to the programme spend, would suggest therefore that some €11.9m of the €25m spend, or 47.5% was effective. This should be contrasted with other innovative or even mainstream programmes where the proportion of highly positive results can be just as low, often for the deployment of significantly greater resources.

Taking all this into account, along with the degree of influence Pacts have achieved over local and nationl programmes, we would draw the following conclusions:

• the investment of €25m succeeded in creating a distinct network of partnership networks, each of which in its different way sought to tackle issues of local and employment development

• where Pacts found little room for manoeuvre, or lack of support from key authorities; or were poorly managed, they achieved little, and the investment made can be considered a drag on the programme, achieving little in the way of demonstration effects or wider influence. These experiences can however be seen as instructive in highlighting the difficulties which this sort of approach might face, and therefore give an insight into how future partnership based territorial development might need to be structured to avoid barriers to progress

• where Pacts chose to focus on employment action which could have been funded under other mainstream EU or national programmes there was substantial deadweight at the programme level. This is not to say that the specific beneficiaries of the actions taken were not worthy of support, nor that they necessarily would have benefited from another programme had the Pact not existed. However the limited funds available to Pacts added very little indeed to the overall pool of programme funds, and the achievement of significant absolute outputs (jobs created etc) was not an objective of the programme. These Pacts tell us little about the value of a partnership-based approach to the devising of territorial employment strategy, nor of the value of collaborative approaches to joint action. What they do tell us is that in circumstances where local bodies and partnerships feel themselves to have

75 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

insufficient funds to tackle the social issues which confront them there may be a tendency to use whatever funding is available to support direct action, with little regard to the overall objectives of the funding source. The reasons for this can lie in a number of factors, including lack of clear communication through the channels from programme creator to individual implementer; lack of alternative opportunities for the Pact to pursue the programme’s intended objectives; local pressure to tackle immediate pressing needs rather than devote time to abstract strategic planning. The benefit of spending Pact resources in these circumstances lies once again in the lessons it generates for the problems encountered in attempting to stimulate particular types of action at local level when relatively little direct leverage can be exerted. In our opinion however these lessons are hardly new and have been recognised in many previous experimental programmes, not least those promoted by the Commission. Did we need to be told again that poor specification of the task; ineffective communication with providers; the absence of effective monitoring and the need to reserve powers to require promoters to alter their behaviour may lead to local actors using funds for their own priorities? The Pacts programme tells us that the inclination to capture funds for “bottom drawer” local projects will occur particularly where the policy itself comes up against circumstances where key players are unsympathetic, or where the organisational form intended already exists to a significant degree.

Where Pacts did add value, in the ways outlined earlier in this chapter, the cost effectiveness of the programme is rather higher. Despite the limited funds available to each Pact, there were many instances of significant change occurring as a result of their work. When combined with effective dissemination (see section 5.4 below), we have been able to identify good examples of partnership working coming into effect where it was underdeveloped previously, and of its adoption more widely by other bodies and territories. The circumstances most conducive to this might simply add up to ‘being in the right place at the right time’, that is; where local issues were clearly identifiable and in need of action; where institutions existed receptive to the idea that partnership working might achieve results; and where the wider context did not impose barriers to action. In these situations the programme achieved its intended catalytic effect, bringing different elements together at low cost and leading to results greater than would otherwise have been achieved. It can therefore be argued that, where conditions were favourable, the programme was in fact highly cost effective. Not only were demonstrable results achieved through use of techniques new to the territory, lessons and influence were generated which led to more widespread adoption of useful practices. It might be argued that in at least some of these circumstances the Pact input was once again deadweight, in that the logic of local conditions would have led to partnership working anyway – it is impossible to know for sure the extent to which this would have been the case. In our view, however, the Pact input in most such examples at the very least enabled development towards partnership working for local and employment development to progress: Ø faster Ø wider, involving more partners Ø deeper, enabling more joint work to be done Ø with more self reflection and evaluation of cause, effect and lessons Ø more publicly

76 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

So was the Pact programme cost effective? • Yes, in that a wide variety of useful and mainstreamed results were achieved for relatively modest outlay. • Partially in that there was plenty of dead weight and substitution in many areas, and in others little or no need for a Pact type institution to be introduced. • No in that many of the results are insufficiently clear cut or consistent for the benefits of this particular approach to likely to be adopted very widely.

5.2.2 Scale

Having drawn conclusions as to the cost effectiveness of the Pact programme configured in the way it was, it is also appropriate to consider whether the scale of EU support was of the right level. We have highlighted the relatively small funds provided by the EU (without on this occasion a requirement for match funding to be made available from Member State sources), and there is no serious prospect that a smaller EU input per Pact could have achieved similar results for less cost.

There is however an argument to be made that a greater level of funding might have achieved proportionately more during the same time period, and more again had the Pacts had longer to do their work. The scale of resource was available was predetermined by the Commission, and was not varied in relation to the specific plans put forward by Pacts. Given the novel nature of the intervention it is of course the case that no one could have known beforehand whether this scale was at the right level or not. We cited above various examples of where Pacts pointed to the low level of funds as a hindrance to their work, and overall €2-300,000 over two years was capable of supporting little more than a couple of full time equivalent posts plus direct costs of support and administration. The Pact programme argument would be that the funds were intended to act as a catalyst for action by others, and that the intention was for a small input to mobilise much greater resources from the contributions of partners. As we have seen this did occur in at least some circumstances, and the fact that some Pacts (particularly those we have identified as the most successful) undertook their work with commitment and resource from partners provides some evidence that the TEP input level had the potential to achieve its ends.

In other cases however we would conclude that a greater input might have been beneficial. A number of issues can be identified: • a relatively small level of resource for the Pact limited the absolute amount of time and associated action which could be reliably input to developing, managing, evaluating and disseminating the work, particularly at the beginning and end of the process. In the setting up phase, before substantial additional resources could be mobilised from partners, Pact managers were reliant on their EU funds, and were therefore limited in the amount of effort which could be brought to bear. At the end of the process, with the Pact in many cases not continuing its work or transmuting into something else, managers could be left with little resource to do the work needed to fully understand what had been achieved or disseminate the results

77 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

• limited funds could also give the Pact operation little leverage, little to ‘bring to the table’ to persuade partners to take part and make their own inputs available. In circumstances where alternative routes to employment development were in existence, particularly in the ‘crowded platform’ countries, the Pact could be seen as adding little to existing processes, and not even bringing significant funds into the equation. Where this occurred we would argue that in many cases it demonstrated either that the Pact had no useful additional role to play, or did not succeed in making a case for being needed in the local context (a reflection perhaps of a poorly conceived application in the first place). In these circumstances we would argue that a requirement for local input to be guaranteed as part of the Pact application (ie co funding) would achieve the twin ends of both ensuring local commitment to support from the beginning, and of providing the Pact operation with a higher level of resource to do its work

• limited funds were often fully committed before the work of the Pact was complete, even allowing for the additional second tranche of input. Whilst this might simply reflect poor planning and management, in other cases it was the result of a desire to meet manifest demand for action. A common consequence of this was the lack of funding available for what the Pact might perceive as discretionary or second priority activity, particularly research, evaluation and dissemination. This as an issue is not unique to the Pact programme, with local promoters often inexperienced in the requirements of an innovation focused initiative, or under pressure to meet immediate demand rather than more abstract needs which bring little of immediate benefit to the project itself. With a small amount of funds available and few staff it is hardly surprising that the amount devoted to these ‘ancilliary’ activities ended up in many cases being very low. This would of course severely limit the likelihood of the Pact fulfilling all the expectations of if. Having said this however we would not argue for additional funds without conditions being attached. More money might simply be used for extra local delivery work, and might even give an incentive for local action outside the strict Pact remit. Instead we would argue for a much greater emphasis to be placed on the ring fencing of project funds for activities which are central to its fundamental purpose, as an experimental programme needing to be fully understood and communicated more widely.

The question of timescale is also important here. A consistent message came from Pacts themselves as well as national experts and correspondents that the time available for their work was too short. This view was evidenced in relation to: • the time needed for initial negotiations and set up • the often slow process of bringing partners into the Pact and of agreeing terms of reference and work programmes • the need for time to enable actions taken to be planned, delivered, evaluated and disseminated • the difficulty of identifying the consequences of work after a relatively short period of time

In many cases the Pacts have a point here, particularly in areas with little tradition of partnership working, where Pact type working was new, and a large proportion of anticipated partners needed to be approached and persuaded to take part. Whilst the Pact could (as we have demonstrated) during this period form effective partnerships,

78 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 there was limited time available to fully demonstrate the benefits of this sort of approach, little concrete evidence to persuade partners or other authorities that the approach was one worth continuing or spreading more widely. With a clock running, it was inevitable that at least some Pacts effectively became ‘lame ducks’ six to nine months before their due end date, with staff concerned about their futures after the end and partners looking to their plans beyond the end of the Pact funding. Without much time to make the Pact a coherent and indispensable part of the local system it was very difficult for many to make a strong case for post EU funding continuation. There are many ways in which additional time could be organised, including simply providing more funds over a longer period; giving a large ‘lump sum’ with flexibility to spend as necessary over say four years; using a taper where EU funds wind down over a longer period, perhaps tied to evidence of local funds being made available to make up the shortfall; or a requirement for firm commitment of matching funds after the initial period. The overall conclusion is however clear. In those situations where the Pact idea was new or obviously additional, a small amount of money in a short time frame did not provide ideal conditions for all the necessary stages to be completed and the Pact embedded. Sustainability was seriously curtailed.

We now turn to consider the wider financial picture of Pacts, as well as the effects of bringing new elements into relationship with each other – the catalytic effect.

5.3 Catalytic effect

The catalytic effect is based on the proper chemical analogy, i.e. an element which creates a new or greater interaction between other elements which otherwise would interact to a much lesser degree. The intention was that the introduction of the Pact, with its resources available to support targeted action within a partnership context, could be the spur for new, wider, more effective, more focused interaction. This can be considered in two different ways, firstly the achievement of influence over a wider pool of resources than received by the Pact itself from the EU; and secondly, the effect of bringing different stakeholders into a new relationship which led to new actions taking place. In practice this latter issue is simply another way of expressing the added value accruing from partnership working, and was detailed in the sections above. From the point of view of a catalytic effect, it is clear that some Pacts did successfully demonstrate the value of bringing different types of partners together in new formulations, and that positive results could accrue as a result.

In concrete terms then, the other catalytic effect can be assessed by the ability of the Pacts to lever in additional resources over and above the amount of Technical Assistance that was received from the European Commission.

5.3.1 Leverage over additional resources

The ability of the Pacts to obtain influence over additional resources has been mixed. It is important to recognise that the ways in which influenced was obtained varied, and a number of models can be identified:

79 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

• responsibility for deployment of other external funding streams being given to Pacts (‘programme management’), with a particular emphasis on EU programmes • the Pact partnership agreeing amongst themselves to use some or all of their ‘own’ resources to take forward common activities (‘pooling’) • the Pact achieving indirect influence rather than direct decision making responsibility over the resource deployment decisions of other stakeholders inside or outside the partnership (‘influence’)

From an examination of the initial budgets of Pacts, the 89 Pacts planned to lever in €1.6 billion, as shown in the following table.

Funds From EU From From Total Funds intended to Sources Public Private be levered (including Sources Sources in: the Pact TA funds) TOTALS 47,727,775 549,385,917 579,602,179 1,609,398,338 for the 89 Pacts Source: EI2000 data on Pact Budgets

In practice no serious attempt was made to quantify the extent to which these figures were achieved in practice, and in any case no analysis was given at the start or end of Pacts as to what ‘leverage’ actually meant. The different models described could lead to very different types of activity, from direct planning and delivery of local projects, to somewhat vague ‘influence’ over strategic spending decisions by stakeholders. Pacts were lacking in any kind of consistent understanding of how to conceptualise their relationship to wider funds, let alone how to evaluate the effectiveness of different approaches. The following paragraphs summarise the available data.

Regions had the opportunity to channel funds from the mainstream Structural Funds programmes through Pacts. This could happen in different ways22. The Pacts were introduced at the time of the mid-term review of assistance for Objective 1,3,4 and 5(b) programmes and the beginning of the second programming period (1997-99) for Objective 2. This meant that: • in Objective 1 regions the authorities had the possibility, when reviewing their Objective 1 CSFs, to draw up support programmes taking account of how the TEPs could contribute to the utilisation of these funds. This happened in Spain and Italy. • the Monitoring Committees were able to set aside specific amounts of existing funding for the Pacts, consisting either of the proceeds of the deflator or funds freed up by reprogramming. This was the approach followed in the sub- programmes drawn up under the regional programmes for Greece, the Objective 2 SPDs for France, Belgium, Luxembourg and the Netherlands and the Objective 3 CSFs or SPDs for Germany, Austria, Denmark, Sweden and Finland.

22 Second Interim Report on the TEPs

80 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

• Pact Action Plans could be resourced progressively, without any particular priority, by using funding available for schemes of this type under regional or national programmes (to limited extent in the UK, and in France and Germany).

Sixty one Pacts planned to mobilise a total of € 580 million from the private sector, and in practice there is evidence that Tottenham and Haringey (UK), Apenino Centrale (IT), Perigord Noir (FR), Amberg-Sulzbach (AUS) were successful in influencing deployment of significant private sector resources. In both Berlin- Neukölln (D) and Sönderjylland (DK), the Pacts were able to lever in additional resources owing to the involvement of businesses. On the other hand some Pacts found the private sector reluctant to commit resources to projects when there was no clearly defined source of match funding.

The Finnish National Report states that the resources levered in by the Pacts contributed to the implementation of projects that would not have been carried out without the support of the TEPs. The additional sources of funding have included Structural Funds, co-financing from national programmes, donations from national voluntary bodies, private-sector contributions, and local authority funding. For other Pacts, like Pays de Valois (FR), there were more limited amounts of funding levered in. Tirol (AUS), St Herblain (FR), Coatbridge & Easterhouse (UK) influenced meagre amounts of additional funding.

Some of the Pacts – such as those situated in Portugal and in the United Kingdom found that problems resulted from the fact that the Pacts command relatively small amounts of their own resources. In the UK Pacts were not entitled to any particular sources of funding and were dependent on other institutions and organisations to provide them with financial support. In fact, with the UK’s system largely oriented towards competitive bidding, Pacts often had to submit competitive bids on a project- by-project basis. Without official recognition from the various tiers of government and a distinct, clearly defined and established place in the institutional structure, it is difficult to see how the TEPs could manage to lever in additional resources more easily in a UK context.

The problems encountered by the TEPs in mobilising additional resources were generally linked to the fact that the Pacts relied upon other actors and sources to provide them with financial support. This can be seen very clearly in relation to the Norte Alentejo and Marinha Grande Pacts. Both have been identified as lacking a 'defined status'; they suffered from the absence of finance specifically destined to finance their own actions. It is also a problem that was openly acknowledged by the Turku Pact. For Turku, it is stated in the EI2000 Final Report of the National Experts that one of the weaknesses was related to the difficulty in getting funding for the projects.

This particular variable was often linked to the commitment of the partners and of superior authorities to the TEP. For example, the Peine (Niedersachsen) Pact only had the ‘limited commitment of local actors concerning financial contributions’. The Tongeren Pact was weakened by the lack of resources at the provincial and regional level. Different countries also displayed varying levels of financial commitment to the TEPs. This was particularly apparent for the trans-border Öresund Pact. The

81 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

Danish Government was unwilling to provide support despite requests being made while the Swedish supported the TEP with national funding. When successful in levering in significant additional resources, this strength is linked to other variables, such as the commitment of the partners or the attitude of superior authorities to the actions of the Pact. For instance, the Albertville Pact (FR) gained financial support from the Conseil Général.

A Pact might be able to lever resources into an area in a more broadly defined sense. An example of this is provided by the Catania Sud Zona Pact (IT). Through its actions, the Pact claims to have made the area more attractive for investors. It is stated in the Italian National Report that many companies have relocated to their area to take advantage of its attractions. The Pact co-ordinator interviewed judges that most private resources redirected on the Playa zone financed new services related to tourism (hotel, sport equipment, etc.), impacting on the general development of the area.

Analysis of the available evidence leads to a number of conclusions:

• where Pacts were well integrated with the wider EU funding process in member states they were more likely to be used as a channel for EU funds from other sources. This had the potential to bring greater co-ordination to the deployment of EU funds, and also had the advantage of generating synergy between programmes • in these circumstances Pacts tended to be regarded as part of the EU funding system. This might lead in some cases to less emphasis on the acquisition of other public or private funds, since large amounts of EU funds were available (much with its own co-funding); on the other had, where match funding was still required the Pact could act as a means to obtain it through the partnership • if use of EU funds was a significant element in Pact work, less pooling or generalised influence would take place (except where needed for provision of match funding). As a lesson for the future therefore this might be considered to be a less useful outcome of Pacts. Use as a implementation method for EU programmes could make sense in prevailing circumstances, but would not necessarily yield an information gain for the potential of the Pact approach in the likely absence or serious reduction of EU funds in the future. • pooling of resources from whatever source provided those Pacts which achieved this with a means of initiating direct actions in the locality, and yielded many good examples of how new and interesting activities were generated. As a demonstration of the value of the Pact approach this could give very useful lessons of the possibilities of a genuine catalytic effect, where different elements were brought creatively together. If enabled to continue after the end of funding for the Pact this could be a very positive development, but only a few examples of this are identifiable. The more common experience was for Pacts to have a set pool of resources to use for one off activities, with little longer-term continuation, primarily due to their short life span. • achievement of less clear cut ‘influence’ over the deployment of other resources is by definition more difficult to assess. Pacts which were very active in assembling, processing and dissemination local information, and in generating strategic approaches, would point to better concentration of local efforts on important issues as an indicator of success. There are however few ways to confirm the extent to which it was the Pact which caused this to happen, as opposed to other

82 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

influences. In any case, the value of influence was again hampered where Pacts did not continue beyond their EU funding period • leverage of any sort over significant resources was not a sine qua non of Pact success. Whilst some such as in the UK had few alternative routes, others managed to focus on increasing co-operation in the field of labour-market interventions. The Norte Alentejo Pact adopted such a role, which did not require significant additional resources, since the emphasis was on improving the quality and effects of existing programmes without necessarily carrying out a large number of new projects with the Pact’s own resources.

5.4 Dissemination, Mainstreaming and Replication

Pacts were expected to consider the lessons their work would have for other actors, and to view their activities as experimental. The limited funds disbursed were to be regarded as an investment in the creation of new approaches and ideas. As we have detailed above, the work of Pacts turned out to be highly diverse, in many cases novel and innovative. It is not feasible to quantify the outputs of Pact’s dissemination given the lack of readily useable indicators, however the evidence does enable us to draw some conclusions as to the dissemination and mainstreaming activity of Pacts and its relative effectiveness. Specific dissemination work included the normal means: websites, newsletters and events such as conferences being the most common forms. In general, the Pacts that seem to have been most successful in the dissemination of information were the ones that used a variety of channels of communication. The Agro Nocerino Pact (IT), for example, used a number of different methods to disseminate information on the Pact, including conferences, a website, publications, flyers, brochures, broadcasts in a regional TV channel, direct marketing with beneficiaries, and fairs and events. The success of these activities was seen in the high participation of bidders responding to invitations to tender, in the increase in the number of partners, and in the consolidation of the partnership.

Across the board however a mixed picture emerges. In some TEPs, dissemination was an important and integral part of their overall strategy, for others it played a less significant role. Three broad models of dissemination can be identified, although any one Pact might include elements of each in their work. They are: • ‘internally focused’ dissemination, largely concentrated on the distribution and processing of information throughout the Pact partnership itself, and directly between partners and other local bodies. This type was most often related to the sustainability of the Pact itself as an institution in its context • ‘local profile’ dissemination, attempting to gain recognition for the Pact widely in the local area, sometimes involving direct communication to the local general population • ‘externally focused’ dissemination aimed at communicating the lessons of the Pact beyond the territory itself, to audiences including superior authorities; actors in other areas working at the same level or on the same issues; and for some, other Pacts. This can, at least in part, be related to mainstreaming.

83 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

5.4.1 Internally focused

The majority of Pacts attempted to communicate with relevant organisations and stakeholders close to their level of work and interests. In early work this might be to attract partners to the Pact idea itself, later the focus turned more towards letting such bodies know about the work of the Pact and what it had achieved – this often in the context of a search for continuation funding. The content of dissmenation in these circumnstance tended to concern emphasis on the local achievements of the Pact, pointing out what work had been done, often in very concrete terms. The message would often concern the value of the specific Pact in question, and the merit of its continuation. Implicit in such a message is of course the need for further funds. The Halle-Vilvoorde Pact (B), for example, had an internal communication system (for administrative and organisational purposes) but did not really follow an external communication strategy. There was a website but no general newsletter or media presentations. The main channel for dissemination was the publication of the project results, such as descriptions of projects and studies by either the TEP or the operational partners. Hence, the benefits of the dissemination activities occurred in an ‘internal’ context; it was felt that the continual dissemination of project results served to maintain the motivation of the partners.

5.4.2 The development of Pact networks

It is necessary to consider under the dissemination heading the networking which took place between the Pacts funded under the programme. In one sense this could be considered internal dissemination, in that the bodies involved were part of the programme itself, talking to each other, although it could also be seen as capacity or even solidarity building.

Several Pacts emphasised the value of the exchange of experiences and the dissemination of best practice that occurred through the transnational events organised by the European Commission23. The Commission organised a number of such events:

Rome, 4-6 May 1997, seminar Ostersund, 18 & 19 September 1997, thematic seminar Viareggio, 2 & 3 October 1997, thematic seminar Porto, 13 & 14 October 1997, thematic seminar Bremen, 25-27 January 1998, seminar Turku, 8-10 October 1998, seminar & exchange mart Brussels, 8-10 November 1999, conference & exchange mart Naples, 6-7 July 2000, seminar

The exchange marts were perceived by the Pacts as especially valuable.

23 Information about the programme as a whole can be found at: http://europa.eu.int/comm/regional_policy/innovation/innovation

84 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

The Iisalmi Pact (SF) also drew attention to the benefits of transnational co-operation. Iisalmi was active in international co-operation and had gained experience from running other international projects. The partners within the TEP had the chance to visit TEPs in other European countries and felt this to be an advantage, since they were able to learn new ways of acting and finding solutions to the problem of unemployment. The Dudelange Pact (LUX) saw the “networking offered by the European Commission was seen as a positive instrument in general”. However, for this Pact, few networking activities were undertaken. In particular, it was seen as a disadvantage that there were no other TEPs in Luxembourg (such as for the exchange of ideas and experience).

In this area, it is also worth mentioning the establishment of the MetroNET, which brought together five northern European urban TEPs to disseminate best practice and exchange experiences of general applicability. The MetroNET aimed to develop common themes and share good practice in employment strategy and implementation. Although most Pacts developed transnational contacts at some point, few went as far as the MetroNET. In fact, the network developed a number of transnational projects, including one designed to address social exclusion. Among the Pacts belonging to this network were Berlin-Neukölln (DE), Öresund (DK), Stockholm (SE), and Tottenham and Haringey (UK).

Some Pacts also actively co-operated with each other. For instance, Dundalk and Drogheda (IRL) established a cross-border project with Newry and Mourne (UK) that was intended to transfer lessons.

This activity was of most value therefore in supporting Pacts in their own work, and helping each other to work more effectively. There is little evidence to suggest that such activity had wider implications.

5.4.3 Local profile

Achievement of a high local profile was usually sought during the course of the work itself. Some Pacts drew specific attention to the development of a TEP ‘brand’ in their locality. The Marinha Grande Pact (PT) acquired “a good reputation and became a symbol of work well done"24 in the area, whilst the Sangro Aventino TEP (PT) was able to acquire “a certain authority in relation to the regional and national authorities” through active publicity seeking.

The Bruxelles-capitale Pact (B) is one example of a Pact using TV advertisements as a means to create awareness of the Pact among a wider audience. Press conferences were also held and brochures produced and widely distributed. The view was expressed that, through these dissemination activities, the Pact became known in business and administrative circles – as well by the public at large. In other Pacts, websites were designed to attract the interest of a wider public audience.

In Ceuta (ES), the Pact envisaged a dissemination plan addressed mainly to the target groups. However, it was not finally developed. The reason for this was that

24 Case study

85 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 dissemination among the general public took place ‘by itself’. Among other reasons, such as the small size of Ceuta, it was felt that this was due to the relevance of unemployment for the whole city.

In Vallès Occidental (ES), priority was given to the implementation of the Pact rather than to dissemination activities, as it was felt that the limited budget available would not allow for a comprehensive dissemination strategy. However, it was believed that the dissemination of the Pact’s activities also took place ‘by itself’. Despite this positive appraisal, the Pact felt that in future more resources should be devoted to the dissemination of activities.

The benefits of local profile can be contrasted with other situations where its lack was felt to be a hindrance - the Stockholm (SW) Pact “did not have any real visibility”, which was seen as one reason for its limited achievements, whilst the Dudelange Pact (LUX) would have benefited from a higher profile: “The partners were aware of the Comité Local and the Service Emploi, but generally not of the TEP structure that lay behind it.” Therefore whilst the direct work of the Pact was valued, the structure and process which had led to it was not.

5.4.4 Externally focused

Target audiences here included both superior authorities, and other actors from different areas, working to tackle the same sort of issues in similar circumstances. Whereas the first type, the ‘internally’ focused approach can be seen as mainly to do with issues for the locality, this second type had the potential to be more clearly a mainstreaming activity, in that it concerned the use of the Pact approach in new situations.

The dissemination of a bottom-up approach has been highly relevant in France, where employment policy is traditionally ‘top down’. The French National Report argues that one of the main sources of added value from the French TEPs comes from the dissemination of completely new instruments in a territory where labour-market actions had only been organised through traditional administrative channels. This reflects the openness of the traditional system to new ideas and the sympathetic attitude of authorities at various levels – the factors which were identified earlier as conducive to positive outcomes in congested policy environments. Although the Pacts have not been systematically mainstreamed, the ability to set up Pacts has been provided as an option for Measure 10, Axis 6 for European Social Fund interventions. One of the French Government’s agencies, the Délégation à l'aménagement du territoire et à l'action régionale (DATAR), hopes to establish approximately 40 TEPs to use sources of funding linked to Objective 2. Several regions have expressed an interest in pursuing this opportunity.

In Italy, the Ministry of the Economy and Finance established a programming instrument, entitled P.I.T (Territorial Integrated Projects) to allow the ten EU Pacts to consolidate the experience they acquired during the TEP Programme. It was left up to each region to choose to include a PIT in their Regional Operational Programme. For example, the Agro Nocerino Pact, where the Pact Partnership and structure is still

86 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 fully active in the area after the end of the TEP Programme, was able to place its actions within the framework of the new Community programme for the period 2000- 2006 through the PIT option.

In other Member States, the Pacts have not succeeded in continuing as an integral part of the Structural Funds Programme for the period 2000-2006. In some cases, such as Portugal and Greece, central government has not been supportive in inserting measures providing for Pact-style approaches in the mainstream 2000-2006 programmes. While all three Portuguese Pacts pressed for continuing their operations, the Portuguese government did not support their request. This was partly due to confusion with respect as to who the national level interlocutor would be to enable the Pacts to negotiate their mainstreaming (this was eventually decided to be the Ministry for Regional Development). Pact interviewees have reported that in some of the cases where it did not prove possible for the Pact to find its place in the mainstream Structural Funds, the partnership established by the Pact continued in an informal way. This may be possible from the momentum of the co-operation achieved during the TEP Programme. However, without the process being fuelled by any TA funding, it is difficult to see those partnerships being sustained well into the future.

In Finland, the Iisalmi Pact had an important influence on the eastern Finland Objective 1 Programme for 2000-2006. The Pact’s management played an active role in the preparation of the Programme. The Pact’s experiences affected the strategy and measures that were developed. The idea of a partnership, third-sector opportunities for employment, and pilot projects are all explicitly mentioned in the Single Programming Document, and the monitoring committee of the Eastern Finland Objective 1 Programme has reserved 5% of the financing for the measure ‘employment and know-how’. This will act as a global grant for local employment pacts. The Iisalmi Pact has also played a significant part in the preparation of regional development strategies and in the nationally-led regional policy programme for the development of towns.

Therefore Pacts in the form envisaged in the Programme have in the main not been continued in the mainstream Structural Funds Programmes 2000-2006, but as shown above this does not mean that they have failed to influence initiatives at national level in other ways. Their lasting effects have been marked in several Member States. In general, either the Pact methodology has been adopted in other initiatives, other sectors and even in countries outside the EU (Poland) or some of the actions that were implemented by the Pacts have been carried forward in some way. The concept of a local employment pact has been simulated in national programmes, in other sectors or, in some cases, taken up by regional governments.

For example, the Valles Occidental (ES) pact methodology was adopted at the level of the region of Catalonia. This constitutes a reality with 19 employment pacts currently already operating in the rest of Catalonia (funded with € 30 million from objective 2 ESF and 50% co-funding from the local administrations).

In the Netherlands, the idea of a Pact-style partnership has been applied to areas other than employment. The Province of Limburg has established a similar body to deal with tourism in the territory.

87 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

In Greece, the influence of the TEP methodology can be seen in an initiative sponsored by the national government to establish Pact-style partnerships at the municipality level. Signs of the Pact methodology can also be seen in the region Provence Alpes Côte d’Azur (FR). Although this region has not made specific reference to the TEPs set up with the help of European Commission Technical Assistance, it is going to develop its own network of Pact-like partnerships. These partnerships will be territorial in their scope and will cover areas that take in an average of 100,000 people.

The Berlin-Neukölln (DE) Pact says that the approach caught on and was adopted in other parts of Berlin and even in Poland. In a similar way, the Pact methodology has proved useful in Appennino Centrale (IT) where similar pacts for the agricultural sector have emerged using the same method.

The Pacts have also had a lasting effect on the mainstream through the continuation of their actions. A notable example is provided by the Dudelange Pact (LUX). The Pact’s main activity, the Service Emploi, was mainstreamed as part of the Objective 3 Programme (at least until 2003). In the Värmland Pact (SE), some of the TEP’s projects were merged with – or moved to – Structural Funds programmes, the Regional Growth Agreements and national labour-market policy programmes. Likewise, a number of projects carried out by the Viborg Pact have continued. This illustrates a general point: the kind of innovative projects undertaken by the Pacts have often had a lasting effect. If they are perceived to have been of great value, mainstream programmes have taken them up in some way. There is one problem that results from this: if the TEPs have largely been discontinued – at least in the form envisaged by the European Commission – the kind of innovative projects that they were able to support might not be undertaken in the future.

88 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

CHAPTER 6: CONCLUSIONS AND RECOMMENDATIONS

We have indicated the conclusions arising from the evaluation at various points in this report, and in this section we summarise the main findings. We believe that it is appropriate to offer two discrete kinds of conclusions. On the one hand, there are those that refer to the Pact process as a generic policy device in its own right. The evaluation has given us insights into what the advantages and disadvantages might be in different sorts of contexts and with different sorts of form and management. On the other hand, there are conclusions about the Territorial Employment Pacts Pilot Action itself. The evaluation here tells us about the appropriateness of the action as a vehicle to support an appropriate experiment from which policy lessons can effectively be drawn. In both cases we are able to offer suggestions for the future.

6.1 Pacts as a Policy Instrument for Local Employment and Development

Territorial Employment Pacts offer an opportunity to insert a process of co- ordination, integration, innovation and partnership at local level in the context of employment and local development. What the evaluation most powerfully demonstrated was that the extent to which this opportunity can be grasped will differ substantially from place to place, and in predictable ways based on a number of key criteria.

As a starting point, the evaluation examined whether the basic underlying Pact concept (as represented by the four attributes the Pacts were intended to embody) was achieved in practice. We concluded that sufficient ‘genuine’ Pacts were created, enabling the evaluation to consider the performance of the Programme as a policy instrument, in the knowledge that the sort of entities envisaged were indeed created: • some 62% (55) of Pacts were either fully consistent with the model, or were close to it. • a bottom up approach, and a broad based partnership, were strongly associated one with another, and were also both the most frequently achieved characteristic • integration of strategy was attempted and to a degree achieved by most Pacts • most attempted to be innovative, but what counts as innovation varied considerably while in some cases there were limited expectations beyond the novel experience of partnership working • there was a distinct though not absolute association between the achievement of Pact attributes, and the quality and consistency of Pact management.

The evaluations’ conclusions as to the achievements of Pacts in practice can be summarised as follows:

• solid, reliable, quantified and unambiguous evidence for the achievements of Pacts is scarce. Little consistent information is available across Pacts as a whole, and few recorded substantial data concerning their own activities • most Pacts were however able to provide qualitative and subjective information, as well as alternative perspectives. This gives a reasonable understanding of the areas where Pacts felt themselves to be successful or not, and patterns do emerge

89 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

• Pacts’ own views of their achievements are dominated by the creation and nurturing of partnerships and the associated benefits, consistent with the purposes of the programme, • Pacts have also been able to give examples of positive wider effects, which are diverse and include better resource deployment, stimulation and operationalisation of project ideas, realignment of policy, better articulation of supply and demand • some direct creation of employment has taken place but is of little importance as an achievement of the programme as a whole. It does however offers some models of collaborative approaches to new project development and operation • the gaining of influence over resource distribution has taken place, bringing a more strategic approach the benefits of less overlap and greater clarity • the wider lessons of Pacts have not been disseminated widely. This is due to the diffuse and locally specific nature of the achievements, lack of understanding that this was a significant activity, and lack of resources devoted to dissemination.

What was needed for the Pact to be taken up successfully was a positive combination of the following:

• 'space to act' within the prevailing policy and practice framework in each Member State; • the right kinds of people in place to design, deliver, and reflect upon the work of the Pact; • the availability of the right institutional capacity both within the organisation responsible for managing the Pact and, crucially, within some wider set of influential stakeholders above, below and alongside it

These were by no means exclusive requirements since we were able to identify well- organized and creative Pacts even where there appeared to be a plethora of agencies and partnerships operating over the same terrain. We also found many examples where “space to act” was not a problem but where the competency needed to run the Pact was weak, or institutional structures were insufficiently flexible to respond. The “golden” combination, as we have set it out above, would represent the most fertile ground for Pacts to grow. It has been part of the detail of the evaluation to reveal (chiefly from the case studies) the sorts of capacities that the best Pacts had put in place so that others can learn from them.

The policy task is to facilitate a process, by which Pacts across the variety of contextual circumstances we have explored can seek to optimize how they can design themselves to be more “fit-for-purpose”, effectively managed and properly embedded with the right stakeholders and external contact networks. What emerges here is yet another application of the “law of requisite variety” where, for success, the Pact itself must have sufficient internal flexibility and variety to match the complexity of the real world system on which it seeks to have an impact. The best of the Pacts had this capability and were able to respond well even to the least promising situations. They were characterised for example by: • an ability to think creatively about the labour market issues by which they were confronted, recognising the causes and effects of different circumstances, and the levers which might make a difference in practice

90 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

• flexibility in approach whereby successful approaches were duplicated elsewhere, the unsuccessful being quickly dropped • an openness of membership and flexibility in bringing in new partners when appropriate, and in changing structures to suit changed conditions • management strength in depth

Where, however, there was space for the Pact to operate and a policy problem that was relatively straightforward, it was possible to achieve good results with a lower level of competency and institutional flexibility.

The power of the underlying concept in those cases where these combinations of conditioning properties were in place was demonstrated by the best of the Pacts. These did succeed in: • achieving good results consistent with the objectives of the programme; • being able to articulate what they had done and why it was of value; • being capable of communicating the lessons of their work to wider audiences; • leaving a legacy in their localities and more widely to carry the process forward;

In this context is should be borne in mind that good results here relates to achievements consistent with the objectives of the programme – the establishment of partnership based and collaborative approaches to local development, with an associated variety of impacts on the labour market and employment conditions. This does not refer specifically to direct employment results such as job creation or training delivery, which were not a requisite indicator of Pact success (or at least should not have been, even if some interpreted the programme in this way).

These best practice cases have established the Pact methodology as an effective means for adding value to local employment and development actions. It is clear, however, that the value of Pacts goes much wider. They are also capable of being applied to a wider set of public policy issues. Indeed, wherever the different actors in a given territorial context are insufficiently coordinated and orientated towards a strategic and focused set of objectives, the Pact principles applied in practice can offer a governance vehicle to bring them together in an effective way.

The recommendations implied by this would be:

• the value of a Pact type approach should be recognised as having potential in different ways and with regard to a wider set of local development issues. These could include, inter alia: Ø employment, naturally, this being the subject of the pilot Ø development of territorial competitiveness, bringing together issues around training, skills needs, development of sector expertise, cluster development, infrastructure inefficiencies etc Ø local economic development, where spatial planning decisions require better co-ordination with labour markets, competitiveness and the market itself Ø business support, particularly with regard to supply chain management, support for entrepreneurialism and SMEs, and better joint planning and coordination with other stakeholders Ø planning and integration of education and further learning and training

91 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

• but that for the approach to achieve results, the need for a number of conditions to be in place be recognised (space to act; management capacity; organisational capital). In programme terms, applicants should either be required to demonstrate that these are at least potentially in place; or the programme should be designed in such a way as to develop them as part of the establishment and capacity building process

6.2 Pacts from a National Perspective

The evaluation did reveal that, for many Member States in the EU, the Pact process may look to be an essentially a mature policy product. Mature in the sense that the basic principles are already well understood at national government level and already in place in homegrown organizational structures and policy instruments. This applies in the main to the Member States we classify as ‘crowded platforms’ (FR, UK, NL, DK, SE), although also includes to a lesser degree at least some of the ‘receptive space’ countries, specifically those with partial experience of local partnership implementation even if not the full blown model (for example IRE and IT) It follows that Pacts per se can appear to be of less interest for these Member States. This, of course, serves not to deny the intrinsic value of the Pact method but in most cases to confirm it as being already mainstreamed. Even in these situations, however, the evaluation shows that the Pact process can still add value. This arises chiefly from the capacity of bottom-up partnerships to be a fruitful source of ongoing policy innovation. In the “crowded platform” States it has still been possible to demonstrate that local Pacts can act as a fountainhead of new policy ideas. The innovations that we could identify here applied either to creating new activities or applying existing ones in more sensitive ways. They could, for example, address the needs of particular target groups; stimulate new combinations of actions combining a supply side with a demand side view; and help to finding more creative ways to achieve effective partnership. A broader outcome for Member States where Pacts are seen as a mature policy product is to use them to offer a stimulus for widening and deepening the application of bottom-up partnership throughout the body politic, and not just in relation to employment and local development.

• This has clear implications for the future of the generalised structural funds such as Objectives 2 and 3, where we would recommend that a Pact like approach be considered as the norm for territorially based implementation, but that in the main this should be mapped on to existing partnerships rather than created afresh. • In a context of reducing structural funding available for the current Member States in the context of enlargement, the Pact approach could in fact provide a vehicle for the most effective use of limited resources. As funds reduce, their ability to achieve significant change through support for direct action will lessen to the point where its value will be in question. Two strategies for use of available funds then present themselves. One is to use them for innovation and experimentation funding, and such a role is likely to continue. The other however would be to focus funds on adding value to the partnership approach as it currently exists. This could involve a variety of actions, drawing on the better TEP example, such as the involvement of a wider diversity of partners and sectors;

92 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

encouraging more sharing of good practice and transnational experience; giving more scope for research and reflection on the effectiveness of different types of working • This also implies that project/promoter selection should pay more attention to the availability of the key features, prioritising those which can demonstrate sophisticated understanding of the policy and practice context within which they are operating; can offer the level of management competence required; and have evidenced links with the necessary institutional players above, below and alongside

By contrast, in some Member States and regions Territorial Pacts per se can be characterised as the innovation. In these cases there are three sorts of conditions that might prevail:

• there is, as yet, no appreciation of the merits of the bottom-up partnership and of inclusive and coordinated local partnership (Portugal for example); • some blockage in the system makes adoption too costly or too politically or administratively challenging (parts of Spain and France for example) ; • there is simply not the competency or institutional capacity in place (such as Greece);

These conditions, of course, interact with each other. Not seeing the merits of a particular approach will normally be powerfully associated with being overwhelmed by the perceived blockages to its application and the competency and capacity needed to overcome them. While it is very possibly the case that Territorial Pacts can make their greatest value added contribution in these Member States, the evaluation demonstrates that the process of both making the case and of simultaneously addressing the blockages is non-trivial. In particular, it is clear that making measurable progress here may have to be reserved for the long term.

We would argue that there is a necessary threshold capacity for effective Pact insertion and that the process of installing such capacity can be challenging. This is not least because the prior conditions will normally lie within the competency of the Member State itself and will demand adjustments to an evolving system of national and territorial governance that has its own historical antecedents and future trajectory. The key question for an initiative by the European Commission in this sort of case is whether Territorial Pacts as currently conceived can deliver sufficient net value added over a reasonable time to cover both the real and perceived costs of helping to create the necessary threshold conditions. From the evidence presented in the evaluation and the fact that so many regions and Member States have already mainstreamed something similar, there seems to be a strong case for arguing that Pact-type processes can indeed add value to European systems of governance in general. For the Member States in this group, however, the bias would need to be toward supporting a long- term process of domestic evolution of the idea and helping to build the capacities for a gradual implementation. This would need to be done in close association with the Member State itself from which much of the necessary prior infrastructure would need to emerge.

93 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

Therefore: • a programme based on Pacts in countries lacking the threshold conditions would need to be actively planned and delivered in collaboration with Member State authorities at different levels, and should be expected to take some considerable time to achieve recognisable results • the evaluation showed that the more successful, Portuguese Pacts differed from their less successful Greek counterparts primarily with regard to their good relations with key strategic authorities. In the main these were in place from the beginning, and were further nurtured and developed during the Pact’s life. Most of the Portuguese Pacts were also well managed and organised. The Greek examples lacked these essential features – they had poor strategic resources, and also suffered from weak and inexperienced delivery. • this is of particular relevance to the new / accession member state as structural funds begin to apply to them. These countries tend to have little experience of locally determined action and in many cases only have emerging institutional structures and little tradition of partnership working. A recommended strategy would be for a Pact type process to be promoted as a desirable element in to the development of local policy and practice. There is a clear opportunity to embed a partnership approach, coordinated strategic planning, and the capacity necessary for it to be effective, from the very beginning. This would enable it to become a norm without having to go through the process of later introduction into an established situation

6.3 Pacts from a Territorial Perspective

• The Commission Guidance

It is clear from the evaluation that the application of Pacts as instruments of territorial policy is variable both in terms of the scale at which they are best applied and in relation to the problems that they are best suited to address. The questions that are most important here are clearly “at what spatial scale are Pacts best able to operate and what sorts problems do they best address?” In launching the pilot action the Commission offered no firm prescription. The guidance was set out in general terms in order to give the widest scope to the examples that later became the subject of the evaluation. In terms of spatial scale, it was indicated that Pact could be “promoted at regional or local level, in towns, cities or employment catchment areas which share socio-economic characteristics”. From the point of view of the problems to be addressed the clear focus, consistent with the Luxembourg process, was employment. The proposed mission for Pacts was to define the issues better, mobilize available resources, integrate and co-ordinate actions and share best practice (SEC, 1999, 1933).

• Reflections on the Spatial Scale and Role of Territorial Employment Pacts

94 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

So what have we learned? First, there are some clear cases where sheer distance or the geographical scale at which a Pact has to operate makes for a loss of effectiveness. The remote rural context, with its constraints on accessibility for face-to-face contact and where the supply and demand for labour pivots mostly around the home or farm poses obvious limits for a Pact model. Equally, and at the other extreme, a Pact set at the scale of an entire densely populated urban-industrial region or major metropolitan city a Pact has a struggle to deal with the complexity of such an environment where accessibility is coloured by more complex factors than the friction of distance. From a partnership viewpoint the greatest challenge is for the Pact to have a distinct and important enough mission to capture attention in such a competitive environment. The concept of such a region having “shared socio-economic characteristics” is also hard to sustain except in the most generalized sense.

The original conception that Pacts might best operate at a level somewhere above the municipality and below the region seems on the basis of the evaluation to retain its persuasiveness. In the specific case of Territorial Pacts for employment there is clearly both a theoretical and practical logic that would privilege a spatial scale that would both capture the bulk of the available job slots (demand) and the people able within a reasonable travel-to-work time to be available to capture them (supply). Thus:

• it is the travel-to-work area,“bassin d’emploi” or the “continuous urban labour market” that offers the best-fit scale for a Pact with an employment focus

We are able, however, on the basis of the evaluation to offer some extensions and limitations to this proposition that has such a sound basis in theory. We have already identified the remote rural case as one having limitations for Pacts. However, even in these cases it is possible to envisage a role for a Pact-style partnership to organise out- of-season work opportunities or job-assembly to package together part-time working opportunities over a wide region. Therefore

• A wide spatial scale for a Pact with a narrow focus has the potential to thrive, where it can engineer scale economies through coordination of local actions

The Pact process may also find room for itself even in those densely populated urban- industrial or metropolitan regions described earlier. Handled as a single marketplace for employers and workers, the problems arise in the sphere of public administration and partnership. As the evaluation has indicated, it is here that the “crowded platform” effect is likely to be most in evidence, with government departments, state agencies and stakeholder partnerships of various kinds falling over each other to exercise their influence. In these complex situations it is normal to find a significant degree of specialization - with each player tackling a manageable part of the problem, ideally with some sensible coordination among the players. Here it is possible to imagine a Territorial Employment Pact as having a clear role. The presence of some overarching co-ordinating body - preferably a multi-stakeholder partnership with government, the private sector, the social partners, and civil society representatives to exercise this role was clearly envisaged for some Pacts by their original sponsors within the Commission. The pilot action produced few complete examples of this process in action though, as we shall show later, the terms under which the

95 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002 programme was installed militated against Pacts being initiated on this sort of scale unless they were already established under domestic arrangements. However,

• The Pact process has considerable potential for improved labour market action in dense metropolitan areas, where it is able to bring a new level of strategic co- ordination to specialised or narrowly focused institutions

A second feature of the way Pacts could operate in densely populated, continuous urban industrial labour markets that was identifiable in the evaluation was for a Pact to operate as a specialist organization not at the macro-level but at the micro-scale. In these cases, the bottom-up partnership approach was used to tackle a specific target group or problem area. As we pointed out earlier, their primary role was to come together to design and implement innovative solutions to particular problems where the existing arrangements left gaps in provision.

The evaluation showed that the Pact approach appeared to have significantly more leverage in coordinating employment policy at a spatial level between the two extremes of the remote rural, and the densely populated, continuous urban industrial. In these cases, Territorial Pacts seemed to be much more effectively positioned to deal with the sorts of issues that needed to be addressed. They were a better fit to the spatial context – a labour market that captured both available jobs and available workers without the complexity of the larger conurbations. They were more justifiably able to claim to be operating in a more common socio-economic context. The ability of a Pact to exercise a degree of leverage through coordinating key partners against an agreed strategy was more feasible.

• Controlling for the variability of competence and capacity, the Territorial Pact principles are more easily adopted in strictly practical terms in the context of the semi-rural, freestanding town or in an economically coherent sub-region.

In these cases, they also seem to be more easily adopted to fit in with pre-existing institutional frameworks. However, even in these cases, where the implementation of the Pact principles – co-ordination, integration and partnership at local level – needed a significant adaptation of pre-existing institutional and policy delivery structures, the requirements for success were much more challenging.

• The need therefore is for a longer timeframe and the installation of the necessary resources and human and institutional capital. Programmes designed to promote a Pact approach need therefore to either include development of these in their actions, or find means to ensure they are in place before support is given

6.4 Summary of conclusions regarding Pacts as a policy instrument in action

Using the terms that we have applied throughout the report, we can summarise the findings that bear upon Pacts as a value added policy instrument. As we shall go on to show in the next section we found ourselves unable in many cases to go as far as we would have liked in fleshing out and giving evidence for good Pact practice. On the basis of the analysis we are, however, confident of the following findings:

96 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

• Pacts were able to be successful at a variety of territorial scales, but only where they were able to find an appropriate spatial or governance niche to fill;

• In what we defined as full spectrum governance or crowded platform cases, Pacts could be successful if they understood the context they were facing, had good leadership, institutional capacity, adequate resources and good integration with wider policy. They were more likely to be successful in terms of innovation, enhanced integration and targeting particular target groups. However, they needed to have a clear space in which to add value, and needed to avoid duplicating existing structures and responsibilities. Therefore in this type of areas the approach needs to be much more focused. Pact type initiatives should only receive funding if they can demonstrate they have identified a problem that needs solving, they have a policy proposal to address them and also that they can tap in the appropriate human resources. This approach in crowded platforms would also require substantial funds, to cover the opportunity cost of the very high calibre people needed to make it work in such contexts

• In what we defined as established contexts for local action or receptive spaces the general probability of success tended to be greatest since there were more opportunities for innovation and creativity in the governance structure. This was particularly the case where the Pact managed to map onto a clearly defined, manageable territory and labour market. Pacts need to have leadership, institutional capacity, adequate resources and a policy role. Here the TEP model could carry on as it has been applied so far, and lead to identifiable outputs and success, but there would still be a likelihood of redundancy and double counting. In these cases, commitment of smaller levels of resource (though not shorter time scale) might be appropriate, but care would be needed to avoid duplication. Pacts would need to demonstrate how they added value in detail.

• In the category described as having open spaces but some resistance to local action there was a lower probability of short-term success due to a lack of the necessary start-up conditions. In these cases a long-term investment would be needed, tied to a general development process. This will be the anticipated situation in the accession states. The Pact approach has potential here and should be promoted, but only if there is recognition that it will require a long term outlook, committing significant resources, and working closely with central governments in parallel with wider institutional developments.

• We were unable within the constraints of the analysis to find a robust association of Pact success with other factors, such as the state of the labour market or geographical conditions. Whilst such conditions could clearly impose greater or lesser challenges for Pacts, the issues of institutional opportunity, quality of management, and organisational capital were always of over-riding weight among the principal components that governed outcomes.

• Fortunately, it would be possible to pay direct attention to these issues in designing and implementing programmes, leading to higher likelihood of successful outcomes. With the reduction of structural funds in the future, the targeting of resources on these issues as opposed to support for direct delivery

97 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

would be a sensible strategy, and would have the potential for considerable added value for lower inputs than required for large scale mainstream delivery, at least in those receptive areas and at those scales where the opportunities for coordination exist and are not being exploited

6.5 The Territorial Pacts Programme as a Pilot Action

We now make a distinction between the Pact concept per se and its materialisation in the European Commission’s TEP Programme. We would argue that, in execution, the 89 Pacts in the pilot action were a difficult set of cases from which to make a determination of the likely success and failure of Territorial Employment Pacts. We say this because we do not wish to have the outcomes of the evaluation used as a judgement on the Pact concept in general under circumstances where the pilot could not be regarded as a fair trial. The evaluation revealed, for example, that only a relatively small number of Pacts were able unequivocally to demonstrate the successful application of the basic concept and therefore of its merit as a policy intervention. We would argue that this had more to do with the way the pilot action was conceived, designed and delivered than the way Pacts can perform in the right conditions. We present our conclusions on the programme itself in a critical but constructive light. While there are lessons to be taken about Pacts as a concept we believe there are important lessons here also about how best to conduct a pilot designed to explore the merits of a new policy instrument. First the critique. We offer the following as our key findings in this context:

• the overall concept was insufficiently integrated with delivery of the programme. It was poorly explained in the application materials, and selection of Pacts did not use sufficiently robust criteria to ensure they were fully consistent with its intentions. During the course of the programme there was too much variation in interpretation of the concept by key parties, notably between the three responsible DGs; the 15 national experts; and those responsible for managing and understanding Pacts

• the use to which EU funds were to be put was insufficiently specified and monitored, enabling some Pacts to undertake work of little value for the intentions of the programme. There was also insufficient insistence on the use of resources for internal evaluation, and for dissemination activities

• the process of selection by Member States, whilst intended to secure a higher level of commitment to the programme, in practice was undermined by variable attitudes by Member States. Some displayed little interest in the programme, whilst other selected Pacts which were some way distant from the intentions of the programme

• few clearly defined indicators were provided to Pacts to enable them to judge their achievement of programme specific outputs and outcomes. There were therefore unable to have a consistent view as to whether they were achieving what was expected of them. Indeed, there is evidence that some were under the

98 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

impression that they would be judged against indicators wholly inappropriate for the intentions of the programme, such as job numbers or costs per job

• the programme did not recognise that attempts to build partnerships and co- ordinate strategic plans and activities would be crucially dependent on the existing policies and practices in Member States. It could be said that this is a finding of this evaluation and was not therefore known beforehand, but we would argue that this is not in fact a surprising finding, and the requirements asked of Pacts to indicated how they were going to add value and achieve mainstreaming at least imply recognition of their place within a wider context. In any case, Pacts were not in practice asked for sufficient detail at to how their work would integrate with, and add value to, the contextual situation

• additionally, the relationship between Pacts and key stakeholders was insufficiently strong in many cases. More commitment from those with most influence over their likelihood of success was required from the beginning, most importantly the National Authorities, and regional or local authorities with most control over the resources and policies alongside which Pacts needed to work. The attempt to achieve a higher level of national buy-in to the process through Member State selection of Pacts did not achieve its objective of securing the active engagement of national authorities in the programme, the departments or sections responsible frequently having little engagement with the wider programmes and processes which would have the most decisive influence over the work of Pacts. In practice the main result of Member State selection of Pacts was to increase the variability of models selected and to dilute the consistency with which selection criteria were applied.

• the resources made available to Pacts proved to be too limited, and those which were successful were so in spite of rather than because of their limited funds. The actual amount required could not have been known in advance (although it would not be difficult to work out just how much practical resource €200,000 would buy, and hypothesise as to how effective this might be). In retrospect the pilot might have been more revealing had different amounts of resource been made available for work in definably similar areas, to assess the amounts needed for effective action to take place. This could have been based more on the bottom up principle, with applicants required to cost the work needed to undertake the actions they perceived as necessary in their localities.

• The same applies to the time scale for operation, which was too short for the full ‘cycle’ necessary for effective design, implementation, reflection and dissemination. The Structural Fund timeframe of 6-7 years would be more appropriate. Commitment of resources to the Pact from other sources was also too uncertain, and its availability was hampered for many by lack of belief that the Pact would have any significant value beyond its EU funded life

• Many Pacts had insufficient organisational capacity and experience to be effective. Greater recognition of the importance of experience, imagination and administrative competence in the planning and selection process would give Pacts a greater likelihood of success

99 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

• Pacts were too limited in scope to be able to influence macro economic conditions. It is unlikely that any public programme of this nature could do so, but this was insufficiently recognised, leading some Pacts to set wholly unrealistic objectives and targets.

Reading these critiques from a positive perspective we can offer some suggestions for future practice:

• Part of the controlling process has to come from a greater clarity of pre-definition and pre-selection. This implies that in launching a pilot initiative it is vital to spend sufficient time and resources in the pre-planning stage to be able to offer clear guidance to bidders and Pact selectors. • The process of selection should be reconsidered with regard to the involvement of the Commission and/or the Member States. The principle of Member State participation in decisions has potential value in embedding programmes more firmly in their priorities, particularly when the pilot action will need to interact with other national programmes. However, it is necessary for the intentions of a programme to be more clearly spelled out, and for these to be agreed to and implemented by Member States. A ‘double lock’ whereby Member States and the Commission both have a role in selection, with both required to agree to selection against previously determined criteria would be one method to consider here • Part of this process (and a normal one for Commission programmes) is to set relatively clear indicators so that both those commissioning the initiative and those carrying out the actions are focused on what it is they are to achieve. Where flexibility is required this should be built in to the indicators. • Where (as in the case of Pacts) the success or failure of the action bears heavily on the partnership bodies that are to carry it out, there needs to be some guidance (while of course allowing for regional variations) on what these sorts of bodies might look like. Post-selection from an open call at least gives the opportunity for judgements to be made. • There needs to be a real calculation of the minimum level of resource needed to give each Pact a fair chance of convincing its constituency that the exercise is worthwhile and to carry out the necessary tasks. • The bureaucratic process needs to be wary of being slow to sanction spend at the beginning but quick to demand commitment, spend and accountability towards the end of the programme. The confidence and goodwill of the project sponsors is a vital ingredient in having them try hard to achieve objectives. Failure may arise as much from frustration and loss of trust as from the absence of drive or competency • The overall purpose of pilot actions needs to be more clearly defined. There are two quite distinct strategies here, which pull in different directions unless the programme is of such huge size that the distinction becomes irrelevant. One is where the objective of a pilot action is to capture as wide a range of possible examples as possible, with the aim of demonstrating a diversity of approaches, encouraging significant innovation and generating a wide range of new ideas and approaches. The other is where the programme is intended to test out and demonstrate in robust and clearly identifiable terms the effectiveness of a particular policy direction, approach, or methodology. This evaluation was set the objective of attempting the latter, whilst the TEPS programme in practice looked much more like the former. Unless a very large programme supports sufficient

100 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

numbers of examples to allow for meaningful segmentation into sub groups (in practice this would require as a minimum several hundred examples), it will be very difficult for robust results to be obtained from a programme which, by accident or design, has allowed the funding of a large diversity of types. As an issue this is by no means insuperable, but it does require careful consideration of the underlying objectives of the programme, and its conscious design and implementation in such a way as to give the best opportunity for the objectives to be realised.

Summary: Pacts and the Pilot Action

The pilot action and its evaluation enabled us to make some relatively clear judgements on the value of Territorial Employment Pacts as a policy instrument. By way of a final summary we believe we can offer the following statements with a good degree of certainty;

• The local level represents a significant terrain of action for employment and local development policy; • There is evidence that operating at the local level can lead to the creation of strong opportunities for job generation, for particular supply-side actions especially in relation to the insertion of excluded groups into employment and for addressing market failures by more effective matching of supply and demand. • Making these latent opportunities solvent requires the mediating actions of some form of local partnership organisation with a broad stakeholder base and with some degree of effective policy leverage and legitimacy. • Effective local partnership organisations of this nature can represent a vital catalyst for policy development and, in particular, a key source of innovation and creativity • Other things being equal, the evaluation indicates that the most likely spatial scale at which the right sort of conditions may be expected to prevail is the sub-regional level and in the case of an employment focused activity this is most effectively associated with the travel-to-work area, the bassin d’emploi or the continuous urban labour market. • However, these conditions in relation to pacts represent an average situation around which there can exist a very considerable range of other possibilities where successful interventions can be made and where success can be achieved in employment and local development. • This variety in the possible local partnership forms and spatial scales observable for territorial pacts is a product of the fact that they are embedded differentially across member states and within regions according to prevailing circumstances. • Pacts and similar local partnership structures are emerging across what can be described as a highly variable European opportunity surface where their reception ranges from enthusiastic acceptance through relative indifference to a degree of resistance and this serves to explain the variety of ways they are embedded. • Among the many issues for Pacts revealed by the pilot five particular ones are worthy of emphasis: o The need for space to act – a foothold in local structures is essential

101 ECOTEC Research and Consulting Ltd Evaluation of the Territorial Employment Pacts – Final Report –October 2002

o The need for reality checks on ambition – correctly addressing problems where they have leverage is critical o The need for size and scope to fit the mission – a balance is vital where the costs of partnership are commensurate with the added value to be achieved in delivering real outcomes o The need for requisite capacity and competency – leadership and management is challenging for pacts and local partnerships and a key variable in success o The need for focus – pacts and local partnerships need to have clear and deliverable objectives

• For territorial policy the evaluation indicates that the application of the pact principles can have generic value and are worthy of promotion in themselves, but: o The mechanisms for them to be successfully applied are not going to be available everywhere o A prior stage of indeterminate length and resource demands will be essential to create the necessary threshold conditions and for capacity to be built in those places where the starting position is low o For some states and regions the pact principles are already strongly embedded in domestic policy and the additional value of a Commission initiative is likely to be limited except where it gives the opportunity for further experiment and innovation o Across the board the unequivocal value of the pilot action was the scope it offered for innovation and experiment regardless of the circumstances in which the Pacts found themselves and in this respect it has to be considered as being both a valuable exercise and good value for money

102 ECOTEC Research and Consulting Ltd