sign in sign up
<<
Home , Evaluation

ADE – ENTERPRISE PLC - ZENIT

2. EVALUATION

2.1 OBJECTIVES AND SCOPE OF THE EVALUATION

Objective of the evaluation: the purpose of this study is to provide a thorough and systematic analysis of the contribution and impact of the Structural Funds RTD and innovation actions in the Member States and regions from an economic development perspective. The study should also contribute to establishing guidelines for future investments by Structural Funds in this field in the assisted regions, based on the experience of the past and current intervention.

In defining the objectives of the evaluation, the Commission services noted that in developing RTDI for Objective areas, “it is imperative to have a detailed understanding of the current regional situation in order to determine what policies might be most appropriate. A region- specific analysis, according to the characteristics of the Objectives, enables current innovation and technology trends to be assessed and gaps to be identified in the framework”. This observation was taken as the starting point by the evaluation team in developing the methodology since it both underlines that there may be need to consider more fully the specific economic and innovation profiles of the Objective 2 zones when devising policie s; and the need for a thematic evaluation of interventions in favour of RTDI to take due account of the wider of support agencies and the culture of innovation in the regions concerned. As such, the basic building blocks for the evaluation are the 15 regional case studies through which the evaluation attempted to provide a richer assessment of the different factors which influence the means of delivery and the effectiveness of Objective 2 RTDI measures.

The Terms of Reference for the evaluation set out seven key issues and five key aspects for the comparative analysis which are resumed in the following tables.

TABLE 2.1: The Seven key issues identified by the TOR

Core issues identified by the Commission Ancillary Questions 1. How effective were the Structural Funds in promoting RTDI Pertinence of actions funded; capabilities Coherence with regional strategy 2. To what extent and through which mechanisms have the Private sector participation in RTDI Structural Fund programmes contributed to reinforcing links to local measure design; business and industry? Good practice examples and transferability 3. To what extent have the Structural Funds programmes ERDF/ESF synergies; contributed to overcome skill shortages and improve qualifications Graduate retention / integration; in the field of RTDI? Equal opportunities 4. To what extent has the new RTDI capacity provided with the Impact on business performance and assistance of Structural Funds, enhanced competitiveness and employment economic performance in the areas concerned? Sustainable development Diversification of economy 5. What has been the impact of CSFs and SPDs on national and Change in regional innovation regional RTDI policies and funding, especially on regional distribution of RTDI capacity? Network and linkage effects 6. Has there been synergy and co-operation or rather duplication Exchange of good practice; with other relevant Community policies, in particular the RTD Complementarity of actions. framework programmes? 7. What are the main policy issues for the future in regard to the Programme design development needs of the Objective 2 regions Strategy & implementation mechanisms

Thematic Evaluation RTDI in Objective 2 – Synthesis Report 10 ADE – ENTERPRISE PLC - ZENIT

Type of measures

Thematic Evaluation RTDI in Objective 2 – Synthesis Report 11 ADE – ENTERPRISE PLC - ZENIT

TABLE 2.2: Five Key Aspects of the Comparative Analysis

Theme Focus of analysis RTDI funding Amount and proportion of funds allocated to RTDI The importance assigned to different Breakdown of expenditure on RTDI related actions basis of categorisation of types of projects of RTDI organisation and Evaluation and structures structures The effectiveness of instruments and Appraisal of key types of mechanisms instruments through case studies The impact on human resources and Analysis of ESF measures in training certain regions

In order to make these issues more explicit, the evaluation team developed a full set of research questions which were presented in the inception report. Both the original issues and the research questions are recalled at appropriate occasions during the remainder of this report.

2.2 METHODOLOGICAL APPROACH

In order to carry out the evaluation, the evaluation team proceeded in three main stages in line with the terms of reference of the study:

· definition and submission of a detailed methodological framework and work-programme to the Commission in the form of an Inception Report; · a comparative analysis of trends in funding, objectives and programme design (e.g. single priority against horizontal distribution of RTDI measures across programmes) in all 86 Objective 2 zones carried out on the basis of desk-research; · an in-depth review of RTDI measures, projects, partnership and programme management procedures in 15 case study regions. These case studies were used to both confirm the trends identified in the initial desk research and to analyse key thematic issues based on an extensive series of with the main programme participants and surveys. An impact on companies as the final beneficiaries of RTDI measures was implemented in certain Member States.

A series of methods and tools were developed in order to gather and analyse the and qualitative at each stage of the evaluation:

· a categorisation of RTDI measures with a view to facilitating the analysis of funding trends; · a logical framework type table which allowed the structuring of key information on RTDI measures in each of the SPDs; · a template for the country reports with synthetic financial and analytical tables including regional innovation profiles; · an SME impact survey questionnaire; · good practice templates used to summarise measures and projects providing a novel approach to supporting the innovation activities of regional firms or with an identifiable impact on regional competitiveness. · an guide and questionnaire for the case studies.

Thematic Evaluation RTDI in Objective 2 – Synthesis Report 12 ADE – ENTERPRISE PLC - ZENIT

Rather than producing stand-alone regional case study reports, the information obtained from the field work in the case study regions enriched the final versions of the Country Reports.

Thematic Evaluation RTDI in Objective 2 – Synthesis Report 13 ADE – ENTERPRISE PLC - ZENIT

TABLE 2.3: Sources of information

COUNTRY CSFs/OPs/ CSFs/OPs/RP SPDs SPDs Evaluations Evaluations Evaluations Annual Reg. Other RPDs/SPDs Ds/SPDs 94-96 97-99 89-93 94-96 97-99 Reports RTD Sources 89-91 91-93 Reports (RIS, RTP..) Austria N/E N/E 4/4 4/4 N/E 4/4 4/4 4/4 2/2 4/4 Belg./Lux. N/R N/R 5/5 5/5 4/5 5/5 5/5 N/A N/E Denmark 2/2 2/2 2/2 2/2 2/2 2/2 N/E 2/2 Finland N/E N/E 7/7 5/7 N/E N/E N/E N/E N/A N/A France 1/19 N/A 19/19 19/19 9/19 15/19 19/19 19/19 3 12/21 Germany 6/6 6/6 9/9 9/9 6/6 9/9 9/9 9/9 Italy N/R N/R 11/11 11/11 9/9 5/5 N/E 7 1 3/3 Netherlands N/R N/R 5/5 5/5 3/3 5/5 5/5 N/A 2/2 Spain 4/6 6/6 7/7 7/7 6/6 5/5 6/7 N/A N/E Sweden N/E N/E 5/5 5/5 N/E N/E N/E S S United N/A 1/9 12/12 12/12 6/12 3/12 9/12 5/12 Kingdom N/E Note: For detailed information on available data please see the respective country reports. N/E: document non existent N/R: document not requested N/A: document not available

Thematic Evaluation RTDI in Objective 2 – Synthesis Report 14 ADE – ENTERPRISE PLC - ZENIT

Comparative analysis

As a first step, desk research was carried out in each of the eleven1 Member States covered. The documentation used at this stage consisted of the available SPDs and CSFs, the annual reports, ex- ante, interim and ex-post evaluations, etc. Table 2.3 gives a general overview of the data that has been employed for the desk research. In most of the countries, this material was accessible either through the Commission or direct contacts with regional authorities. Data availability varied across the Member States, e.g. for Germany and Spain, previous programming documents and annual reports were fully available. However, some national evaluation teams suffered from a lack of source material, mainly due to difficulties in obtaining requested information through official channels.

The documentation available was used to analyse each Objective 2 region in detail concerning baseline data, the RTDI strategy and measures in the SPDs and the actual and planned expenditures on RTDI measures from 1989 (where applicable) to 1999. A key emphasis was placed on completing the logical-framework and financial tables for the 1994-96 period on the basis of a categorisation of RTDI measures.

Due to the increasingly all encompassing definition of the term innovation, such a categorisation is complex since there is inevitably overlap with traditional policy domains, notably business support. Moreover, each Member State tends, for historical or institutional reasons, to define the scope and remit of RTDI measures/actions in a different manner (e.g. in Germany, investment grants are considered to promote "innovation" in the recipient companies through the acquisition of new equipment, etc.). In the Terms of Reference for the evaluation, the Commission suggested three broad categories of measures: scientific potential; technological potential and support to innovation. In order to establish a more precise definition of the parameters and limits of the analysis, these categories were broken down into a series of "standard RTDI measures" with the aim of:

· allowing an adequate level of demarcation between different policy options, instruments and delivery mechanisms; and · permitting an inter-regional and across country comparison with respect to funding flows.

The categorisation is reprinted in Table 2.4 below with a series of examples, which are not necessarily intended to represent "good practice", drawn from the Country Reports in which further information can be found. In the case of the examples given for “actions”, they are used illustrate the way in which the ERDF/ESF intervened and are not necessarily the unique mission of the centres/programmes mentioned.

It should be noted that the categorisation is an analytical tool not a model on which RTDI measures should be structured in the context of a programme. By its very nature the classification separates out different types of actions. In implementation, the need to integrate and build measures with significant critical mass means that in practice the sub-measure or project types are often put together.

1 For the purposes of this evaluation the single Objective 2 zone in Luxembourg was included in the Belgian Country Report.

Thematic Evaluation RTDI in Objective 2 – Synthesis Report 15 ADE – ENTERPRISE PLC - ZENIT

TABLE 2.4: Categorisation of measures

Types of measures Example from evaluation Enhancement of scientific potential Infrastructure S.1.1. Grant / loan for construction of research facilities · Centre of excellence in electromechanics/ aerospace (Liège - Belgium Equipment S.1.2. Grant / loan for purchase of new equipment · Chemistry Laboratory of Horticulture Engineering (La Rioja - Spain) Actions S.1.3.1 Grant for research projects · Technology Centres (Pais Vasco) S.1.3.2 Grant for collaborative research projects · Technology Centres (Pais Vasco)

Actions S.2.3.1 Grant to cover costs of preparing submission to EU RTD FP · Re-Craft (Yorkshire & Humberside) S.2.3.2 Grant for human capital mobility n.a. S.2.3.3 Grant for funding of researchers n.a.. Enhancement of technological potential Infrastructure T.1.1 Grant for construction of technology resource centres; · Advanced Manufacturing Technology Centre (East London & Lee Valley) T.1.2 Grant for construction of science & technology parks · Creation and development of four technology parks in Piemonte (Italy)

Equipment T.2.1 Grants for equipping technology resource centres · Jyväskyla Centre of Excellence for pulp and paper industry (Central Finland)

Actions T.3.1 Grants/loans to stimulate contract research (technology transfer); · Mechatronica (Twente - Netherlands) T.3.2 Grants for development of technology brokering services; · Technology Centre North-Netherlands (Groningen-Drenthe) T.3.3. Grants / financial engineering schemes for the promotion of spin-offs; · Competence Centre for Energy, Environment and Construction (Vorarlberg - Austria) · Centre de Valorisation des Produits de la Mer (Nord-Pas de Calais - France) T.3.4 Funding for services of technological resources centres · Interface Enterprise-Université (Liège - Belgium) T.3.5 Funding for University interface services · Coteach (East London and Lee Valley - UK) T.3.6. Grants for mobility of researchers to SMEs · n.a. T.3.7 Grants for mobility of business personnel to research facilities · Technofutur (Liège- Belgium) T.3.8. Funding of training in new technologies

Thematic Evaluation RTDI in Objective 2 – Synthesis Report 16 ADE – ENTERPRISE PLC - ZENIT

TABLE 2.4 (ctnd): Categorisation of measures

Support to innovation Infrastructure I.1.1 Grants for construction of innovation centres · Lee Valley Business Innovation Centre (East London and Lee Valley)

Equipment I.2.1 Grants for equipment of innovation centres n.a.

Actions I.3.1. Support to stimulation actions (audits, business innovation · CASIMIR (Auvergne - France) counsellors, demonstration projects, networking of operators, etc.) Bremen Innovation Agency (Bremen - Germany) Sheffield Regional Technopole (Yorkshire & Humberside - UK) I.3.2 Inter-firm networking initiatives (clusters, etc.) · NOR-COM (North Jutland - Denmark) Mission-Régionales Textiles Techniques (Nord-Pas de Calais) MEDI-Link (Yorkshire & Humberside) I.3.3 Aid for the recruitment of non-scientific specialised personnel · Innovative Assistant Scheme for in-house ecological supplies (Styria - Austria) I.3.4. Incentives schemes to use specialised consultants; · Fonds Régional d’Aide au Conseil (all regions - France) I.3.5 Promotion of co-operation with non-scientific resource centres · Prodesign (East London and Lee Valley) (technical colleges, design institutes, etc.) 1.3.6 Training schemes for the management of innovation n.a. I.3.7 Subsidies for industrial R&D · Applied Research Grants (Piemonte - Italy); I.3.8 Subsidies for collaborative industrial R&D; · Cluster projects (Zuid-Oost Brabant - Netherlands) I.3.9 Support to financial engineering schemes/funds · Venture capital scheme (Pais Vasco - Spain) I.3.10 Support to strategic planning initiatives for RTDI · Regional Technology Plan (Twente - Netherlands)

Thematic Evaluation RTDI in Objective 2 – Synthesis Report 17

ADE – ENTERPRISE PLC - ZENIT

BOX 2: Difficulties in Identifying RTDI content

Measure 1.1 "Support for business initiatives" in the 1994-96 SPD for Alsace gathers under a single measure accounting for 15% of the SPD, actions as different as:

· Aids for capital investment in SMEs (which often have a very limited technology and innovation content) · Aids for intangible investment and access to counselling (same remark as above) · Support for exports actions (not relevant for analysis) · Support for joint business initiatives (which might have an innovation or technology transfer content) · Support for acquisition of technologies by SMEs (relevant to analysis) · Support for innovation projects in firms (relevant to analysis) · Recruitment of technicians in SMEs (probably relevant to analysis).

Source: Country Report France

In retrospect, it is clear that in a number of Member States (France, Finland, the Netherlands), this categorisation was difficult to apply due to two main reasons:

· firstly, in many SPDs, RTDI actions are included in measures of a generic nature supporting, in particular, SME development (see boxed example below); · secondly, even where a specific measure contains only RTDI type actions the content is often defined only at a very aggregate level and during implementation diverse projects may be funded altering considerably the real content and (more critically) the objectives of the measure.

Despite these caveats this tool enabled the evaluation team to analyse the evolution of RTDI content using the broad categories and to examine good practice using an action type framework. This first attempt at a categorisation of RTDI measures in Europe has allowed a significant body of evidence to be constituted concerning planned funding allocations across the 86 Objective 2 zones. This data allowed the testing of a number of postulates such as the existence of a trend over the periods 1989-93, 94-96 and 97-99 from a concentration of resources on the first two categories (science and technology) towards innovation support measures.

The main output of the desk research phase was the production of 11 Country Reports on the basis of a common template. These reports were the basis for both an interim synthesis report submitted in July 1998 and for the comparative analysis section of the final report.

In-depth analysis in 15 case study regions

Following the submission of the interim report, an in-depth analysis were carried out in fifteen case study regions (see table below) as the basis for responding to the issue linked to efficiency in programme management, effectiveness and impact. In each case study region, the analysis was carried out at several levels: firstly at the programme management level with face-to-face interviews with the programme managers focusing on the importance assigned to RTDI and the quality of RTDI organisation and structures. Secondly, at the partnership level with interviews and addressed to the most important RTDI project promoters (the organisations responsible for managing specific measures or sub-measures) in order to analyse a series of issues around partnership, programme design, effectiveness of measures, etc..

In addition, final beneficiaries were asked to answer a questionnaire on the RTDI projects funded under Objective 2 that they had carried out. This company survey could be carried out in five case study regions in three Member States United Kingdom, Austria and Germany. The analysis of the case

Thematic Evaluation RTDI in Objective 2 – Synthesis Report 19 ADE – ENTERPRISE PLC - ZENIT study regions allowed the identification of good (and bad) practice and innovative approaches and served as basis for key sections of the Country Reports.

It is important to underline that over and above the impact survey the evaluation approached the issue of effectiveness and impact essentially through case studies using good practice or exemplar projects funded under Objective 2 to illustrate the types of impact that funding RTDI activities can have on the growth prospects of a regional economy. Although it would have been preferable, and indeed it had been foreseen, to back up these illustrative examples with quantified indicators of impact, the evaluation team was faced with a dual problem:

· investments in certain types of projects (notably larger infrastructure based initiatives, e.g. technology parks in Piemonte, Italy) although begun as early as the 1989-93 period has only become fully operational during the 1997-99 period; and more fundamentally, · project and programme level monitoring and evaluation of RTDI activities is extremely poor (this issue is developed in section with the consequence that little or no data on impact on the final beneficiaries (i.e. regional firms) has been collected by Objective 2 programme managers over the last decade.

Table 2.5 provides a list of the case study regions for each of the 11 Member States concerned by Objective 2 and indicates the company in the core team responsible for overseeing the preparation of the Country Report and the national expert (where different).

TABLE 2.5: list of case study regions and members of evaluation team

Country No. of case Case study regions Country co-ordinator - studies national expert Austria 1 Steiermark ZENIT Joaneum Research Belgium/ 1 Liege ADE Luxembourg Denmark 1 North Jutland Enterprise - Hanne Tanvig Finland 1 Central Finland Enterprise VTT France 2 Nord-Pas de Calais ADE - Auvergne MERIT Germany 2 North Rhine Westphalia ZENIT Bremen Italy 1 Piemonte ADE The Netherlands 1 Twente ADE Capio Consult Spain 2 Pais Vasco Enterpris e La Rioja Infyde Sweden 1 Blekinge Enterprise Hanne Tannvig UK 2 Yorkshire and the Humber Enteprise - East London CRESR

In order to avoid duplication and overlap, meetings were held with the evaluators for the Thematic Evaluation of RTDI in Objective 1 zones (consortium led by Circa, Ireland) and with the Ernst & Young team carrying out the SME Thematic Evaluation. Close co-operation was also maintained with the Commission services and notably the desk officer in the evaluation unit of DGXVI, Mr Andrea

Thematic Evaluation RTDI in Objective 2 – Synthesis Report 20 ADE – ENTERPRISE PLC - ZENIT

Mairate, who was instrumental in ensuring access to required documentation at EU level and in informing and guiding the work of the evaluation team.

Thematic Evaluation RTDI in Objective 2 – Synthesis Report 21