Commission of the European Cqmviunities

COMMISSION OF THE EUROPEAN CQMVIUNITIES C0M(94) 69 final Brussels, 20.09.1994 COMMUNICATION FROM THE COMMISSION TO THE COUNCIL AND THE EUROPEAN PARLIAMENT Final evaluation of the results of Eurotra: a specific programme concerning the preparation of the development of an operational Eurotra system for Machine Translation. COMMMUNICATION OF THE COMMISSION TO COUNCIL AND THE EUROPEAN PARLIAMENT Subject: Final evaluation of the results of Eurotra: a specific programme concerning the preparation of the development of an operational Eurotra system for Machine Translation I. INTRODUCTION 1. This communication concerns the evaluation of the results of the Eurotra research programme. The evaluation was carried out through a panel of independent experts, in accordance with Article 4 of the Council Decision 90/664/EEC of 26.11.90 concerning "the preparation of the development of an operational Eurotra system". The Decision also stipulates that the evaluation should be transmitted to Council and the European Parliament. The evaluation report entitled "Final Review Panel Report, February 1993" and the Opinion of the Eurotra Advisory Committee are annexed to this Communication. 2. This communication gives a short overview of the Eurotra programme, the main conclusions and recommendations of the final evaluation and the position of the Commission. H. THE EUROTRA PROGRAMME 3. In November 1982, the Council decided to launch the Eurotra research and development programme (Council decision 82/752/EEC of 4.11.82). The objective was to overcome language barriers: "the multilingual nature of the European Community is of high cultural value, but is also in practice an obstacle to closer ties between the peoples of the Community, to communications and to the development of the internal and external trade of the Community". 4. The programme was staged over five and one half years (1982-1987) at an estimated cost oi 16 M ecus. It comprised three phases: preparatory actions, basic and applied linguistic research, and stabilisation of the linguistic models.and evaluation of the results. A The expected result of the programme was an operational prototype for a machine translation system in a limited field and for limited categories of text dealing with all official languages of the Community. This prototype would provide the basis for development on an industrial scale in the period following the programme. 5. Following the enlargement of the Community, the Council decided in 1986 to add the Spanish and Portuguese languages to the system. At the same time, the budget was increased by 4.5 Mecus and the duration was prolonged for two and one half years to 1989. 6. In 1988, Eurotra was reviewed by an Evaluation Committee of independent experts headed by Dr. A.E. Pannenborg (ex- Vice-Chairman of Philips). Their Final Report was transmitted to Council and Parliament (COM(88) 270 final). 7. The Evaluation Committee reached the following conclusions : Eurotra had succeeded to generate substantial cooperation between Member States, in a field of growing importance. Eurotra had contributed substantially to strengthen the human resource base for research in computational linguistics and for the emerging language industry. Substantial progress was made towards achieving the scientific and technical objectives of the programme. However, efforts should be made to involve industry and the area suffered clearly from a lack of long term policy at the Community level. 8. Following this evaluation, the Council decided the transition of the Eurotra Programme to the third phase (Council Decision 88/445/EEC of 25.7.88), and in 1989, the Council decided to accord another 7 Mecus for the completion of a machine translation system of advanced design (Eurotra). (Council Decision 89/410/EEC). This programme aimed to implement a system prototype, improve the Eurotra software, linguistic specifications and training methods, prepare for the industrial development of Eurotra, and set out evaluation objectives and criteria. 9. In March 1990, a second evaluation on the Eurotra Programme was carried out by independent experts, chaired by A. Danzin (ex-Director IRIA and Vice-President of Thomson-CSF) and transmitted to Council and Parliament (COM (90) 236 final); lO.The Danzin report confirmed the findings of the Pannenborg evaluation of 1988. The original expectations of the mid 1980s appeared to be over-ambitious. Genuine progress had been made in the project since the Pannenborg evaluation, however. 11.The Danzin report stressed that, by the very met of its existence, Eurotra has laid the foundations for a Community achievement in the field of language technologies. The report strongly reiterated the need for a long term Community strategy in the field. 12.Based on the findings of the Danzin report the final phase of the Eurotra programme was approved by Council (Council Decision 90/664/EEC of 26.11.90). This phase was allocated 10 Mecus and aimed at "the development of a high-level scientific prototype in the field of automatic translation". This phase ran from 1990 to 1992. In line with recommendations from the evaluators, shared-cost projects were launched, notably for system development, testing and research. New avenues of work were opened up on advanced system architectures, lexical and terminology resources, standards, education and training. III. THE FINAL REVIEW: AIMS AND SCOPE 13.When Eurotra came to an end in December 1992, a final evaluation of Eurotra was subsequently carried out by a panel of independent experts (Final Review Panel Report, February 1993). The evaluation took place in the first quarter of 1993 in accordance with Article 4 of the Council Decision 90/664/EEC of 26 November 1990. The panel was led by Prof. Brian Oakley, former Director of the UK Alvey research programme and retired chairman of Logica Cambridge. 14.The final evaluation had two complementary aims: " to appreciate the achievements of the Eurotra programme in the years 1991-1992, or more precisely, in the period since the last evaluation (i.e. 1990)", and " to appreciate the outcome of the programme which was conceived in the late seventies and had lasted ten years". Furthermore, the final evaluation should look at the way in which the recommendations of the previous evaluators have been taken up both in the 1990-1992 phase of the Eurotra programme and in follow-up programmes. IV. CONCLUSIONS AND RECOMMENDATIONS OF THE FINAL REVIEW 15. The Final Review Panel Report examines three distinct aspects of the Eurotra programme: - its main research phases between 1983-1990; - the final phase preparing the development of an operational system during 1990-1992; - and future options for Community action in the area of language technology. 16.Main conclusion The main conclusion of the Final Review Panel report is that the language area is of highest importance. The report states that: "The problems of language are amongst the largest challenges racing the European Community. We are divided by our different languages and the resulting communication failures...The cost, both in direct economic terms and in loss of cohesion generated, is very heavy, especially compared with our main competitors in the USA and Japan...With Eurotra...a base has been constructed on which future programmes can build, in the struggle to bring technology to bear on the language problem of the Community". The report makes a series of detailed conclusions on different aspects of the programme, as set out below: 17.1nitial Objectives: The evaluators conclude that the difficulty and scope of the initial objective of the Eurotra programme (especially that of developing a system for handling all languages in parallel) have not been adequately recognised at the start of the programme. A longer-term plan would have been needed, with a more pragmatic aim and with strong involvement of industry in the mainstream development work. 18.Technical considerations: It is noted by the report that the programme revised its original objectives in line with the previous evaluations, and worked towards a scientific prototype, rather than an operational system. The programme has now achieved the scientific basis for industrial developments in this area. The scientific quality of the work is felt to be an achievement of the programme and may well turn out to be influential in future systems designs in Europe and elsewhere. The research work on semantics, for example, is considered of very great importance The technical outputs include notably the production of a language specification for each of the official Community languages. The Eurotra Reference Manual, together with the specifications, is a remarkable record of'the outcome, containing a description of the largest linguistic effort ever made at a multilingual level. The objectives of the work on the software development platform (ALEP) and on re-usable lexical resources are considered by the evaluators to be eminently desirable and practical. Further development and exploitation by the research community make excellent sense It is felt that more attention should have been given to dictionary development, to user interfaces, to interactivity and to producing demonstrators at the appropriate stages. The creation of a very coherent community of computational linguists in every country of the European Community is a very considerable achievement, where the credit lies largely with the Eurotra programme and the supportive governments. The shift from contracts of association with Member States to cost-shared projects with growing involvement of industry is to be welcomed. On the whole, however industry participation remains low. The report notes that this situation is being remedied in die follow up programmes to Eurotra. i9.ResuKs and exploitation; The Final Review Panel points to several results of the Eurotra programme. At the technical level, the Reference Manual, the language specifications, software systems developed centrally or in individual centres, are prime examples. But probably the most important output firom the programme is the manpower trained in computational linguistics and in machine translation.

Load more