Scope of thequality criteria same timerejecting andopposingcriteriathat are inappropriate. ­scientists are encouraged toapplythesecriteria whilstat the improve quality inthetechnologicalsciences.Technological funding agencieswithasetof criteriadesignedtomaintain and other research institutions,research policymakers andresearch technological sciencesthemselves. Itprovides and universities This acatech POSITION PAPER andforemost directed isfirst at the ously discussed,reviewed andinformedbyresearch practice. and across different cultures. Accordingly, they mustbecontinu value systems.Furthermore, quality criteriawill vary over time tain preferences that are themselves anexpression of particular ply betheproduct of research findingsalone–they reflect cer theless, itshouldbestressed that quality criteriawillnever sim The criteriaare theresult of abroad consultation process. Never sciences asawhole. oped thefirst ever set of quality criteriaforthetechnological of the technological sciences”,acatech“voice has now devel cal engineeringandIT. Inkeeping withitsmissiontoactasthe ommendations, thesewere confinedtothedisciplines of electri- and Humanities workinggroup produced avaluable setof rec technological sciences.AlthoughaGerman Councilof Science Appropriate quality criteriahave yettobeformulated forthe tal impact onresearch quality. cally geared towards thedisciplineinquestion have adetrimen world-class research. Inappropriate criteriathat are notspecifi- community andmustsetoutthestandards required todeliver ity criteria should be established by the corresponding scientific needs itsown quality criteriaforevaluating Thesequal itswork. There isaconsensusinacademiathat every fieldanddiscipline acatech (Ed.) Success ­Research Recommendations forAssessing Sciences the Technological Quality Criteriain cal sciencesasawhole.Individual sub-disciplinesmay wishto The quality criteria recommended below applytothetechnologi­ Executive Summary andRecommendations acatech POSITION PAPER ------research. They primarily relate toresearch, includingthe ­appropriate modifications –fortechnicalcollegesengaged in they are alsovalid fornon-university research and–withthe and goals.Whilethequality criteriaare aimedat universities, weighting of criteriawillalsodependonthespecificcontext add specificcriteriaorweightthecriteria.Theselectionand § § § § § § The key criteriaare asfollows: agencies. tutions, research policymakers andresearch funding They are directed at and other researchuniversities insti- whole. of quality criteriaforthetechnologicalsciencesasa This acatech POSITION PAPER presents thefirst ever set At aglance § § § § § § tion has practical relevance. Innovation prizes indicate that theresearch inques - ships shouldbepromoted. Participation inlarge, established research partner ceive ahigherweightingthan patents. been successfullymarketed andshouldtherefore re- Licences are anindication that innovations have cations shouldbeevaluated positively. high number of doctorates and postdoctoral qualifi- account forasignificantproportion of research. A Doctorates incollaboration –often withindustry– cured third-party funding. private fundingwhenitcomestosuccessfullyse- No distinctionshouldbedrawn betweenpublicand sciences. tive performance evaluation inthetechnological Publications are only of limited value for quantita - -

Foto: iStock/mediaphotos development of young researchers and technology transfer. developing know-how with practical applications. Consequent- They are unweighted and free to use. ly, especially in the technological sciences, no distinction should be drawn between public and private funding (e.g. from the It is not the intention of this acatech POSITION PAPER to pre- German Research Foundation or industry), provided that part- scribe which criteria are best suited to maintaining and improv- nerships with industry serve to generate or apply new knowl- ing the quality of the technological sciences in each specific edge. The material resources required to carry out a project vary case. Researchers and research institutions must set their own from one discipline to another. To prevent any resulting bias, priorities for the individual quality criteria. material resources should be excluded and only human resourc- es should be taken into account when using these criteria.

Recommended quality criteria for the (3) Number of doctorates and postdoctoral qualifications technological sciences Doctorates account for a significant proportion of the research carried out in the technological sciences. They often involve (1) Publications collaboration with industry and thus also equip people for the Publications are only of limited value for quantitative perfor- labour market. The postdoctoral “Habilitation” qualification mance evaluation in the technological sciences. The h-index helps to develop young academics. In principle, a high number and impact factor may perhaps have some importance for cer- of doctorates and postdoctoral qualifications should therefore tain individual sub-disciplines. However, they should not be the be evaluated­ positively. However, it is important to ensure high deciding factor in the appointment of professors with industry supervision standards. experience to their first university post, since most of these indi- viduals will have had limited opportunities to publish while (4) Leadership of and involvement in research partnerships they were working in industry. Modern research questions call for collaboration between scien- tists from different disciplines. Since this is a time-consuming In the technological sciences, it is important to consider the process, participation in large, established research partner- ­value and impact of publications not only as far as academia is ships should also be promoted. concerned, but also for the national innovation system. Refer- ence works, textbooks, monographs and significant contribu- (5) Patents and licences tions to conference reports or to standardisation work and regu­ Patents are a measure of innovation and the fundamental de- lation drafting should all be taken into account, rather than sire to translate innovations into practical solutions. Licences, concentrating solely on articles for scientific journals. Until this on the other hand, document the cases where these solutions happens, the assessment of publications should be carried out are successfully delivered. Accordingly, licences are a more im- by experts on a qualitative basis. portant quality criterion than patents in the technological sciences. English is the lingua franca of global research. Nevertheless, publications in other languages aimed at national innovation (6) Spin-offs system actors also play a prominent role in the technological Spin-offs are indicative of an effort to translate research findings sciences. It is important not to lose sight of this fact. into practical solutions. To quantify spin-offs, it would be neces- sary to employ parameters such as number of employees or The overriding importance attached to articles in English-­ turnover. language journals as a performance criterion results in re- search findings being split up and published in several parts. (7) Organisation of prominent scientific events This is at odds with the quality requirements for the technologi­ The organisation or contracting of scientific events (e.g. confer- cal sciences. ences or summer schools) is an indication of the status and rec- ognition that researchers and research institutions enjoy within (2) Third-party funding measured in terms of the number of the scientific community. posts funded The ability to successfully secure third-party funding indicates (8) International research exchanges that the applicant is able to deliver the desired project goals, Research quality is also reflected in international recognition. whether these involve expanding our theoretical knowledge or This should be evaluated using parameters such as the number

2 of long-term visiting researchers in both directions (i.e. visiting and engineering associations, standards bodies, science self-­ German technological scientists abroad and visiting foreign sci- regulation bodies or funding bodies. entists at German institutes). (10) Science and innovation prizes and awards (9) Posts/appointments in scientific organisations and research Prestigious prizes and awards show that researchers are held in funding bodies high esteem by the scientific community. Innovation prizes in Posts and appointments reflect the status of researchers within the technological sciences also indicate that the research has the scientific community. These roles include e.g. the editorship practical relevance. of scientific journals and positions in scientific societies, science

Methodological approach

The quality criteria set out in this acatech POSITION PAPER are the result of a broad consultation process within the tech- nological sciences. As well as representatives of the key technological science fields, the project group also included social scientists and humanities scholars engaged in research into the technological sciences. The interim findings were discussed by a wide group of experts and stakeholders from academia and industry.

Editor: acatech – National Academy of Science and Engineering, 2018

Munich Office Office Office Karolinenplatz 4 Pariser Platz 4a Rue d’Egmont /Egmontstraat 13 80333 | 10117 Berlin | Germany 1000 Brussels | Belgium T +49 (0)89/52 03 09-0 T +49 (0)30/2 06 30 96-0 T +32 (0)2/2 13 81-80 www.acatech.de F +49 (0)89/52 03 09-900 F +49 (0)30/2 06 30 96-11 F +32 (0)2/2 13 81-89 [email protected]

This executive summary is based on: acatech (Ed.): Qualitätskriterien in den Technikwissenschaften. Empfehlungen zur Bewertung von wissenschaftlichem Erfolg (acatech POSITION PAPER), ­Munich: Herbert Utz Verlag 2018. The original version of this publication is available at www.acatech.de/publikationen or www.utzverlag.de.

3