The Evaluation of Research In October 2008 the Italian National University Council (CUN, Consiglio Universitario Nazionale) requested all scholarly societies in the humanities to express their views on the evaluation of research. The paper below, produced by the Board of AIA (Associazione Italiana di Anglistica) outlining AIA’s position and circulated to all its members, was also signed by Compalit (Associazione per lo Studio della Teoria e Storia Comparata della Letteratura), by AIS (Associazione Italiana degli Slavisti) and by SUSLFF (Società Italiana per gli Studi di Lingua e di Letteratura Francese). The Research Dimension in Evaluation by all.2 Although its promoters tend to say that the classification does not have an evaluative As we start it needs to be remembered that the function, it has inevitably led to category “C” evaluation of research products is just one taking on a negative valence as the “recipient” dimension of evaluation. Other dimensions of everything which has not been placed in the include teaching and the contribution made to “higher” categories.3 The panels responsible the discipline in the form of participation in for the ERIH classification then argue that the colloquia and conferences, preparation of merit of each contribution is totally unconnected academic events etc. It is to be hoped that, in to the categories where the journal is “placed”: deciding the criteria for the evaluation of every category can host contributions whose products, those factors are not sidelined which intrinsic value is highly diverse. This is one of can prove useful in tracing the continuous the many reasons why the classification has professional development of scholars in relation given rise to considerable criticism from to the scientific community they belong to and researchers working in the humanities,4 one thus in relation to the growth of their own cogent example being the finely documented students, considered as people and not, in the and caustic article published in Le Monde words of one university President, as “credit Diplomatique (December 2008, p. 32). crunching machines”.1 There are a wide range of possible publishing The Product Collocation Scenario venues for the products of disciplines included This is how we choose to call the listing of the in English Studies in Italy: journals and the sources of publication held to a) journals/series of Department volumes (or be qualifying in each disciplinary sector. There former Institutes, now Departments); exist classifications of journals, such as ERIH b) journals/volume series published in Italy (European Reference Index for the (in Italian, which are also published in English); Humanities), which we do not consider, at least c) journals/series of volumes published in in their current form, to be particularly valid/ Italy, only in English; useful. The members of the ERIH panels d) journals/ volume series of Italian University themselves point out that the classification is (English) Departments published abroad; not aimed at providing instruments to evaluate e) international journals/series published research projects, or the research products by outside Italy in English; the individual researcher. Indeed it is important to emphasise that the ERIH criteria of f) international journals/series published distribution and circulation, which place any outside Italy in a foreign language other than journal in a language other than English in English; category “B” or “C”, automatically create an g) journals/series published online. imbalance (not to say a distortion) in favour of Any exhaustive and detailed “picture” of all the English which is not necessarily to be shared available journals, series and publishers may in 13 The European English Messenger 18.1 (2009) a sense seem a paradox. The world of publish- even in its most heated moments, unanimously ing is dynamic and constantly changing, making claims that the worth of the single contribution any attempt to “freeze” every source of publica- is not indissolubly linked to its venue, and rather tion in a set position or category not only difficult has stressed that in the construction of know- to achieve but above all pointless: ledge it is not unusual for the more radically a) there are journals and/or series which innovative work to appear in publications which appear locally and then grow rapidly in terms are not mainstream (cf. Le Monde Diploma- of quality, selectivity and prestige; tique, cit.). b) well-known international publishers are There are 2,200 sources of publication regularly involved in takeovers, mergers or the mentioned in the Bibliographie Linguistique5– launching of new series – Pergamon is a case and linguistics is just one of the many research in point, rapidly entering and then leaving the domains covered by English Studies in Italy. In market; the light of what has been stated so far, it would c) there is a network of large and small seem unwise to produce a selection or a publishers in Italy which require protection classification of the publishing scenario for (these days there are frequent press reports of English Studies in Italy – or other non English- their going out of business), so as to avoid the speaking countries. Researchers in the field of tradition of scientific publication in Italian being English Studies in these countries typically simply annihilated by the hegemony of English- operate in a wide variety of academic contexts language publishers; and scholars are often under pressure from the d) finally (or rather above all) there is, and it start of their university career not just to publish is expressly emphasised here, the prestige and in English, but also to constantly try and improve originality of the single product which for the quality of the English language journals various reasons is published in Italian and at a produced “at home”, to make them competitive local level, but this should detract nothing from internationally. its worth. Giving way weakly before criteria If, then, each evaluation board takes due which at all costs reward international account of the value of internationally publication compromises or prejudices in every recognised indicators such as the ISBN of a sense the value of the single research product. volume or series or the ISSN of a journal, of its No evaluation board can, in the end, shirk the refereeing policy and of its international editorial responsibility of evaluating the individual piece board etc., this does not release it from the of work as a contribution to the construction obligation of reading – reading also work that of knowledge, independently of the language is published in “local” journals which do not have or of the kind of publication in which it appears. an international refereeing apparatus – and of The history of science itself reveals how the expressing an evaluation. In this regard we feel strictest parameters can be spoiled or obscured that the debate on refereeing needs to be by human actions in which straightforward mentioned, nor can we ignore the comments common sense would have rendered judgment raised over the role of international journals as simpler and clearer. The history of the invention gatekeepers.6 The English language is of the telephone is a good example, with its undoubtedly an instrument which provides recent attribution to the Italian Meucci after visibility, yet visibility, and the power connected decades of legal battles which had long with it, is no unquestioned guarantee of primacy favoured the American Graham Bell. in the construction of knowledge: judgment on It seems quite dangerous for the future of the individual product cannot give such publishing, as we have noted above, to squeeze misleading weight to its place of publication. out the so-called minor journals (minor in terms Brilliant Ph.D students and young researchers, of circulation, subscriptions etc.) to prop up a in the haste imposed by job application deadlines market which broadly speaking favours English and national selection procedures, have language journals. Further, the current debate, published with local publishers, “gifting” them 14 products which would have merited publication, which have recently been circulating it is clear but at the cost of a long waiting list, in that bibliometric criteria are: international publications. a) difficult to use in an appropriate way;8 Within English Studies, for over a decade b) far from being completely reliable.9 there has been a lively debate over the criteria It is worth noting that the Humanities Committee which give centrality to knowledge, leading to of the European Science Foundation has stated marginalisation and nearly silencing great unequivocally that these criteria are not suited scientific traditions. Publication in English to knowledge in the humanities.10 involves adopting English models and patterns The limitations of the bibliometric approach of scientific writing - this interweaving or the to evaluation should be highlighted, above all in Gordian knot of language, knowledge and 7 the humanities, to reduce the risk of hasty power has been widely studied and debated. consensus, already lingering in the air: in several Italian writing on English Studies in Italy is Faculties, humanities scholars have been put well-placed internationally. Perhaps, as well as under pressure by statisticians and more being the rigorous custodian of the standards generally quantitative disciplines to espouse of their own journals and series, we should also, bibliometric parameters and apply them to their together with other foreign language and own work. literature scholars and with academics in Italian It would therefore be useful for colleagues in Studies, support Italian sources of publication these areas to read the argumentation of the and journals both in Italian and in other European major world exponents in their respective languages. A ranking system mainly based on disciplines, before brandishing bibliometrics as quantitative parameters will likely place them a useful sword with which to adjudicate the in the “waste paper basket” of category “C”, sharing out of research funding.