COME T., ROUET G., 2016, « How to evaluate academic staff? Efficacy and performance of the structures and schemes», Gestion et Management Public, vol.5, n°1, 2016/3, p.57-80 [ISSN : 2111-8865]

How to evaluate academic staff? Efficacy and performance of the structures and schemes Thierry COME1, Gilles ROUET2

Introduction: Why and how evaluate the academic staff? Résumé

Cet article propose une mise en perspective de The question of how to evaluate agents deux systèmes d’évaluations des is a recurrent question concerning human universitaires, français et slovaque, dans le resource policies, more particularly within the cadre théorique de la « justice framework of service management (Chanut & organisationnelle », pour tenter d’apporter Rojot, 2011). It is nevertheless inextricably des éléments sur la problématique de linked to the issue of why. In an organisation, l’équilibre entre contrôle et validation the main aim of evaluation is to improve nationaux, d’une part, et autonomie locale de performance resulting in compliance with the recrutement et de performance, d’autre part. objectives set and their adaptation to the changing environment as well as the Mots-clés incorporation of the social and cultural Évaluation, Personnel académique, Processus standards in force within the organisation and de Bologne, Autonomie des établissements, the sharing of a common vision. To this end, Nouveau Management Public evaluation must ensure the sustainability of the regulation and limit any risk of internal conflict while it must be designed in Abstract: accordance with a principle of neutrality. How to evaluate academic staff? Efficacy and Evaluating agents must make it possible to performance of the structures and schemes understand and improve the response of the This article puts into perspective two systems organisation to the real expectations and evaluations academics, French and Slovak, in the needs of consumers, to check the validity and theoretical framework of “organisational justice”, execution of the objectives and their suitability to contribute to explore the issue of the balance in light of the practice and context and to set between national control and validation on the one their level according to the available resources hand, and local autonomy of recruitment and of that could and have been implemented. It performance, on the other. must be deemed fair by all agents while reflecting the reality of their actions. In a Key words service economy, and more particularly within Evaluation, Academic staff, Bologna Process, Autonomy of HEIs, New Public Management a knowledge-driven economy in a “learning society” (Béjean, Monthubert, 2015), the question of evaluating service providers is inextricably linked to the measurement of the system’s performance. Evaluating academic staff is therefore part of a broader process of

1 Thierry Côme est maître de conférences habilité en sciences de gestion, membre du CNU, VP CNU, CNESER, directeur du SIOU-BAIP de l’Université de Reims Champagne-Ardenne, [email protected]. 2 Gilles Rouet est professeur en sciences de gestion, directeur de l’Institut Supérieur de Management de l’Université de Versailles St-Quentin-en-Yvelines, Chaire Jean Monnet ad personam en études interdisciplinaires sur l’Union européenne, [email protected]. How to evaluate academic staff? Efficacy and performance of the structures and schemes / Thierry CÔME & Gilles ROUET evaluating universities, their efficacy and their Universities (CNU). Today, the improvement in performance. training for academic staff, the enhancement of their potential and their performances Between the Bologna process and the through motivation and involvement and the generalisation of the new public management organisation of their career path management (NPM) principles, the change in the higher facilitating the introduction of anticipatory education system in Europe forces universities competence management as part of an – which are now independent but subject to an accreditation policy are major objectives in injunction of competition (Zeller, 2015) – to terms of human resources for any public implement a system of self-assessment and to higher education system. Evaluation must be a comply with the quality control procedures tool conducive to the continuous relating to the education provided and the improvement of the system and an integral research conducted as their resources now part of a quality procedure within the primarily depend on their performances. As a establishment. fundamental element of production, academic staff can no longer escape evaluation. The evaluation system must ensure that Nevertheless, numerous stakeholders in academic staff members are as involved and higher education believe that while it is motivated as possible so that their behaviour legitimate vis-à-vis the citizens to subject satisfies the objectives set. The notions of the teaching and research staff to evaluation, in transparency, fairness, equity and particular within the framework of the acceptability of the evaluation system become organisation of a public service, numerous the necessary conditions for the success and aspects are called into question: how, why and sustainability of the system. “Perceptions of for whom should these professionals be justice allow employees to assess the quality of evaluated and who should conduct the the relations binding them to their employer” process? Just how effective is an evaluation (Manville, 2008, p. 530). The system for process in terms of the certain costs of evaluating academic staff must therefore be measurement compared to potential benefits: deemed fair to be accepted and more the removal or return to work of inactive or particularly to contribute to increasing their suffering staff, the increased incentive for the involvement. More than their desire, vast majority of teaching and research staff to acceptance of the evaluation by the person comply with this model at the risk of being evaluated is a necessary prerequisite of eliminating any original approaches or the effectiveness of the systems as a factor of pedagogical innovation (Côme & Rouet, involvement and continuous improvement. 2014)? The objective of improving the university system as a priority for its users must certainly Evaluating teaching and research staff is be used as a basis for any evaluation, although nevertheless in no way a new invention. Long the prior communication of the objectives and before the objective – observation or criticism procedures is essential. If evaluation, even if it – of implementing a “culture of evaluation” in is fair and transparent, is perceived by the staff teaching establishments, promotions, the evaluated merely as a system for justifying creation of chairs, prizes, research sanctions, it cannot be accepted without a programmes and the opening of courses were justification and legitimisation of the not only a means of encouraging and involving objectives assigned to it. Both the message academic staff but also underpinned their and the method become conditions for the reputations and served as forms of evaluation. success of any evaluation system introduced. This argument is recurrent in France in the criticisms and strong opposition to evaluation To establish public managerial as stipulated in the LRU law of 2007 and recommendations in terms of human implemented by the consolidated decree of 2 resources policy in the field of higher September 2014 and the decree of 31 August education, it is useful to compare evaluation 2015 relating to the National Council of systems in a benchmarking process in order to 58 Revue Gestion et Management Public Vol.5, n°1 Septembre/Octobre 2016 understand both what makes it possible to the customs and practices, of simultaneously accept and incorporate the system and what determining a price and estimating a value generates increased involvement and leads to a subjective approximation approach motivation of the staff evaluated. Analysing (this “estimation”, linked to “esteem”, either another European example thus enables us to of oneself or of others, mimetic desire) being understand how systems deemed “fair” and combined with a precise procedure of “equitable” by those concerned prove to be determining a price, score, result or indicator, less contentious, cheaper and, ultimately, a procedure objectified by the shared more effective. Comparing the French system, judgement of the other – the buyer, the based on a balance between the local consumer and, more generally speaking, the authority and the national structure, with the market. Slovak system of evaluation – a system that is Evaluation is thus a relationship with accepted and incorporated by the academic value (Vial, 2012), or event values, nurtured by staff evaluated – offers some initial lessons incorporating the economic meaning, of which are particularly useful in implementing course, but also the symbolic, ideological and an evaluation system in France and thus give social meanings. These values are called on to rise to a thought process relating to the establish and legitimise a measurement of the principles of organisational justice which value of objects, services or products by should underpin its construction with a view to presenting it as the accepted, rational, creating an effective system. regulatory or universal value of these objects, This article calls on theoretical works on services or products. In particular, they serve evaluation, the management of higher to reinforce and verify the conformity of education establishments and organisational practices, writings and behaviours (for justice as well as on the French and Slovak example whether or not a teacher/researcher experiences relating to evaluation in order to is “compliant” with a disciplinary field, a key determine which system would be fairest. question in France in the staff evaluation system). When evaluating people (or rather In the initial theoretical section, we when evaluating the file of a person in relation begin by explaining the elements which to their activity or their own explanation of this constitute a fair evaluation, in terms of both activity, posture or representation, etc.), this the criteria and the procedures adopted, and value is not, of course, materialised through which make it acceptable to the staff the determination of a monetary value (at evaluated. We then present a comparative least not directly, even if the financial aspects approach of the French and Slovak systems of are important both to those concerned and to evaluating teaching and research staff and the public fund managers in the event of an principles of organisational justice evaluation of teachers/researchers with a view implemented. to changing category or to obtaining training or research leave), but through a judgement, a 1. A fair evaluation? choice, a score or a ranking. Surprisingly, while the notion of value added is widely used to 1.1 Criteria, judgements and evaluate the performances of schools mechanisms (Felouzis, 2005) or even universities, the The etymology of the term “evaluation” assessment of a member of the academic explains, illustrates or summarises both a staff’s potential and the evaluation of their current problem and recurrent controversies. work in light of this potential is never In old French, avaluer conventionally means envisaged. Peer judgement also involves taboos and in no way excludes unequal “determining the value” of something (or practices, as confirmed by the continued someone?). Value is at the heart of a existence of a strong glass ceiling in higher fundamental element of the history of economic thought and the dual meaning, in 59

How to evaluate academic staff? Efficacy and performance of the structures and schemes / Thierry CÔME & Gilles ROUET education in France and Europe, unfortunately positions with regard to the subject, even if a to the clear detriment of women4. certain change in position can be observed when the evaluation becomes part of an The development of normative operative support and advisory mechanism evaluation procedures based on criteria that facilitating personalised and quota-free must be defined and selected (given that handling of any difficulties encountered. This certain criteria cannot, in fine, be included in evaluation-prevention, designed in particular an assessment objectification approach) to reduce occupational psychosocial risks appears to mirror the very significant (OPR), is in particular advocated by the CFDT. development of the conditions of these The career monitoring mechanism for teaching evaluations: in many fields, massive use is and research staff in France is largely based on made of self-assessment procedure, thus this approach as presented by so-called making it possible to “prepare” an evaluation reformist unions. It also reflects a rationale of or to legitimise, a priori, results or both. Self- balancing the power and decisions between assessment, based on analysis by those local and national levels typical of the concerned, of their compliance with management of teaching and research staff. predetermined (and clearly explained) criteria may be a factor of social peace and of the There is, of course, no perfectly prevention of conflicts and tool for evaluators. “objective” evaluation, simply mechanisms It does not, however, replace evaluation by enjoying a greater or lesser degree of recognised external operators (whose acceptance on the part of those concerned. recognition must be established by those Irrespective of whether it is a case of being evaluated). Within the Bologna Process, quantifying (a fact which is often contested the evaluation or accreditation agencies have compared to “qualification” by peers or therefore introduced recruitment procedures experts), a key question is that of the criteria: and transparency (online) concerning the CVs which questions must an evaluator answer of the evaluators who must comply with a when providing an opinion concerning a charter and a code of ethics. course to the High Council for the Evaluation of Research and Higher Education (HCERES)5 1.2 Fairness and evaluation and which criteria must candidates satisfy to The mechanisms, conscious or be qualified for a post of associate by otherwise, used by the evaluators to measure the CNU? the “value” based on the files submitted to When evaluating members of a them are linked to mental processes resulting professional body, it is important to ensure in a judgement or “evaluation”, the latter term that the criteria are legitimised by those often being preferred by its initiators as it does concerned. Nevertheless, establishing criteria not lead to (or less so) the subjectivity which is does not in itself constitute objectification but nevertheless a clear element of the process. must be part of an approaching involving the Suspicion of this subjectivity is nevertheless stakeholders, although this is often not the characteristic of current social change. The case. rejection of all new forms of staff evaluation, perceived as a form of control or additional Evaluation, as a decision-making tool oppression and in particular as a individualistic facilitates judgements and questions element of the collective regulation of wage concerning value issued by “experts”, relations, necessarily characterises most union evaluators deemed to be legitimate due to

4 The GCI (Glass Celling Index) is 1.7. For more 5 The HCERES is the French independent body which details on the inequality between men and produces evaluation reports on research teams, women in higher education, see the report university courses and establishments used by the published by MENESR “Vers l’égalité femmes- Ministry of Higher Education for accreditation hommes, les chiffres clefs”, September 2016. purposes. France has not chosen to authorise independent agencies to grant accreditations. 60 Revue Gestion et Management Public Vol.5, n°1 Septembre/Octobre 2016 their experience, maturity and competences. giving rise to behaviour dictated by a desire to Globally speaking, even if the objectives, satisfy a standard, although this would appear approaches and tools vary, it is a question of to contradict any creative or innovative highlighting a management panel, a “limited approach which in fact drives research set of indicators (5 to 10) designed to offer forwards. managers an overview of the changes in the The discussions concerning its systems they manage and enabling them to organisation and its use are certainly identify the trends influencing these systems symptomatic of a malaise and even of a within a timeframe consistent with the nature difficulty to ensure compatibility between of their posts” (Bouquin, 2008, p. 241). professional identities (Dubar, 1991) and some The indicator must demonstrate institutional developments. The foundations operational relevance, although it informs the underpinning the professional identity of manager not only of how the organisation teaching and research staff is still linked to a works; its very creation provides a framework peer-based recruitment method which can for developing, implementing and monitoring vary (Pigeyre & Sabatier 2012) and to a “self- a strategy enabling the stakeholders perception” which is barely fostered by a themselves to assimilate the performance coherent and shared self-assessment indicators (Derujinski-Laguecir, Kerne & approach (Fave-Bonnet, 2010) or even by the Lorino, 2011). If French public universities lack of a reflective stance in relation to the adopt this rationale, the introduction of results obtained, in particular in the field of evaluation mechanisms is not the result of the research (Alter, 2001). The changing status of marketisation or privatisation of the public teaching and research staff and the sector but represents an attempt to ensure the diminishing recognition of “university a new “human resource management” is ” occurring at the same time as a fall operational for a changing administration that in purchasing power help to explain a falling is endeavouring to monitor processes, results back on the discipline (Fave-Bonnet in and impacts in the organisation of a public Bourdoncle & Demailly, 1998, p. 434) which service based on the element that constitutes can very often be observed in the debates its strength and unity: the academic staff. relating to the CNTi. 1.3 Acceptance of the evaluation and The professional identity of teaching organisational justice and research staff is based on their pedagogical freedom (there is no inspection) While not actually posing a problem, all and their autonomy, specific in the case of a of these questions are certainly at the heart of public official, thereby enabling them to discussions, disputes and denunciations, part exercise their profession of researcher in of classic union delaying tactics but possibly 6 complete academic freedom. The evaluations even including a refusal to accept the law . As following their request are otherwise teaching and research staff would already be legitimate, or at least accepted (in particular subject to numerous evaluations (for within the framework of these peer recruitment and promotions in particular), evaluations) in the event of recruitment or what would be the point in adding a new promotion, although the introduction of a mechanism? Another argument is linked to the regular procedure poses a problem. In such a actual research approach which for many mechanism, the work of teaching and research researchers is collective. Collective evaluations staff is linked to set objectives often perceived instead of individual ones would thus be as a restriction of their ability to make a necessary with an approach that would avoid

6 Cf. the letter from Christophe Mileschi explaining . systematic and recurrent individual evaluation” at: 61 How to evaluate academic staff? Efficacy and performance of the structures and schemes / Thierry CÔME & Gilles ROUET judgement, a denial of their level of decision, in fine. The latter must also be competences and thus an obstacle to legitimised. In many countries, incorporating fundamental liberties. external evaluators is a means of avoiding too high a degree of endogamy. In the most common representation of academic staff, public policies use (public) Furthermore, self-assessment can also management tools sometimes without their be conducive to the adoption of a reflective having been legitimised by the stakeholders. posture, increase the accountability of the They often denounce a public policy defined by stakeholders and encourage a collective the results of evaluations and by the search for meaning and the intelligibility of implementation of benchmarking, a tool activities, for example in the administrative designed to identify efficient practices in a field. Finally, self-assessment can be competitive market which, by calling on incorporated into a benchmarking approach indicators subject to very little discussion or which is often implied in the process of contestation (in particular when evaluating highlighting and communication “best” research), results in a loss of power for the practices in the Education, Audiovisual and decision-makers (and the researchers Culture Executive Agency (EACEA)7. themselves). The contribution of research on Thus, even if it seems that, as a matter “organisational justice” provides an of principle, a regular and mandatory interesting theoretical framework concerning evaluation of all public service officers is the procedures and staff evaluations in any acceptable, if the objective is linked to the organisation. This research focuses on the role performance of these services and to the of equity in the world of work (Adam, 1963; career and recognition of the officers’ work, Greenberg, 1990; Cohen-Charash & Spector, the fundamental issue of the method remains 2001). In particular, it involves exploring (Meriade, 2011) all the more so if the very employee perception of the equitable nature notion of performance is not explained. of evaluation and promotion procedures and responses to their demands, for example in Globally speaking in Europe, while terms remuneration and work conditions teaching and research staff are recruited by (Sweeney & McFarlin, 1993) and examining peer selection and the institutional how this perception can impact attitudes and mechanisms are increasingly subject to behaviours or what are the positive and evaluations conducted by external experts, negative impacts on employee productivity or self-assessment measures are becoming more individual and collective performance. It is also generalised, both when preparing interesting to study how the perception of the accreditations and recruiting teaching and equity of situations determines, at least in research staff. The question is then to part, a change in professional behaviour. determine how to link the self-assessments with final decisions? As the perception of the level of equity is naturally subjective and linked to specific Several aspects are important: situations which can be described by - the self-assessments must be based on numerous characteristics, this notion of criteria that are understandable, clear and organisational justice is often only highlighted legitimate in relation to regulatory texts, for when “fairness”, or equity”, is violated. This is example the definition of the missions of the the case, for example, for wage differences people concerned or a shared understanding between men and women performing the of a professional body or a disciplinary field; same work in equal positions within the same company or for arbitrary dismissals. - the use of self-assessments by commissions or evaluators before taking a

7 Cf. . 62 Revue Gestion et Management Public Vol.5, n°1 Septembre/Octobre 2016

The authors have highlighted three the weight of equity in perspective, in distinct types of organisational justice: particular in relation to a principle of distributive, procedural and interactional. “equality”. “Distributive justice” (Adams, 1965) is “Procedural justice” is linked to the based on the theory of equity and concerns perception of the equity of the decision- results distributed proportionally to inputs, making process leading to a particular result. level of education, experience, production, Employees may indeed be willing to accept an etc. In a labour context, this can involve undesirable result if they feel that the remuneration, social recognition, job safety, decision-making process was conducted in career opportunities and promotions. The key accordance with the principles of issue is to be able to establish a relationship organisational justice. It is therefore possible between contributions and results, in that the perception of the equity of the particular in a constrained situation (for decision-making process results in the example the number of possible promotions legitimisation of a results which is nevertheless for university professors). An evaluation is not equitable from the standpoint of therefore necessary but is often similar to a distributive justice. Greenberg (1994), for judgement in relative terms which, when example, observed that employees who formed between peers elected to this effect, smoked more readily accepted a ban on can result in a political dimension being smoking at their place of work when they felt prioritised and therefore departing from a that they have received full information rationale of equity. concerning the change in the company’s policy. An not qualified by In numerous organisations, human the CNTi to assume the role of university resource policies are largely standardised professor can therefore accept the situation if (collective agreements, industry agreements, he believes that the system used is transparent etc.) and employees are familiar with – and and impartial. One of the main aspects of this often legitimise – the procedures, including type of analysis lies in the information relating when it means benefiting from a promotion or to the procedures and the application thereof. increase in salary based on their evaluated The involvement of employees in participatory results. rationales increases the quality of reception of An employee may nevertheless perceive this type of information (Storey, 2000) which, situations to be inequitable such as the if it remains hierarchical, can be seen as a form promotion of a colleague deemed to be of manipulation. The perception of the undeserved. The transparency of procedures is equitable nature of the procedures is thus therefore not enough: it is necessary to prove linked to the modes of command, human that the contributions of one employee are resource policy, the culture of the significantly different from those of another. organisations and the participation of the When the rules relating to the evaluation of employees concerned. The procedures must the contributions are clearly established, the be perceived as being “fair” and be accessible, employee who has not been promoted will effective and impartial. attempt to adopt a behaviour designed to Leventhat et al. (1980) highlighted other satisfy the criteria for promotion. If the rules criteria that can contribute to employee are unclear or are interpreted in relation to perception of an equitable decision-making other higher texts, it is possible, in particular process: the permanence of the procedures for a civil servant, to abandon all hope of a applied systematically to everyone; neutrality, promotion and to adopt a specific behaviour whereby decisions must be based on verifiable designed to provide a poor performance. Any factual elements, sometimes obtained by international comparison must take account of different sources; precision, as the information intercultural variations (Storey, 2000, used to justify a decision must not be Friedberg, 2005) and it is necessary to place approximate; a means for appeal, with 63 How to evaluate academic staff? Efficacy and performance of the structures and schemes / Thierry CÔME & Gilles ROUET provisions available to contest decisions; The behaviour most harmful to the representativeness, whereby the process must organisations is withdrawal or renunciation as concern all employees corresponding to the not only is the situation not resolved, but the criteria and, of course, ethics. contribution of the employees also stagnates or diminishes. In countries such as Slovakia “Interactional justice”, refers to the where academic staff are not civil servants, quality of interpersonal treatment of the withdrawal is rare because the employees, employees within the organisations, in most of whom have fixed-term contracts, are particular through official decision-making subject to mandatory evaluation in relation to procedures (Tyler & Lind, 1992). The objectives which determine the renewal of perceptions linked to the authority of a their contracts. manager or administrator can affect procedural justice judgements in relation to The contributions of works on trust, neutrality and compliance with rights. organisational justice are very interesting for The perception of the equitable nature of the issue addressed in this article. In France, decisions will thus be linked to the level of perceived injustices and inequities do not trust granted to decision-makers and the appear to be compensated by loyalty towards employee appraisal of their intentions and universities and an approach should be behaviour. Furthermore, acknowledged adopted enabling the systems and procedures neutrality pushes employees to consider to be revised in order to eliminate cases of decisions to be impartial. serious injustice completely (Brumback, 2005). In particular, it is essential to ensure the use of Certain organisational configurations appropriate performance criteria (for and certain types of behaviour of managers academic staff in France, for example in are conducive to distributive, procedural relation to the official missions of the and/or interactional justice, thereby university) and the competence of the benefiting both the employees who feel they evaluators, which is not necessarily the case in are being treated equitably and the the situation where the members of the CNU organisation, which can implement accepted are elected by their peers. procedures which are therefore more likely to be effective. Some works, in particular by Sharpe (2006), show that the perception of equity Individuals can react differently to a affects the way in which employee accept and perceived injustice with varying degrees of adjust to organisational change. This can be impact on the organisation and numerous illustrated by analysing the implementation of factors can influence the decision to act in light the architecture of university courses within of a situation perceived to be unfair (DiFabio & the framework of the Bologna Process in the Bartolini, 2009). Turnley & Feldman (1999) different countries of Europe. According to highlight four possible behaviours in response different research works (Côme & Rouet, to a situation perceived to be unfair: 2011, Lips, 2016), it is clear that Slovak departure, either permanent or temporary, academic staff, for example, legitimised and from the organisation; withdrawal or organised a radical change in their courses far renunciation, with reduced involvement and quicker than their French counterparts. This is, effort; loyalty to the organisation which allows of course, only one of several possible causes. the injustice to be ignored or rationalised; and denunciation of the injustice which may or may not lead to compensation. In the case of 2. Evaluating teaching and research decision-making bodies such as the CNTi, the staff: a French-Slovak comparative means for appeal, for example with regard to approach promotions, might not exist but the parties concerned may attempt to take action through This empirical research is based on the unions or represented groups, which numerous official reports on the French amounts to a political game. system, both political (Senate reports) and 64 Revue Gestion et Management Public Vol.5, n°1 Septembre/Octobre 2016 administrative (in particular IGAENR reports). the most renowned universities where the Interviews conducted with Slovak academic selection of candidates is more intense, staff in 2015 enabled us to refine our essentially based on an evaluation of their knowledge of their evaluation system and, scientific activity and calling on an examination more particularly, to understand their of applications by peers. The introduction of appraisal of it. The authors’ experiences within national commission to verify appointments is French and Slovak universities and in the regularly requested by certain political national evaluation bodies such as the CNU, opponents, but without success. AERES (now HCERES) and the CNESER or in The debates on this issue reflect the European programmes are also called on in a dogmatic relationship between the autonomy reflective approach, in particular with regard of higher education establishments, at least in to evaluation practices. We have also studied terms of recruitment, and the development of the literature produced by these evaluation a liberal democracy, a debate which has also bodies along with the analyses and comments been relatively intense in Poland where a of union organisations. Cross-referencing system for verifying appointments has these data has enabled us to analyse and ultimately been reinstated, as in all Central understand the differences in the approach to European countries. Unlike in France, the and understanding of the evaluation system by measures taken to break with the Soviet French and Slovak academic staff. A final, more administrative heritage provide for a national subjective, source allowed us to perform a verification (and/or appointment) after triangulation by comparing the data collected recruitment and not before, thereby raising with our experience as international experts the problem of the validity of applications a having participated in numerous institutional priori. In contrast, France chose in 1992 to twining arrangements financed by the introduce a “qualification”, an a priori control European Commission with the aim of of applications for university posts (after reforming university regulations, in particular having experimented with an a posteriori the evaluation of teaching and research staff system of control). and accrediting programmes and establishments in a co-construction approach 2.1 The evaluation of teaching and between experts from different countries. research staff in France: a blocked With this external view, we could more easily system? identify the particularities of the French and Slovak systems. In France, one of the elements of this approach, the evaluation of teaching and The means of evaluating and validating research staff, is at the centre of polemics and applications and/or appointments of teaching controversies. French public universities have and research staff in Europe would appear to followed a trend, relatively widespread in differ greatly from one country to another. It is Europe, of “autonomy” and since the nevertheless possible to identify different enactment of the law of 10 August 2007, types of mechanism using two variables: French universities have become “managers” national and/or local validation and the of their own jobs and payrolls (Balme & method of evaluating applications. Solely local Cytermann, 2012). This transfer of personnel validation can consolidate the reputation management has been something of a effect of establishments when there is little or poisoned chalice and “the most formidable poor preparation for such as rapid change. This is the case in Bulgaria where the national commission inherited from the former regime was abolished in 2009 (Rouet, 2012). The recruitment of academic staff now depends on the establishments’ strategies, thereby favouring localism in most institutions except 65 How to evaluate academic staff? Efficacy and performance of the structures and schemes / Thierry CÔME & Gilles ROUET challenge of autonomy”8. Elected by their monitoring has become effective in boards of directors, chancellors now play a connection with the five-year evaluations of more important role in terms of human the establishments. For the first time, an resource management9 thereby implying the evaluation mechanism for teaching and introduction of a regular evaluation of the research staff was to be generalised and different members of staff, a mechanism mandatory, albeit within the framework of the implemented everywhere for administrative criteria defined by the CNU sections. and technical personnel. The professional This systematic measure does not interviews conducted for the latter categories fundamentally change the mechanism which is have now become a reality in all universities. still based on the definition of criteria, not For researchers and teachers-researchers, necessarily transparently, by elected members whose remits are broad and prescribed by or those appointed by the government of the law10, the introduction of regular and, more different disciplinary sections of the CNU. This importantly, accepted evaluation remains a organisation is naturally inconsistent with the problematic objective. Some missions may development of multidisciplinarities and does indeed be subject to evaluation within the not allow a GPEC mechanism to be legitimised establishment while others can, as chosen, by within the establishment, as there is no conducted by the National Council of homogenous treatment in accordance with Universities whose structure and organisation the CNU sections (a French member of the are nevertheless founded on “disciplines”, academic staff can only be affiliated to a single often historically built on a rationale of section), or to avoid withdrawal or teaching rather than research. renunciation behaviour. Until recently, a member of the The changing management methods in academic staff in France could, in practice, French universities bring an element tension avoid any evaluation during the course of their to this issue. Regular evaluation is now career if they were never a candidate for intended for chancellors, who are managers of promotion, a change in pay grade, a research public amenities, and it is necessary to apply sabbatical or a doctoral supervision bonus (in this regulatory measure although the which case their file is evaluated by the CNU “profession” is not ready to accept a supposed and/or the local authorities in accordance with sacrifice of its “liberty” and any mechanism conditions proper to the CNU sections and the primarily controlled by the establishment is elected councils). The decree of 23 April 2009 likely to be deemed yet another management introduced a periodic evaluation procedure for instrument. It is, of course, possible to object teaching and research staff specified, that introducing a benchmarking practice in a following strong contestation by all budget situation which, if not comfortable, is departments of the CNU, by the decree of 2 at least acceptable cannot be seen in the same September 2014 which introduced “career way as in a situation of permanent restriction. monitoring” and its experimental However, this external restriction on public implementation. Since 2016, this career institutions in terms of promotion quotas or

8 Gillot D., Dupont A., “L’autonomie des recherche.gouv.fr/cid55933/presentation- universités depuis la loi LRU; le big band à l’heure autonomiedes-universites.html. du bilan”, RI no. 446, Senate, 26 March 2013, IV, B 10 Article L952-3 of the Education Code specifies 9 “It is a question of improving the living and that, “The functions of teaching and research staff working conditions of the entire university fall into the following domains: 1° education, both community, of strengthening courses to ensure initial and continuing, tutoring, orientation, greater success among students and of advice and assessment; 2° research; 3° the developing the attractiveness of higher education dissemination of knowledge and liaison with the and research professions”, excerpt from the economic, social and cultural environment; 4° presentation of the law of 2007, at international cooperation; 5° the administration http://www.enseignementsup- and management of the establishment.” 66 Revue Gestion et Management Public Vol.5, n°1 Septembre/Octobre 2016

PEDR11, for example, makes it impossible to they elected or appointed, of deputies and implement an autonomous HR policy. that of the colleagues present on the lists is lower than the average rate of members of the There is therefore a high level of section? The fears expressed by the academic distortion between the legitimate introduction staff concerning the introduction of career of comparative evaluations, often focusing on monitoring and its management by the CNU institutions in the case of public universities, relate as much to the real utility of the system and the implementation of a regular as to their representation of the institution’s evaluation of the staff in these establishments. operations. Through its Permanent However, is it really possible to perform the Commission (CP CNU) and its bureau, the CNU first without implementing the second? has reacted and introduced new rules of In France, the CNU is based on elections conduct (Chappoz, Côme, Dorbaire, Pupion, between peers and not a selection based on 2016), made its procedures more transparent competences. Despite the criticism and and introduced a process to standardise the numerous attacks it has faced, because it is the practices of its different sections. Despite this, symbol of academic conservatism for some CNU career monitoring as an evaluation and for others because it is a terrible, process is far from receiving unanimous mandarin institution or because it promotes approval. The legitimacy of an institution does clan-like procedures, the CNU is a legitimate not guarantee acceptance of the evaluation institution for the profession. This was clearly procedures it introduces. This can only be demonstrated by the success of the petition obtained through the active participation of launched to maintain its role, which was the person being evaluated in their own signed by one-third of the academic staff in evaluation. Volunteering and election or less than one week (17,000 signatures out of appointment by the ministry do not call on the the 51,000 members of teaching and research same mechanism, driven by the process of staff). The alternative credible option of professional identification described above. abolishing the national qualification procedure and replacing it with a solely local procedure 2.2 Self-assessment in Slovakia at the was categorically rejected. This compliance heart of a rapidly changing with a principle of a convention, as described mechanism by Lewis (Côme, Diemer, 1995) clearly The example of the Slovak mechanism in underlines the legitimisation of the institution terms of recruiting and evaluating teaching among the teaching and research staff. Its real and research staff illustrates a dual choice of role and organisation are nevertheless poorly procedure: introducing an a posteriori control understood, its procedures are as vague as of appointments that is more or less severe ever and the slightest discrepancy in its depending on the country and government practices (self-promotion, plagiarism, network and because it was relatively difficult to design effect, disciplinary infringement, bibliometrics an a priori system of control in countries as the single criterion, etc.) is immediately attempting to break with a centralised commented, denounced and criticised on all administrative heritage. Slovakia is also an the social networks without these supposedly interesting example because the country fraudulent practices actually being analysed. experience a relatively short period of Can we, for example, really talk about self- administrative reconstruction between 1990 promotion in the sections when the rate of and 1992 (within the framework of the new promotion of the members of the CNU, be Czechoslovakia), before introducing a new

11 For each CNU section, the candidates’ files are satisfy the criteria and thus receive an A for each classified in three groups: A for the best 20% who criterion adopted, but the quotas imposed on the fully satisfy the criteria, B for the next 30% and C section requires a classification and prevents for the remaining 50%. The latter can also fully them from obtaining a global A grade. 67 How to evaluate academic staff? Efficacy and performance of the structures and schemes / Thierry CÔME & Gilles ROUET state from 1 January 1993 in a very pragmatic research projects, welcoming foreign manner, drawing inspiration from the students from outside the EU who are different European models. required to pay tuition fees of at least 6,000 euros per year); Slovak universities are still for the most • part public; education is free and guarantees An original calculation method: different certain services for students (catering, criteria are used (number of students, accommodation conditioned by distance, surface areas of the premises, etc.) and in particular the publications and scientific resources and academic results) (Kosova & works of each member of each university Rouet, 2013). Fewer than 20% of students are are evaluated in budgetary terms and enrolled in the private sector. In this respect, condition the state allocations to the the situation would appear relatively similar to establishments. This collectivisation of that in France, all the more so as the regions, scientific activity only directly benefits the autonomous communities, also have an teaching and research staff through the interest in the universities. Public institutions operations of each faculty and each are under the direction of the ministry which, university. for example, controls and authorises the internal structures (for example modification Globally speaking, implementing the or creation of components), but they have main axes of the Bologna Process has not been been independent, for about ten years, in opposed, as it is seen as an instrument of EU managing buildings and personnel and in membership by all stakeholders in favour of defining the strategy for their teaching offer Europe (Gura & Rouet, 2011), and Slovakia has (subject to accreditation by the independent drawn considerably on certain mechanisms to agency). Accreditation for courses and introduce several means of evaluating both diplomas is in particular founded on the the establishments and the staff, and in criteria of “sustainability” and academic particular the teaching and research staff for coherence: only teaching and research staff their recruitment, their activity appraisals, who have actually published in their domain their contract renewals or their promotion to can run a course. a higher category. Successive governments have avoided budget increases and have The Slovak higher education system is gradually introduced budget control tools and characterised by: regulatory options enabling public institutions • student mobility from the very first year: as to generate their own resources (for example students undergo a selection process by authorising them to rent out their (application and/or competitive exam), premises). candidates apply to several universities One of the major breaks between the chosen according to their reputation (an old and “new” regime after the fall of the wall annual national ranking is published) and was the abolition of the civil service and the other criteria (services). As student adoption of fixed-term contracts for all accommodation is relatively easy to find university staff. Academic staff are regularly (except in Bratislava) and transport is easy required to compete for their job, which does and relatively cheap, students are quite not prevent them from breaking an ongoing willing to leave the family environment to contract to be recruited in another university. pursue their studies. This early mobility in Like contract renewal, recruitment is only part explains the students’ interest in the open to candidates who satisfy specific criteria Erasmus scheme; relating to their scientific, teaching, value • relatively low state funding (less, creation, administrative, project management proportionally speaking, than in France: and international outreach activities. We 0.60% of GDP compared to 0.76%): the observe the same missions as those define by establishments attempt to generate their regulation in France, but with a particular own resources (national and international emphasis on international outreach and 68 Revue Gestion et Management Public Vol.5, n°1 Septembre/Octobre 2016 especially conditions for creating criteria grids that does not allow some fields to be opposed incorporating the actors concerned, both to others (for example exact sciences to locally and per research field. human sciences). Like elsewhere, recruitment often Table 1 is the translation of the official begins with the doctorate. Post-doctoral table of criteria adopted by the Economics students can apply for a post of teaching Faculty at the University of Matej Bel in Banska assistant and must satisfy certain criteria Bystrica12. To be validated, applications must (publications, scientific activities, etc.) which satisfy all the mandatory criteria (M), at least are, de facto, easily satisfied as they are two optional criteria (O) and at least 6 already included in the doctoral programme. additional criteria (A). It is therefore a question After a few years, a can gain of satisfying the conditions corresponding to accreditation before competing, if satisfying the different missions of the teaching and another series of criteria, to become a research staff while retaining the possibility of docentura (associate professor). fitting different profiles. Appointments are controlled a posteriori by All forms of publication are considered. the ministry. Access to the post of university Every candidate must provide proof of professor follows the same procedure except publication of articles in international reviews that the President of the Republic appoints the with reading committees referenced in a professors upon submission of their files by database (SCOPUS, Web of Science, DOAJ, the ministry and therefore has a right of veto. EBSCO, Index Copernicus, etc.). Proof is all the A ceremony is organised at the Presidential easier to provide as the teaching and research palace on a regular basis (twice a year) to staff must regularly provide a summary of their formalise these appointments). publications and update a verified database The ministry defines a regulatory basis managed by each faculty (part of the faculty for the criteria (more or less restrictive budgets is calculated according to the intensity depending on the political options of the of the teams’ publishing activity). different governments) and each The validity of applications is not based establishment can adapt these lists, although a on publications in specific lists of scientific control may be organised at any time and journals, disciplinary field by disciplinary field. there is nothing to be gained from a university The disciplinary qualification of the journals making an exception for or trying to appoint a can be taken into consideration at candidate who does not satisfy the criteria. recruitment, but not when compiling the It is interesting to analyse the criteria criteria. On the one hand, no commission in themselves and the means of defining these Slovakia defines a list of journals or publishers, criteria. In the first case, this analysis helps only taking into consideration the existence of identify the expected profiles of the a reading committee, referencing databases candidates and their activity. The lists of and possibly (increasingly) the impact factor criteria are defined, in fine using a general base while on the other hand, the posts on offer are (regulation and the missions of teaching and not linked to “disciplines” but to “fields”. The research staff), by the teams and validated by weight of the books and chapters in books is the academic councils of the faculties and important and pedagogical publications are universities who adapt them to the also considered. Thus instead of forcing institutions’ policies and to the characteristics academic staff to publish in a limited list of of the fields of research with a pragmatism journals by defining specific criteria, Slovakia

12 Text adopted by the academic council on 6 on 15 June 2008 by the Slovak parliament. The November 2008 in accordance with § 82, original document is available at paragraph 7 of law no. 131/2002 concerning . universities and the modifications implemented 69 How to evaluate academic staff? Efficacy and performance of the structures and schemes / Thierry CÔME & Gilles ROUET has chosen to define general publication journals remains costly and problematic, in categories which also incorporate professional particular with regard to language problems. or technical publications, catalogues or notices The trend is this to attempt to publish rather which therefore correspond to all the missions than contribute to conferences. Furthermore, of the academic staff. As the criterion relating with regard to activities abroad, many Slovak to publications in international journals is academic staff members validate activities in difficult to satisfy, the regulations nevertheless neighbouring countries, although far fewer offer the possibility of adapting the criteria can declare activities in other countries, which (and reducing the number from 10 to 6, for is desired by the ministerial policy. example for an application for the post of Slovak academic staff are more than professor), on condition that a sufficient happy to quote their colleagues who repay this number of articles has been published in in kind as it is necessary to demonstrate national journals with a reading committee. quotations and avoid quoting oneself as this is The candidates must provide proof of easily detected. Each time a member of the their participation in (or coordination of, for academic staff succeeds in being published in candidates for professorships) national and a referenced journal, it has become common international scientific projects. This criterion practice to insert colleagues’ quotations, even is certainly more difficulty to satisfy for if the subjects are not the same. This numerous candidates, hence the constant behaviour does not seem to have any serious search for scientific partnerships and the effects. promotion of projects awarded, in particular Implementing this criteria-based European projects. approach has several significant With regard to previous experience, consequences: candidates must justify 5 or 10 years of • on the profile and activity of the academic teaching depending on the grades within the staff: the minimum criteria require an “field” of the desired post or a “related field”, activity and its validation across all the with both the candidate and the members of missions without specialisations that are the commission given the freedom to appraise too specific: a teacher-researcher is not a (or debate) this related nature. As with researcher and the lists of criteria take publications, it is not a question of recruiting his/her different missions into by disciplinary section but by profiles which consideration. This situation is all the more can be (and often are de facto) inter- or multi- significant as, unlike in France13 Slovak law disciplinary in nature. The “fields” are not concerning higher education does not precisely defined but are part of a dynamic specify the different functions of teachers- approach. researchers. Adopting criteria therefore Generally speaking, publishing scientific leads to a general job profile which does monographs in Slovakia remains an easy task not de facto prioritise the pedagogical, as each university has its own publication research or project coordination aspects. programme (the faculties also develop their The operations of the CNU sections in own scientific journals which are now subject France generally prioritise publications in to an obligation of referencing and thus of reviews listed above and do not, publication in foreign languages), but therefore, take other aspects of the participation in foreign conferences and teacher-researcher’s missions into

13 Article L123-3 of the education code defines dissemination of knowledge and liaison with the the mission of public higher education service economic, social and cultural environment; 4° while article L952-3 divides the posts of teacher- international cooperation; 5° the administration researchers into 5 fields: “1° education, both and management of the establishment”, of which initial and continuing, tutoring, orientation, only points 1, 2 and 5 are subject to a genuine advice and assessment; 2° research; 3° the evaluation by the National Council of Universities. 70 Revue Gestion et Management Public Vol.5, n°1 Septembre/Octobre 2016

account, or only at the margin leading to relative terms than in France as there is no specific profiling. meeting of a national university • on the preparation of applications: the commission either before or after the criteria grid, validated and legitimised by recruitments, but only a single local the academic community (through the commission and an administrative control academic committees) is published and of the criteria conducted by the used by candidates to prepare their administrative staff based on proof application, often with the help of provided locally when validating the colleagues14. Once the candidates feel that applications and then possibly by the they are ready, the applications are central administration once the submitted when a competitive exam is recruitment process is complete. opened. Self-assessment is therefore The mechanism therefore combines generalised in Slovakia while the three rationales: self-assessment, verification of criteria falls under the aegis administrative verification (which is not an of the administration. The cost of evaluation) and peer evaluation. recruitment is therefore much lower in

14 Interviews conducted with 14 academic staff February 2015, drawn from 4 different faculties (assistants, and professors) at the (humanities, political science and international University Matej Bel in Banska Bystrica in relations, economics and natural sciences). 71 How to evaluate academic staff? Efficacy and performance of the structures and schemes / Thierry CÔME & Gilles ROUET Table 1: Validity criteria for applications for the posts of (MCF HDR) and university professor Activities Criteria Docent Professor Academic scientific diploma or academic degree (PhD., science M Yes Yes candidate) M Authorisation to direct research (docent) - Yes M Experiences of pedagogical activities in the field 5 years 10 years M Theses supervised and defended - 1/1* activities O Masters dissertation supervised (defended) 10 20 A Bachelors dissertations supervised (defended) 10 0 A Scientific officer in a teaching unit No Yes Teaching A Responsible for a study programme No Yes A Preparation of a new study programme No Yes M Scientific monograph 1 2 M Articles in international scientific journals with a reading committee 5 10 M National/international research projects (including European research 2 5 programmes) M Works quoted in national journals, monographs, 20 45

university manuals and manuals for secondary education M Works quoted in international journals, monographs, university manuals 10 20 and manuals for secondary education M Author or co-author of a manual for universities or a monograph 0 1 publications M Manuals for secondary education 1 2 O Articles in national scientific journals with a reading committee 20 45 20 2 25 Conferences during international O including in classified scientific journals, 3 1 7 scientific eventsincluding in scientific journals, 12 O Conferences during national scientific eventsincluding in conference proceedings 10 20 A Notices in encyclopaedias or dictionaries 2 5 Scientific research and A Reviews, reports, critiques in journals with reading committees 10 20 A Prize or distinction for a book No Yes A Participation in international conferences Yes Yes A Participation in national conferences Yes Yes A Editor of scientific publications 2 5 A Appraisal and evaluations of monographs, professional books, doctoral 2 10 theses, dissertations, research projects Other scientific research activities (doctoral study conferences, A Yes Yes doctoral thesis evaluations, etc.)

A Research or teaching visits abroad (total time) 1 month 2 months

A Member of foreign or national editing committees of scientific journals with No Yes a reading committee A Member of juries to award scientific titles (PhD.) No Yes

A Member of scientific committees of conferences Yes Yes A Member of juries for bachelor’s and master’s degrees Yes Yes Other activities A Scientific prize No Yes * 1 supervision of a doctoral student and one thesis defended Source: Matel Bel University, Banska Bystrica

72 Revue Gestion et Management Public Vol.5, n°1 Septembre/Octobre 2016

2.3 What lessons to be learned from a be met. From the very outset, the approach comparison between France and requires the academic staff to demonstrate a Slovakia? certain level of responsibility that they will have to maintain throughout their career. An experiment was carried out to place Since 2002, self-assessment has been a the Slovak process (representative of the general and legitimised practice which would recruitment processes in Central Europe) and appear to contribute to playing down a French process into perspective. Three CVs of singular situation: academic staff in Slovakia academic staff (two candidates for a are subjected to two types of evaluation, professorship and one candidate for a post of independent of the 1-, 3- or 5-year contract associate professor) were submitted to this list renewal procedures and therefore conduct a of criteria (corresponding to the field of self-assessment of their activity at each step of “economics and management” including, for the way. example, sports management). It was impossible to use the CVs produced for the First, each university organises a French CNU (and formatted in relation to the mandatory evaluation of the teaching staff by criteria announced by the disciplinary the students and the results are sections) and the analysis was conducted using communicated to the faculty and university the candidates’ complete CVs. The two authorities. Naturally, the members of the candidates for the post of university professor teaching staff are notified of the results and if would have validated the criteria in Slovakia they are problematic, they are summoned for and could have hoped to apply while the an interview. This evaluation can have candidate for the post of associate professor repercussions on contract renewals (as can satisfied the majority of the criteria to apply objectives that have not been met). The other for a post of docent (outside the HDR)15. This evaluation is annual and systematic and is experiment highlights the limits of the French more like an administrative verification. Every system of checking the validity of applications year, each teacher-researcher must report (“qualification”) based on a partial appraisal of their publications (cf. annexe), their scientific the mission of academic staff: the three activities, their distinctions and their candidates had themselves checked that they consulting activities. This work can be satisfied the criteria announced by the CNU completed continuously as the elements are section and were surprised by the final result. entered in the publications database and as It is indeed difficult to reduce a “peer” checked by an administrative department in procedure to an administrative verification of liaison with the central administration. Thus criteria and the commissions necessarily while in France the relations between the evaluate the candidates based on their central services of the universities and those of application files whereas these candidates the central administration generally do not might better correspond to profiles proposed exist for teacher-researchers, the situation is by local universities than to a more theoretical very different in Slovakia where each academic national profile. staff member directly documents a database used to verify their activity, document the It is also a system based on trust and not accreditation file and calculate a part of the suspicion: CVs are not dissected to find budget allocated by the state16. It would assumed flaws but to highlight the criteria to

15 There is no equivalent of an associate professor 16 As the publishing activity of the members of a in Slovakia, rather a position of teaching assistant university contribute to determining part of the for which there is no list of criteria: only a PhD is public subsidy, each component will benefit from required before a hearing by a recruitment encouraging the academic staff to publish as much commission. as possible. Each university does not receive a direct premium linked to this activity, but has the 73 How to evaluate academic staff? Efficacy and performance of the structures and schemes / Thierry CÔME & Gilles ROUET appear that this system helps make the this standpoint, implementing a system of stakeholders more responsible. administrative verification based on quantitative implementation of criteria is not In the French system, it is difficult to only advantageous from a financial point of improve the situation: maintaining a process view but also with regard to the candidates of peers examining the applications, even if themselves who can anticipate and develop framed by a list of criteria corresponding to the their activity in relation to a professional functions of the teacher-researchers should project. Self-assessment ensures commitment rely on a criteria-based approach which is and increased responsibility. difficult to standardise as this means defining a national profile and not a set of profiles taking account of the different functions. From

Table 2: Comparisons of positive and negative elements of the French and Slovak evaluation mechanisms France Slovakia Positive elements Negative elements Positive elements Negative elements * Status of civil * No real incorporation * All missions of the * Salary partly servant and stable of all missions of the academic staff taken linked to the Distributive justice salary grid academic staff into account policy of the establishments * Weight of disciplinary rationales with criteria proper to each CNU * Evaluation criteria * Legitimacy of section set nationally and * Verification the CNU as a * Suspicion of favouritism adapted locally deadline by the national and clientism (for * Transparency of national level evaluation body example, self-promotion criteria (no * Any change of * Effort by CP CNU to in certain sections) interpretation of grade is linked Procedural justice standardise practices * Demagogic behaviour criteria) to a vacancy * Relative effort by in certain sections (very * Systematic use of self- and thus sections to provide different qualification assessment depends on the information rates from one section to * Neutrality of policy of the * Emergence of a another) verifications of establishment code of conduct * Absence of means of the criteria appeal in numerous procedures * Role of unions or * Major political games * Weak external * No arbitration associations (local and national constraint for HR or external (depending on the CNU elections - CNU -) promotion policy remediation sections) for their * HR policy of * Team effect with the system members chancellors under major self-assessment Interactional * Possible effect constraint procedures justice against local * Absence of interviews * Importance given to arbitration by and meetings in many thesis and generalising career procedures. qualification juries monitoring

Source: Authors

possibility of negotiating service obligations, for time to enjoy a better income (an experienced example, or work conditions, a very important university professor has a net monthly wage of element when numerous academic staff in less than 1,000 euros. Slovakia work in several universities at the same 74 Revue Gestion et Management Public Vol.5, n°1 Septembre/Octobre 2016

In France, the system for evaluating assessment based on a grid of criteria that teacher-researchers is based more on have been discussed and validated has been procedural justice and the procedures are generalised for activity monitoring as well as ensured by the legitimacy of the CNU which is for recruitments and promotions (for which governed by a peer-based elective system. scientific publications assume greater However, numerous aspects (cf. the table importance). The international dimension is above) demonstrate the limitations of this central to any general approach and the legitimisation with regard to promotions, incentive to be mobile is thus of the utmost changes in grade, etc. as each teacher- importance. This rationale is not only linked to researcher must attempt to understand and the size of the country but also to political adapt to criteria that are often unclear and choices which have resulted in a very rarely fluctuating and which cannot always be contested criteria-based approach (perhaps applied. The collective institutional appraisal is due to the “administrative” treatment of the globally positive (judging by the results of the criteria) which gives rise to self-assessment numerous petitions defending the CNU) while practices being combined with stakeholder at an individual level, the number of strategies enabling academic staff to compare withdrawal and renunciation behaviours themselves locally with their peers for remains very high (for example, in section 6, recruitment or career advancement. management sciences, less than 10% of the Contestation lies more in the resources members request a PEDR whereas 20% could allocated to the public universities, and thus to possible obtain it). possibility of opening academic staff positions, than in the processes used for recruitment and In Slovakia, the evaluations – prepared regular evaluation, even if this contestation de by self-assessments often shared within the facto rarely finds its way outside academic teams – are founded on a national base in a circles. process organised at local level, which is more a case of interactional justice whereby all the The approach also makes it possible to missions of the academic staff are taken into find a balance between local autonomy and account. The academic staff members are all national control while not sacrificing a the more ready to accept the procedure if it disciplinary rationale which would quickly gives rise to discussion (and adaptation) at have found its limitations in the Slovak local level, a situation which is diametrically context. As stakeholders of the system, the opposite in France where the fear of local authorities have certainly envisaged that the arbitration encourages the national disciplinary fields could not be defined solely mechanism to legitimised, even if it is often in reference to Slovak scientific activities but in criticised. a far broader scope. Globally speaking, therefore, it would be Conclusion: evaluation, self-assessment necessary to verify precisely the hypothesis and administrative verification that – in Slovakia (as in most other countries of Central Europe) – this generalisation of the Putting the situations in France and criteria grids, leading to self-assessment Slovakia into perspective provides elements practices and administrative verification prior concerning problem of balance between to the evaluation by a peer commission, national control and validation on the one facilitates recruitment and activity monitoring hand and local recruitment and performance for academic staff while minimising the autonomy on the other (Côme & Meskova, procedural costs and leaving the discussion of 2011). disciplinary boundaries open without disturbing the mechanisms of professional In Slovakia, the establishments manage identification and enabling each academic an subject to objectives staff member to manager their own career and thus to an obligation of results. Self- development. 75 How to evaluate academic staff? Efficacy and performance of the structures and schemes / Thierry CÔME & Gilles ROUET This mechanism is clearly perceived by effects, conformism effects and reputation those concerned as a guarantee of effects are not in themselves a guarantee of professional identity, all the more so as the increased involvement of the academic staff, criteria used would seem to take all the but it is the evaluation mechanisms and their missions of teacher-researchers into account. alignment with the characteristics and The academic staff members appear to have behaviour of teacher-researchers in each incorporated (or even accepted) this rationale country and, more broadly speaking, the that they do not consider particularly “neo- national contingency factors which determine liberal”. Ultimately, analysis of recruitments how effective they are. and contract renewals over ten years in several Beyond the issues of the performance of universities within the country shows that few these evaluation and professional identity academic staff are subject to sanctions (non- mechanisms, analysing the change in practices renewal of their contract) following difficulties resulting from these mechanisms could quickly in satisfying their objectives and that the constitute a source of evaluation of the latter, collective financial incentive is a useful and this time in relation to the role of the university relatively effective motivation tool. A in constructing and disseminating knowledge contrario, this mechanism pushes academic at a global level. staff to act according to the criteria and, for example as participation in conferences without any recognised publication (with a reading committee) is rarely taken into References account, opportunist behaviours are easily identified. ADAMS J. S., (1965), “Inequity in social exchange”, The academic staff members are this in Berkowitz L (ed.), Advances in Experimental Psychology, 2, pp. 267-299, Academic Press. subject to (or contribute to) evaluation systems that differ considerably from one ADAMS J. S., (1963), “Toward an understanding of country to another and it would be interesting inequity”, Journal of Abnormal and Social to extend this comparison of mechanism by Psychology, 67, pp. 422-436. means of an analysis of their effects on the ALTET M., (2001), « L’analyse de pratiques. Une behaviour of academic staff, in particular the démarche de formation professionnalisante ? », youngest among them. While Slovak academic Recherche et Formation, 35, pp. 25-41. are less encouraged to take part in ANDRO A., BACQUÉ M.-H., GAUDILLIÈRE J.-P., conferences, French academic staff are MURARD N. & TISSOT S., (2008), « Que faire pour primarily driven to publish articles in journals l’Université ? », Mouvements, 3 (55-56), p. 7. included on lists drawn up by national BALME P. & CYTERMANN J.-R., (2012), L’université commissions. It is unproductive for a young française : une nouvelle autonomie, un nouveau associate professor to publish a book, either in management, Grenoble, Presses universitaires. France or elsewhere, or an article in a Russian, BECCARIA C., (2009) [1764-1766], Des délits et des Polish, Brazilian, Indian or German journal peines, trad. Philippe AUDEGEAN, Lyon, ENS even if it has a significant impact factor (as this Éditions. criterion is not taken into account). Conforming to the criteria of the different BÉJEAN S., & MONTHUBERT B., (2015), Pour une mechanisms naturally has rapid effects on the société apprenante, propositions pour une stratégie nationale de l’enseignement supérieur, STRAENES behaviour of academic staff and consequently Report, septembre. on the very attractiveness of the university. BOUQUIN H., (2008), Contrôle de gestion, Paris, Equity, the transparency of decisions Presses Universitaires de France. and recognition of their validity naturally contribute to this incorporation of the BOURDONCLE R., & DEMAILLY L. (dir.), (1998), Les professions de l’éducation et de la formation, system’s rules and objectives by the academic Villeneuve d’Asq, Presses Universitaires du staff and to their acceptability. The network Septentrion. 76 Revue Gestion et Management Public Vol.5, n°1 Septembre/Octobre 2016

BRUMBACK G., (2005), “A performance appraisal FELOUZIS G., (2005), « Performances et “valeur that’s better than ratings and rankings”, Innovative ajoutée” des lycées : le marché scolaire fait des Leader, 14, October-December. différences ». Revue française de sociologie, 46(1), pp. 3-36. CHANUT V., & ROJOT J., (2011), « Réinventer l’évaluation. Les nouvelles règles du jeu RH dans FRIEDBERG E., (2005), « La culturel ‘‘nationale’’ une administration publique », Management & n’est pas tout le social. Réponse à Philippe Avenir, 49(9), pp. 195-222. d’Iribarne », Revue française de Sociologie , 46(1), pp. 177-193. CHAPPOZ Y., Côme T., DORBAIRE P., & PUPION P. C., (2015), « Valeurs et régulation de systèmes GIBERT P., (2009), Tableaux de bord pour les universitaires: l’élaboration des codes de organisations publiques, Paris, Dunod. déontologie en France et dans les pays du CAMES », GILLOT D., DUPONT A., (2013), « L’autonomie des Gestion 2000, 32(5). universités depuis la loi LRU ; le big band à l’heure COHEN-CHARASH Y., & SPECTOR P., (2001), “The du bilan, RI n° 446, Sénat, 26 mars, IV, B. Role of Justice in Organizations: A Meta-Analysis”, GREENBERG J., (1994), “Using socially fair Organizational Behavior and Human Decision procedures to promote acceptance of a work site Processes, 86(2), pp. 278-321. smoking ban”, Journal of Applied Psychology, 79, CÔME T., & ROUET G., (2014), « Évaluation et pp. 288-297. management de la recherche dans les universités, GREENBERG J., (1990), “Organizational Justice: le cas français en perspective » in FALLON, C., & Yesterday, today and tomorrow”, Journal of LECLERCQ, B. (dir.), Leurres de la qualité dans Management, 16, pp. 399-432. l’enseignement supérieur ? Variations internationales sur un thème ambigu, pp. 301-316, HURTEAU M., & HOULE S., (2013), L’évaluation de coll. Thélème, Paris, Academia. programme axée sur le jugement crédible, Québec, Presses universitaires. CÔME T., & DIEMER A., (1995), « Conventions et institutions », Les Cahiers du CERAS, 4, PUR, Reims. KAHN A., & LOUART P., (2010), « Le management de l’université », Humanisme et Entreprise, 4(299), CÔME T., & MESKOVA L. (dir.), (2011), Les pp. 5-12. Universités en Europe centrale, 20 ans après, volume 2. Économie et management, Bruxelles, KOSOVA B., & ROUET G., (2013), « L’enseignement Bruylant. supérieur et le Processus de Bologne en Slovaquie : bilan et critiques », Revue internationale CÔME T., & ROUET, G. (dir.), (2011), Processus de d’éducation de Sèvres, avril, 62. Bologne, construction européenne, politique européenne de voisinage, Bruxelles, Bruyland. LEVENTHAL G., KARUZA J., & FRY W., (1980), “Beyond fairness: A theory of allocation DERUJINSKI-LAGUECIR A., KERNE A., & LORINO P., preferences”, in MIKULA G. (ed.), Justice and Social (2011), « Une approche instrumentale des Interaction, Springer-Verlag. indicateurs de performance », Management & Avenir, 42, pp. 111-132 LIPS, C., (2016), L’appropriation du Processus de Bologne par les acteurs de l’Université, enjeux et DI FABIO A., & BARTOLINI C., (2009), “The impact of perspectives, Thèse de doctorat en sciences de organizational justice on job satisfaction and gestion, Université de Versailles St-Quentin-en- affective commitment in an Italian hospital”, Yvelines, octobre 2016. Psychologie du Travail et des Organisations , 15(4), pp. 420-434. MANVILLE C., (2008), « Perceptions de justice et implication : les salariés contingents et permanents DIVAY S., (2011), « L’évaluation qualitative seraient-ils différents les uns des autres ? », .Objectiver sans compter », Bref-Cereq, n° 286 mai. Relations industrielles / Industrial Relations, 63(3), DUBAR C., (2010), La socialisation : Construction pp. 529-554. des identités sociales et professionnelles, Paris, MERIADE L., (2011), L’évaluation de la performance Armand Colin. des universités françaises : entre simplification de la FAVE-BONNET M.-F. (dir.), (2010), L’évaluation complexité et complexification de la simplicité, dans l’enseignement supérieur en questions , Paris, Sarrebruck, Éditions Universitaires Européennes. L’Harmattan.

77 How to evaluate academic staff? Efficacy and performance of the structures and schemes / Thierry CÔME & Gilles ROUET MEYER J.-P., & HERSCOVICH L., (2001), assessment of justice”, Journal of Organizational “Commitment in the workplace: toward a general Behavior, 18, pp. 83-98. model”, Human Ressource Management Review, THEVENET M., (1992), Impliquer les personnes dans 11, pp. 299-326. l’entreprise, Paris, Éditions Liaisons. PIGEYRE,F., & SABATIER M., (2012), “Recruter les TURNLEY W., & Feldman D., (1999), “The impact of professeurs d’université : le cas du concours psychological contract violations on exit, voice, d’Agrégation du supérieur en sciences de gestion », loyalty, and neglect”, Human Relations, 52, July. Revue Française d’Administration Publique, pp. 399-417. TYLER T., & LIND E., (1992), “A relational model of authority in groups”, in ZANNA M. (ed.), Advances ROUET G., (1993), L’invention de l’école , Nancy, in Experimental Social Psychology, pp. 151-191, Presses Universitaires. Academic Press. ROUET G., (2012), « Réformes et enjeux de VIAL M., (2012), Se repérer dans les modèles de l’enseignement supérieur en Bulgarie », Revue l’évaluation, dans les modèles de l’évaluation, internationale d’éducation de Sèvres, n° 61, pp. 25- méthodes dispositifs, outils, Bruxelles, De Boeck. 32. ZELLER C., (2015), Mobilisation collective des SHARPE A., (2006), The psychological contract in a professionnels et représentations sociales de la changing work environment, Work Institute paper. performance organisationnelle : le cas des SHEPPARD B., Lewicki R., & Minton J., (1992), enseignants-chercheurs dans les universités Organizational Justice: The Search for Fairness in françaises, Thèse de Doctorat en Sciences de the Workplace, Lexington. Gestion, Aix Marseille Université, Aix-en-Provence, décembre. STOREY J., (2000), Human Resource Management: A Critical Text, Thomson Learning. SWEENEY P. D., & MCFARLIN D. B., (1993), “Process and outcome: Gender differences in the

78 Revue Gestion et Management Public Vol.5, n°1 Septembre/Octobre 2016

Appendix: Publication evaluation form

Matej Bel University, Banska Bystrica, Faculty of Humanities and Social Sciences The evaluation is based on the list and on the proof of publication recorded by the university and submitted to the ministerial subsidies section of the ministry. Cf. http://www.library.umb.sk/index.php?module=articles&id=105&language=1 rotations. NB: the originals and at least photocopies of the covers and first and last pages, of the summary and of the ISBN no. or other; they are recorded in a national database. Part of the subsidy is calculated according to the scores of each member of the faculties. Each university and each faculty can adapt the mechanism using the criteria established at the nation level. This procedure is conducted at the end of each period. Maximum number of points (100%) and professor and docent = 100 points; teaching assistant = 80 points.

Category € Points (agency) ty Difficul AAA Scientific monograph published by a foreign publisher A 100% 6 130 AAB Scientific monograph published by a national publisher [in one of the main world languages B 100% 6 115 –UK, FR DE, ES, RUS] AAB Scientific monographs published in national publications C 100% 6 100 ABA Collections, scientific anthologies published abroad A 100% 6 130 ABB Collections, scientific anthologies published in the country 100% 6 100 ABC Chapters in scientific monographs abroad B -- 2 30 ABD Chapters in scientific monographs in the country -- 2 25 ACA University manuals published abroad B 20% 6 100 ACB University manuals published in the country C 20% 6 80 BAA Popularisation monographs published abroad 20% 4 50 BAB Popularisation monographs published in the country 20% 4 50 BCB Manuals for primary and secondary schools (C) 20% 6 80 BCI Course documentation (C) 20% 6 80 CAA Artistic monographs, catalogues published abroad 20% 4 50 CAB Artistic monographs, catalogues published in the country 20% 4 50 EAJ Translations of popularisation works 20% 4 50 EAI Exercise books 20% 3 40

FAI Other editorial work (bibliography, encyclopaedia, catalogues, dictionaries, collective 20% 3 40 reviews, etc.)

ADC Scientific articles published in international scientific journals with a reading committee A 65% 2 50 ADD Scientific articles published in national scientific journals with a reading committee A 65% 2 50 BDC Popularisation article published in international scientific journals with a reading committee 65% 1.5 30 BDD Qualified work in national scientific journals with a reading committee 65% 1.5 30 CDC Works of art and translations in foreign scientific journals with a reading committee 65% 1.5 30 CDD Works of art and translations in national scientific journals with a reading committee 65% 1.5 30

79 How to evaluate academic staff? Efficacy and performance of the structures and schemes / Thierry CÔME & Gilles ROUET

ADE Scientific articles published in international journals without a reading committee [if A 1% 2 40 recorded in international databases / index] ADF Scientific articles published in national journals without a reading committee [if recorded A 1% 2 40 in international databases / index] AEC Overviews of scientific articles published in international journals, scientific conference 4.5% 1.5 20 reports, monographs. AFA Guest contributions at international scientific conferences A (B) 4.5% 1.5 35 AFC Other contributions at international scientific conferences A (B) 4.5% 1.5 30 ACC Chapters in manuals published by a foreign publisher 2.5% 2 20 BEC Professional documents published abroad, conference proceedings 2.5% 1 10 AED Overviews of scientific articles published in national journals, scientific conference reports, 2% 1.5 20 monographs AFB Guest contributions at national scientific conferences C 2% 1.5 30 AFD Other contributions at national scientific conferences C 2% 1.5 25 ACD Chapters in manuals published by a national publisher 1% 2 20 BED Professional documents published domestically, conference proceedings 1% 1 10 BBA Chapters of other monographs published abroad 0.5% 1.5 15 BBB Chapters of other monographs published domestically BCK Chapters of manuals and pedagogical texts 0.5% 1.5 15 BDA Lexicons, dictionaries, terminology works and encyclopaedias published by foreign publishers BDB 0.5% 1 5 Lexicons, dictionaries, terminology works and encyclopaedias published by national publishers BDE Popularisation publication in foreign journals without a reading committee 0.5% 1 5 BDF Popularisation publication in national journals without a reading committee CDE Works of art and translations of foreign works in foreign journals without a reading committee CDF Works of art and translations of foreign works in national journals without a reading 0.5% 1 5 committee

The € column shows the financial advantage of ministry funding: group A1 (6270 € = 1 = 100%), group A2 (1243 € = 1/5 = 20%), group B (4121 € = 2 / 3 = 65%), group C - sub-group C1 (4.5% - 2.5%), sub-group C2 (2% - 1%), sub- group C3 (1%), sub-group C4 (0.5%). Difficulties: the degree of difficulty depends on the work provided, the time and the number of people involved in preparing the monographs (min. 60 p.). Ultimately, the points are allocated on the basis of these three different categories of indicator.

80