Global University Rankings and Their Impact

EUA report on rankings 2011 global university rankings and their impact a ndrejs rauhvargers Copyright © by the European University Association 2011 All rights reserved. This information may be freely used and copied for non-commercial purposes, provided that the source is acknowledged (© European University Association). Additional copies of this publication are available for 10 Euro per copy. For ordering information, please contact publications@eua.be or write to: European University Association asbl Avenue de l’Yser 24 1040 Brussels, Belgium Tel: +32-2 230 55 44 Fax: +32-2 230 57 51 A free electronic version of this report is available at www.eua.be. ISBN: 9789078997276 “Six Blind Men and an Elephant” image Copyright © 2011 Berteig Consulting Inc. Used with permission. EUA report on rankings 2011 global university rankings and their impact a ndrejs rauhvargers t he blind men and the elephant by John Godfrey Saxe (1816-1887) It was six men of Indostan To learning much inclined, Who went to see the Elephant (Though all of them were blind), That each by observation Might satisfy his mind. The First approach’d the Elephant, And happening to fall Against his broad and sturdy side, At once began to bawl: “God bless me! but the Elephant Is very like a wall!” The Second, feeling of the tusk, Cried, -”Ho! what have we here So very round and smooth and sharp? To me ‘tis mighty clear This wonder of an Elephant Is very like a spear!” The Third approached the animal, And happening to take The squirming trunk within his hands, Thus boldly up and spake: “I see,” quoth he, “the Elephant Is very like a snake!” The Fourth reached out his eager hand, And felt about the knee. “What most this wondrous beast is like Is mighty plain,” quoth he, “Tis clear enough the Elephant Is very like a tree!” The Fifth, who chanced to touch the ear, Said: “E’en the blindest man Can tell what this resembles most; Deny the fact who can, This marvel of an Elephant Is very like a fan!” The Sixth no sooner had begun About the beast to grope, Then, seizing on the swinging tail That fell within his scope, “I see,” quoth he, “the Elephant Is very like a rope!” And so these men of Indostan Disputed loud and long, Each in his own opinion Exceeding stiff and strong, Though each was partly in the right, And all were in the wrong! MORAL. So oft in theologic wars, The disputants, I ween, Rail on in utter ignorance Of what each other mean, And prate about an Elephant Not one of them has seen! c ontents EDITORIAL 7 AcROnyms 9 GLOssARy 10 ExEcuTIvE summARy 11 Purpose and principles of this review 11 Summary of observations and findings 12 I. InTRODucTIOn 18 Purpose of the report 18 Brief summary of the history of rankings 19 Implications of the rankings in brief 20 II. MethODOLOGIEs Of ThE mOsT pOpuLAR GLObAL RAnkInGs 23 1. International rankings producing league tables 24 2. Rankings concentrating on research performance only (with or without producing league tables) 38 3. Multirankings 44 4. Web Rankings 57 5. Benchmarking based on learning outcomes 59 III. AnALysIs Of REsults (What DOEs IT mEAn fOR unIvERsITIEs) 60 Which universities are considered in world university rankings that produce league tables? 60 Rankings and the research mission of universities 61 Rankings and the teaching mission of the universities 63 Biases and flaws 64 The dangers of taking ranking positions too seriously 66 Iv. Main cOncLusIOns 68 v. GuIDAncE TO InTERpRETInG RAnkInG REsults 69 General guidance 69 What should be noted regarding each university ranking? 70 REfEREncEs 76 6 editorial EUA commissioned this report in response to the growth in international and national rankings, as a result of increasing questions from member institutions requesting information and advice on the nature of these rankings, because of the interest shown by national governments in ranking exercises, and finally in light of the European Commission’s decision to develop a ‘European ranking’. An ad hoc working group of Council was first established in 2009 to consider how the association could best serve its members in this respect. This resulted in a decision of the EUA Council to launch a pilot project to publish, in 2011, the first in a series of EUA reviews of rankings. This project was entrusted to an Editorial Board chaired by EUA President, Professor Jean-Marc Rapp, former Rector of the University of Lausanne, and including: Professor Jean-Pierre Finance, President of the Henri Poincaré University, Nancy 1 and EUA Board member; Professor Howard Newby, Vice-Chancellor of the University of Liverpool; Professor Oddershede, Vice-Chancellor of the University of Southern Denmark and President of the Danish Rectors’ Conference. Professor Andrejs Rauhvargers, Secretary General of the Latvian Rectors’ Conference accepted our invitation to carry out this analysis, and is the author of the Report. We are honoured that the Gulbenkian Foundation and the Robert Bosch Foundation have, together, agreed to support this project over a two year period. The report focuses on international rankings and also refers to a number of other ongoing projects seeking to measure university performance. It describes and analyses the methodologies used by the main international rankings using only publically available and freely accessible information. It is intended as a service to EUA members, often under pressure to appear in the rankings, or to improve their position in one way or another. It is clear that despite their shortcomings, evident biases and flaws, rankings are here to stay. They ‘enjoy a high level of acceptance among stakeholders and the wider public because of their simplicity and consumer type information’ (AUBR Expert Group, 2009). For this reason it is important that universities are aware of the degree to which they are transparent, from a user’s perspective, of the relationship between what it is stated is being measured and what is in fact being measured, how the scores are calculated and what they mean. However, it is important to underline that international rankings in their present form only cover a very small percentage of the world’s 17,000 universities, between 1% and 3% (200-500 universities), completely ignoring the rest. They are of direct relevance for only around half of EUA members, situated in a small percentage of those countries in which EUA has members, and strongly correlated with the wealth of those countries. The report confirms that most international rankings focus predominantly on indicators related to the research function of universities. Attempts to measure the quality of teaching and learning generally involve the use of proxies, often with a very indirect link to the teaching process, and are rarely effective. The importance of links to external stakeholders and environments are largely ignored. Where existing data is used, it is often not used consistently, and reputational factors have in many cases disproportional importance. Taken together, this leads to an oversimplified picture of institutional mission, quality and performance, and one that lacks relevance for the large majority of institutions, especially at a time when diversification and individual institutional profiling are high on agendas across Europe. 7 On a more positive note, the arrival of global rankings over the last few years has focused considerable attention on higher education, and put the spotlight on universities that are increasingly being compared nationally and internationally. Rankings have certainly helped to foster greater accountability and increased pressure to improve management practices. They have encouraged the collection of more reliable data and in some countries have been used to argue for further investment in higher education. Although it is said they can be used to guide consumer choice, there is little convincing evidence that they do so (except in Asia or via the CHE Ranking in Germany). It is our view that at present it would be difficult to argue that the benefits offered by the information they provide, given the lack of transparency that we have observed, are greater than the ‘unwanted consequences of rankings’. For there is a danger that time invested by universities in collecting and using data and statistics in order to improve their performance in the rankings may detract from efforts to progress in other areas such as teaching and learning or community involvement. Looking to the future, measures are being taken by the ranking providers to try to improve the methodologies they use, which we can only encourage, and which we will follow up in subsequent reports. The International Rankings Expert Group (IREG) has announced that it will conduct an audit of the various rankings. We hope that this exercise will include the use of independent experts as this would add considerably to its credibility. Among the open questions for future consideration is that of the ‘democratisation’ of rankings to allow more of the world’s universities the opportunity to find their place, and what this would entail. With regard to some of the more recent European initiatives that seek to broaden the focus of rankings to cover the different missions of the university experience suggests that lack of internationally comparable data is a challenge. The debate will continue and EUA will take up these and other questions in future reports. AC knowledgements The considerable and detailed analysis for this report has been carried out by Professor Andrejs Rauhvargers. The Editorial Board would like to thank Professor Rauhvargers most sincerely for his commitment, for the enormous amount of time he has invested in researching, carefully describing and analysing the various rankings, ratings and classifications included in the review. It has been a challenging enterprise, not least given the initial decision made to take account only of publically available information on the various rankings included.

Load more