THE Japan University Rankings 2019 methodology | Times Higher Education (THE)
METHODOLOGY FOR THE TIMES HIGHER EDUCATION JAPAN UNIVERSITY RANKINGS 2019 March 2019
1
THE Japan University Rankings 2019 methodology | Times Higher Education (THE)
Times Higher Education Japan University Rankings: Times Higher Education (THE, part of THE World Universities Insights Limited) is the data provider underpinning university excellence in every continent across the world. As the company behind the world’s most influential university ranking, and with almost five decades of experience as a source of analysis and insight on higher education, we have unparalleled expertise on the trends underpinning university performance globally. Our data and benchmarking tools are used by many of the world’s most prestigious universities to help them achieve their strategic goals.
The annual Times Higher Education (THE) Japan University Rankings, started in 2017, aims to provide the definitive list of the best universities in Japan, evaluated across four key pillars of Resources, Engagement, Outcomes and Environment. Times Higher Education’s data is trusted by governments and universities and is a vital resource for students, helping them choose where to study. Benesse Corporation is a publisher of educational materials in Japan, and has strong relationships throughout the Japanese education community. These rankings have been prepared by THE together with Benesse Corporation, and are published by Benesse Corporation in Japan and by THE across the world
To help demonstrate the integrity of the Rankings, we have asked PricewaterhouseCoopers LLP UK (“PwC”) to undertake independent limited assurance over our application of the specific procedures (i) - (xiv). Their independent limited assurance opinion on our application of specific procedures (i) – (xiv) is set out on page 16 and 17 of this document. The independent assurance opinion from PwC covers the top 100 universities by rank, and banding allocation below the top 100. It does not include individual pillar scores.
The specific procedures (i) – (xiv) are split into four sections: 1) Data collection and sources 2) Criteria for exclusion, inclusion, and data processing 3) Calculation, scoring and ranking 4) Publication and reporting
Red boxes will document the methodology area and context for the specific procedure (i) – (xiv) marked with the symbol “Ω” and bolded, which has been independently assured by PwC.
Important links:
JUR 2019 Final Rankings: https://www.timeshighereducation.com/rankings/japan-university/2019
JUR 2019 Methodology: https://www.timeshighereducation.com/world-university-rankings/methodology-japan-university-rankings-2019
2
THE Japan University Rankings 2019 methodology | Times Higher Education (THE)
Directors’ Statement: This document (the “Methodology”) sets out our end-to-end process for generating the THE Japan University Rankings 2019 (the “Rankings”). As directors and management of Times Higher Education, we state that we have followed our Methodology and correctly applied the specific procedures denoted by (i) - (xiv) as marked with the symbol “Ω” in the report.
Signed: ……………………………………………..
Print: ………………………………………………..
Role: ………………………………………………..
Date: ………………………………………………..
For and on behalf of THE World Universities Insights Limited
3
THE Japan University Rankings 2019 methodology | Times Higher Education (THE)
Summary of the Rankings methodology:
The THE Japan University Rankings score universities across four key pillars that we believe are important when applying to universities. These are:
Resources: does the university have the right resources? Engagement: does the university engage its students? Outcomes: does the university produce good results? Environment: does the university have a supportive environment?
THE uses 16 carefully calibrated performance metrics, listed below, to provide comprehensive and balanced comparisons. The methodology makes use of data sourced by THE, Benesse Corporation and Elsevier as described in section 1. Core university data submitted by the universities is validated by THE. Each of the metrics are normalised and weighted according to our methodology to generate the final Rankings as set out in the calculation of metrics section.
The 16 performance metrics are grouped into the four pillars:
Resources o Finance per student o Faculty per student o Mock university entrance exam score o Research output per member of staff o Research grants per member of staff Engagement o Student engagement o Student recommendation o Interaction with teachers and faculty o Global talent development o Student ability development Outcomes o Employer reputation o Academic reputation Environment o Proportion of international students o Proportion of international staff o International exchange programmes o Courses in a foreign language
4
THE Japan University Rankings 2019 methodology | Times Higher Education (THE)
1) Data collection and sources
Benesse Corporation
Mock exam results
Benesse Corporation has carried out mock exams and collected test scores for over 5.5 million high school students and approximately 90% of high schools in Japan for the last 60 years.
The score used in the Rankings is the mock exam result at which 60% of the applicants with that particular score would be offered admission to the university. The score was created to help high school students to determine which universities would be likely to admit them based on their mock exam results. The overall score for a university is based on scores created at the subject/major level. The mock exams cover all academic subjects in high school.
The mock exam data used for the 2019 Japan University Rankings is from 2018.
High school survey
Benesse Corporation prepared a survey that targeted high school career advisors in Japan over two years between October and December 2017 (for year 1) and September and October 2018 (for year 2). 5,287 high schools were surveyed with a response rate of 39%. There are two questions in the survey for the career advisors from each high school to nominate up to 15 universities for their capability to i) develop students’ abilities and ii) focus on global talent development.
Employer reputation survey
The employer reputation survey was prepared by Nikkei Human Resources and ran from 19 February to 23 March 2018. All 4,750 public companies (including those listed in JASDAC and other exchanges) were targeted with a response rate of 20%. Companies were asked to share the top ten universities they have hired people from, and what they thought of students coming from each university, based on 12 questions across four areas.
Accreditation data
University accreditation comes from public lists developed by the Japan Institution for Higher Education Evaluation, the National Institution for Academic Degrees and Quality Enhancement of Higher Education, and the Japan University Accreditation Association (all three certified by the Japanese ministry of education). All universities must receive accreditation from one of three associations every seven years.
Student perceptions survey
THE worked with Benesse Corporation to gain insight into the perceptions of currently enrolled students about their university, across any subject and level of study. The survey ran between July and October 2018. The survey is distributed to universities by Benesse or self-administered by individual institutions under THE guidelines.
Following analysis of the samples we decided to rebalance the student perceptions survey responses according to the gender split of each institution’s student population.
To perform this we have reweighted the average scores for each university according to the average score by gender and the actual gender balance. In doing this we have not included in the rebalancing any responses that have no identified gender or where gender was stated as other – these are reincorporated without weighting.
All student survey responses were gender-weighted according to the gender split of each institution’s student population. Ωi
5
THE Japan University Rankings 2019 methodology | Times Higher Education (THE)
Elsevier
Bibliometric data
The bibliometric indicator used for the Rankings is calculated as the total scholarly output (articles, reviews, conference papers, books and book chapters) between 2013 and 2017, divided by number of staff. Total scholarly output is calculated by Elsevier.
Academic reputation survey
An annual survey was sent to a sample of academics randomly selected by Elsevier asking them to nominate the top 15 universities for teaching and research globally. They were asked to nominate 6 additional universities in their country (for teaching and research) who they had not previously nominated in the global section. Only the teaching votes coming from academics who are affiliated with Japanese universities were considered for the metric calculation in these Rankings.
The academic reputation score for a university were the number of mentions they received for the 2017 and 2018 surveys in the world and country teaching sections from respondents associated with a Japanese institution. Where a university received no votes, they were allocated a zero score.
All vote totals were aggregated from the individual respondent level data provided by Elsevier to a total per university. Ωii
Institutional data – self-submitted on the THE Portal
A named representative from each institution submits and authorises their institutional data for use in the Rankings Ωiii, via THE’s designated online portal, with confirmations that they have:
Provided true and accurate information for their institution for 2017; and Understood and complied with the THE terms and conditions https://www.timeshighereducation.com/terms-and-conditions;
In global terms, the most complete data available for all institutions has been found to be from two years ago, therefore all institutions report 2017 data (defined as the appropriate annual cycle for the client that ends within the calendar year 2017).
Prior to submission of data within the portal, the draft data undergoes certain automatic validation checks to ensure that data is complete and accurate, which is reviewed by the named representative. Ωiv
Times Higher Education will not self-submit data for an institution without positive confirmation from the named representative of the institution. Ωv
Japan Statistical Yearbook 2019
Regional price parities 2017 data (for prefectures) in Japan was provided from Table 20.7, all items less imputed rent, at link http://www.stat.go.jp/english/data/nenkan/68nenkan/1431-20.htm
6
THE Japan University Rankings 2019 methodology | Times Higher Education (THE)
2) Criteria for exclusion, inclusion, and data processing
Exclusion and inclusion criteria
There are five key criteria for universities to be includes in the Rankings:
1. Have their data submitted by a university representative through THE Portal.
AND
2. Have non-zero / non-null values for the following data points: a. Institutional income b. Number of students c. Number of academic staff d. Number of courses e. At least one exchange field (may have a zero value, not null)
AND
3. Be accredited (see data sources section for the definition of accreditation).
AND
4. Not have any outstanding issues resulting from THE’s manual validation on submitted Portal data.
Management reviews and approves all institution submissions data for appropriateness and accuracy, based on prior year values and gaps within datasets. Ωvi
AND
5. Have at least 50 eligible and valid survey responses.
Universities meeting the five key inclusion criteria are included in the rankings. Ωvii
Data collected through either stream of the student perceptions survey (Benesse distributed or self-distributed by institutions) individually or the sum thereof must reach 50 eligible and valid responses for the university to be included in the Ranking.
Universities with fewer than 50 respondents to the survey were excluded from the Rankings Ωviii
A total of 226 met the five criteria defined above i.e. had sufficient validated data to be included in the Rankings.
Data processing and mapping
Regional price parities data in Japan obtained from the Japan Statistical Yearbook 2019 is used to convert university finance data by the Japanese prefecture that each main campus of the university resides in. The university address for prefecture mapping is obtained from data provided and validated by Benesse Corporation.
Finance data has been converted correctly using regional price parity. Ωix
7
THE Japan University Rankings 2019 methodology | Times Higher Education (THE)
All datasets provided by Benesse Corporation and Elsevier, including the employer survey, high school survey, mock exam results, academic reputation survey, and bibliometric data are accurately mapped using the unique five-digit university identifier.
The datasets used in the rankings have been accurately mapped by university name and ID. Institution-level bibliometric (Scopus and/or SciVal) and reputation survey data obtained from Elsevier is mapped to THE institution data via THE’s institution ID. Ωx
8
THE Japan University Rankings 2019 methodology | Times Higher Education (THE)
3) Calculation, scoring and ranking
Calculation of metrics
The pre-weighted indicators are calculated for each university Ωxi based on the definitions below:
1. Resources
Finance per student: o This metric is calculated as institution income (include all forms of income, such as general university funds, grant income, research income, contract income, teaching income, donations, investments and commercialization) divided by full-time equivalent (FTE) number of students in all years and of all programmes that lead to a degree, certificate, university credit or other qualification, as adjusted for regional price parity at the prefecture level. This metric uses a logarithmic scale to incorporate outliers prior to normalisation.
Faculty per student: o The student-to-faculty ratio is defined as total FTE number of staff employed in an academic post divided by FTE number of students in all years and of all programmes that lead to a degree, certificate, university’s credit or other qualification.
Mock exam entrance level score: o The mock exam metric requires no calculation before normalisation as it is extracted directly from Benesse Corporation mock exam results data.
Research output per member of staff: o This metric captures the number of papers per member of staff and is a measure of research presence. It is calculated as the total scholarly output between 2013 and 2017 (from Elsevier) divided by number of academic staff. This metric uses a logarithmic scale to incorporate outliers prior to scaling.
Research grants per member of staff: o This metric captures the research presence which is calculated as the number of research grants divided by number of academic staff. This metric uses a logarithmic scale to incorporate outliers prior to scaling.
9
THE Japan University Rankings 2019 methodology | Times Higher Education (THE)
2. Engagement
Student engagement: o This metric is generated from the average scores per College (gender-weighted) from four questions on the student survey: o To what extent does the teaching at your university or college support CRITICAL THINKING? o To what extent did the classes you took in your college or university so far CHALLENGE YOU? o To what extent does the teaching at your university or college support REFLECTION UPON, OR MAKING CONNECTIONS AMONG, things you have learned? o To what extent does the teaching at your university or college support APPLYING YOUR LEARNING to the real world?
Student recommendations: o This metric is generated from the average score per College (gender-weighted) from the following question on the student survey: o If a friend or family member were considering going to university, based on your experience, how likely or unlikely are you to RECOMMEND your college or university to them?
Interaction with teachers and faculty: This metric is generated from the average scores per College (gender-weighted) from two questions on the student survey: o To what extent do you have the opportunity to INTERACT WITH THE FACULTY and teachers at your college or university as part of your learning experience? o To what extent does your college or university provide opportunities for COLLABORATIVE LEARNING?
Global talent development: o This metric is generated from the raw scores per university from the following question on the high school survey: o “Please choose universities in Japan that focuses on global talent development and fill in the university codes up to 15 (in any order).” o This metric is the mean score across two years and uses a logarithmic scale to incorporate outliers prior to normalisation. Only non-zero values will be standardised. Universities that received no votes will score zero.
Student ability development: o This metric is generated from the raw scores per university from the following question on the high school survey: o “Please choose universities in Japan that develop students’ abilities and fill in the university codes up to 15 (in any order)” o This metric is the mean score across two years uses a logarithmic scale to incorporate outliers prior to normalisation. Only non-zero values will be standardised. Universities that received no votes are scored zero.
10
THE Japan University Rankings 2019 methodology | Times Higher Education (THE)
3. Outcomes
Employer reputation:
o This metric is accounting for the average score (across a series of questions regarding the quality
of the graduates a company has hired from them) given to a university by employers, and the
amount of votes received by a university. This metric is the mean score across the last two years.
Academic reputation: o This metric is the number of votes obtained from the Elsevier reputation survey from the last two years, and is calculated as the number of global teaching votes from Japanese respondents of the reputation survey and number of country teaching votes from Japanese respondents to the reputation survey. Only non-0 values will be standardised, and universities that received no votes are scored a zero for this metric.
4. Environment
Proportion of international students: o This metric is the percentage of students of international or overseas origin divided by the number of students. Only non-zero values will be standardised, and universities with no international students are scored a zero for this metric.
Proportion of international staff: o This metric is calculated as the number of FTE academic staff of international or overseas origin divided by the FTE number of staff employed in an academic post. Only non-zero values will be standardised, and universities with no international staff are scored a zero for this metric.
International exchange programmes o This metric is calculated as a weighted score of the number of students on an international exchange programme with exchanges over 1 month being weighted higher. Only non-zero values will be standardised, and universities whose students did not go on any exchange programme scored a zero for this metric.
Courses in a foreign language o This metric is calculated as the number of foreign courses taught by an institution divided by the total number of courses taught at a university, with a foreign course being defined as not being taught in Japanese. Only non-zero values will be standardised, and universities with no foreign- language courses are scored a zero for this metric.
Moving from a series of specific data points to metrics, and finally to a total score for a university, requires us to match values that represent fundamentally different data. To do this we use a standardisation approach for each indicator, and then combine the indicators in the proportions indicated below.
The standardisation approach we use is based on the distribution of data within a particular indicator, where we calculate a cumulative probability function, and evaluate where a particular university’s indicator sits within that function. A cumulative probability score of X in essence tells us that a university with random values for that indicator would fall below that score X per cent of the time.
For all indicators except research grants per member of staff and research outputs per member of staff, we calculate the cumulative probability function using a version of Z-scoring.
11
THE Japan University Rankings 2019 methodology | Times Higher Education (THE)
Weightings of metrics to final scores and rankings
The above 16 performance metrics representing the four pillars are weighted according to THE’s assessment of relative importance.
Once the final population of universities and indicators has been prepared, the scores for each university are generated by weighting the metrics and the Final Rankings are calculated Ωxii according to the following percentage breakdowns:
Pillar Metric % weighting
Finance per student 8%
Faculty per student 8%
1. Resources Mock exam entrance level score 6%
Research output per member of staff 7%
Research grants per member of staff 5%
Student engagement 6%
Student recommendations 6%
2.Engagement Interaction with teachers and faculty 6%
Global talent development 6%
Student ability development 6%
Employer reputation 8% 3. Outcomes Academic reputation 8%
Proportion of international students 5%
Proportion of international staff 5% 4. Environment International exchange programmes 5%
Courses in a foreign language 5%
12
THE Japan University Rankings 2019 methodology | Times Higher Education (THE)
4) Publication and reporting
Final rankings preparation
The universities ranked 1 – 150 overall are published in the final Rankings on the THE website. On the website only, the pillar scores for the universities ranked 1 – 150 for each individual pillar are also listed. Pillar scores are only displayed if they are scored in the top 150 for that pillar, and overall scores/ranks are only displayed if the university is in the top 150 of all universities overall.
For the universities ranked 1 – 100 overall, an individual rank position is listed. The next 50 universities are assigned to bands of 10 (e.g. 101 – 110) and universities in each band are sorted alphabetically. Subsequent to these, a banding of 151+ is assigned to all universities that are not in the top 150 overall. There are a total of 226 universities ranked.
Review and sign off
The Rankings are formally signed off by THE World Universities Insights Limited management prior to being published in print and online.
The Rankings results are reviewed and signed off by THE’s editorial team. Ωxiii
Reporting
The Rankings for the top 100 universities and banding allocation below top 100 are accurately reported on the THE website. Ωxiv
The Rankings are listed together with the Rankings methodology on the Times Higher Education website at:
https://www.timeshighereducation.com/world-university-rankings/japan-university-rankings-2019- methodology
13
THE Japan University Rankings 2019 methodology | Times Higher Education (THE)
Appendix 1. THE Japan University Rankings 2019 universities ranked 1 – 150
Below is the listing of universities ranked 1 – 150 in the Japan University Rankings that are subject to independent assurance by PwC LLP (exact rank for top 100 and banding allocation below top 100 – not overall scores)
Rank Name Overall score 1 Kyoto University 82 2 The University of Tokyo - UTokyo 81.9 3 Tohoku University 80.2 4 Kyushu University 79.5 5 Nagoya University 79.3 5 Hokkaido University 79.3 7 Tokyo Institute of Technology 79 8 Osaka University 77.9 9 University of Tsukuba 77.5 10 Akita International University 76.7 11 ICU International Christian University 72.7 12 Hiroshima University 71.8 13 Waseda University 71.5 14 Keio University 70.4 15 Hitotsubashi University 68.5 16 Kobe University 68 17 Sophia University 67.9 18 Kanazawa University 66.6 19 Chiba University 66.4 20 Tokyo University of Foreign Studies 65.9 21 Nagaoka University of Technology 65.6 22 Tokyo University of Agriculture and Technology 65.5 23 Tokyo Medical and Dental University (TMDU) 65.4 24 Okayama University 64.4 25 Ochanomizu University 64.2 26 The University of Aizu 64.1 27 Ritsumeikan Asia Pacific University (APU) 63.7 28 Yokohama National University 63.4 29 Toyohashi University of Technology (TUT) 63.1 30 Yokohama City University 62.9 31 Kyoto Institute of Technology 62.4 32 Tokyo University of Marine Science and Technology 62 33 Ritsumeikan University 61.7 34 Kanda University of International Studies (KUIS) 61.3 35 Doshisha University 60.4 36 Tokyo University of Science 60.3 37 Kwansei Gakuin University 60 38 Osaka City University 59.9 39 Tokyo Metropolitan University 59.7 40 Nagasaki University 59.6 41 Rikkyo University 59.4 41 Toyota Technological Institute 59.4 43 The University of Electro-Communications 58.7 44 Shibaura Institute of Technology, Tokyo 58.6 45 Niigata University 58.4 46 Shinshu University 58 46 Fukuoka Women's University 58 48 Gakushuin University 57.3 48 Osaka Prefecture University 57.3 48 Akita University 57.3 51 Kobe City University of Foreign Studies 57.2 52 Yamaguchi University 56.6 53 Chuo University 56.5 54 Meiji University 56.4 55 Gifu University 55.9 56 Hosei University 55.8 57 Tsuda University 55.6 57 University of Fukui 55.6 59 Yamagata University 55.5 60 Tottori University 55.4 60 Kansai University 55.4 62 Gunma University 55.1 63 Saga University 54.9 64 Ehime University 54.6 65 Kagoshima University 54.5 65 Kindai University 54.5 67 Nagoya City University 54.1 68 Shizuoka University 54 68 Future University Hakodate 54 70 Kansai Gaidai University 53.9 71 Aichi Prefectural University 53.5 71 University of Shizuoka 53.5 73 Toyo University 53.2 14
THE Japan University Rankings 2019 methodology | Times Higher Education (THE)
74 Nagoya University of Foreign Studies 53 75 Utsunomiya University 52.9 75 Meiji Gakuin University 52.9 75 Saitama University 52.9 78 Aoyama Gakuin University 52.8 79 Kyoto University of Foreign Studies 52.7 79 Akita Prefectural University 52.7 79 The University of Kitakyushu 52.7 79 Soka University 52.7 83 Kochi University of Technology 52.6 84 Tokushima University 52.4 85 Iwate University 52.1 86 Ibaraki University 51.8 87 Juntendo University 51.6 88 Kagawa University 51.2 89 University of the Ryukyus 51 89 University of Toyama 51 91 Kyoto Sangyo University 50.8 92 University of Miyazaki 50.7 92 Musashi University 50.7 92 St. Luke’s International University 50.7 95 Showa Women's University 50.2 96 Mie University 50.1 96 Hamamatsu University School of Medicine 50.1 98 Tokyo University of Agriculture 50 99 Dokkyo University 49.9 100 Shimane University 49.6 101-110 Kitasato University 48.4-49.5 101-110 Konan University 48.4-49.5 101-110 Nagoya University of Commerce & Business 48.4-49.5 101-110 Otaru University of Commerce 48.4-49.5 101-110 Sapporo Medical University 48.4-49.5 101-110 Seikei University 48.4-49.5 101-110 Seinan Gakuin University 48.4-49.5 101-110 Showa University 48.4-49.5 101-110 Tokyo Woman's Christian University 48.4-49.5 101-110 University Of Hyogo 48.4-49.5 111-120 J. F. Oberlin University 46.8-48.2 111-120 Kogakuin University of Technology and Engineering 46.8-48.2 111-120 Nara Women's University 46.8-48.2 111-120 Nihon University 46.8-48.2 111-120 Oita University 46.8-48.2 111-120 The Jikei University School of Medicine 46.8-48.2 111-120 Toho University 46.8-48.2 111-120 Tokai University 46.8-48.2 111-120 Tokyo International University 46.8-48.2 111-120 University of Yamanashi 46.8-48.2 121-130 Aichi Medical University 45.6-46.7 121-130 Fukuoka University 45.6-46.7 121-130 Japan Women's University 45.6-46.7 121-130 Meijo University 45.6-46.7 121-130 Muroran Institute of Technology 45.6-46.7 121-130 Nara Medical University 45.6-46.7 121-130 Reitaku University 45.6-46.7 121-130 Ryukoku University 45.6-46.7 121-130 Tamagawa University 45.6-46.7 121-130 Tokyo City University 45.6-46.7 121-130 University of Niigata Prefecture 45.6-46.7 131-140 Chiba Institute of Technology 44.2-45.5 131-140 Chukyo University 44.2-45.5 131-140 Doshisha Women's College of Liberal Arts 44.2-45.5 131-140 Fukuoka Institute of Technology 44.2-45.5 131-140 Fukushima University 44.2-45.5 131-140 Nippon Medical School 44.2-45.5 131-140 Shiga University 44.2-45.5 131-140 Teikyo University 44.2-45.5 131-140 Tokyo Denki University 44.2-45.5 141-150 Aichi University 42.0-44.1 141-150 Fujita Health University 42.0-44.1 141-150 Fukui Prefectural University 42.0-44.1 141-150 Kanagawa University 42.0-44.1 141-150 Kansai Medical University 42.0-44.1 141-150 Kyoai Gakuen University 42.0-44.1 141-150 Miyazaki International College 42.0-44.1 141-150 Osaka Jogakuin University 42.0-44.1 141-150 Prefectural University of Hiroshima 42.0-44.1 141-150 The University of Shiga Prefecture 42.0-44.1 141-150 Yamaguchi Prefectural University 42.0-44.1
15
THE Japan University Rankings 2019 methodology | Times Higher Education (THE)
Independent assurance report to the directors of THE World Universities Insights Limited for the Times Higher Education Japan University Rankings 2019
The directors of THE World Universities Insights Limited engaged us to perform an assurance engagement over the application of specific procedures (i) – (xiv) in production and reporting of the top 100 universities by rank, and banding allocation below the top 100 for the THE Japan University Rankings 2019 (the “Rankings”).
Our conclusion Based on the results of our work, nothing has come to our attention that causes us to believe that THE World Universities Insights Limited management has not applied, in all material respects, the specific procedures (i) – (xiv) outlined in their report.
This conclusion is to be read in the context of what is stated below.
Scope of our work We have performed a limited assurance engagement over the procedures (i) – (xiv) as marked with the symbol “Ω” set out in the report (‘THE’s procedures’) within the THE Japan University Rankings 2019 methodology (the ‘Methodology’), which outlines THE’s production and reporting of the Rankings.
Our work has been performed in accordance with the agreement between us dated 29 November 2018.
Professional standards applied and level of assurance We performed a limited assurance engagement over application of THE’s procedures in accordance with International Standard on Assurance Engagements 3000 (Revised) Assurance Engagements other than Audits and Reviews of Historical Financial Information issued by the International Auditing and Assurance Standards Board.
Our independence and quality control We complied with the Institute of Chartered Accountants in England and Wales (ICAEW) Code of Ethics, which includes independence and other requirements founded on fundamental principles of integrity, objectivity, professional competence and due care, confidentiality and professional behaviour.
We also apply International Standard on Quality Control (UK) 1 and accordingly maintain a comprehensive system of quality control including documented policies and standards regarding compliance with ethical requirements, professional standards and applicable legal and regulatory requirements.
Inherent limitations Our assurance procedures are limited to assessing the application of THE’s procedures and are subject to the following inherent limitations:
Reliance has been placed on data obtained from third parties. These data sets include: o Mock Exam result data provided by Benesse Corporation; o Accreditation data provided by Benesse Corporation; o High school survey data provided by Benesse Corporation; o Employer reputation survey provided by Nikkei Human Resources (part of Nikkei Inc.); o Regional price parity data obtained from Japan Statistics Bureau; o Bibliometric data for universities provided by Elsevier (part of RELX Group); and o Academic reputational survey response data provided by Elsevier (part of RELX Group). Our responsibility is to assess whether management has applied specific procedures (i) – (xiv) of the Methodology, not to express any view on the resulting rankings.
16
THE Japan University Rankings 2019 methodology | Times Higher Education (THE)
Work performed Our limited assurance procedures primarily comprised: Examining the Methodology and THE’s procedures in order to obtain an understanding and assessing any key assumptions and limitations. Obtaining an understanding of the third party surveys and data. Assessment of management’s application of THE’s procedures (i) – (xiv) as marked with the symbol “Ω” in the sections: o Data collection and sources; o Criteria for exclusion, inclusion, and data processing; o Calculation, scoring and ranking; and o Publication and reporting. Testing accurate reporting of institutions in the THE Japan University Rankings 2019 report and on the Times Higher Education website. (The top 150 Universities have been reproduced in Appendix 1). Enquiries of relevant management.
The procedures performed in a limited assurance engagement vary in nature and timing from, and are less in extent than for, a reasonable assurance engagement. Consequently, the level of assurance obtained in a limited assurance engagement is substantially lower than the assurance that would have been obtained had a reasonable assurance engagement been performed.
Directors’ responsibilities The directors of THE World Universities Insights Limited are responsible for: establishing an appropriate Methodology and specific procedures for producing the Rankings and reporting the results on THE’s website; designing, implementing and monitoring policies, activities, processes and controls to comply with the procedures; their Methodology, including the application of the procedures set out in this Methodology; supporting the Directors’ Statement with sufficient evidence, including documentation; and the maintenance and integrity of THE’s website.
Our responsibilities We are responsible for: planning and performing the engagement to obtain evidence to support our assurance conclusion; forming an independent conclusion, based on the procedures we have performed and the evidence we have obtained, on management’s application of THE’s procedures as described in the report; and reporting our conclusion to the directors of THE World Universities Insights Limited.
We believe that the evidence we have obtained is sufficient and appropriate to provide a basis for our conclusion.
Intended users and purpose This report is produced in accordance with the terms of our agreement dated 29 November 2018 and is intended solely for the use and benefit of the Board of Directors of THE World Universities Insights Limited, and solely for the purpose of reporting to them on the application of THE’s procedures within the Methodology in preparation and publication of the Rankings and no other purpose. We do not, in giving our conclusion, accept or assume responsibility (legal or otherwise) or accept liability for, or in connection with, any other purpose for which our report including the conclusion may be used, or to any other person to whom our report is shown or into whose hands it may come, and no other persons shall be entitled to rely on our conclusion.
We permit the disclosure of our report, in full only and in the company of the Methodology, to enable the directors to demonstrate that they have discharged their governance responsibilities by commissioning an independent assurance report over procedures (i) – (xiv) of the Methodology, without assuming or accepting any responsibility or liability to any third parties on our part. To the fullest extent permitted by law, we do not accept or assume responsibility to anyone other than the directors of THE World Universities Insights Limited for our work or this report save where terms are expressly agreed and with our prior consent in writing.
PricewaterhouseCoopers LLP Chartered Accountants London, UK 13 March 2019
17