Guidelines for Tenure and Promotion Decisions 8 June 2020

Home , Journal of Econometrics

Department of Econometrics and Operations Research, Tilburg University Guidelines for Tenure and Promotion Decisions 8 June 2020

Tenure and promotion policies at the Department of Economics 1 • Introduction

I. Summary applying for (all tenured associate and full professors for a tenure and promotion to associate professor decision, This document establishes guidelines for tenure and promo- etc.) are invited to participate. tions in the Department of Econometrics and Operations c. The criteria should be consistent with TiSEM’s strategy Research (EOR), which consist of for 2018–2021, which specifies the ambition to “remain in the European top three in all fields [of business and a. departmental criteria for tenure and promotions and economics] (as measured by rankings).” b. a departmental procedure for tenure and promotions. d. The criteria should be consistent with TiU’s, TiSEM’s, and in particular the department’s vision to stimulate II. Background Data Science related research. e. Point of departure is that research and teaching are The guidelines are informed by the following background treated symmetrically. This is apparent from section VI documents: below, where “very good” and “excellent” qualifications are mapped into requirements for tenure and promo- a. “TiSEM Academic Careers: Guiding principles & proce- tion. Needless to say, the elaboration of what is “very dures for tenure and promotions,” version 2.3, revised good” and what is “excellent” is very different for re- December 2018. search and teaching (and other activities). b. “TiSEM Strategy 2018–2021: Excel – Innovate – Connect,” f. Important roles are assigned to ad hoc review commit- 12 October 2017. tees for research (RRC) and teaching (TRC). g. Bibliometric analysis and student evaluations may The TiSEM guidelines (a; from here on: TAC) provide the provide an objective basis for assessing a candidate’s context for and binding restrictions on the departmental cri- research and teaching, but peer review of the candidate’s teria and procedure. TiSEM’s strategy document (b) sets the actual research and teaching is leading in their assess- ambition and goals that the tenure and promotion guidelines ment. should serve. h. The document does not describe the transition rules for current tenure track assistant professors or current col- III. Scope leagues who are already working towards a promotion. It is obvious that a smooth transition will have to take The guidelines concern the departmental input into decisions place, but what this exactly entails will have to be decid- taken by the School’s Assessment Committee (AC). Accord- ed (by the head of department) on a case by case basis. ing to TAC, Section 5, these include V. Procedure • tenure and promotions to associate professor, professor 2, and professor 1; “Steps” refer to the six steps of TiSEM’s assessment proce- and exclude (among other things) all other promotions and dure (TAC, Section 4). the appointment of lecturers.

Throughout, we assume that all candidates for tenure and promotion are affiliated with either Econometrics or OR (we use capitalized “Econometrics” and the acronym “OR” to refer to the department’s two research groups, not the eponymous fields of research).

IV. General principles a. The departmental procedure and criteria should meet- TiSEM’s requirements (TAC), in order to minimize going back and forth with the AC. b. The procedure should be transparent and consistent. Therefore, we specify an explicit and inclusive procedure with clear criteria. Specifically, we use the same proce- dures for Econometrics and OR, in which all department members who have at least the position the candidate is

Tenure and promotion policies at the Department of Economics 2 • Procedure

Step 1: Preliminary request i. Initiate tenure and/or promotion procedure (a) Candidate and HoD ii. Submit preliminary request to dean HoD Decide on financial and legal feasibility Dean iii. If ok, continue with iv; if not, inform candidate HoD

Step 2: Departmental review iv. Prepare preliminary portfolio for departmental review (TAC, Appendix A, documents 2,4,6, 7; and all Candidate relevant research papers, preferably as hyperlinks in documents 2 and 7) v. Appoint ad hoc departmental teaching review committee (TRC; b) HoD vi. Appoint ad hoc departmental research review committee (RRC; c) HoD vii. Provide HoD with written review of teaching, including Statement of Academic Director (TAC, Appen- TRC dix A, document 3) viii. Provide HoD with written review of research program and papers, including nomination of external RRC reviewers, and outreach activities (if applicable) ix. HoD writes an evaluation of the candidate’s citizenship HoD x. Organize a departmental meeting with department members to decide whether to continue and (if HoD, department members so) which external reviewers to ask for letters, based on the preliminary portfolio and the teaching, EOR research, outreach (if applicable), and citizenship reviews, and the criteria in Section VI.

The department members invited to the meeting are those who have at least the position the candidate is applying for (all tenured associate and full professors for a tenure and promotion to associate professor decision, etc.) xi. If ok, continue with xii; if not, inform candidate HoD

xii. Obtain letters from at least three external reviewers (TAC, Appendix A, document 5); see (d) HoD xiii. Consult RRC on letters and decide on bringing case to AC HoD xiv. If ok, continue with xv; if not, inform candidate HoD Steps 3-6: Submission request promotion/award of tenure to AC; final decision xiv. Write recommendation to AC (TAC, Appendix A, document 1), compile final portfolio, and submit HoD portfolio to dean Make final decision on request following TiSEM procedure AC, Dean, Exec. Board (promotion to full prof. only) xv. Inform candidate and department HoD

Notes: a. Following TAC, Section 4, a candidate can ask the HoD of three or more (if many papers need review) tenured to start the tenure and promotion procedure at any TiSEM faculty members who have not been the candi- stage in their career. The HoD should proactively ap- date’s PhD advisor or coauthor. It is chaired by a member proach candidates if they do not timely ask for tenure or of the EOR department and at least half of the other com- promotion. In particular, the tenure process should start mittee members are from the EOR department as well. no later than the fifth year of the tenure track. d. TRC and RRC are separate committees, but may have b. The ad hoc teaching review committee (TRC) consists of members in common. three Academic Directors and/or tenured TiSEM faculty e. Following TAC, Section 4, the external reviewers “should members, including at least one Academic Director of a be scholars from top universities who are leaders in their program in which the candidate has taught and at most field, but preferably not supervisors or co-authors.” The one non-EOR member. It is chaired by a member of the HoD will make sure to obtain letters from at least three EOR department. reviewers who are not supervisors or coauthors. c. The ad hoc research review committee (RRC) consists

Tenure and promotion policies at the Department of Economics 3 • Criteria: Research

VI. Criteria criteria for “very good” and “excellent” research, teaching, and outreach in Sections VI.1 – VI.3 and describe a “good” The department evaluates the portfolio in four dimensions: citizen in Section VI.4. The final decision combines the four research, teaching, outreach, and citizenship. We discuss the dimensions as follows.

For award of tenure and promotion to associate professor, candidates must be • very good in both research and teaching, • excellent in at least one of research or teaching, and • good citizens. For promotion to professor 2 or professor 1, candidates must be • very good in both research and teaching; • excellent in at least one of research, teaching, and outreach; and • good citizens.

VI.1. Research finance, and the School’s other fields. Group members may specialize in one of these areas, but often find value Following TiSEM’s strategy, quality trumps quantity of re- in combining two or more of them and always keep an search. This requires a substantive assessment of the candi- eye on applied relevance to economics and business. date’s actual research output based on careful reading (by the Therefore, the journal list contains the highest ranked RRC) of all research papers in the candidate’s portfolio. To (see 2) journals in the Web of Science (WoS) category offer objective guidance while leaving room for such a sub- Economics and selected high ranked journals in the WoS stantive assessment, we specify category Statistics and Probability. The OR group has the eponymous focus. Its journal list contains the highest 1. necessary bibliometric conditions for tenure and promo- ranked journals in the WoS category Operations Re- tion and search and Management Science and selected journals 2. examples of research profiles of (un)successful tenure in the WoS categories Mathematics and Mathematics, and promotion cases, Applied. but no sufficient conditions. For simplicity, the necessary 2. Journals are ranked by Article Influence Score AIS;( conditions are stated in terms of “first tier” and “second http://www.eigenfactor.org). tier” journals, which are selected, at five-year intervals and 3. The AIS is an easily available and objective measure of separately for Econometrics and OR, based on the following impact, which well reflects the perceived journal quality principles. in the profession (Abbring et al., 2014). 4. First tier journals are the best relevant journals. They in- 1. We include only, and enough, journals relevant to each clude those journals to which we would typically expect research group. The Department’s research focuses on researchers in each group to submit first. quantitative methods in economics and business. The 5. The publication process is noisy and good and import- Econometrics group covers the quantitative-empirical ant work may be rejected by a first tier journal. Second and quantitative-theoretical approaches to (primari- tier journals are the best outlets for such work. ly) economics and finance. To this end, its members Based on the 2018 AIS (see Appendix A), this gives the fol- study econometric and statistical theory and methods, lowing journal lists. economic and finance theory, computational economics and finance, and their applications in economics, VI.1.1. Journal list Econometrics (see Appendix A.1 for journal numbers and AIS)

Tier Web of Science category Journals AIS range 1st Economics 1–41 20.925–2.506

Statistics and Probability 1–4 12.053–2.533 2nd Economics 42–70 2.441–1.523

Statistics and Probability 5-8 1.844-1.533

Tenure and promotion policies at the Department of Economics 4 • Criteria: Research

Economics has a widely accepted “top five” of general inter- The best journals in statistics regularly publish papers on or est journals: QJE, Ecma, JPE, RES, and AER. directly related to current research in econometrics. There- fore, we complement the first tier field journals in economet- Over time, these journals have become so selective that rics with the relevant statistics journals in WoS Statistics and other (very good) general interest journals (notably REStat, Probability with AIS in the same range: JRSSB (AIS=5.671), JEEA, EJ) and top field journals now also play a key role in AOS (4.807), JASA (3.369), and Biometrika (2.533). publishing important and impactful research (Heckman and Moktan, 2018). In the first tier, we include all these (and The second tier covers the 29 journals in WoS Economics with some more) by selecting all 41 journals in WoS Economics 1.500 £ AIS < 2.500. It includes very good field journals in with AIS ³ 2.500. Among these are top field journals in econo- econometrics (JAE, ET, EctJ, JFEctr), theory (JET), and other metrics (QE, JBES, JOE), theory (TE, AEJ-Micro), finance (JF, fields (JRU, JMCB, etc.). It also includes 4 WoS Statistics and RFS, JFE), applied micro (AEJ-Applied, JHR, JOLE, JHE), IO Probability journals in this AIS range that are relevant to eco- (Rand), etcetera. nomics and business.

VI.1.2. Journal list OR (see Appendix A.2 for journal numbers and AIS)

Tier Web of Science category Journals AIS range 1st Operations Research & MS 1–12 2.718–1.223

Other (SIOPT, JCTB, JCTA) 1–3 3.774–1.155

2nd Operations Research & MS 13–43 1.080–0.607

OR has shorter lists than Econometrics because it has fewer the Data Science/Machine Learning field is developing rapid- journals (e.g. in 2018, there were 363 journals in WoS Eco- ly and we have to build up expertise in this field, we will grad- nomics, but only 84 journals in WoS Operations Research & ually build up these first and second tier journal lists. In the MS). This is also evident from the fact that the marginal AIS meantime, until this list has been completed, each member in OR are smaller than those for Econometrics, even though of the department can submit a proposal to add a specific fewer journals are included. journal to the lists. The head of the department together with the Research Coordinator of the OR group, in consultation Since the OR group wants to stimulate research on OR and with all department members, decide on this list of first and Data Science, we also intend to add a list of relevant first and second tier Data Science journals. second tier journals in the Data Science area. However, since

VI.1.3. Necessary conditions for “very good” and “excellent” research profiles

Tenure/associate professor Full professor 2 Full professor 1 Very good 1 first tier & 2 first tier publ. 4 first tier publ.

1 first or second tier publ. (of which at least 1 in the last 5 years) (of which at least 1 in the last 5 years) Excellent 2 first tier publ. 4 first tier publ. 6 first tier publ.

1 first or second tier publ. (of which at least 1 in the last 5 years) (of which at least 1 in the last 5 years)

necessary conditions. Review articles, book reviews, Notes: comments, letters to the editor, and non-refereed 1. For all candidates, the necessary conditions are checked articles in special conference proceedings (e.g. AER using the combined Econometrics and OR journal lists Papers & Proceedings) do not count. In ambiguous (the typical profiles in VI.1.4 and VI.1.5, however, are cases, the RRC decides. group specific). • The RRC may, exceptionally and with motivation, also 2. There are two types of exceptions: count publications in outlets not listed as first or sec- • Only original research articles count towards the ond tier journals. Examples include articles in new

Tenure and promotion policies at the Department of Economics 5 • Criteria: Research

journals with rising reputation but relatively low AIS, as a guide and judge a candidate’s publication record on its articles in other Statistics and Probability journals substance and actual impact. In doing so, the RRC can take or in journals in other WoS categories, and refereed relevant publications in other outlets, e.g. in related fields like publications in renowned conference proceedings, mathematics and statistics, into account. provided they are relevant to TiSEM’s research areas. VI.1.4.1. Econometrics VI.1.4. Examples of “very good”, “excellent”, and neither “excellent” nor “very good” research profiles We would typically expect Econometrics candidates to mainly publish in journals on the Econometrics list in Appendix A.1 A typical excellent research profile should be consistent with (with obvious exceptions, e.g. for candidates who are under- the stated ambition to be a top three school in Europe in all stood to work on the boundary with OR). The Econometrics fields of economics and business. This requires the RRC’s lists of first tier and, to a lesser extent, second tier journals judgment beyond simple bibliometric assessment. For exam- are long and of heterogeneous quality. Therefore, we do not ple, excellence at the top European level may be evidenced specify examples of research profiles in terms of first and sec- with a relatively small number of very novel, important, and ond tier journals, but more specifically in terms of (i) top-five impactful publications, resulting from a candidate’s own journals, (ii) top field journals and very good (outside top- research program. Thus, even though we specify the typical five) general interest journals, and (iii) very good (outside the profiles partly in terms of numbers of publications in vari- top) field journals and (other) good general interest generals. ous types of journals, the RRC is expected to use this only See VI.1.1 for details and examples of these categories.

Econometrics Tenure/associate professor Full professor 2 Full professor 1 Very good (but 2 top field/very good general interest & 6 top field/very good general interest 10 top field/very good general interest not excellent) 1 very good field/good general interest publ. publ. publ.

or or or

1 top field/very good general interest & 4 top field/very good general interest & 6 top field/very good general interest &

3 very good field/good general interest 4 very good field/good general interest 8 very good field/good general interest publ. publ. publ. Key author of some pubs; equal author Key role in internationally recognized Key roles in multiple internationally on the other pubs (or more pubs) research program recognized research programs

100 GS cites 500 GS cites 1000 GS cites

Regular presentations at main confer- Regular invitations to main confer- Regular invitations to main conferences ences and seminars ences and seminars

Invited seminars at good schools Advised 1 PhD grads; is advising 2 Advised 2 PhD grads; is advising 3 more more Advising 1 PhD student Good grades on Vidi (or equivalent) Good grades on Vici (or equivalent) Good grades on Veni (or equiv.) appli- application application cation Some other evidence of quality (e.g. Some other evidence of quality (e.g. Some other evidence of quality (e.g. prizes, membership of 2nd tier editori- prizes, handling editor 2nd tier j., prizes, invited research visits, invited al board, invited longer research visits) visiting professorships) editorial activity) Ongoing research activity: A pipeline of several papers with the potential to be published in top field or very good general interest journals

Reference letters from renowned researchers in the field that allow the conclusion that the candidate’s research profile is on par with those tenured and/or promoted to the same level at the best European institutions.

Tenure and promotion policies at the Department of Economics 6 • Criteria: Research

Econometrics Tenure/associate professor Full professor 2 Full professor 1 Excellent 1 top-five, Publications: Publications:

1 top field/very good general interest, 1 top-five & 2 top-five & & 6 top field/very good general interest 8 top field/very good general interest 1 very good field/good general interest publ. publ. publ. or or or 8 top field/very good general interest & 1 top-5, 3 top field/very good general interest & 2 very good field/good general interest 10 top field/very good general interest, 1 very good field/good general interest publ. & publ. 4 very good field/good general interest publ. Key author of some impactful pubs; Pursuing impactful own research Has been and is pursuing multiple equal author on the other pubs (or program impactful own research programs more pubs) 1000 GS cites 2000 GS cites 200 GS cites Regular invitations to main confer- Regular invitations to main confer- Regular presentations at main conferences and seminars ences and seminars; including (semi-) ences plenary talks Advised 2 PhD grads; is advising 3 Invited seminars at good schools more Advised 4 PhD grads; is advising 3 more Advising 2 PhD students Top grades on Vidi (or equivalent) application Top grades on Vici (or equivalent) Top grades on Veni (or equivalent) application application Acquired funding for 1 PhD/postdoc position Acquired funding for 3 PhD/postdoc Some other evidence of excellence (e.g. positions prizes, invited research visits, invited Some other evidence of excellence (e.g. editorial activity) prizes, membership of 1st tier editorial Some other evidence of excellence board, invited longer research visits) (e.g. prizes, handling editor 1st tier j., visiting professorships) Ongoing research activity: A pipeline of several papers with the potential to be published in first tier journals, with a shot at the very best

Reference letters from renowned researchers in the field that clearly confirm that the candidate’s research profile is at least on par with those tenured and/or promoted to the same level at the very best European institutions

Tenure and promotion policies at the Department of Economics 7 • Criteria: Research

Econometrics Tenure/associate professor Full professor 2 Full professor 1 Neither 1 top field/very good general interest & 4 top field/very good general interest & 6 top field/very good general interest & excellent nor very good 2 very good field/good general interest 4 very good field/good general interest 8 very good field/good general interest publ. publ. publ. All publications joint with senior co- No own research program Active (but not leading) in multiple author internationally recognized research 250 GS cites programs 50 GS cites Regular presentations at conferences 500 GS cites Some presentations at conferences and in seminars and in seminars Regular presentations at conferences Advised 1 PhD grads; is advising 2 and in seminars Advising 1 PhD student more Advised 2 PhD grads; is advising 3 more Only limited other evidence of quality Only limited other evidence of quality (e.g. funding, prizes, invited research (e.g. funding, prizes, invited research Only limited other evidence of quality visits, invited editorial activity) visits, invited editorial activity) (e.g. funding, prizes, invited research visits, invited editorial activity) Ongoing research activity: A pipeline of several papers, but with checkered publication potential.

Reference letters from renowned researchers in the field suggest that those tenured and/or promoted to the same level at the best European institutions have better research profiles than the candidate.

VI.1.4.2. Operations Research in Appendix A.2 and the Data Science list that will be estab- lished in the future. Below we specify typical research profiles We would typically expect Operations Research candidates in terms of first and second tier journals. to mainly publish in journals on the Operations Research list

Econometrics Tenure/associate professor Full professor 2 Full professor 1 Very good (but 2 first tier & 4 first tier & 6 first tier & not excellent) 1 second tier publ. 3 second tier publ. (of which 2 in the 5 second tier publ. (of which 2 in the last 5 years) last 5 years) Leading author of some pubs; equal Leading role in internationally recog- Leading roles in multiple internation- author on the other pubs nized research program ally recognized research programs

100 GS cites in total 500 GS cites in total 1000 GS cites in total

Regular presentations at main confer- Last 3 years: 50 GS cites per year Last 3 years: 100 GS cites per year ences Regular invitations to main confer- Regular invitations to main confer- Invited seminars at good schools ences and seminars ences and seminars

Advising 1 PhD student Advised 1 PhD grads; is advising 2 Advised 2 PhD grads; is advising 3 more more Good grades on Veni (or equiv.) application Good grades on Vidi (or equivalent) Good grades on Vici (or equivalent) application application Some other evidence of quality (e.g. prizes, invited research visits) Some other evidence of quality (e.g. Some other evidence of quality (e.g. prizes, membership of 2nd tier editori- prizes, handling editor 2nd tier j., al board, invited longer research visits) visiting professorships) Ongoing research activity: A pipeline of several papers with the potential to be published in first tier or second tier journals

Tenure and promotion policies at the Department of Economics 8 • Criteria: Research

Econometrics Tenure/associate professor Full professor 2 Full professor 1 Excellent 3 first tier & 6 first tier & 9 first tier &

1 second tier publ. 2 second tier publ. (of which 2 in the 3 second tier publ. (of which 2 in the last 5 years) last 5 years) Leading author of some impactful Pursuing impactful own research Has been and is pursuing multiple pubs; equal author on the other pubs program impactful own research programs

200 GS cites in total 1000 GS cites in total 2000 GS cites in total

Last 3 years: 100 GS cites per year Last 3 years: 200 GS cites per year

Regular presentations at main confer- Regular invitations to main confer- Regular invitations to main conferences ences and seminars ences and seminars; including (semi-) plenary talks Invited seminars at good schools Advised 2 PhD grads; is advising 3 more Advised 4 PhD grads; is advising 3 Advising 2 PhD students more Top grades on Vidi (or equivalent) Top grades on Veni (or equivalent) application Top grades on Vici (or equivalent) application application Acquired funding for 1 PhD/postdoc Some other evidence of excellence (e.g. position Acquired funding for 3 PhD/postdoc prizes, invited research visits, invited positions editorial activity) Some other evidence of excellence (e.g. prizes, membership of 1st tier editorial Some other evidence of excellence board, invited longer research visits) (e.g. prizes, handling editor 1st tier j., visiting professorships) Ongoing research activity: A pipeline of several papers with the potential to be published in first tier journals.

Econometrics Tenure/associate professor Full professor 2 Full professor 1 Neither excel- 1 first tier & 3 first tier & 4 first tier & lent nor very good 2 second tier publ. 5 second tier publ. 9 second tier publ. All publications joint with senior co- No own research program Active (but not leading) in multiple author internationally recognized research 250 GS cites in total programs 50 GS cites in total Regular presentations at conferences 500 GS cites in total Some presentations at conferences and in seminars and in seminars Regular presentations at conferences Advised 1 PhD grads; is advising 2 and in seminars Advising 1 PhD student more Advised 2 PhD grads; is advising 3 Only limited other evidence of quality Only limited other evidence of quality more (e.g. funding, prizes, invited research (e.g. funding, prizes, invited research visits, invited editorial activity) visits, invited editorial activity) Only limited other evidence of quality (e.g. funding, prizes, invited research visits, invited editorial activity) Ongoing research activity: A pipeline of several papers, but with checkered publication potential.

Tenure and promotion policies at the Department of Economics 9 • Criteria: Teaching

• Supervision of masters and bachelor theses and/or oth- VI.2. Teaching er student projects; • Course administration: scheduling forms, exam forms, As for research, we propose a substantive assessment of the course descriptions, learning goals, exams, grading, candidate’s actual teaching performance based on careful canvas, and timeliness of these; evaluation (by the TRC) of the candidate’s portfolio and other • Communication and involvement with other lecturers relevant input. We stress that traditional signals such as good and program director: covering of course material, set- teaching evaluations and teaching awards are certainly not ting within program, prerequisites, coordination of ma- perfect indicators of good or excellent teaching, and form terial with other courses, planning of assignments, etc.; only a limited part of the evaluation by the TRC. The TRC will • Sharing of teaching experience with other staff; take into account a variety of teaching aspects (and not just • Mentoring of freshmen; lecturing), listed as follows (among others): • Coordination of courses that involve more than one program or several teachers or teaching assistants; • Lecturing: preparation, organization, interaction, use of • Teacher training: scouting of teaching assistants, su- media (e.g., blackboard, whiteboard, slides, phones); pervising teaching assistants, mentoring (junior) staff in addition to student evaluations and sounding board members; reports, TRC members may attend some lectures in • Contributions to publications on teaching and educa- person to form their opinion; tional reports; • Program contents: contributions to development of • Contributions to educational projects; courses and/or curricula within a program or school; • Holding the position of program director, chair of the • Organization of course and course materials: course exam committee, vice dean of education, or a similar syllabus, lecture notes, exercises, videos, internet links, position. example exams, timing of communication of these to students, exams, solutions, and grading instructions; • Communication with students outside the classroom We specify necessary conditions for and example profiles of (e.g., office hours, canvas, forum, email); very good and excellent teachers.

VI.2.1. Necessary conditions for “very good” and “excellent” teaching profiles

Tenure/associate professor Full professor 2 Full professor 1 Very good Proven expertise in teaching for small and large groups (60+) at bachelor and (research) master’s level. (but not excel- Proven ability to teach for different programs, for example Econometrics & OR and Business Administration lent) Demonstrated ability to coordinate a Demonstrated ability to coordinate a course and to design or substantially restruc- course or to design or substantially ture a course restructure a course Obtained BKO (basic teaching qualification) or equivalent Excellent Proven excellence in teaching for small and large groups (60+) at bachelor and (research) master’s level. Proven ability to teach for different programs, for example Econometrics & OR Proven ability to teach for a variety of and Business Administration different programs Demonstrated ability to coordinate a course and to successfully design or substantially restructure a course

Obtained SKO (senior teaching qualification) or equivalent

Development of educational programs or other activities important for the department

Substantial contribution to innovation in education and/or degree programs

Notes: to “compare individual faculty members to each other or to a departmental average,” but rather “as part of 1. ASA (2019) notes that “[a] large body of research has a holistic assessment … to document patterns in an demonstrated that student evaluations of teaching … instructor’s feedback over time.” It also recommends are weakly related to student learning and are often that quantitative scores are reviewed in an appropriate biased against women and people of color.” ASA interpretative context (sample sizes, response rates, …). recommends that teaching evaluations are not used The TRC should take this into account.

Tenure and promotion policies at the Department of Economics 10 • Criteria: Teaching

VI.2.2. Examples of “very good”, “excellent”, and neither “excellent” nor “very good” teaching profiles

Below we describe nine teaching profiles: three that are very good, three that are excellent, and three that are neither very good nor excellent. The last three profiles demonstrate, in particular, that marginally meeting the necessary conditions is not enough. The first six profiles illustrate how much more is expected.

Tenure/associate professor Full professor 2 Full professor 1 Very good Taught courses for small and large groups (60+) at bachelor and (research) master’s level in Econometrics & OR as well (but not ex- as Business Administration; cellent) Lecturing of the different courses taught is ‘very good’ or sometimes ‘excellent’.

Has successfully supervised BSc and MSc theses.

Has obtained BKO (basic teaching qualification)

Has completely redesigned one MSc Has substantially restructured one course. course. Has coordinated a large course in BSc Business Administration

Is a member of an educational com- Substantial contribution to innovation Has been an Academic Director for mittee in education and/or degree programs several years

Proven ability to attract outside funding for education related projects or acquisition of applied/executive teaching

Tenure/associate professor Full professor 2 Full professor 1 Excellent Taught courses for small and large groups (60+) at bachelor and (research) master’s level in Econometrics & OR as well as Business Administration;

Lecturing is ‘excellent’ without exception for all courses taught.

Has successfully supervised several BSc and MSc theses.

Has substantially innovated an existing BSc course and has developed a new MSc course.

Has obtained SKO (senior teaching qualification)

Has been a member of several education related committees Actively contributes to innovation of Noticeable contribution to the organization and development of degree pro- one program grams, like holding the post of Academic Director for several years

Proven ability to attract funding for education related projects or acquisition of applied/executive teaching

Has written a textbook for one or more courses. Has been very active in teacher training Has been very active in teacher train- and mentoring junior colleagues ing and mentoring junior colleagues

Chairs a national or international network on teaching innovation

Tenure and promotion policies at the Department of Economics 11 Criteria: Outreach & Citizenship

Tenure/associate professor Full professor 2 Full professor 1 Neither very Taught courses for small and large groups (60+) at bachelor and (research) master’s level in Econometrics & OR as well good, nor ex- as Business Administration; cellent Lecturing of courses taught qualifies as ‘very good’, on average.

Has substantially restructured one course.

Has obtained BKO Has avoided accepting other education Has coordinated one large course in BSc Business Administration related tasks Has been an Academic Director for several years

VI.3. Outreach Some comments that explain this in more detail:

Outreach will be assessed in line with the faculty guidelines • The societal impact generated should be based on sci- (TAC, Section 2.1): entific research. Clear proof of this would e.g. be that the researcher has published scientific papers on the topic “Outreach, which is incorporated into all our teaching in top applied journals. and research programs, implies that we expect faculty members to actively participate with government • Just being very visible in the media is not enough for partners and/or business partners, in activities where being considered as “excellent in outreach.” knowledge can be applied in practice (e.g. developments • The societal impact could be expressed, e.g. in money and improvements in the private sector, societal issues, or lives saved, more profit, or improvements in line with knowledge portals, and advisory boards). Faculty mem- the Sustainable Development Goals. The societal impact bers may also be expected to be actively involved in should be significant and provable. academic associations (e.g. editorships, (vice) president position, treasurer). Outreach activities may also include • Proofs of significant societal impact could be, e.g., a submitting grant proposals aimed at multidisciplinary letter by the beneficiary (CEO of the company, minster research, sharing contacts with organizations from the or other high official) that clearly states the contribution public and private sectors in order to enrich courses, and impact of the researcher. Moreover, several top and offering opportunities for internships and place- applied journals (as, e.g., Omega and INFORMS Journal ments. on Applied Analytics) have societal impact as a strict criterion for publication, and hence a paper published in Staff members may also be asked to actively share such journals is a clear signal of impact. Finally, winning academic findings with society through professional a major “practice or valorization award” could also be a publications or newspapers, by participating in public proof of significant societal impact. debates or other public appearances in the media or at conferences. They may also participate in joint activities with e.g. CentERdata and TIAS that focus on applied VI.4. Citizenship research and executive teaching.” Candidates are either “good” citizens or not. Good citizens These guidelines describe important aspects of good out- actively participate in academic life in the department and reach, but for “excellence in outreach” the societal impact of TiSEM, which includes (but is not limited to) participation the research is most important. A researcher is considered to in seminars and conferences, recruitment and placement be “excellent in outreach” if and only if her/his research has activities, information sessions for bachelor or master’s pro- been adopted by companies or government and has led to grams, etcetera. For more senior promotions, in addition, the significant improvements. department expects willingness and demonstrated ability to contribute to administration and management.

Tenure and promotion policies at the Department of Economics 12 • References

VII. References

Abbring, Jaap H., Bart J. Bronnenberg, Pieter A. Gautier, and Jan C. van Ours (2014): “Dutch Economists Top 40,” De Economist 162(2), 107-114. https://doi.org/10.1007/s10645-014-9227-7

ASA (2019): “Statement on Student Evaluations of Teaching,” online statement, American Sociological Association, https://bit. ly/2lWA3Gh.

Heckman, James J., and Sidharth Moktan (2018, revised March 2019): “Publishing and Promotion in Economics: The Tyranny of the Top Five,” NBER Working Paper 25093, National Bureau of Economic Research, Cambridge, MA. https://www.nber.org/papers/w25093.pdf

Tenure and promotion policies at the Department of Economics 13 • Appendices

Appendices: Journals by Web of Science (WoS) category and 2018 Article Influence Score (AIS)

Source

The 2018 Journal Citation Reports® (https://jcr.clarivate.com) provided all journal and citation data in this report.

Legend

Tier 1st first tier journals

2nd second tier journals

Field general interest Quarterly Journal of Economics

econometrics Journal of Econometrics

theory Theoretical Economics

finance Journal of Finance

Type review (special status) Statistical Science Current TiSEM journal catego- *** topscore ries ** top

(for comparison only) * very good

(***)/(**)/(*) as categorized by other group etc.

Tenure and promotion policies at the Department of Economics 14 • Appendices

A.1. Econometrics

WoS Economics AIS

1 QUARTERLY JOURNAL OF ECONOMICS *** 20.925

2 ECONOMETRICA *** 12.053

3 JOURNAL OF POLITICAL ECONOMY *** 11.826

4 JOURNAL OF FINANCE *** 10.619

5 REVIEW OF ECONOMIC STUDIES *** 10.293

6 American Economic Journal-Macroeconomics 8.351

7 AMERICAN ECONOMIC REVIEW *** 8.346

8 JOURNAL OF ECONOMIC LITERATURE (***) 7.869

9 American Economic Journal-Applied Economics 7.029

10 JOURNAL OF ECONOMIC PERSPECTIVES (***) 6.918

11 REVIEW OF FINANCIAL STUDIES *** 6.751

12 BROOKINGS PAPERS ON ECONOMIC ACTIVITY 6.679

13 JOURNAL OF FINANCIAL ECONOMICS *** 6.555

14 Annual Review of Economics 6.260

15 JOURNAL OF HUMAN RESOURCES 6.198

16 Journal of the European Economic Association (*) 6.136

17 JOURNAL OF LABOR ECONOMICS (*) 5.993

18 REVIEW OF ECONOMICS AND STATISTICS * 5.943

19 American Economic Journal-Economic Policy 5.855

20 JOURNAL OF MONETARY ECONOMICS ** 5.515

21 JOURNAL OF ECONOMIC GROWTH (*) 4.288

22 Quantitative Economics 3.663

23 JOURNAL OF ACCOUNTING & ECONOMICS 3.631

24 Theoretical Economics 3.620

25 ECONOMIC JOURNAL ** 3.457

26 JOURNAL OF BUSINESS & ECONOMIC STATISTICS * 3.335

27 American Economic Journal-Microeconomics 3.303

28 Annual Review of Financial Economics 3.284

29 ECONOMIC POLICY (*) 3.141

30 RAND JOURNAL OF ECONOMICS ** 3.140

31 Review of Environmental Economics and Policy (*) 3.130

32 JOURNAL OF POLICY ANALYSIS AND MANAGEMENT 3.049

33 INTERNATIONAL ECONOMIC REVIEW (*) 2.985

34 WORLD BANK RESEARCH OBSERVER 2.881

Tenure and promotion policies at the Department of Economics 15 • Appendices

35 REVIEW OF ECONOMIC DYNAMICS (*) 2.881

36 JOURNAL OF PUBLIC ECONOMICS ** 2.718

37 JOURNAL OF DEVELOPMENT ECONOMICS (*) 2.648

38 JOURNAL OF INTERNATIONAL ECONOMICS ** 2.596

39 JOURNAL OF ECONOMETRICS ** 2.546

40 JOURNAL OF HEALTH ECONOMICS (*) 2.537

41 JOURNAL OF FINANCIAL AND QUANTITATIVE ANALYSIS 2.506

42 Review of Finance (*) 2.441

43 JOURNAL OF APPLIED ECONOMETRICS * 2.366

44 JOURNAL OF URBAN ECONOMICS 2.252

45 ECONOMIC GEOGRAPHY 2.221

46 EXPERIMENTAL ECONOMICS 2.144

47 IMF Economic Review 2.086

48 VALUE IN HEALTH 2.082

49 ECONOMETRIC THEORY * 2.080

50 JOURNAL OF RISK AND UNCERTAINTY * 1.971

51 JOURNAL OF ECONOMIC GEOGRAPHY 1.970

52 Econometrics Journal * 1.966

53 QME-Quantitative Marketing and Economics 1.945

54 JOURNAL OF MONEY CREDIT AND BANKING (*) 1.914

55 MATHEMATICAL FINANCE (*) 1.891

56 JOURNAL OF ECONOMIC THEORY ** 1.879

57 Review of International Organizations 1.859

58 JOURNAL OF ECONOMIC SURVEYS 1.826

59 JOURNAL OF LAW & ECONOMICS (*) 1.773

60 Annual Review of Resource Economics 1.738

61 Socio-Economic Review 1.736

62 JOURNAL OF ENVIRONMENTAL ECONOMICS AND MANAGEMENT 1.700

63 Economic Systems Research 1.637

64 American Journal of Health Economics 1.626

65 ECONOMIC DEVELOPMENT AND CULTURAL CHANGE 1.603

66 EUROPEAN ECONOMIC REVIEW (*) 1.589

67 JOURNAL OF ECONOMIC HISTORY 1.587

68 Journal of Financial Econometrics 1.561

69 REVIEW OF INTERNATIONAL POLITICAL ECONOMY 1.551

70 SCANDINAVIAN JOURNAL OF ECONOMICS 1.523

Tenure and promotion policies at the Department of Economics 16 • Appendices

71 PHARMACOECONOMICS 1.499

72 WORLD DEVELOPMENT 1.469

73 Education Finance and Policy 1.461

74 JOURNAL OF LAW ECONOMICS & ORGANIZATION 1.461

75 ECONOMICA 1.436

76 WORLD BANK ECONOMIC REVIEW 1.400

77 JOURNAL OF POPULATION ECONOMICS 1.352

78 JOURNAL OF ECONOMIC BEHAVIOR & ORGANIZATION (*) 1.340

79 Econometric Reviews 1.340

80 LABOUR ECONOMICS 1.337

81 Cambridge Journal of Regions Economy and Society 1.291

82 INTERNATIONAL JOURNAL OF FORECASTING 1.285

83 HEALTH ECONOMICS 1.264

84 ECONOMICS OF EDUCATION REVIEW 1.262

85 FOOD POLICY 1.257

86 Journal of Agrarian Change 1.238

87 TRANSPORTATION RESEARCH PART B-METHODOLOGICAL 1.220

88 Journal of Human Capital 1.213

89 REAL ESTATE ECONOMICS 1.211

90 AMERICAN JOURNAL OF AGRICULTURAL ECONOMICS 1.195

91 POST-SOVIET AFFAIRS 1.189

92 INTERNATIONAL JOURNAL OF INDUSTRIAL ORGANIZATION 1.173

93 JOURNAL OF INDUSTRIAL ECONOMICS (*) 1.170

94 GAMES AND ECONOMIC BEHAVIOR (*) 1.148

95 JOURNAL OF ECONOMIC DYNAMICS & CONTROL (*) 1.147

96 Journal of Demographic Economics 1.137

97 EXPLORATIONS IN ECONOMIC HISTORY 1.122

98 REGIONAL SCIENCE AND URBAN ECONOMICS 1.114

99 ECOLOGICAL ECONOMICS 1.113

100 ECONOMY AND SOCIETY 1.112

101 JOURNAL OF BANKING & FINANCE (*) 1.100

102 NATIONAL TAX JOURNAL 1.079

103 NEW POLITICAL ECONOMY 1.072

104 ENERGY ECONOMICS 1.071

105 REGIONAL STUDIES 1.067

106 JOURNAL OF ECONOMICS & MANAGEMENT STRATEGY 1.056

107 ECONOMIC THEORY (*) 1.053

Tenure and promotion policies at the Department of Economics 17 • Appendices

108 Journal of Economic Inequality 1.025

109 ENERGY POLICY 1.013

110 JOURNAL OF RISK AND INSURANCE 1.008

111 ECONOMIC INQUIRY 1.005

112 ECONOMIC HISTORY REVIEW 1.002

113 OXFORD BULLETIN OF ECONOMICS AND STATISTICS 1.001 < 1 or 114-363 [250 more journals in Economics] missing

Other (selection from WoS Statistics and Probability) AIS

1 JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-STATISTICAL METHODOLOGY *** 5.671

2 ANNALS OF STATISTICS *** 4.807

3 JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION *** 3.369

4 BIOMETRIKA 2.533

5 Stata Journal 1.844

6 Annals of Applied Statistics 1.827

7 Finance and Stochastics (*) 1.713

8 JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES A-STATISTICS IN SOCIETY ** 1.533

A.2. Operations Research

WoS Operations Research and Management Science AIS

1 MANAGEMENT SCIENCE *** 3.774

2 MATHEMATICAL PROGRAMMING *** 2.965

3 JOURNAL OF OPERATIONS MANAGEMENT (***) 2.663

4 OPERATIONS RESEARCH *** 2.031

5 M&SOM-Manufacturing & Service Operations Management (***) 1.918

6 MATHEMATICS OF OPERATIONS RESEARCH ** 1.745

7 OMEGA-INTERNATIONAL JOURNAL OF MANAGEMENT SCIENCE 1.390

8 TRANSPORTATION SCIENCE 1.387

9 SYSTEMS & CONTROL LETTERS * 1.384

10 PRODUCTION AND OPERATIONS MANAGEMENT (***) 1.342

11 TRANSPORTATION RESEARCH PART B-METHODOLOGICAL * 1.220

12 INFORMS JOURNAL ON COMPUTING (***) 1.155

Tenure and promotion policies at the Department of Economics 18 • Appendices

13 COMPUTATIONAL OPTIMIZATION AND APPLICATIONS 1.080

14 EUROPEAN JOURNAL OF OPERATIONAL RESEARCH * 1.055

15 JOURNAL OF QUALITY TECHNOLOGY 1.052

16 TECHNOVATION 1.018

17 DECISION SUPPORT SYSTEMS 1.013

18 4OR-A Quarterly Journal of Operations Research 1.010

19 TRANSPORTATION RESEARCH PART E-LOGISTICS AND TRANSPORTATION REVIEW 0.980

20 RELIABILITY ENGINEERING & SYSTEM SAFETY 0.959

21 INTERNATIONAL JOURNAL OF PRODUCTION ECONOMICS 0.933

22 COMPUTERS & OPERATIONS RESEARCH 0.901

23 IIE TRANSACTIONS 0.898

24 JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS * 0.880

25 IEEE Systems Journal 0.878

26 OPTIMIZATION METHODS & SOFTWARE 0.863

27 Fuzzy Optimization and Decision Making 0.834

28 Flexible Services and Manufacturing Journal 0.794

29 EXPERT SYSTEMS WITH APPLICATIONS 0.790

30 JOURNAL OF GLOBAL OPTIMIZATION 0.764

31 NETWORKS & SPATIAL ECONOMICS 0.760

32 SAFETY SCIENCE 0.758

33 Discrete Optimization 0.725

34 JOURNAL OF MANUFACTURING SYSTEMS 0.690

35 MATHEMATICAL METHODS OF OPERATIONS RESEARCH 0.688

36 OR SPECTRUM * 0.659

37 OPTIMIZATION 0.659

38 QUEUEING SYSTEMS 0.635

39 Optimization Letters 0.620

40 DISCRETE EVENT DYNAMIC SYSTEMS-THEORY AND APPLICATIONS * 0.614

41 ANNALS OF OPERATIONS RESEARCH * 0.611

42 OPTIMIZATION AND ENGINEERING 0.609

43 OPERATIONS RESEARCH LETTERS 0.607

44-84 [41 more journals in Operations Research and Management Science] < 0.6

Tenure and promotion policies at the Department of Economics 19 • Appendices

Other (selection from WoS Mathematics and WoS Mathematics, Applied) AIS

1 SIAM JOURNAL ON OPTIMIZATION 2.718

2 JOURNAL OF COMBINATORIAL THEORY SERIES B ** 1.387

3 JOURNAL OF COMBINATORIAL THEORY SERIES A ** 1.223

Tenure and promotion policies at the Department of Economics 20 • Appendices

Transition policy

To: Dean of TiSEM, Geert Duysters Chairman Faculty Assessment Committee, Peter Essers

From: Head of Department Econometrics and Operations Research, Herbert Hamers

Date: June 8 ,2020

Concerns: Transition policy “Guidelines for Tenure and Promotion Decisions”

Dear Geert and Peter,

Enclosed you find the report “Guidelines for Tenure and Promotion Decision” of the department Econometrics and Operations Research. This report describes the procedure and criteria for future tenure appointments and promotions to full professor in our department.

However, I want to emphasize that applying these new criteria and ignoring the old criteria would not be fair to all current members of our department. It is agreed upon in our department that the Head of Department makes a personal agreement with these members were a transition policy is eligible. In case a member of our department is nominated for tenure or promotion to full professor and to this member a transition agreement applies, the Head of Department will explain and motivate this in the accompanying letter.

The assessment procedure will be identical to all department members as soon this report becomes effective.

Best regards

Herbert

Tenure and promotion policies at the Department of Economics 21