A Pluriannual, Nationwide Survey of Italian Medical Universities
Total Page：16
File Type：pdf, Size：1020Kb
Italian Journal of Medicine 2021; volume 15:99-106 Do local curriculum scores correlate with national residency test results? A pluriannual, nationwide survey of Italian Medical Universities Livio Tarchi,1 Stefano Damiani,1 Simone Marini,2 Claudio Cappelli,3 Giammaria Liuzzi,4 Massimo Minerva,5 Pierluigi Politi1 1Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy; 2Department of Epidemiology, University of Florida, Gainesville, FL, USA; 3Primary Care, ASUR MARCHE - AV5, San Benedetto del Tronto (AP), Italy; 4Department of Clinical Chemistry, ASST Grande Ospedale Metropolitano Niguarda, Milan, Italy; 5Department of Anesthesiology and Intensive Care, ASST Melegnano e della Martesana Sede di Cernusco, Cernusco Sul Naviglio (MI), Italy ABSTRACT In Italy, to match medical doctors to specialty residencies, the Minister of Education organizes a national multiple choice ques- tions exam composed of 140 items. The curriculum grants additional points based on the candidates’ grade average and graduation mark. Both national and international policies assume that curriculum scores can be evaluated in absolute terms, irrespective of the course attended by the candidate or its University of graduation. A fair inquiry over the current criteria of selection is not available to this day. Hence, this study aimed at investigating the degree of association between results in the Italian national residency test and the score given for each candidate’s curriculum. We Correspondence: Livio Tarchi, Department of Brain and Be- calculated Spearman’s correlation coefficients between the havioral Sciences, University of Pavia, via Bassi 21, 27100 score obtained at the 2019 test and curriculum scores. A mod- Pavia, Italy. erate/high degree of correlation (rho=0.600) was reached. Tel.: +39.0382.986456. Spearman’s rho ranged from 0.447 to 0.788, considering the E-mail: [email protected] single Universities. We thus measured a modest but non-neg- ligible between-University variability. As differences between Key words: Testing; curriculum evaluation; postgraduate train- test and curriculum scores across institutions may determine ing; admission; selection. the admission or not to the residency for many candidates, the Acknowledgments: the authors would like to thank the asso- authors call for broader attention on the topic. ciation of medical doctors ‘ALS Associazione Liberi Special- izzandi - fattore 2a’ for facilitating this work. Contributions: LT and SD contributed equally to this work, devised the project, the main conceptual ideas, and proof out- Introduction line, drafted the initial version of the manuscript; CC, GL, and MM collected the sample; LT conducted the analysis; SM pro- The Italian Medical Residency test vided the theoretical formalism for a statistical account of the Medical residencies in Italy are attributed to grad- sample; PP verified the analytical methods and supervised the findings of this work. All authors have seen and approved the uates from medical schools after a national test. Ac- final version of the manuscript. cording to the ranking, each candidate might then Non-commercialchoose its usetraining program only until the fulfillment of po- Conflict of interests: the authors declare no potential conflict sitions. In Europe, centralized and national residency of interests. tests are not uncommon. For instance, a knowledge- Data availability: data supporting the findings of this study are based test adjusted for academic merits has been the available from the corresponding author, LT, upon reasonable adopted method for matching medical graduates in request. France, Spain, and Portugal. As the labor mobility be- See online Appendix for Supplementary materials. tween countries is one of the tenants of the EU project, several treaties have been enacted to regulate the re- Received for publication: 26 January 2021. ciprocal recognition of medical degrees and specialty Accepted for publication: 9 February 2021. titles.1 However, the organization and definition of This work is licensed under a Creative Commons Attribution post-graduate programs still fall under national juris- NonCommercial 4.0 License (CC BY-NC 4.0). diction. While the test is defined at the national level, a high degree of autonomy is given to each of the © Copyright: the Author(s), 2021 forty-two Italian Medical Universities in defining Licensee PAGEPress, Italy Italian Journal of Medicine 2021; 15:99-106 courses and exams for their students. No evaluation doi:10.4081/itjm.2021.1470 system has yet been adopted to evaluate the level of preparation each University should match. [page 99] [Italian Journal of Medicine 2021; 15:1470] [page 99] Article The national Italian residency test (SSM) comprises the available choices as fit.2 This evidence may also 140 questions, granting 1 point each for a correct an- partially explain why many doctors decide to seek a swer and –0.25 for mistakes. On top of the test results, position abroad right after their graduation.3 In fact, individual ranking positions are adjusted with 7 addi- from 2005 to 2015, 10,104 Doctors in Medicine from tional points based on the outcome obtained in the Uni- Italy applied to the UE scheme for professionals mov- versity of origin and the individual merits. The factors ing abroad and left the country.3 Also noted, 52% of determining curricula scores can be found in Table 1. all European doctors who moved abroad through sim- ilar schemes were from Italy.4 Although impossible to The current scenario differentiate between specialized and unspecialized doctors within these numbers, the magnitude of the While the residency test is national, the University phenomenon is deemed relevant for the discussion. No curriculum depends on the University of origin, and it study has previously attempted to validate this con- shows a high grade of autonomy for what concerns struct of interest to the authors’ knowledge. Grade Average and Graduation Mark scores. Even though curriculum scores give only 4.7% (7 out of Validity framework 147) of the final score, this small percentage is crucial for selecting the candidates in the SSM. Italian SSM As the study aims to evaluate an established fash- is highly competitive, as only a considerably reduced ion in assessing education for medical graduates, an fraction of the candidates will obtain a residency po- overall judgment is needed on the degree of the ade- sition each year. In 2019, only 52.3% of candidates quacy and appropriateness from which actions and in- were matched to a residency position. In total,17,565 terpretations about scores derive.5 This includes the candidates competed for 9183 residencies, 8075 of current use of the score,only which is under study and both which were indiscriminately available to all of them implicit and explicit meanings attributed to it. (46.0% of the candidates) in the form of national The explicit meaning attributed to SSM exams is grants, publicly funded.2 Another important factor to that a better result directly translates into a better can- consider is that the candidates with the highest final didate, irrespectiveuse of the specific specialties under SSM score will have priority in choosing both the city evaluation, as the exam is one and does not differen- in which to conduct the residency and the specializa- tiate based on results in sub-fields of medicine. The tion. The available slots will progressively reduce so implicit meaning of the curriculum scores is that the that candidates may even withdraw as they do not see grading system used throughout Italy, in different Table 1. Composition of Curriculum scores. Grade average Points Between 27 and 27.50 0.5 Between 27.50 and 28 1 Between 28 and 28.50 1.5 Between 28.5 and 29 2 Between 29 and 29.50 Non-commercial 2.5 Above 29.50 3 Note: grades range from 0 to 30, 18 being the minimum passing score. Graduation mark Points Between 105 and 108 0.5 Between 108 and 110 1 110 1.5 110 and honors 2 Note: grades range from 0 to 110, 66 being the minimum passing score. Other qualifications Points Experimental Thesis 0.5 Ph.D. 1.5 Note: virtually all Italian Medical Graduates defended an experimental thesis on their graduation session. [page 100] [Italian Journal of Medicine 2021; 15:1470] Do local curriculum scores correlate with national residency test results? medical schools, is either unanimous or that, although context, the primary aim of this study was to investigate different, does not introduce bias. Another implicit whether differences in grading between the Universities meaning attributed to these curriculum scores is that could introduce a selection bias in the candidates due higher grades should correct for potential variance in to discrepancies in evaluating and administering med- results through different years so that a candidate with ical programs in a system of high local autonomy. We good knowledge and marks might correct with addi- hence evaluated: i) the association between the curricu- tional points a result taken in a single point in time. lum score and the residency test score; ii) changes of To the authors’ knowledge, no literature was found this association across the different Italian Universities, on the correlation between the level of preparation of a hypothesizing the presence of at least a modest/moder- single medical school and its impact on the alumni for ate degree of heterogeneity. future matching into residency programs. No literature, study, or report was also found about the correlation be- tween the different levels of preparation offered by sin- Materials and Methods gle medical schools in Italy and what SSM exams are Sample concerned. Furthermore, for what concerns the specific case of the Italian residency exams, no literature, study, SSM results are public but semi-anonymized to or report was found on the potential bias introduced by protect the privacy of candidates. We were able to at- the different grading systems adopted by the single au- tribute specific variables to the 2019 cohort and asso- tonomous medical schools and the curriculum score at- ciated results.