TESTING PRACTICES IN THE 21ST CENTURY 1 Testing practices in the 21st century. Developments and european psychologists’ opinions Arne Evers¹, José Muñiz², Dave Bartram³, Dusica Boben⁴, Jens Egeland⁵, José R. Fernández-Hermida⁶, Örjan Frans⁷, Grazina Gintiliené⁸, Carmen Hagemeister⁹, Peter Halama¹⁰, Dragoș Iliescu¹¹, Aleksandra Jaworowska¹², Paul Jiménez¹³, Marina Manthouli¹⁴, Krunoslav Matesic¹⁵, Mark Schittekatte¹⁶, H. Canan Sümer¹⁷, and Tomáš Urbánek¹⁸ ¹University of Amsterdam, The Netherlands ²University of Oviedo, Spain ³SHL Group, Thames Ditton, UK ⁴Drustvo Psihologov Slovenije, Ljubljana, Slovenia ⁵Vestfold Mental Health Care Trust, Tønsberg, Norway ⁶Spanish Psychological Association, Madrid, Spain ⁷University of Uppsala, Sweden ⁸Vilnius University, Lithuania ⁹Technical University of Dresden, Germany ¹⁰University of Trnava, Slovakia ¹¹Universitatea Babes-Bolyai, Cluj-Napoca, Romania ¹²Psychological Test Laboratory of the Polish Psychological Association, Warsaw, Poland ¹³University of Graz, Austria ¹⁴Association of Greek Psychologists, Athens, Greece TESTING PRACTICES IN THE 21ST CENTURY 2 ¹⁵University of Zagreb, Croatia ¹⁶University of Gent, Belgium ¹⁷Middle East Technical University, Ankara, Turkey ¹⁸Institute of Psychology, Academy of Sciences, Brno, Czech Republic Corresponding author at: Faculty of Social and Behavioral Sciences University of Amsterdam Roetersstraat 15 1018 WB Amsterdam The Netherlands, Tel. +31 20 525-6751 , Fax +31 20 639-0531. E-mail address: [email protected] (Arne Evers). SELF-ARCHIVED PREPRINT Evers, A., Muñiz, J, Bartram, D., Boben, D., Egeland, J., Fernández-Hermida, J.-R., Frans, O., Gintiliené, G., Hagemeister, C., Halama, P., Iliescu, D., Jaworowska, A., Jiménez, P., Manthouli, M., Matesic, K., Schittekatte, M., Sümer, H. C., & Urbánek, T. (2012). Testing Practices in the 21st Century: Developments and European Psychologists’ Opinions. European Psychologist, 17(4), 300-319. TESTING PRACTICES IN THE 21ST CENTURY 3 Abstract The main goal of the European Federation of Psychologists’ Associations (EFPA) Standing Committee on Tests and Testing (SCTT) is the improvement of testing practices in European countries. In order to reach this goal, the SCTT carries out various actions and projects, some of which are described in this paper. To better inform its work, it decided to survey the opinions of professional psychologists on testing practices. A questionnaire of 33 items was administered to a sample of 12,606 professional psychologists from 17 European countries. The questionnaire was based on, but not identical to, one used in 2000. The new data show that the positive attitude of the respondents toward the use of tests that was obtained in 2000 has increased in most countries, with a high percentage of the surveyed psychologists using tests regularly. Five main dimensions explained 43% of the total item variance. The dimensions involve items relating to: Concern over incorrect test use, regulations on tests and testing, Internet testing, appreciation of tests, and knowledge and training relating to tests and test use. Important differences between countries were found on these five dimensions. Differences were found according to gender for four of the five dimensions and in relation to field of specialization for all five dimensions. The most commonly used tests are the classic psychometric tests of intelligence and personality: WISC, WAIS, MMPI, RAVEN, 16PF, NEO-PI-R, BDI, SCL-90. Finally, some future perspectives are discussed. Keywords: Psychometric testing, Testing practices, Survey, Europe, EFPA TESTING PRACTICES IN THE 21ST CENTURY 4 Testing practices in the 21st century. Developments and european psychologists’ opinions The European Federation of Psychologists’Associations (EFPA) investigated psychologists’ attitudes toward various aspects of testing in the year 2000 using a comprehensive survey across six European countries (Mun˜iz et al., 2001). The aim was to gather information about attitudes and usage that could later contribute to training and awareness campaigns to improve psychologists’ use of tests across Europe. A review of the projects and actions recently carried out by EFPA and the International Test Commission (ITC) can be found in Mun˜iz and Bartram (2007). As the first survey was completed over a decade ago, the EFPA Standing Committee on Tests and Testing (EFPASCTT) considered it appropriate to reassess European psychologists’ opinions of tests. Besides performing a followup of the results obtained at that time, technical advances that have emerged over this last decade had to be taken into account. These advances are having a strong impact on the way psychologists practice their profession in general and, in particular, on the use of tests. We refer in general to the developments in computer-based testing and more specifically to the possibility of testing by the Internet and related issues of unproctored (or unsupervised) testing and computer generated feedback. Therefore, in the new survey, in addition to questions concerning the key issues of 10 years ago, various questions were included about the incidence of these two factors in professionals’ daily testing practices, such as: Are computerized tests replacing paper-and- pencil tests in the psychologist’s daily practice? Is the Internet a clear advance in contemporary psychological assessment? Before describing the new survey and the results obtained, an overview of the recent actions carried out in Europe with the aim of improving test use is presented. Correct test use is based on two essential pillars: on the one hand, tests should have adequate psychometric TESTING PRACTICES IN THE 21ST CENTURY 5 properties and, on the other, they should be used correctly, from the decision made about when or if to use them, the time they are administered and scored up to the use that is made of the scores. The organizations that dedicate effort to improving test use, both national and international, perform various actions and carry out projects structured around two main strategies that can be described as either ‘‘restrictive’’ or ‘‘informative’’. The restrictive strategy refers to the actions performed to limit the use of tests to the professionals who are qualified and competent to use them. The systems used vary from country to country (Bartram, 1996; Bartram & Coyne, 1998; Mun˜iz & Ferna´ndez-Hermida, 2000; Mun˜iz, Prieto, Almeida, & Bartram, 1999). One of the most commonly used in various countries is to classify tests into three categories, according to the old APA criteria (A, B, and C). Tests in category A are educational achievement tests and simple inventories and accessible to most people. The use of tests from categories B (collective cognitive and personality tests) and C (individually administered tests, such as intelligence tests and projective tests) is restricted exclusively to psychologists in some countries while just category C is restricted in others, with specific training or qualification requirements being placed on other people for tests in category B. Another option is for professionals who use testing, whether psychologists or not (e.g., human resources professionals in work and organizational assessment, or teachers involved in assessing children for special educational needs), to obtain a specific qualification that accredits their competence to use the instruments they need to use in their work. In this sense, the project currently being undertaken by the EFPA-SCTT, under the direction of Dave Bartram, is especially noteworthy (Bartram, 2011). This project is briefly described below. Test-User certification In 2007, under the auspices of EFPA a Working Group chaired by Dave Bartram was set up to look into the implementation of a procedure for accreditation of national testuser TESTING PRACTICES IN THE 21ST CENTURY 6 certification schemes. The group has established a set of standards for test users and defined three levels of test use: Level 1 relates to use under direct supervision (e.g., test administration); Level 2 to independent use of tests of a limited range within a limited set of situations; and Level 3 to being a specialist in test use (the Level which would be expected of a psychologist who had specialized in tests and testing). In 2009, the SCTT presented the following proposals to the EFPA General Assembly: (a) to complete its current work by carrying out a pilot Level 2 implementation project within UK, Norway, and Sweden, the three countries which are currently awarding qualifications in test use that are aligned with the Level 2 EFPA standards. In parallel with this activity, the SCTT will move ahead with developing the Level 3 (expert level) qualification as the basis for a Special Register associated with EuroPsy. The SCTT has also developed a standard nomenclature for qualifications, aligned with the European Qualification Framework (EQF) to aid public understanding of the qualifications. This should also help provide recognition of these test- user qualifications within the EQF as well as by EFPA. The Working Group has now established a European Test User Accreditation Committee for test-user qualifications and has documented the procedure for accrediting UK, Norway, and Sweden’s current qualifications (with the detailed accreditation of the UK procedures having been completed at the time of writing). All these activities are aimed at the improvement of testing practices in Europe. The accreditation of the professionals properly trained in tests and testing is a strategic