Vol. 17, June 2007 Editor: Jan Bogg

TESTING INTERNATIONAL

PRESIDENT Prof. Jacques Grégoire Université catholique de Louvain Faculté de Psychologie Place du Cardinal Mercier, 10 CONTENTS 1348-Louvain-la-Neuve Belgium ITC PRESIDENT BIOGRAPHY 2

PRESIDENT-ELECT Prof.. M. Born Erasmus University Rotterdam Institute of Psychology ITC GUIDELINES IN COMPUTER/INTERNET TESTING—HAVE YOUR SAY 2 Woudestein, T13-15 P.O. Box 1738 3000 DR Rotterdam COMPUTERISED ADAPTIVE TESTING 2 SECRETARY Prof. R.K. Hambleton University of Massachussetts U.S.A PIONEER OF ITC JEAN CARDINET RECALLS EARLY DAYS 3 TREASURER Prof. Em. Barbara M. Byrne School of Psychology University of Ottawa CAN YOU HELP - RESEARCH ON COST OF E-ASSESSMENTS 3 145 Jean-Jacques Lussier Ottawa, Ontario Canada K1N 6N5 NEWS 3 PAST-PRESIDENT Prof. Jose Muniz University of Oviedo, Spain ITC MEMBER QUESTIONS AND ANSWERS 4 COUNCIL MEMBERS Elected Members Prof. Fanny Cheung Chinese University of Hong ARTICLES Kong, Hong Kong Prof. Cheryl Foxcroft PSYCHOLOGICAL TESTS AND TESTING IN SWEDEN 5 University of Port Elizabeth, S.A Prof. Frederick Leong CURRENT ADVANCES IN TEST ADAPTATION IN ROMANIA 7 Michigan State University, USA Co-Opted Members Dr Iain Coyne University of Nottingham, UK Dr David Foster Kryterion, USA Prof. Tom Oakland University of Florida, USA

REPRESENTATIVES Dr. M. Bullock, IUPsyS American Psychological Association, U.S.A. Prof. D. Bartram, IAAP SHL Group plc U.K

EDITORS International Journal of Testing Prof. John Hattie University of Auckland School of Education Fisher Building 18 Waterloo Quadrant Private Bag 92019 Auckland New Zealand

Testing International Dr. Jan Bogg University of Liverpool Department of Clinical Psychology The Whelan Building LiverpoolL69 3GB AN OFFICIAL PUBLICATION OF THE INTERNATIONAL TEST COMISSION UK

ITC PRESIDENT Jacques Grégoire ITC GUIDELINES HAVE YOUR SAY

Jacques Grégoire, Ph.D., is Full Professor in Computer/Internet Testing the Faculty of Psychology As part of its ongoing activities, the Interna- and Educational Sci- tional Test Commission is reviewing the ences at the Catholic Uni- work it has done on developing the Guide- versity of Louvain, Bel- lines on Computer-based and Internet- gium. His research inter- delivered testing. ests include assessment Please provide us with your views from your of learning and learning countries perspective. This will be invaluable disabilities, methods for for the ITC in enhancing its activities and psychological diagnostic, intellectual assess- engaging with individuals around the world ment, and relationship between learning and who have an interest in tests and testing. emotions. In collaboration with Catherine We would be grateful if you could spare 10 Van Niewenhoven and Marie-Pascale Noël, minutes to have your say. he developed a test for the diagnostic of Complete the questionnaire online by mathematical disabilities, the TEDI-MATH, MAY 31st, 2007. already adapted in Spanish, Dutch and Ger- http://www.surveymonkey.com/s.asp?u=395993312214 man. He was scientific adviser for the French adaptation of several tests, including the WAIS-III, the WISC-IV and, currently, the Computerized Adaptive Testing K-ABC-III. He is also member of the scientific panel for the development of the future WAIS-IV and WMS-III. His academic publi- Internet and CBT Testing Guidelines cations include over 100 journal articles, http://www.intestcom.org/ book chapters, and books/monographs. He is author of two popular French books on CAT Central a computerized adaptive intelligence assessment: “Clinical assess- testing site ment of child intelligence” (2006) and http://www.psych.umn.edu/psylabs/catcentral/ “Clinical assessment of adult intelligence” (2004), and a reference book on the CAT Publications methodology of test development, written Williamson, David M., Mislevy, Robert J. and Bejar, with Dany Laveault: “Introduction to test Isaac (2006). Automated Scoring of Complex Tasks in theories in Psychology and Education”. He is Computer Based Testing. Mahwah, New Jersey: Law- Consulting editor of several scientific Jour- rence Erlbaum Associates Publishers. nals and Associate Editor of the Journal of PsychoEducational Assessment. He has Van v van der Linden, W.J. (2005). Linear models for also served as secretary of the International optimal test design. New York: Springer-Verlag. Test Commission (1995-2004). With the collaboration of Ron Hambleton, he organized Williams, David D., Howell, Scott L. and Hricko, Mary the 5th ITC international conference in Brus- (2006). Online Assessment, Measurement and sels (July, 2006) on “Psychological and Edu- Evaluation. Hershey, PA: Information Sciences Pub- cational Test Adaptation Across Languages lishing, and Cultures”. Bunderson, C.V., Olsen, J.B. and Greenberg, A. The International Journal of Testing Institute for Computer Uses in Education (1993). Com- Call for Reviewers: the ITJ are continually puters in educational assessment: An opportunity to looking for suitable reviewers for manu- restructure educational practice" PB92-127604, Con- scripts submitted to the International Journal gress of the United States Office of Technology As- of Testing (IJT). If you are interested in re- sessment. viewing articles on a range of topics related to testing and assessment with international Bunderson, C. V., Inouye, D. K. and Olsen, J. B. perspective, please e-mail your name, quali- (1989). Four generations of computerized educational fications, CV and areas of expertise to the measurement. In R.L. Linn (Ed.) Educational Meas- editor, John Hattie at [email protected] urement, 3rd Edition. New York: Macmillian.

Pioneer of ITC Jean Cardinet recalls NEWS early days

Modestly Dr Cardinet said Tests standards, development “Of course, I have no merit in and use in Brazil its (ITC) success, since I left Solange Wechsler the baby into the hands of Pontifical Catholic University of Campinas, other psychologists soon after Brazil and Past President of the Brazilian it was born. The reason for Institute of Psychological Assessment which I left the field is that my career took abruptly a differ- Psychological assessment in Brazil is ex- ent direction. I left the Univer- periencing revolutionary changes. A deci- sity of Neuchâtel to take a position as head sion in 2003 by the Federal Council of Psy- of a research section in a newly created In- chologists, the national institution that regu- stitute of Education. I had to seize this sud- lates the practice of psychology, required all den opportunity and I never regretted it.” psychological tests used in Brazil to be evaluated and approved based on their sci- Dr Cardinet retired in 1990 and in recalling entific quality within the context of the Brazil- his early days he cited that his professors ian reality. In an effort to implement this de- included Thurstone and Cronbach. cision, the Federal Council of Psychologists organized a national commission of 40 Bra- “From my first career, I kept an interest in zilian researchers to establish and then ap- Generalizability Theory, as I had worked ply standards to verify whether published with Lee Cronbach on that subject. I devel- psychological tests met at least minimal sci- oped new methods for the exploitation of entific requirements, which were: a) a de- the symmetry of ANOVA designs. A special scription of its theoretical foundations, b) evi- computer program has recently been offered dence of the test’s validity and reliability on Internet to make such G and D Studies.” based on recent studies with Brazilians, and c) recently acquired norms on Brazilians if The EduG program and guide is available the test is intended to be norm-referenced or from the website. This program will be re- the empirical foundation for tests when their viewed in the International Journal of Test- interpretation procedures do not rely on ing, at a later date. norms (e.g., as with some projective tech- niques). The reaction to this decision among EduG program and guide available from: Brazilian psychologists was somewhat http://www.irdp.ch/edumetrie/englishprogram.htm mixed. Many welcomed the imposition of higher standards to test development and use while others were surprise to learn that CAN YOU HELP? tests they had been using for several years Research on cost of e-assessments in their practice were considered to not meet minimum standards. Three years after the Ângela Guimarães Pereira, Action Leader: 2003 decision, most Brazilian psychologists Quality of Scientific Information for EU policy agree that the imposition of these standards formulation from the European Commission was needed and have greatly enhanced the is enquiring to ITC members about studies quality of psychological assessment in Bra- on cost of e-assessments, including com- zil. Moreover, this decision strongly im- parisons with traditional pencil and paper pacted the scientific basis of Brazilian psy- ones. They are particularly interested is Lan- chology. For example, many new tests have guage testing. been developed by Brazilians and normed and validated on Brazilians. Please write to: Ângela Guimarães Pereira, Action Leader: Quality of Scientific Informa- This progress toward best practices in test tion for EU policy formulation, European development has resulted in Brazil becom- Commission DG Joint Research Centre ing preeminent in Central and South Ameri- IPSC, TP 361, 21020 Ispra (VA), Italy. can in reference to test development and E-mail: [email protected] use. The formation of Brazilian Institute of Psychological Assessment (IBAP) in 1997, a

national professional association dedicated to the Psychology International Newsletter advancement of test development and use, also has Psychology International, the newsletter of the been important to this success. More than one thou- APA office of international affairs is available sand psychologists have attended Brazilian Institute online from: http://www.apa.org/international/ of Psychological Assessment sponsored conferences.

The next IBAP conference will be held July 25- British Psychological Society 28, 2007 in Joao Pessoa, in the northeast of Bra- Psychological Testing Centre zil. Information can be obtained from www.ibapnet.org.br The British Psychological Society's Psychological Testing Centre (PTC) provides a focus for all So- ciety activities in relation to psychological testing. ISPA - The Israeli Psychometric Association The Psychological Testing Centre on-line pro- Avi Allalouf vides access to information about tests and test- National Institute for Testing and Evaluation ing. Whether you are a psychologist, a test user [email protected] or a member of the public interested in knowing more about testing, the PTC Online is a valuable The Israeli Psychometric Association (ISPA) was resource. founded recently as an initiative of the National Insti- tute for Testing and Evaluation (NITE). The founding Read best practice guidelines, link to other committee of ISPA includes the following members: sources of information, access general informa- Yoav Cohen (Chair), Naomi Gafni, Chanan Gold- tion on tests and link to test distributors/ schmidt, Carmel Oren, Sara Shapira and Avi Al- publishers. In addition, you may subscribe to the lalouf. complete test reviews, discuss good practice in relation to test development and read articles on The Israeli Psychometric Association consists of issues in testing, research and ethics. professionals from the areas of educational and psychological measurement and evaluation. It aims Visit the Psychological Testing Centre (PTC) at to: www.psychtesting.org.uk

• encourage activity in the development and application of standardized tests and performance as- ITC Q+A sessment evaluation; • promote research in the fields of educational and psychological measurement; Q. Do you know what The Brazilian Federal Psy- • improve assessment methods and measurement chological Association rules are associated with the administration of psychological tests to Bra- tools and facilitate their application in an erudite zilian children if the use of such test results are fashion; limited to research purposes? • construct, evaluate and distribute updated meth- Relana Fitzgerald, PhD, University of Virginia, odological instruments and innovative assess- Center of Global Health ment methods; • warn about test misuse and incorrect interpreta- A. Our regulation does not control any tests for tion of tests results; support the development of research purposes. In this case, any tests can professional contacts, the dissemination of knowl- be used. The present regulation only controls the edge and collaboration among experts in the use of psychological tests for professional pur- measurement and evaluation field. poses. In these cases, only tests which have presented validity, reliability and norms to the ISPA members receive (and are invited to contribute Brazilian population can be used. A list of tests to) the ISPA Newsletter, which is published twice a allowed to be used for professional purposes year. The newsletter presents short research notes, can be found at the following address: disseminates information about assessment pro- http://www.pol.org.br/satepsi/sistema/admin.cfm jects, recently published books and articles, upcom- Solange Wechsler, ITC member and former ing conferences and current issues in the field. ISPA president of Brazilian Federal Council has already held three scientific conferences, each of Psychologists of which attracted some 150 participants. For more information please visit www.ispa.org.il

ARTICLES development was not considered research, and both academic and economic incentives were Psychological tests and testing in Sweden lacking. Additionally, Swedish psychologists con- Eva Tideman stituted a very small group and thus a small mar- Department of Psychology, Lund University, ket. For example, in 1960, there were approxi- Sweden mately 500 psychologists in Sweden. The total E-mail: [email protected] population in Sweden, Norway and Denmark ap- proximates the population of the state of New The last two decades have seen steadily growing York. However, Swedish psychologists were not interest among psychologists and other behav- completely lacing of tests. For example, in the ioural scientists in their use of psychological tests 1960s, they had Swedish versions of the Stan- within both the clinical and I/O fields. Various rea- ford-Binet, Wechsler Intelligence Scale for Chil- sons account for this growth. Following a substan- dren, Griffiths’ Mental Development Scale, and tial decline in interest in assessments and clinical the Leiter International Performance Scale. In ad- diagnostics during the 1970s and 1980s, Swedish dition, a number of measures developed by psychologists to a large degree have resumed Swedish researchers were used to assess cogni- their central position as diagnosticians and recog- tive functions in children and adults. nize the value of test use to support their other services. The supply of psychological tests also The 1970s saw a decline in clinical test develop- has increased in line with the increase in demand ment and interest for clinical test application in for and interest in tests. The supply of psychologi- Sweden (as was true in Europe and the US). The cal tests in Sweden is relatively good, relying low interest in assessment and diagnostics and somewhat heavily on Swedish versions of Ameri- therefore in psychological tests among psycholo- can and British tests. Very few tests developed by gists continued until the latter part of the 1980s Swedish researchers achieve significant applica- and had considerable consequences for Swedish tion. This is especially true for clinical tests and test development and test sale. In 1991, Psyko- also applies in a somewhat lesser degree to tests logiförlaget, still the only test publisher in Sweden, for I/O purposes. was on the edge of bankruptcy and thus had no money to invest in test development despite the The development and use of psychological tests in fact that existing psychological tests were 20 or Sweden has two tracks: one for clinical tests and more years old. From the mid-1990s the develop- another for work (i.e., I/O) psychology. This in turn ment of psychological tests acquired renewed vig- has resulted in different tests, practices, and user our in response to the increased interest in and groups. Although in many places they are parallel demand for proper diagnostics as a basis for more and partially overlapping, the two areas of applica- effective treatment Psykologiförlaget acquired a tion are regarded as being almost completely dif- private owner and became successful in its efforts ferent.The description below mainly concerns psy- in part thought successful collaborative efforts with chological tests for clinical use yet also relates to individual researchers and clinicians to develop the significant work in I/O psychology spear- and publish a series of psychological tests for both headed by the Swedish Psychological Association the clinical and I/O markets. From the mid-1990s (Sveriges Psykologförbund). onwards, a large number of clinical tests have been adapted for use in Sweden, including the From the end of the 1950s onwards, the Swedish Wechsler scales, neuropsychological tests (e.g., Psychological Association had played a prominent Wisconsin Card Sorting Test, the Rey Complex role in both the development and marketing of Figure Test, the Delis-Kaplan Executive Functions psychological tests. Between 1954 and 1991 the Systems Test and NEPSY) personality tests (e.g., Association operated as Sweden’s only test pub- NEO PI-R and MMPI-2) as well as the classic lisher, Psykologiförlaget. The Swedish Psycho- Beck scales for depression. logical Association’s interest initially was a desire to control the use of psychological tests and to Psychological tests intended for clinical use are limit their access only to licensed psychologists. reserved exclusively for psychologists. However, The Association also attempted through Psyko- there are no official regulations governing access logiförlaget (by means of collaboration with uni- to tests or who may use tests in their professional versities) to stimulate the development of new capacity. Instead, access is controlled partly tests with the goal to provide psychologists with through the providers’ sales regulations and partly modern instruments. However, successes were through the successful efforts of the Swedish Psy- fairly modest. The Association found it difficult to chological Association and psychologists to limit engage the university-based professionals. Test (Continued on page 6)

their use to their own group, arguing that they gether with their professional associations gener- alone hold unique abilities within the field of psy- ally agree on the need to find pragmatic methods chological assessment. The Swedish Psychologi- the lead to the development of tests that satisfy cal Association has adopted the International Test high scientific standards, meet clinical needs, and Commission (ITC) guidelines for test use. The in- reflect economic and practically reality. tent of its use mainly has been to influence practices of groups who use tests outside the psychol- During the last ten years a fairly large number of ogy profession. The psychologists’ own profes- tests have been developed for use in work psy- sional ethics code, one that is common to psychology, especially measures of personality. chologists in the three Scandinavian countries, These include both Swedish developed and concurs with the ITC guidelines in applicable parts. American and British versions of tests. However, Approximately 8,000 professionally active psy- the quality of I/O tests has varied greatly and, de- chologists serve Sweden, among whom approxi- spite their widespread use, many lack documenta- mately 3,500 work in various clinical setting with tion on their development and application. In the children and adults, 900 work in schools/ mid-1990s the Swedish Psychological Association preschools, 400 work within the employment ser- recognized this shortcoming and, through the ef- vice, and 500 are within higher education. Approxi- forts of the Swedish Foundation for Applied Psy- mately 600 work as privately practising psycho- chology, has been very successful in increasing therapists and 500 as I/O consultants. the quality of available tests, the users’ knowledge of tests, and psychological assessment in a Although the number of psychologists in Sweden broader sense. The standards for using tests in I/O is somewhat large, the market for test use remains tests generally are lower than those for clinical ap- relatively small. Thus, the economic base for test plications. For example, persons who complete a development in Sweden is relatively weak and the brief course on testing can access psychological development of psychological tests for clinical ap- tests from various providers. plication is not a high priority of the State or universities. These economic realities have influenced The work of Swedish Foundation for Applied Psy- both the range and level of development work. chology has been inspired by similar initiatives by The development of each test has been based on the British Psychological Society. Sweden has an individual considerations that are determined by established system for establishing the quality the fundamental nature of the test, the need for control of tests in accord with guidelines issued by cultural adaptation and standardisation, possible the European Federation of Psychological Asso- interested parties to partner in their development, ciations. The Swedish Foundation for Applied Psy- and a precise cost/benefit analysis associated with chology also has supported the implementation of their development. As is true in other Scandina- International Test Commission guidelines for test vian countries and elsewhere, Sweden experi- use and has produced an examination procedure ences problems with unauthorised translations and for test users. Although the work of the Foundation multiple illegal versions of the same test. Unau- has received criticism from both commercial and thorized translations of tests can be found in clini- academic quarters, few dispute that the status of cal practice and university use. Also, there is a test development and use in the Swedish work general lack of knowledge of and respect for copy- psychology market has improved dramatically, in right issues and a wide-spread belief that re- part through the Foundation’s efforts. As the level searcher are entitled to translate and use tests in of knowledge and demands generally have in- their own research without proper permits from the creased within this area, inadequate tests and test copyright holder. On the other hand, individual re- users are finding it more difficult to establish them- searchers often play a very important role in dis- selves in the marketplace. A number of clinical seminating results of their research, enhancing specialty groups continue to increase their knowl- test use in clinical practice and thus the viability of edge and use of psychological tests in their prac- tests. tices. For example, there is a healthy interest among neuropsychologists engaged in both clini- Swedish tests have varying degrees of Swedish cal practice and research who have actively con- standardisation, ranging from pure translations, no tributed to the availability of a relatively large num- adaptations to Swedish culture, and no Swedish ber of neuropsychological tests. In reference to norms to extensive development work that has children, successful collaborative projects between resulted in tests with exhaustive documentation of publishers and universities have contributed to the their psychometric properties in different clinical development of WISC-III, WPPSI-III, Leiter-R, and non-clinical populations with Swedish norma- Merrill-Palmer Scales of Development–Revised, tive data. Test development often strides these and the Beck Youth Inventory. two extremes. Practitioners and academics to- (Continued on page 7) 6

virtual destruction of the discipline. Thus, its more Today a single commercial player, Harcourt As- recent history is rather bleak. Psychology sur- sessment, established in Sweden since autumn vived only by being implicitly tolerated by and em- 2006, carries out most clinical test development bedded within other academic departments, in- activities. The presence of Harcourt Assessment cluding history, philosophy, education, as well as in Sweden can have a long-term positive impact, in few personnel departments (e.g. those with in- enabling Scandinavia as a whole to obtain faster dustrial and organizational orientations). Although access to updated and modern instruments, in- psychology made a fresh start after 1990, it has cluding collaborative efforts between the Scandi- not yet succeeded in bridging the serious profes- navian countries that will give rise to valuable syn- sional gap (David, Moore & Domuta, 2002). ergy effects leading to large and comprehensive test development projects. From this point of The lack of nationally developed measures to as- view, the future of clinical test development in sess psychological and educational qualities ex- Sweden seems very promising. emplifies this gap. Many of the renowned measures used internationally (e.g., 16 Personality All Nordic countries have national language trans- Factors, California Psychological Inventory, NEO lations of the International Classification of Func- Personality Inventory Revised, Myers-Briggs Type tioning, Disability and Health (ICF). The Swedish Indicator, Eysenck Personality Questionnaire, National Board of Health and Welfare published it State-Trait Anxiety Inventory, and State-Trait An- in 2003. Shorter versions as well as the more re- ger Expression Inventory, 2nd Edition) have been cent comprehensive International Classification of translated into the Romanian langauge. Most Functioning Disability, Functioning, and Health translations have been associated with the vision- (ICF) core and brief core sets also have been de- ary work of a handful of psychometricians at the veloped. Swedish field trials for the preliminary University of Babes-Bolyai at Cluj-Napoca, as well Swedish version of ICF-CY, for children and as the Bucharest University. On the positive side, youth, published in 2004, are managed by the the availability of these instruments has facilitated research program Children-Health-Learning- the teaching of these instruments in academic Development. settings and their use in research. On the nega- tive side, the tests do not reflect common profes- Sweden also is participating in the Measuring sional standards. For example, test adaptations Health And Disability In Europe project sponsored were not consistent with the International Test by the European Union, to measure health and Commission guidelines on test adaptations, trans- disability in Europe. This project also will validate lations are sometimes poorly done, cultural adap- the IFC-CY for planning, implementing, and tations occur marginally, tests rarely are vali- evaluating early childhood intervention and habili- dated, and normative samples either are too small tation services. However, at present, neither the or display seriously skewed distributions. Thus, IFC nor the IF C-CY is being used in clinical prac- reliable interpretations when making high-stake tice. Hopefully, these instruments will prove to decisions is hazard. Additionally, copyright in- have various clinical applications and not be re- fringement occurs commonly by psychologists, an stricted only to academic interest. issue that constitute a serious problem.

An estimated 4000 and 5000 psychologists serve Current advances in test adaptation a population of 22 million. The number of psy- in Romania chologists is growing. During the last four to five years a small number of Romanian test compa- Dragoş Iliescu, PhD nies have formed. The principle purpose of this Associated Professor, National School for Political article is to describe the work of one. and Administrative Studies, Bucharest TestCentral™ (www.testcentral.ro) this was founded in 2003 as a private initiative, forming Margareta Dincă, PhD relationships between a major local research Professor, Titu Maiorescu University, Bucharest company (D & D Research, Ltd.), many academic and professional psychologists and researchers, Scientific psychology in Romania was established as well as international test companies, thus rap- at the beginning of the 20th century through the idly establishing itself as the main Romanian pub- efforts of Florian Stefanescu Goanga, a pupil of lisher of psychological tests. The company dis- the German psychologist Wilhelm Wundt, and plays acceptable standards for adapting psycho- other pioneering psychologists. However, psy- logical tests in Romania that are widely regarded chology was among the humanistic sciences nationally and internationally. TestCentral™ pub- banned by the communist regime, resulting in the (Continued on page 8) 7

lishes a large variety of tests from such publishers Establishing Test Norms as Psychological Assessment Resources, Sigma TestCentral™ follows two major principles when Assessment Systems, Mindgarden, Hogrefe, Or- norming tests: the sample should be nationally ganizzazioni Speciali, Management Research In- representative and stratified as well as sufficiently stitute, and Robertson-Cooper. Tests include large. All normative research during the last three those that assess personality, educational, clinical years has involved randomised samples of Roma- and quasi-clinical, values and social axioms, and nians, stratified by gender, age, education, urban/ work/industrial/organizational qualities. TestCen- rural residence and ethnic group member in light tral™ used the University of California, Berkeley’s of the most recent national census data. Institute of Personality Assessment and Research Additionally, the standardization samples are efforts during the early 1950s as a model for its large. For example, the California Psychological work. Within this model, the company attends to Inventory-260 was normed on 3200 (50% males); financial goals and other issues keenly important the State-Trait Anxiety Inventory (Spielberger, to international publishers who look upon Roma- 1983) on 1400; the State-Trait Anger Expression nia and other small national emerging markets Inventory -2 on 1200; the Survey of Work Styles with both interest and scepticism. The company (Jackson & Mavrogiannis-Gray, 1988) on 1500 also is committed to engage in scholarly activities (750 males and 750 females), with a supplemen- that lead to high quality products. These activities tary sample of 150 Romanian top managers; the recognize the importance of making suitable cul- Nonverbal Personality Questionnaire and the tural adaptation, adequate norming, and conduct- Five-Factor Nonverbal Personality Questionnaire ing post-norming validation studies. Cultural adap- on 1800 subjects (50% males); the Social Axioms tation and norming of tests at TestCentral™ al- Survey (Leung & Bond, 1989) on 1600; and the ways have been consistent with professional stan- Student Style Questionnaire (Oakland, Glutting & dards for test development and adaptation, includ- Horton, 1996) on 2400 students (100 boys and ing those from the International Test Commission 100 girls from each of the age groups from 7 (www.intestcom.org). through 18 years) . Validation Translation Romanian psychologists generally strongly dis- Translations use consecutive and back- trust tests that only have been translated. The ori- translations methods with dyads of translators, gins of their reservations stem, in part, from copy- sometimes also coupled with panels of subject right infringement and illegal usage, the poor qual- matter experts. Consistent with the International ity of the translations, and the circulation of many Test Commission guidelines for test adaptation, forms of the same test. Due to these and other the objective of this phase is to ensure the reten- reservations, TestCentral™ was committed to the tion of the original meaning of the test items when broader issue of working to ensure the tests were translating them into Romanian and to work to properly validated, not merely suitably translated ensure the translated items are suitable for use in and normed. Given the limited resources of Test- the Romanian culture and reflect the same trait as Central™ and the relatively small market for in that found in the culture in which the test first tests, extensive validation that may have been was developed. These translations are fairly used on tests developed in the host counties (e.g. straightforward for some tests, including the Non- the hundreds of research designs that have been verbal Personality Questionnaire (Paunonen, used to validate the California Psychological In- Jackson & Ashton, 2002) and the Five-Factor ventory or the NEO Personality Inventory, Re- Nonverbal Personality Questionnaire (Paunonen, vised) is not possible. Nevertheless, every test is Jackson & Ashton, 2002). Their items use nonver- subject to a thorough criterion-validation process bal stimuli and drawings that do not need transla- using observer-evaluations of the targeted behav- tion. Moreover, research with these measures in iour and additional empirical research in order to Romania (Iliescu, Minulescu & Nedelcea, 2006a, determine whether the results of a test and its 2006b) and in other countries (e.g. Korea; Lee, scales are consistent with predicted behavioural Ashton, Sungjin & Park, 2000) found them to be outcomes. For example, in Romania, extensive fairly free of cultural content. Qualities found in studies have provided strong support that in rela- other tests published by TestCentral (e.g., Califor- tion the California Psychological Inventory’s nia Psychological Inventory, Gough, 1987, 1996; Achievement via Conformance and Achievement State-Trait Anger Expression Inventory, 2nd Edi- via Independence scales predict academic tion, Spielberger, 1988, 1996; NEO Personality achievement, the Social Conformity and Self- Inventory, Revised, Costa & McCrae, 1992) re- control scales predict frequency and intensity of quire more attention because their cultural adap- delinquency, and its work-related scales predict tation is more complex. leadership behaviour (Pitariu, Pitariu & Ali Al Mu- (Continued on page 9) 8

tairi, 1998, Pitariu & Iliescu, 2004 etc.). Other re- amples are provided. search has investigated challenging issues in reference to the reliability of the Nonverbal Personal- The Romanian Anxiety Scale on the California ity Questionnaire (Iliescu, Minulescu & Nedelcea, Psychological Inventory better fits Romanian cul- 2006) and the Student Style Questionnaire ture than the Anxiety secondary scale originally (Iliescu, Dincă & Dempsey, 2006) to establish the contained in the inventory (Albu & Pitariu, 1999). minimum acceptable age of administration. Re- The revision of the Romanian Female/Male scale search with the Freiburger Persönlichkeitsinven- of the California Psychological Inventory also im- tar, Revised (Fahrenberg, Hampel & Selg, 2002) proves the scale (Pitariu, 1981; Albu & Pitariu, correlated data from the Somatic complaints and 1991). The Jackson Vocational Interest Survey Health concerns scales with self-reports and ob- (Jackson, 2000) assessment of vocational inter- jective measures of physical well-being and be- ests has been combined with Holland’s codes. haviours such as medication intake (Pitariu, The Fleishman Job Analysis Survey is used in 1986). This research also demonstrates the value research that examines the development of occu- of using this test for professional selection in the pational profiles in Romania and could easily be- military (Pitariu & Iernuţan, 1984). come the backbone of a national occupational network, mirroring the O*Net used in the United Research with the State-Trait Anger Expression States. Inventory-2 has demonstrated its ability to detect patterns of unsafe and endangering driving be- Issues Associated with Fairness haviours (Tureanu & Iliescu, 2006). Data from the Fairness is an ongoing concern in Romania, es- Survey of Work Styles and the Shortened Stress pecially given the emergence of the profession of Evaluation Tool (Cooper & Cartwright, 2002) pre- psychology. Issues associated with fairness war- dict Type A behaviour pattern (Tureanu, 2006). rant special care when adapting tests and later Data from the Social Axioms Survey has also when selecting and using them. For example, the been validated using cultural anthropological data California Psychological Inventory-462 has been and large-sample analyses (Mihalcea, Dincă & reworked and republished in 1996 as a 434 item Iliescu, 2006). The inclusion of social axioms data version because of legal pressures in light of the increases the prediction of stress coping patterns Americans with Disabilities Act (Gough & Bradley, and job effectiveness when coupled with value 1996). However, the test infringes only marginally graphic research (Gheorghe, Dincă & Târziu, in light of Romanian law. Nevertheless, the 462 2001). Mountains of test data await analysis and item version of the California Psychological Inven- publication. TestCentral conceptualizes validation tory was dropped, even though this is the most studies in terms of long-term perspectives. One researched version of the famous measure in Ro- year ago it established its own panel of subjects mania. The attempt to maximize fairness when on whom longitudinal research is being con- using psychological and educational tests repre- ducted. Data from this panel of about 250 indi- sents a main goal when adapting, developing, se- viduals are used to examine the co-occurrence of lecting, and using tests. Due to the large number personality traits assessed concurrently by two or of the Hungarian ethnic minority within Romania more personality tests as well as measures of (>7% of the population), TestCentral has begun various behavioural outcomes and indicators translating tests reports and other test materials in (e.g., health and health-related behaviours such Hungarian. Supplementary special norms on spe- as coronary heart disease and Type A behaviour cial samples for this ethnic population are under pattern; smoking; medication ingestion; frequency development. of medical visits; frequency and duration of hospi- talization and therapy, including psychotherapy Conclusions and counselling), safety in various environments National test publishers can take an active role in and situations, work and work-related behaviours, the development of psychology within their coun- including tenure and work performance, family try. This is occurring in Romania. Although some and social relationships, quality of life, and aca- may believe locally developed measures may bet- demic achievement. ter fit Romanian culture, evidence to support this belief is sparse in countries in which test develop- Further Test Development Efforts ment is being born. Proof of a better fit must be At times, research needs require one to not sim- based on comparisons with existing and widely ply adapt and norm tests and instead to make ma- used measures. Judgements as to whether new jor changes, including the development of new locally developed measures are superior to adap- scales that best reflect Romanian culture and tation of widely used measures should be based needs. This work occurs with the permission and on research findings. Tests that are internationally assistance of the original test authors. Some ex- (Continued on page 10) 9

renowned constitute a resource that, when properly Gough, H. G. (1987). California Psychological Inventory administra- adapted, normed, and validated, serve the interests tor's guide. Palo Alto, CA: Consulting Psychologists Press. of the Romanian people and those who serve them, Gough, H. G., & Bradley, P. (1996). California Psychological Inventory including Romanian psychologists. manual (3rd ed.). Palo Alto, CA: Consulting Psychologists Press. Iliescu, D. (2006, May). Traducerea, adaptarea, etalonarea si vali- darea FPI-R în România [Translation, adaptation norming and valida- The development of indigenous measures at this tion of the FPI-R in Romania]. Presented at the National Conference point in Romania’s history would require the devel- of Psychology, organized by the Romanian Psychological Association, opment of international perspectives and technol- Cluj-Napoca. ogy, qualities that may be in somewhat short supply. Iliescu, D., Dincă, M., & Dempsey, A. (2006, July). Challenges in the In addition, measures developed there are unlikely Romanian indigenization and norming of the Student Style Question- to find acceptance among the international commu- naire. Presented at the Annual Conference of the International Test nity. The example of other European countries such Comission (ITC), Bruxelles, Belgium. as in Germany where the Freiburger Persönlich- Iliescu, D., Nedelcea, C., & Minulescu, M. (2006a). Noi alternative în keitsinventar (Fahrenberg, Hampel & Selg, 1970) evaluarea personalităţii: NPQ – Chestionarul Nonverbal de Personalitate [New alternatives in personality assessment: The was developed in the early 1970s, or in Italy where Nonverbal Personality Questionnaire], Adaptare şi etalonare la the Big Five Questionnaire (Caprara, Barbanelli & populaţia României. Psihologia Resurselor Umane, 11 (4), 49-61. Borgogni, 1993) was developed in the late 1980s Jackson, D. (2000). Manual for the Jackson Vocational Interest Sur- are striking: both countries had at least 20 years of vey. London, ON: Sigma Assessment Systems. valuable research with validated international meas- Jackson, D. N., & Mavrogiannis-Gray, A. (1988). Manual for the Sur- ures behind them—conditions that lead to the devel- vey of Work Styles. London, ON: Sigma Assessment Systems. opment of measures that gained international ac- Lee, K., Ashton, M. C., Sungjin, H., & Park, K. B. (2000). Psychomet- ceptance and acknowledgement and thus were able ric properties of the Nonverbal Personality Questionnaire in Korea. to further spark the development of indigenous psy- Educational and psychological measurement, 60 (1), 131-141. chological research in psychometrics at an effective Leung, K., & Bond, M. H. (1989). On the empirical identification of dimensions for cross cultural comparisons. Journal of Cross Cultural world-class level. Although Romania is not yet at Psychology, 20, 133 151. this level, it aspires to achieve this in the foresee- Mihalcea, A., Dinca, M., & Iliescu, D. (2006). Axiomele sociale: studiu able future. pe un eşantion naţional [Social axioms: Study on a national sample]. Presented at the National Conference of Psychology, organized by References the Romanian Psychological Association, Cluj-Napoca, May 2006. Albu, M. & Pitariu, H. (1991). Algoritm de construire a unei scale pen- Nedelcea, C., Iliescu, D., & Minulescu, M. (2006b, May). Măsurarea tru un test psihologic: Contributii la reproiectarea scalei F/M a Inventa- personalităţii prin chestionare nonverbale. Experimentarea NPQ şi FF rului Psihologic California (CPI) [An algorithm for constructing a psy- NPQ în România [Personality assessment through nonverbal ques- chological scale: Some contributions to a revision of the F/M scale of tionnaires, Experimentation of the NPQ and FFNPQ in Romania]. the California Psychological Inventory]. Psychologia-Paedagogia, Presented at the National Conference of Psychology, organized by Universitatea Babes-Bolyai, Cluj-Napoca, Romania, 36, (2), 30-35. the Romanian Psychological Association, Cluj-Napoca, Romania. Albu, M. & Pitariu, H. D. (1999). Evaluarea anxietatii cu ajutorul Inven- Oakland, R., Glutting, J., & Horton, C. (1996). Manual for the Student tarului Psihologic California (CPI) [Assessment of anxiety with the Style Questionnaire. San Antonio, TX: Harcourt Assessment. California Psychological Inventory]. Studii de Psihologie, 4, 19-32. Paunonen, S. V., Jackson, D., & Ashton, M. C. (2002). Manual for the Caprara, G. V., Barbanelli, C., & Borgogni, L. (1993). Manual for the Nonverbal Personality Questionnaire (NPQ) and the Five-Factor Non- Big Five Questionnaire. Firenze, Italy: Organizzazioni Speciali. verbal Personality Questionnaire (FF-NPQ). London, ON: Sigma As- Cooper, C. L., & Cartwright, S. (2002). ASSET Management Guide. sessment Systems. Manchester, UK: Robertson Cooper. Pitariu, H. & Iliescu, D. (2004). Inventarul Psihologic California – Costa, P. T. Jr., & McCrae, R. R. (1992). Revised NEO Personality CPI260-Ro [The CPI-260 Inventory]. Psihologia Resurselor Umane, 5 Inventory (NEOPI-R) and NEO Five-Factor Inventory (NEO-FFI) pro- (2), 40-49. fessional manual. Odessa, FL: Psychological Assessment Resources. Pitariu, H. (1981). Validation of the CPI femininity scale in Romania. David, D., Moore, M. & Domuta, A. (2002). Romanian Psychology on Journal of Cross-Cultural Psychology, 12, 111-117. the International Psychological Scene: A Preliminary Critical and Em- Pitariu, H. (1986). Adaptarea si experimentarea inventarului de per- pirical Appraisal. European Psychologist, 7 (2), 153-160. sonalitate Freiburg (FPI) [Adaptation of and experimentation of the Dincă, M. & Iliescu, D. (2006, July). Validation of the Romanian ver- Freiburg Personality Inventory]. Revista de Psihologie, Bucharest, sion of the Student Style Questionnaire. Presented at the Annual Con- Romania, 32, 282-293. ference of the International Association of Applied Psychology (IAAP), Pitariu, H. D. & Iernuţan, L. (1984). Utilizarea inventarului de Athens, Greece. personalitate Freiburg (FPI) in investigarea capacitatii de adaptare la Dincă, M., Iliescu, D. & Mihalcea, A. (2006, July). A map of social viata militara [Using the Freiburg Personality Inventory for the axioms in Romanian culture. Presented at the 18 Congress of Cross investigation of adaptation to the military life]. Revista sanitara Cultural Psychology, Spetses, Greece. militara, 1 (1), 47-55. Fahrenberg, J., Hampel, R., & Selg, H. (1970). Das Freiburger Pitariu, H. D., Pitariu, H. A., & Ali Al Mutairi, M. (1998, August). Persönlichkeitsinventar. Göttingen, Germany: Hogrefe. Psychological assessment of managers in Romania: Validation of a Fahrenberg, J., Hampel, R., & Selg, H. (2002). Das Freiburger test battery. Presented at the meeting of the American Psychological Persönlichkeitsinventar FPI-R: Revidierte Fassung. Göttingen, Association, San Francisco, CA. Germany: Hogrefe. Spielberger, C. D. (1983). Manual for the State Trait Anxiety Inven- Gheorghe, M., Dincă, M., & Târziu, R. (2001, July). Social axioms tory. Palo Alto, CA: Consulting Psychologists Press in the Romanian culture. Presented at the Second Convention of the Spielberger, C. D. (1988, 1996). Manual for the State-Trait Anger International Association for Cross-Cultural Psychology (IAACP), Win- Expression Inventory. Odessa, FL: Psychological Assessment Re- chester, UK. 10sources.

www.intestcom.org

The International Test Commission – Join us?

If you are reading this newsletter and are not a member, why not join the ITC?

The International Test Commission is an organization with a variety of members. Its full members include most of the major national psychological associations; its affiliate members include test publishers and developers, and its individual members include researchers, practitioners and test developers. The ITC has become most widely known for its Guidelines in the areas of test adaptation, test use and computer-based testing, and for its international conference every two years. The fifth such conference, held in Brussels in 2006, focused on test adaptation.

The Sixth ITC Conference will be held in Liverpool, England, 14th-16th July 2008. Its focus is on the impact of testing on people and society. The conference, with the theme “Enhancing the value of test use” will bring together researchers, educators, psychologists, policy experts, testing specialists and those who use tests on a regular basis to discuss the impact of testing on people and society. Topics to be addressed include methodological, technical, professional and ethical issues in relation to policy and practice issues within testing, and will include talks on the following: the value and utility of testing, policy and practice issues in testing, advances in testing, the availability and use of tests in small and developing countries and the impact of tests and testing on people and society. As in the past, the conference will include invited lectures, workshops, symposia, paper sessions and posters and will cover the areas of work, health, leisure and educational contexts.

ITC Conference 2008: The Public Face of Testing 14-16th July 2008, Liverpool, UK

ITC NEWSLETTER: EDITOR JAN BOGG [email protected]