LTRC 2016 Exe Abb.Indd 1 03/06/16 15:42 More Than Four Decades of Research Supports the Validity of the TOEFL® Family of Assessments
Total Page:16
File Type:pdf, Size:1020Kb
38th Annual Language Testing Research Colloquium June 20-24, 2016 Palermo, Sicily (Italy) LANGUAGE CONSTRUCTS, CONTEXTS, AND LTRC CONTENT IN CLASSROOM AND 2016 LARGE-SCALE ASSESSMENTS libro_convegno_LTRC 2016_exe_abb.indd 1 03/06/16 15:42 More than four decades of research supports the validity of the TOEFL® Family of Assessments Read the latest journal articles and reports on the TOEFL® Family of Assessments. Analyzing and Comparing Reading Stimulus Materials Across the TOEFL® Family of Assessments (2015). Chen, J., & Sheehan, K. M. TOEFL iBT Research Report No. 26. Princeton, NJ: ETS. Comparing Writing Performance in TOEFL iBT ® and Academic Assignments: An Exploration of Textual Features. (2016). Riazi, A. M. Assessing Writing, 28, 15–27. Designing the TOEFL® Primary™ Tests. (2016). Cho, Y., Ginsburgh, M., Morgan, R., Moulder, B., Xi, X., & Hauck, M. C. Research Memorandum No. RM-16-02. Princeton, NJ: ETS. Different Topics, Different Discourse: Relationships Among Writing Topic, Measures of Syntactic Complexity, and Judgments of Writing Quality. (2015). Yang, W., Lu, X., & Weigle, S. Journal of Second Language Writing, 28, 53–67. Effects of Strength of Accent on an L2 Interactive Lecture Listening Comprehension Test. (In press). Ockey, G. J., Papageorgiou, S., & French, R. International Journal of Listening. Enhancing the Interpretability of the Overall Results of an International Test of English-language Proficiency. (2015). Papageorgiou, S., Morgan, R., & Becker, V. International Journal of Testing, 15, 310–336. Evaluating the TOEFL Junior® Standard Test as a Measure of Progress for Young English Language Learners. (2015). Gu, L., Lockwood, J., & Powers, D. E. (2015). ETS Research Report RR-15-22. Princeton, NJ: ETS. Facilitating the Interpretation of English Language Proficiency Scores: Combining Scale Anchoring and Test Score Mapping Methodologies. (2016). Powers, D., Schedl, M., & Papageorgiou, S. (2016). Language Testing. (First view online) Predicting Grades from an English Language Assessment: The Importance of Peeling the Onion. (2015). Bridgeman, B., Cho, Y., & DiPietro, S. Language Testing. (First view online) Test Takers’ Writing Activities During TOEFL iBT ® Writing Tasks: A Stimulated Recall Study. (2015). Barkaoui, K. TOEFL iBT Report No. TOEFL iBT-22. Princeton, NJ: ETS. http://www.ets.org/toefl/research ETS —Measuring the Power of Learning.™ Copyright © 2016 by Educational Testing Service. All rights reserved. ETS, the ETS logo, TOEFL, TOEFL IBT and TOEFL JUNIOR are registered trademarks of Educational Testing Service (ETS) in the United States and other countries. MEASURING THE POWER OF LEARNING and TOEFL PRIMARY are trademarks of ETS. 33893 libro_convegno_LTRC 2016_exe_abb.indd 2 03/06/16 15:42 38th Annual Language Testing Research Colloquium LANGUAGE CONSTRUCTS, CONTEXTS, AND LTRC CONTENT IN CLASSROOM AND 2016 LARGE-SCALE ASSESSMENTS libro_convegno_LTRC 2016_exe_abb.indd 1 03/06/16 15:42 Piazza Marina, 61 90133 - PALERMO www.unipa.it Pubblicazione realizzata in occasione del 38th Annual Language Testing Research Colloquium Palermo, 20-24 Giugno 2016 Realizzazione e coordinamento editoriale: Siena Italtech S.r.l. Progetto grafico e impaginazione: Ol3 S.r.l. - Perugia Stampa e legatura: GPT S.r.l. Finito di stampare nel mese di Giugno 2016 da GPT S.r.l. - Città di Castello (Pg) libro_convegno_LTRC 2016_exe_abb.indd 2 03/06/16 15:42 LTRC 2016 TABLE OF CONTENTS Welcome Message from the President 5 Welcome Message from the Chairs 6 The Samuel J. Messick Memorial Lecture 8 The Davies Lecture 10 The Cambridge/ILTA Distinguished Achievement Award 12 Conference Support 14 ILTA 2016 15 Award Winners 16 Colloquium Venues 18 LTRC 2016 Conference Overview 21 LTRC 2016 Conference Schedule 25 Pre-Conference Workshop Abstracts 39 Paper Abstracts: Tue, June 21 49 Paper Abstracts: Wed, June 22 51 Language Constructs, Contexts, and Content in Classroom 3 and Large-Scale Assessments libro_convegno_LTRC 2016_exe_abb.indd 3 03/06/16 15:42 LTRC 2016 Paper Abstracts: Thu, June 23 68 Paper Abstracts: Fri, June 24 86 Symposium Abstracts 103 Poster Abstracts 119 Work-in-Progress Abstracts 145 Demo Abstracts 167 Index of Presenters 178 Language Constructs, Contexts, and Content in Classroom 4 and Large-Scale Assessments libro_convegno_LTRC 2016_exe_abb.indd 4 03/06/16 15:42 LTRC 2016 Tony Green himself, who passed away earlier this year, a Welcome ILTA President founding father of ILTA and an inspirational teacher, mentor and friend to many of us. Message In total, we received 368 submissions. At from the the time of writing, the programme includes forty-five research papers, four symposia, President twenty-three posters and twenty works- Welcome to in-progress. Palermo sees the introduction LTRC 2016 of an innovative presentation format: the demo. Reflecting the growing role of Benvenuti in Sicilia for the 38th Language new technology in language assessment Testing Research Colloquium on the theme of practices, the nine demos in Palermo Language Constructs, Contexts, and Content profile an intriguing range of software tools in Classroom and Large-Scale Assessments. with the potential to improve assessment This year, the context, at the crossroads of practice. Directly before the colloquium, five the Mediterranean world, is, appropriately workshops are featured, one of which will be enough, the island home of Archimedes: the conducted in Italian—the second time (after grandfather of measurement. The attractions Korea) that an LTRC workshop is given in a of the hospitable climate, cosmopolitan language other than English. Sicilian cuisine, and Palermo world heritage site surrounding us at the conference will Of course, constructing the context for and surely only be displaced by the volume of content of a large-scale conference is a feat eureka moments inside the conference hall! of engineering that would tax even the great Archimedes himself. We owe special thanks It is an encouraging sign of the growing reach to our hosts at the University of Palermo, the of our organisation that we find ourselves in LTRC co-chairs: James Purpura and Monica Italy for the first time and that next year we Barni and to the local chairs, Carla Bagna and will cross the Atlantic to another exciting new Mari D’Agostino. Thanks are also due to the location: Bogotà in Colombia. other members of the organizing committee: Veronica Benigno, Jee Wha Dakin, Barbara This year’s conference theme expresses Dobson, Francesca Gallina, Kirby Grabowski, the broadening interests of ILTA members Giuliana Grego Bolli, Heidi Han-Ting Liu, with increased attention to the place of Lucilla Lopriore, Sabrina Machetti, Paola assessment in the classroom, alternative Masillo, Saerhim Oh, and Lorenzo Rocca. intellectual perspectives on theory and In addition, we are grateful to our student practice, and the importance of preserving volunteers. And finally, thanks to Erik Voss, and reflecting on the history of the who has made an invaluable contribution as development of our field. webmaster. Our three featured speakers all touch Whether you are an ILTA veteran or this is on these themes. James Pellegrino (The your first time at LTRC, I do hope you will find Samuel J. Messick Memorial Lecture inspiration and pleasure in Palermo in equal Award) discusses instructionally relevant measure. assessments, John de Jong (ILTA/Cambridge Distinguished Achievement Award) looks Best wishes, back over his career and forward to a new global standard, and John Norris (The Tony Green, Alan Davies Lecture) explores the familiar ILTA President question of the interface with SLA research through new interdisciplinary perspectives. The conference also gives us an opportunity to share our memories of Alan Davies Language Constructs, Contexts, and Content in Classroom 5 and Large-Scale Assessments libro_convegno_LTRC 2016_exe_abb.indd 5 03/06/16 15:42 LTRC 2016 Welcome Jim Purpura Monica Barni Teachers College, University Message Columbia University for Foreigners from the of Siena Chairs Welcome to producing dubious information, or debates LTRC 2016 in Palermo showing an awareness that tests can be used effectively in decision-making or As the co-chairs of LTRC 2016, we would misused for social, cultural, economic, or like to warmly welcome you to Palermo political purposes. (Palermu in Sicilian; Panormus in Latin; Panormos in Greek; Balarm in Arabic; During our Bologna conversation we also and Ziz—flower, in Phoenician). The idea tried to find a location for the conference. to take LTRC to Italy, and specifically to Naturally, all the Italians proposed Palermo, was hatched over a beautiful their own university or city as the best! lunch, attended by Giuliana Grego Bolli However, this restrained volcano soon led, (Perugia), Lucilla Lopriore (Rome), Marc fortunately, to a spontaneous proposal to Silver (Modena), and us (Monica – Siena take LTRC to the island of Sicily, famous and Jim - NYC of Sicilian parents) during a for her architecture, bearing witness to the language testing meeting in Bologna back aesthetics of the Phonetician, Carthaginian, in 2012. The semi-innocent question, “Why Greek, Roman, Byzantine, Arab, Norman, not an LTRC in Italy” spawned a very lively German, French, and Spanish cultural discussion, made probably even livelier kaleidoscope; also famous for her by a glass (or more) of a dry, ruby-red multilayered cultural tapestry, emotionally Sangiovese. The Italians all agreed that held distinct after three thousand years it was very timely to have LTRC in Italy, under the