September 18 – 19, 2019 WELCOME FROM THE PROGRAM CHAIRS Welcome to Toronto! We’re so happy you’re joining us for the 2019 NCME Annual Meeting. We’ve collaborated with many of you to prepare a program that we hope offers opportunities to learn, grow, connect, and celebrate some incredible work and achievements in the field. This year’s conference theme of “Communicating with the Public about Educational Measurement” allowed us to place some well-deserved focus within the program on how to more broadly communicate with the world about our work, how to present our work in clearer and more compelling ways, and how to reach diverse audiences. We still have many technical sessions, but we also have invited sessions such as Communicating/Depicting Results in Easily Accessible Ways, Across Broad Audiences (Saturday, 12:20pm); Appropriately Interpreting, Comparing, and Communicating Results from International Assessments: Challenges and Opportunities (Monday, 8am); and The Influence of Stakeholder Needs and Values on Assessment Design and Reporting (Sunday, 3:20pm). Of particular note, NCME President Rebecca Zwick has organized a special session for attendees, Communicating Your Research to the Media, with Emily Richmond (public editor of the Education Writers Association) and Holly Yettick (director of the Education Week Research Center), who will share the perspective of editors and journalists combing through various studies and press releases to determine what is newsworthy and describe the process from pitch to published story. This session will be on Saturday at 10:25am. Another timely and relevant invited session being offered is Using the ACT and SAT for Accountability Under ESSA: Appropriate or Inappropriate Use (Sunday, 5pm). The 2019 Annual Meeting program features 22 training sessions, 50 coordinated sessions, 7 Electronic Board sessions (2 of which are Graduate Student sessions), and over 50 individual paper sessions. Another conference highlight will certainly be the featured session for NCME’s Committee on Diversity in Testing, Equity-Centered Design in Assessment (Monday, 10:25am). And we have a session devoted to recognizing the particular contributions of women to the measurement field, Women in Measurement: Their Unique Contributions (Saturday, 8am). In addition, the 2018 NCME Career Award winner, Brian Clauser, will present his research in a special session on Monday at 12:20pm, and the 2019 NCME Career Award winner, Shelby Haberman, will present his research in a special session on Sunday at 3:20pm. The 2019 NCME Award Winners will present their award-winning research on Saturday at 8am. There will also be a session focused on a new NCME award, Excellence in Public Communications, that you will want to check out on Saturday at 4:10pm. If that’s not enough, there’s yoga on Saturday morning at 6:30am, receptions every night, the popular NCME Breakfast and Presidential Address from NCME President Rebecca Zwick on Sunday morning, and the NCME Fitness Run/Walk first thing on Monday. We must acknowledge the incredible and talented cadre of NCME members and colleagues who generously volunteered their time and expertise this year to ensure that we will have a high-quality program in Toronto. So many of you reviewed proposals and provided critically helpful feedback; and so many of you volunteered and are serving as discussants for the program. We can’t thank you enough! We are also grateful to Rebecca Zwick, our NCME President; Jim Roberts, the Training & Professional Development Committee Chair, and his committee; Kevin Krost, the Graduate Student Committee Chair, and his committee; and Nikole Gregg for their work on the program this year. We are so excited the conference is here! Please enjoy! Krista Mattern and Emily Shaw 2019 NCME Annual Meeting Co-Chairs Toronto, Ontario, Canada Table of Contents NCME Board of Directors .............................................................4 Proposal Reviewers ...................................................................6 Future Meetings ......................................................................7 Floor Plans ...........................................................................8 Training Sessions Thursday, April 4 ....................................................................12 Friday, April 5 .......................................................................21 Program Saturday, April 6 .....................................................................34 Sunday, April 7 ......................................................................91 Monday, April 8 ....................................................................129 Index ..............................................................................183 Schedule-at-a-Glance ..............................................................194 3 2019 Annual Meeting & Training Sessions NCME Officers President Rebecca Zwick Educational Testing Service President-Elect Stephen G. Sireci University of Massachusetts Amherst Past President Randy Bennett Educational Testing Service NCME Directors Derek Briggs Rose McCallin University of Colorado Colorado Department of Regulatory Agencies Debbie Durrence Ye Tong Gwinnett County Public Schools Pearson Andrew Ho Walter Way Harvard University The College Board Editors Journal of Educational George Engelhard Measurement University of Georgia Jonathan Templin University of Iowa Sandip Sinharay (incoming) Educational Testing Service Educational Measurement: Howard Everson Issues and Practice SRI International and City University of New York ITEMS Editor Andre Rupp Educational Testing Service NCME Book Series Editor Brian Clauser National Board of Medical Examiners NCME Newsletter Editor Megan Welsh University of California – Davis NCME Website Editors Matthew Gaertner WestEd Brian Leventhal James Madison University 4 Toronto, Ontario, Canada 2019 Annual Meeting Chairs Annual Meeting Program Chairs Emily Shaw The College Board Krista Mattern ACT, Inc. Graduate Student Issues Kevin Krost Committee Chair Virginia Polytechnic Institute Training and Professional Development James Roberts Committee Chair Georgia Institute of Technology Fitness Run/Walk Directors Jill R. van den Heuvel, Ph.D. Alpine Testing Solutions Katherine Furgol Castellano, Ph.D. Educational Testing Service Brian F. French, Ph.D. Washington State University NCME Information Desk The NCME Information desk is located in the Concert Hall Foyer at the Fairmont Royal York. It will be open at the following times: Thursday, April 4 ............................... 7:30am – 4:30pm Friday, April 5 ...................................7:30am– 4:30pm Saturday, April 6 ............................... 8:00am – 4:30pm Sunday, April 7 ................................10:00am – 4:30pm Monday, April 8 ................................ 8:00am – 1:00pm 5 2019 Annual Meeting & Training Sessions Proposal Reviewers Terry Ackerman Fen Fan Leslie Keng Gerald Melican Bercem Akbayin Meichu Fan Eunhee Keum Yu Meng Usama Ali Yu Fang Minsung Kim Yeow Meng Thum Jeff Allen Tia Fechter Se-Kang Kim Stefan Merchant Cristina Anguiano-Carrasco Leah Feuerstahler Seock-Ho Kim Rochelle Michel Alvaro Arce Anthony Fina Young Yee Kim M. Miller Meirav Arieli-Attali Holmes Finch Tim Konold Scott Monroe Ben Babcock Steven Fitzpatrick Jason Kopp Melinda Montgomery Erin Banjanovic Kelly Foelber Patrick Kyllonen Joann Moore Patricia Baron Jean-Paul Fox Emily Lai Kristin Morrison Luz Bay John Fremer Hollis Lai Eric Moyer Jonathan Beard Brian French Joni Lakin James Olsen Kirk Becker Hirotaka Fukuhara Erika Landl Insu Paek Beata Beigman Klebanov Matthew Gaertner Quinn Lathrop Seohong Pak Isaac Bejar Jennifer Galindo Chansoon (Danielle) Lee Tianshu Pan Aarti Bellara Xiaohong Gao Brian Leventhal Richard Patz Dmitry Belov Tracy Gardner Daniel Lewis John Poggio Michelle Boyer Joshua Goodman Jie Li Cornelis Potgieter Nurliyana Bukhari Irina Grabovsky Shuhong Li Sonya Powers Heather Buzick Edith Graf Tongyun Li Jiahe Qian Wayne Camara Raman Grover Yuan-Ling Liaw Justine Radunzel Kevin Cappaert Lixiong Gu Chunyan Liu Heather Rickels Michael Chajewski Kyung (Chris) T. Han Jinghua Liu Michael Rodriguez Jyun-Hong Chen Qiwei Britt He Junhui Liu Jonathan Rubright Yi-Hsin Chen Yong He Yuming Liu André Rupp Edison Choe Dianne Henderson Samuel Livingston Michael Russell Youn-Jeng Choi Amy Hendrickson John Lockwood Leslie Rutkowski Man-Wai Chu Dolores Hidalgo Susan Lottridge Edgar Sanchez Gregory Cizek TsungHan Ho Ru Lu Edynn Sato Amy Clark Steven Holtzman Ying Lu Amy Schmidt Kimberly Colvin Likun Hou Richard Luecht Matthew Schultz Stephen Cubbellotti Chia-Ling Hsu Xiao Luo Bernard Schuster Zhongmin Cui Anne Corinne Yong Luo Carl Setzer Shenghai Dai Huggins-Manley Wenchao Ma S. Kanageswari Laurie Davis Charles Hunter Katerina Marcoulides Shanmugam R. De Ayala Sukkeun Im Jessica Marini Can Shao Robert Dedrick William Insko Scott Marion Benjamin Shear Christine DeMars Paul Jewsbury Kimberly Marsh Mark Shermis John Denbleyker Yue Jia Jose Felipe Martinez Bruce Shotts Ben Domingue Zhehan Jiang Martha McCall Sandip Sinharay John Donoghue Kwanghee Jung Rose McCallin William Skorupski Bryan Drost Pamela Kaliski Catherine McClellan Whitney Smiley Carol Eckerly Priya Kannan Bradley McMillen Jessalyn Smith Howard Everson Shu-Chuan Kao Maria Medina-Diaz Hao Song Maureen Ewing Tzur Karelitz Rob Meijer Dorota Staniewska 6 Toronto, Ontario, Canada Proposal Reviewers (continued) Jeffrey Steedle Jill van den Heuvel Saskia Wools April Zenisky Dubravka Svetina Michael