Toward the Development of a Model to Estimate the Readability of Credentialing-Examination Materials
Total Page:16
File Type:pdf, Size:1020Kb
UNLV Theses, Dissertations, Professional Papers, and Capstones 5-2010 Toward the development of a model to estimate the readability of credentialing-examination materials Barbara Anne Badgett University of Nevada Las Vegas Follow this and additional works at: https://digitalscholarship.unlv.edu/thesesdissertations Part of the Educational Psychology Commons Repository Citation Badgett, Barbara Anne, "Toward the development of a model to estimate the readability of credentialing- examination materials" (2010). UNLV Theses, Dissertations, Professional Papers, and Capstones. 185. http://dx.doi.org/10.34917/1436770 This Dissertation is protected by copyright and/or related rights. It has been brought to you by Digital Scholarship@UNLV with permission from the rights-holder(s). You are free to use this Dissertation in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s) directly, unless additional rights are indicated by a Creative Commons license in the record and/or on the work itself. This Dissertation has been accepted for inclusion in UNLV Theses, Dissertations, Professional Papers, and Capstones by an authorized administrator of Digital Scholarship@UNLV. For more information, please contact [email protected]. TOWARD THE DEVELOPMENT OF A MODEL TO ESTIMATE THE READABILITY OF CREDENTIALING-EXAMINATION MATERIALS by Barbara A. Badgett Bachelor of Science University of Nevada, Las Vegas 2000 Master of Science University of Nevada, Las Vegas 2003 A dissertation submitted in partial fulfillment of the requirements for the Doctor of Philosophy in Educational Psychology Department of Educational Psychology College of Education Graduate College University of Nevada, Las Vegas May 2009 THE GRADUATE COLLEGE We recommend the dissertation prepared under our supervision by Barbara Anne Badgett entitled Toward the Development of a Model to Estimate the Readability of Credentialing-Examination Materials be accepted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Educational Psychology Alice J. Corkill, Committee Chair Gregory Schraw, Committee Member CarolAnne Kardash, Committee Member Mark Ashcraft, Graduate Faculty Representative Ronald Smith, Ph. D., Vice President for Research and Graduate Studies and Dean of the Graduate College May 2010 ii ABSTRACT Toward the Development of a Model to Estimate the Readability of Credentialing-Examination Materials by Barbara A. Badgett Dr. Alice J. Corkill, Examination Committee Chair Professor of Educational Psychology University of Nevada, Las Vegas The purpose of this study was to develop a set of procedures to establish readability, including an equation, that accommodates the multiple-choice item format and occupational-specific language related to credentialing examinations. The procedures and equation should be appropriate for learning materials, examination materials, and occupational materials. To this end, variance in readability estimates accounted for by combinations of semantic and syntactic variables were explored, a method was devised to accommodate occupational-specific vocabulary, and new-model readability formulas were created and calibrated. Existing readability formulas were then recalibrated with the same materials used to calibrate the new-model formulas. The new-model and recalibrated formulas were then applied to sample items extracted from a professional licensing examination and the results were compared. iii ACKNOWLEDGEMENTS I would like to thank Dr. Jack Gerrow Jack D. Gerrow, D. D. S., Marcia A. Boyd, D. D. S., and the National Dental Examination Board of Canada for their continued support. Without the access they provided to curricular and occupational resources as well as examination data, this study would not have been possible. Dr. Boyd’s subject matter expertise proved invaluable. Many thanks also go out to Dr. McCrudden, who directed me to the Buros Center for Testing and introduced me to Dr. Buckendahl. During my internship at Buros, which was granted to me by Dr. Buckendahl, he passed along a research article and said something to the effect of, “Perhaps you can extract an idea for your dissertation from this”. Well, Dr. Buckendahl…I was, in fact, able to find a research idea and thank you so very much for your help. Dr. Buckendahl provided further support by connecting me with the National Dental Examination Board of Canada. Without your support, this investigation would not have been conducted. You pointed me in a direction and when I finally developed a research idea, you provided a means for me to collect the necessary information to bring the idea to full fruition. I am at a loss for words to appropriately express my gratitude for your guidance, support, and patience. I would also like to express my gratitude to my dissertation committee, Gregory Schraw, Ph. D., CarolAnne Kardash, Ph.D., Mark Ashcraft, Ph.D. and Alice J. Corkill, Ph.D., who provided very helpful feedback throughout this process. At the time of the proposal, you offered insight that helped me to better scope the investigation and identified additional steps that were necessary. This resulted in a better-planned study. Thank you for your time and effort. iv Dr. Corkill, my dissertation committee chairperson, advisor, friend, and shepherdess deserves very special thanks. You will never know how much it meant to me that you were willing to sacrifice months of weekends to read, re-read, and provide feedback...only to read, re-read, and provide more feedback. Not many advisors are willing to answer panicked phone calls at all hours of the night to calm the nerves of their hysterical advisees. Thank you for all the encouragement….for assuring me repeatedly that I would eventually succeed. It made such a difference and really helped me keep up my spirits and confidence. What, pray tell, will you do with all of your free time now that you are not inundated with my pestering emails, attachments, phone calls, and text messages? I have learned so much from you over the years...not only about educational psychology…but about life. Thank you for being there...time after time. I would also like to thank Dr. Smith and Dr. Davis for the assistance and support they offered during this process. Dr. Smith, without your macros, I would still be counting words. Dr. Davis, thank you for bouncing around ideas with me early in the process. Finally, I would like to thank my family and friends. Thank you for understanding that I had to go MIA for a while and why I have not “been there”…for understanding why I have missed so many important occasions. Mom, Daddy J, and Dannette, thank you for helping me sort out those silly word lists. It was very helpful. My parents and fiancé have offered the most valued support during this very long process. How many times did you talk me off the ledge? How many times did you listen to me weep and express doubt? You always responded with words of encouragement and helped me realize that I could “do it”. After each of those dozens of conversations, I was able to pick myself up and push forward. I don’t know whether I could have done this v without your love, support, and patience. I don’t know whether I could have done this without knowing that you would be proud, regardless of the outcome. Brian, I especially thank YOU for your patience. Now we can get married!! vi TABLE OF CONTENTS ABSTRACT ....................................................................................................................... iii ACKNOWLEDGEMENTS ............................................................................................... iv TABLE OF CONTENTS .................................................................................................. vii LIST OF TABLES ............................................................................................................. ix CHAPTER 1 INTRODUCTION ................................................................................ 1 Readability ................................................................................................................. 2 Readability in Testing ................................................................................................ 6 Readability of Licensure and Certification Examinations ......................................... 7 CHAPTER 2 LITERATURE REVIEW ................................................................... 13 What is Readability? Definitions and Popular Uses ................................................ 13 Calibration Methods for Readability Formula Development .................................. 15 Measuring Readability ............................................................................................. 33 The Proposed Study ............................................................................................... 114 CHAPTER 3 METHODS ....................................................................................... 120 Variables in the Model ........................................................................................... 120 Formula Calibration ............................................................................................... 125 Phase I: Usefulness of variables ............................................................................ 127 Phase II: Formula creation and calibration ............................................................ 129 Phase III: External