A Study of the Relationship Between Epistemic Style and Evaluation Practice
Total Page:16
File Type:pdf, Size:1020Kb
Western Michigan University ScholarWorks at WMU Dissertations Graduate College 4-2000 A Study of the Relationship Between Epistemic Style and Evaluation Practice Deanna Draze Western Michigan University Follow this and additional works at: https://scholarworks.wmich.edu/dissertations Part of the Educational Assessment, Evaluation, and Research Commons Recommended Citation Draze, Deanna, "A Study of the Relationship Between Epistemic Style and Evaluation Practice" (2000). Dissertations. 1447. https://scholarworks.wmich.edu/dissertations/1447 This Dissertation-Open Access is brought to you for free and open access by the Graduate College at ScholarWorks at WMU. It has been accepted for inclusion in Dissertations by an authorized administrator of ScholarWorks at WMU. For more information, please contact [email protected]. A STUDY OF THE RELATIONSHIP BETWEEN EPISTEMIC STYLE AND EVALUATION PRACTICE by Deanna Draze A Dissertation Submitted to the Faculty of The Graduate College in partial fulfillment of the requirements for the Degree of Doctor of Philosophy Department of Educational Studies Western Michigan University Kalamazoo, Michigan April 2000 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission A STUDY OF THE RELATIONSHIP BETWEEN EPISTEMIC STYLE AND EVALUATION PRACTICE Deanna Draze, Ph.D. Western Michigan University, 2000 The purpose of this study was to examine the differences between personal epistemic styles of professional evaluators, the influence those styles have on evaluation practice, and to determine whether a typology of evaluation practice can be extrapolated from a study of epistemic style. Epistemic style was defined as how one determines what is true (or what constitutes knowledge). The intent was to provide empirical evidence of the relationship between theory and practice in the hope that it will contribute to an increased awareness of the evaluator “as an instrument’' through which data is filtered. The Psycho-Epistemological Profile (Royce, Mos, and Learsley, 1975) was selected to measure dominant epistemic styles among professional evaluators. A reliability study of the instrument was conducted using a sample of evaluators from the Michigan Association of Evaluators. From this group, three evaluators were chosen for semi-structured interviews from each of the dominant epistemic styles represented in the sample. The represented epistemic styles were rational, metaphoric, and eclectic (those evaluators not having a dominant epistemic style). Content analysis of the interviews revealed that the major differences between dominant epistemic styles were not so much in how evaluators determined purpose or use of evaluation but more in the processes involved in evaluation. These processes were (1) roles that evaluators, clients, and stakeholders play and (2) how analysis of Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. data was implemented. There were also differences in theoretical influences among epistemic styles. From the interview data, a typology of evaluation practice was extrapolated. The typology was organized around four themes: (1) theory of action, (2) attitudes toward data, (3) focus on process and outcomes, and (4) theory and practice. The major difference was that the Metaphorics acted as change agents, the Rationals as educators, and the Eclectics as facilitators of improvement. All of these are related to a primary purpose of evaluation chosen by the three styles: to enable program staff to make changes that improve program effectiveness. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. INFORMATION TO USERS This manuscript has been reproduced from the microfilm master. UMI films the text directly from the original or copy submitted. Thus, some thesis and dissertation copies are in typewriter face, while others may be from any type of computer printer. The quality of this reproduction is dependent upon the quality of the copy submitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleedthrough, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send UMI a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note will indicate the deletion. Oversize materials (e.g., maps, drawings, charts) are reproduced by sectioning the original, beginning at the upper left-hand comer and continuing from left to right in equal sections with small overlaps. Photographs included in the original manuscript have been reproduced xerographically in this copy. Higher quality 6" x 9” black and white photographic prints are available for any photographs or illustrations appearing in this copy for an additional charge. Contact UMI directly to order. Bell & Howell Information and Learning 300 North Zeeb Road, Ann Arbor, Ml 48106-1346 USA 800-521-0600 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. UMI Number: 9963063 Copyright 2000 by Draze, Deanna All rights reserved. UMI UMI Microform9963063 Copyright 2000 by Bell & Howell Information and Learning Company. All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code. Bell & Howell Information and Learning Company 300 North Zeeb Road P.O. Box 1346 Ann Arbor, Ml 48106-1346 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Copyright by Deanna Draze 2000 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. ACKNOWLEDGMENTS The writing of my dissertation was not a solo event. It involved many others in terms of support, encouragement, love, and direction. I thank my husband, Jim for his support, love, and advice. I appreciate all the hours he patiently listened to my ideas, giving me feedback when it was needed. I thank my friends Cynthia H., Cynthia P., Sharon, Kate, Lorraine, and Chialin for their enthusiastic encouragement. I thank my colleagues at SAMPI who supported my work. I thank my committee members Dr. Jianping Shen, Dr. Van Cooley, and Dr. Susan Edgerton for their assistance and encouragement. I am grateful to Dr. Uldis Smidchens, for his guidance and encouragement throughout my graduate years at Western Michigan University. Each of these persons played an important role in helping me accomplish my goal. “Little acts of kindness which we render to each other in everyday life, are like flowers by the wayside to the traveler: they serve to gladden the heart and relieve the tedium of life’s journey.” Eunice Bathrick Deanna Draze ii Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. TABLE OF CONTENTS ACKNOWLEDGMENTS....................................................................................... ii LIST OF TABLES................................................................................................... ix LIST OF FIGURES.................................................................................................. x CHAPTER I. INTRODUCTION....................................................................................... 1 Statement of the Problem..................................................................... 1 Theories of Knowledge and Evaluation Practice........................ 1 Influences on Research and Evaluation Practice........................ 1 Epistemic Style............................................................................. 2 Influence of Epistemic Style........................................................ 2 Epistemic Style and Evaluation.................................................. 3 Rationale for the Study........................................................................ 5 Epistemic Style as a Foundation for Evaluation Practice 5 Theory-Laden Observations........................................................ 5 Consciousness of Personal Theories........................................... 6 Lack of Empirical Evidence........................................................ 6 Demystifying of Theoretical Orientations................................... 7 Personal Framework of the Study....................................................... 8 Organization of the Study.................................................................... 8 II. LITERATURE REVIEW.................................................................................. 10 Organization of the Review................................................................. 10 iii Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Table of Contents—continued CHAPTER Influences on Evaluation Practice....................................................... 10 Intervening Variables................................................................... 10 Theoretical Orientations............................................................... 10 Political Context............................................................................ 12 Culture of Organization and Community.................................... 13 Time and Budget............................................................