University of Massachusetts Amherst ScholarWorks@UMass Amherst Doctoral Dissertations 1896 - February 2014 1-1-1970 The operationalization of educational objectives for the evaluation of an ongoing program. Leon Jones University of Massachusetts Amherst Follow this and additional works at: https://scholarworks.umass.edu/dissertations_1 Recommended Citation Jones, Leon, "The operationalization of educational objectives for the evaluation of an ongoing program." (1970). Doctoral Dissertations 1896 - February 2014. 2527. https://scholarworks.umass.edu/dissertations_1/2527 This Open Access Dissertation is brought to you for free and open access by ScholarWorks@UMass Amherst. It has been accepted for inclusion in Doctoral Dissertations 1896 - February 2014 by an authorized administrator of ScholarWorks@UMass Amherst. For more information, please contact [email protected]. THE OPERATIONALIZATION OF EDUCATIONAL OBJECTIVES FOR THE EVALUATION OF AN ONGOING PROGRAM A Dissertation Presented By LEON JONES Submitted to the Graduate School of the University of Massachusetts in partial fulfillment of the requirements for the degree of DOCTOR OF EDUCATION December 1970 Major Subject Educational Evaluation (c) Leon Jones 1971 All Rights Reserved 11 THE OPERATIONALIZATION OF EDUCATIONAL OBJECTIVES FOR THE EVALUATION OF AN ONGOING PROGRAM A Dissertation By LEON JONES Dr. William C. Wolf, Jr. I December 1970 ii i Dedication To My Mother And To the memory of My Father ACKNOWLEDGEMENTS I wish to express my sincere appreciation to Dr. Daniel C. Jordan, Dr. Jimmie C. Fortune, and Dr. Thomas E. Hutchinson for their assistance and guidance in carrying out this thesis work. Without the patience and expertise of these professors, the completion of this thesis would not have been possible. Appreciation is due to Mr. C. Grammaticas, Mr. Ralph Meacham, Mr. Alfonzo Sudentas, and Mr. Leon Kaczman for their assistance in the administration of the field test of the methodology developed. I wish to express special appreciation to my brother, Leander, who rendered unselfishly his editorial talents and moral support. I am greatly indebted to both my wife, Bobbie, for her continuous patience and moral support and daughter, Stephanie, for her sweetness throughout the whole of this endeavor. v TABLE OF CONTENTS Page ACKNOWLEDGEMENTS v LLST OF TABLES ix ABSTRACT x Chapter I. INTRODUCTION 1 Evaluation Background Measurement: Its Relationship to Evaluation Current State of Evaluation Current State of Methodological Research Statement of the Problem Rationale of the Problem II. RELATED LITERATURE 21 The Purpose of Evaluation The Relationship Between Evaluation and Experimental Research III. OPERATIONALIZATION OF FUZZY CONCEPTS: DEVELOPMENT OF AN ASPECT OF EVALUATION METHODOLOGY 39 Introduction Use of a Methodology Transformation of Data into Meaningful Information, As a Process A Way of Identifying Variables Theoretical Treatment Practical Application The Tilings This Methodology Must Accomplish Preservation of the Intent of the Decision-maker at Every Stage of the Process Derivation of the Real Intent of Each Objective Use of Only the Expressly Stated Intent for Evaluative Purposes Identification of the Variables The Range of Relevancy The Domain of Relevancy Quantification of the Variables Critique the Methodology Developed vi Chapter Pape IV. DESIGN OF THE FIELD STUDY . 63 The Importance of Studying Methodology Creation of a Methodology Some Attributes of Methodologies The Import of the Notions "Consistent" and "Complete" Methodological Research Perspective The Methodological Import The Evaluative Import The First Step Conceptual Design of the Field Study Purpose of the Evaluation Design Description of the Design The Number of Objectives Used The Kind of Setting Involved The Kind of Decision-maker Involved The 6H Program Specific Choice of the Decision-maker Cooperation of the Decision-maker Operationalizing the Objectives Designing Observational Techniques V. THE IMPLEMENTATION APPROACH 93 Implementation of the Methodology Introduction of the Methodology to the Decision-maker Establishment of Operational Schedule Operationalization of the Objectives Development of Observational Techniques Implementation of the Observational Techniques Summary of the Implementation of the Methodology VI. INTERPRETATION OF THE OBJECTIVES 107 Interpretation of Short Term and Long Term Type Objectives First Attempt to Operationalize the Objective Second Attempt to Operationalize the Objective Third Attempt to Operationalize the Objective Summary of the Operationalization of the Objective Interpretation of Long Term Objective The Operationalized Components of the Objective Summary of the Operationalization of the Objective Interpretation of Short Term Objective The Operationalized Components of the Objective Summary of the Operationalization of the Objective Interpretation of Non-Anticipated Objectives The Operationalized Components of the Objective Summary of the Operationalization of the Objective vii Chapter Page VII. COLLECTION AND ANALYSIS OF DATA 156 Introduction The First Objective The Second Objective The Third Objective The Fourth Objective Summary Based on the Data Collected Recommendation Based on the Data Collected VIII. CRITIQUE OF THE METHODOLOGY 176 Identification of Quantifiable Variables Completeness of Variable Break Down Statement of Variables in Behavioral Terms Preservation of the Intent of the Decision-maker at Every Stage of the Process Involvement of the Decision-maker Derivation of the Intent of the Objective A Way of Implementing the Methodology Endorsement of the Methodology by the Decision-maker Strategies Used to Implement This Methodology Time Required to Implement This Methodology IX. SUMMARY AND RECOMMENDATIONS 19 7 Summary Recommendations APPENDICES I. EVALUATION DATA INSTRUMENTS 212 218 II. LOG OF FIELD STUDY . 232 REFERENCES . LIST 0 F TABLES Table Page 1 Operationalized Components of Objective No. ] 160 2 Listing of the Present "6H-er's" Component Ratings as They Stand in Relation to the Control Group’s Rating 163 3 Operationalized Components of Objective No. 2 164 A Listing of the Experimental Group's Component Ratings as They Stand in Relation to Their Corres- ponding Control Group's Ratings 166 5 Operationalized Components of Objective No. 3 168 6 Listing of Some of the Present "6H-er's" Component Ratings as They Stand in Relation to a Group of Regular Sixth Graders 169 7 Operationalized Components of Objective No. 4 172 8 Listing of Some of the Present "6H-er's" Component Ratings as They Stand in Relation to a Group of Regular Sixth Graders 173 ix . ABSTRACT Development and Implementation of An Aspect of Evaluation Methodology (December 1970) Leon Jones, B.S., Arkansas A.M.&N. College Directed by: Drs . D. C. Jordan, J. C. Fortune, and T. E. Hutchinson As education and educational involvements have expanded, there has been an accompanying need for new evaluation concepts and stronger evaluation methodology for the new concepts. In fact, the re-emergence of evaluation of the 1960’s continues its upward trend across many educa- tional and social programs. A search of the literature indicates that there is support for the position that whatever else may go on in American education, evaluation will make up a major part of it. In spite of the present evaluation demands, however, there is a gap in existing evaluation methodology The degree to which most educational programs have clearly defined objectives leaves something to be desired. This condition may explain why it is entirely possible for public school evaluations to be meaning- less. These evaluations reflect the confusion of administrators regard- ing educational programs which sometimes may be equally meaningless. Due to the lack of theory and the absence of a tradition of good practice it is also possible that some evaluators fail to fulfill evaluation needs, x conceptually. There may be a relationship among the degree of strength of educational programs, evaluation methodology, and the training pro- vided in institutions for both administrators and evaluators. Ihe authorization and implementation of an evaluation of the 6H program of the West Springfield School System provided an opportunity to demonstrate ways in which school personnel can become aware of how the long range effect of the 6H program compares with its anticipated outcomes. The new piece of methodology should help the superintendent to understand some of the important aspects that may be a consequence of the institutionalization of the 6H program. This type of evaluation gen- erates a need for the measurement of the learning objectives that con- stitute the basis upon which the implementation of the 6H program rests. Ihe 6H program invites the application of measurement methodology to goals or learning objectives aimed at providing certain opportunities for academically talented pupils. The learniiig objectives must be re- stated in operational terms that will provide data that is relevant and meaningful to the people who will, in part, make the decisions that will- affect the program. aspect of I The primary purpose of this research is to develop an evaluation methodology which will demonstrate its worth via applying the methodology in an example situation. Ihe secondary purpose is two-fold. That is, (a) if this methodology is proved to be unworkable, then a con- servative amount of resources would have been expended and (b) if this methodology is proved to be workable, then a rational basis for further research