THE DEVELOPMENT OF A GRADUATE ACHIEVEMENT TEST IN THEATRE DISSERTATION Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School of The Ohio State University By BILL G. HULSOPFLE, B. S. in Educ., M. A. The Ohio State University 1958 Approved by: Department of Speech ACKNOWLEDGMENTS I would like to express m y deep appreciation to all those individuals who have helped to make this study possible. This list of individuals includes professors and students from various depart­ ments at The Ohio State University but especially those professors and students in the Department of Speech. Nfor gratitude is extended to Professor John E. Dietrich who, as my adviser, has offered encouragement and suggestions that have profoundly contributed to the progress and completion of this study. I am especially indebted to Dr. Wallace 0. Fotheringham who has patiently and unselfishly given of his time. Without Dr. Fotheringham*s advice and counsel, this study would not have been possible. I would like also to thank my wife, Vivian, who has been a constant source of inspiration, as well as a captive listener and adviser. ii TABLE OF CONTENTS Chapter Page I. SETTING OF THE STUDY ..................... 1 Statement of the purpose of the study • •••••• 1 The importance of a broad, general background in theatre • .••••••••••••••••••• 4 Need for a graduate achievement test in theatre • • 6 Definitions of terms used • ••••••••••.. 8 Organization of the remainder of the study • • • • • 8 II. SELECTION OF METHODOLOGY TO BE USED .......... 10 Methods used in selecting a master list of vocabulary entries •••••••••••••••• 11 Methods used in pre-validating test items • • . • . 17 Methods used in establishing a background criterion. 20 III. METHODOLOGY OF THE STUDY .................. 25 Procedure used in selecting the master list of words, terms, and symbols •••••••••••• 26 Procedure for the selection of pre-validated entries from the master list ••••••••••••••• 31 Procedure in test item construction and writing the test .•••••••••••••.«••••• 39 Procedure in the development of the criterion . 45 Procedure in the administration of the test .... 51 Procedure in the analysis of the test ••••••• 53 iii p iv Chapter Page IV. PRESENTATION AND EVALUATION OF RESULTS........ 57 Results of procedures used in selecting entries from the master list ••••••••••••••• 66 Results of procedures used in writing the test . 75 Results of procedures in developing the test c r i t e r i o n .......... 81 Results of procedures in scoring backgrounds • • • • 89 Results of procedures in analyzing the test • • • • 93 V. SUMMARY AND CONCLUSIONS.................... Ill Summary •••••••• ........ ••••••••• m Conclusions .......... 112 Implications for further study .............. 116 BIBLIOGRAPHY........................................... 118 APPENDIXES .................... ................... A. Master list of words and terms ..... ... .......... 121 B. Test: Form 51858 153 C. F o r m s ......... .............................. 189 LIST OF TABLES TABLE PAGE 1* Preliminary Bibliography of Test Materials......... 5S 2. Qualifications of Experts Selecting Sources for Words and Terms • ....... 63 3* Frequency With Which Sources Were Selected by Forty-one Experts • •••••.................. 63 4« The Basic Sources of Test Materials ....••••• 65 5* Distribution of General Graduate Sample by Departmental Affiliation ........ 67 6. Distribution of General Graduate Sample by Major and Minor Background Affiliations • •••••.•• 69 7« Use of High Familiarity Ratings on Fictitious Terms at Various Levels of Graduate Standing ••••••• 70 B0 Distributions of Familiarity Indexes in a Sample of 39 Words in General Usage Compared With a Random Sample of Words and Terms from the Master List • ••••••••••••••••••• 72 9. Mean Familiarity Ratings on Infrequent Words and Terms by Doctors and near Doctors in Theatre • • • • 74 10. Per Cent of Course Work Offered by Various Schools in Each of Eleven Sub-Areas in Theatre Included in the Test • ••••••••••*••.••••• 7B 11. Qualifications of Experts Rating Types of Training and Experience ........ 82 v vi TABLE PAGE 1 2 . Means and Standard Deviations of Expert Ratings of Types of Training and Ex rience •••••••••• 83 13.Comparison of Scale Values Resulting from Grouping with the Individual Mean Ratings of Fifteen Types of Training and Experience ••••••••••••• 85 14. Advanced Student Formal Production Experience . • . 87 15. Scale Values Assigned for Formal Production Experience 90 16.Test Items Built around Acceptable Words and Terms Compared with Those not in Respect to Item Validity and Item Difficulty •••••••«•••••••• 92 17. The Total and Major Division Background Indexes and Test Scores •••••••••••••••••••• 94 18. Validity of Total Test and Major Divisions of Test • • 96 19. Tentative Predictions of Background Indexes from Various Test Scores •••••••••••••••• 98 20. Reliability of Total Test and Major Divisions of Test. 99 21. Data on Sub-areas: Tertile Means and F Ratios • • • . 100 22. Item Analysis of Test: Item Difficulty and Item Validity . ......................... 101 CHAPTER I SETTING OF THE STUDY The purpose of this study is to answer the question: nCan a graduate achievement test, based on the vocabulary in the field of theatre, be developed that will satisfactorily estimate the back­ ground of graduate theatre students?” In determining the answer to this major question, the study proposes further to answer a number of subsidiary questions: (1) Can an acceptable criterion be developed? (2) Can this graduate achievement test in theatre meet acceptable standards of item difficulty without sacrificing item validity? (3) Can test validity, reliability, and efficiency be maintained despite the comprehensiveness of a graduate achievement test in theatre? (4) Can this graduate achievement test in theatre meet the commonly accepted standards of good test con­ struction? (5) Can the vocabulary entries in the master list be relatively free from the influence of specific institu­ tions or geographical areas? (6) Can a criterion of the background of graduate theatre students be built which will be relatively free from the influence of specific institutions or geographical areas? Different steps have been taken in many institutions in recent years to determine the background status of graduate theatre students* Attempts have been made to estimate their backgrounds on 1 2 the basis of grades and rank-in-class from official transcript s*^" There have been attests to estimate background from letters of recommendation* Attempts have been made, also, to estimate this background status through testing*^ However, to this author*s knowl­ edge, no published results of a graduate achievement test exclusively in theatre have been made available* A graduate achievement test, for the purposes of this study, is defined as na test designed to measure the results of learning or teaching* • *"3 The development of a graduate achievement test in theatre, based on the vocabulary of the field, is the purpose of this study* Such a test could be submitted to a graduate population and its results analyzed and made available* The idea that special vocabularies, peculiar to specific fields, do exist is not new* A student of astronomy would have a good vocabulary of astronomical terms at his command; a keen baseball fan can discuss the intricacies of the "diamond" in a language quite unintelligible to the layman* • • ^For a discussion of these as methods of estimating back­ ground status, see Chapter II* 2 Hubert Heffner, "Common Grounds for Speech and Theatre," Educational Theatre Journal. V (1953), 344* ^Clarence L* Barnhart, editor, Ths American College Dictionary. (New York: Random House, 1947), p* 10* ^Gladys C* Schwesinger, The Social-Ethical Significance of Vocabulary (New York City: Bureau of Publications, Teachers College, Columbia University, 1926), p* 13* W 3 The fact that this vocabulary Is "quite definitely a func­ tion of the topic" has also been substantiated.5 In the planning and developing of a test which deals primarily with vocabulary, it is recognized immediately that other factors, i.e., intelligence, unusual recall ability, etc., have an effect upon the final results. Those results should not be disturbed too radically, however, in light of studies where comparisons between test scores and intelli­ gence have been made. The value of vocabulary in testing intelli­ gence, for example, was shown in a study by Gerlach who showed that a person with low intelligence could make good grades within a particular field through application. At the same time he noted that a highly gifted person through lack of application could score poorly with respect to that particular field. From these comments, Gladys Schwesinger drew the inference that "whatever measures this general capacity {intelligence], does at the same time tend to predict behaviour within a specific field, if experience within that field has been provHed."^ Miss Schwesinger also concluded from her study, "The Social-Ethical Significance of Vocabulary," that for those individuals who had had emphasized experience in a particular field, "there is found to be an increased ability in verbal expression, ^Walter S. Monroe, editor, Encyclopedia of Educational Research (New York: The Macmillan Company, 1952), p. 389. ^F. M. Gerlach, "Vocabulary Studies," Studies in Education and Psychology. Colorado College, No. 1, 1917# quoted in Schwesinger, op. cit*, p. 9* 7 'Schwesinger, op.