Hintzen, Neil Dental Assistant Test Development Project. Final I
Total Page:16
File Type:pdf, Size:1020Kb
DOCUMENT RESUME ED 287 987 CE 048 596 AUTHOR Laugen, Ronald C.; Hintzen, Neil TITLE Dental Assistant Test Development Project. Final Report. INSTITUTION Florida State Univ., Tallahassee. Center for Instructional Development and Services. SPONS AGENCY Florida State Dept. of Education, Tallahassee. Div. of Vocational, Adult, and Community Education. PUB DATE Aug 86 NOTE 549p. PUB TYPE Reports - Descriptive (141) -- Tests/Evaluation Instruments (160) EDRS PRICE MF02/PC22 Plus Postage. DESCRIPTORS Behavioral Objectives; *Competence; Competency Based Education; *Criterion Referenced Tests; *Dental Assistants; Evaluation Methods; Field Tests; Higher Education; *Performance Tests; Standards; Student Evaluation; Task Analysis; *Test Construction; *Test Items; Test Manuals; Vocational Education IDENTIFIERS Florida State University ABSTRACT A project was conducted to develop two criterion-referenced, multiple-choice tests and a performance test for the dental assistant occupational area. Procedures were recommended for administration of the tests and field test and for revision of the tests. Evaluation experts analyzed the performance standards being tested, drafted test items, reviewed the items, conducted field tests, revised the tests, and finalized the tests and procedures for administering them. (This document contains a report on the drafting of the tests and the following appendixes, which make up the bulk of the document: list of participants, occupational proficiency performance standards, list of competencies, task analysis, item matrices, written tests, and a performance test administration manual.) (KC) , *****************************Wt**#************************************* ,zp , we * Reproductions supplied byi4DRS are the best that can be made * * from the original document. * *********************************************************************** FINAL REPORT for Dental Assistant Test Development Project tp- 7 - 1+0 3-- 3 r5 (A Prepared by: Ronald C. Laugen, Ph.D. and Neil Hintzen The Center for Instructional Development andServices Florida State University August, 1986 "PERMISSION TO REPRODUCE THIS U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement MATERIAL HAS BEEN GRANTEDBY ED CATIONAL RESOURCES INFORMATION CENTER (ERIC) This document has been reproduced as received 'rem the person or organization Originating it. Oulu O Minor changes have been made 10 trnprove reproduction quality TO THE EDUCATIONAL RESOURCES Pnt of view or Opinions stated in this dou- meoint s do not necessarily represent officcial INFORMATION CENTER (ERIC)." OERI position or policy. 2 BEST COPY AVAILABLE TABLE OF CONTENTS I Page ) INTRODUCTION I OBJECTIVES 3 METHODS 3 RESULTS II RECOMMENDATIONS 12 REFERENCES 13 APPENDICES 14 A. List of Participants B. Occupational Proficiency Performance Standards ) C. List of Competencies D. Task Analysis E. Item Matrices, Form A & B ) F. Written Tests, Forms A & B G. Performance Test Administration manual INTRODUCTION The Division of Vocational, Adult, and Community Education(DVACE) of the Florida De- partment of Education (DOE) has been contributing to andsupporting the implementation of individualized competency-based vocational education(CBVE). Many vocational edu- cators think individualized CBVE to be better thantraditional, non individualized, lock- step vocational education in terms of improving studentconfidence and success, and meeting employers' expectations of hiring qualified graduates.Qualified graduates are vocational program completers who possess the required skills,knowledge, and attitudes ( SKA's) to perform successfully in entry-level occupational positions.And when voca- tional program completers are qualified, vocational instructorsand administrators have performed their proper roles. In vocational schools or centers where CBVE has been partially orfully implemented, the conditions are appropriate for the use of written and performance competencytests. 'These tests can be developed and used to determine the degree towhich vocational pro- gram completers have mastered the programrequirements.If the program requirements include the required entry-level SKA's for the corresponding occupations,then two things are possible: (I) vocational programs can providequalified individuals to employers, and (2) competency tests can be developed in order to verify that programcompleters are qualified. The 1984 Florida Legislature has mandated that such testsbe developed. Florida is at the forefront of the movement from traditional teaching and testing to criterion-referenced or 'competency-based teaching and testing. TheDVACE (1983) sponsored and published a position paper for Florida entitled"Individualized Competency- Based Vocational Education." This position paper describes theprinciples and procedures of individualized CBVE and stated the DVACE's endorsement of them. 1 Popham (1978) discussed the movementin terms of its origins and future.He credited Glaser (1963) as one of the first to makethe distinction betweennorm-referenced and ) criterion-referenced measurement strategies.Popham (1978), looking toward the future said " ... we are at the threshold of a neworientation in educational measurement.We can characterized this newperiod of educational assessment asthe criterion-referenced ) era." The role of written and performancecompetency tests in CBVE is toprovide educators completers in the with two kinds of information:(I) the level of proficiency of program occupational area, and (2) the adequacy of theinstruction in which the completers had participated. The latter can serve as the basisfor planning future instruction. It is important to note that these twokinds of information are possibleonly if the tests i are developed using procedureswhich assure that the correct entry-leveloccupational competencies are identified. Popham (1981) states "A criterion-referenced testis used to ascertain an individual's status with respect to a definedbehavioral domain." In order to developcriterion- referenced tests for vocational programs, itis necessary to define thebehavioral domain of each program in terms of the occupation'sentry-level competencies. Once these corn- "petencies have been correctly identified, testdevelopment procedures can begin.The present project utilized OccupationalProficiency Performance Standards(O.P.P.S.'s), de- estab- veloped by the DVACE, as the correctentry-level competencies. O.P.P.S.'s were lished for: (I) educational accountability, and(2) the identification of those minimal competencies students must acquire in order toperform effectively in the student's chosen occupational area. ") The test development project reported herein is in the occupationalarea of Dental Assis- tant. See Appendix A for the list of project participants, including business representa- ) tives, instructors, instructional designers, and test developers. ) OBJECTIVES 1 I. Develop two alternative-form, criterion-referenced multiple choice tests for the Dental Assistant occupational area. 2. Develop a criterion-referenced performance test for the Dental Assistantoccupa- tional area. 1 3.Develop recommended procedures for administration of both the written and perfor- mance tests. 4.Field test and revise the tests. METHODS The Center for Instructional Development and Services (CIDS) of the Florida State Uni- versity was contracted by the Florida Department of Education, Division of Vocational, Adult, and Community Education (DVACE) to develop the written and performance tests for the Dental Assistant occupational program. Items were written and reviewed. Field testing and final revision of the tests were conducted. 3 The test development project was managed by a knowledgeable evaluation specialist called upon the additional expertise of the Center's processional staff. Experienced Florida professionals--both educators and practitioners- - were contracted to conduct analyses of the performance standards being tested, to draft test items, both written and performance, to review test items, and to conduct field tests of the items with program completers. The products of the project are one performance test and two alternate form written tests. Great care was taken to relate each of the items to those performance standards judged necessary for students to enter the occupation, perform in a satisfactory manner, and keep a job. The resulting tests should be useful, valid tools for assessing student competencies and for improving the programs at the institutions that choose to use them. Procedures for Developing Performance Standards As required by Florida Statutes, DVACE developed Occupational Proficiency Performance Standards (O.P.P.S.) for many vocational education occupations. These standards are "developed on the basis of entry level skills needed by vocational program completers to obtain and keep a job" (O.P.P.S.-- Dental Assistant, Appendix B). Identification and verification of the performance standards for a given program include the following nine steps: 1. Acquire task data from workers in the occupation for two years or less. 2. Use task data to determine tasks and duties normally performed by recent employees. 3. Obtain catalogs of performance objectives related to occupation, such as V-TECS. 7 4. Review catalogs to determine minimum orentry-level tasks and prepare lists for statewide verification. 5.Identify ten representative high school,vocational center, and community college programs for use in verification,using the following criteria: a. follow-up data indicateplacement