DOCUMENT RESUME ED 355 195 SP 034 281 AUTHOR Izu, Jo Ann; And Others TITLE Assessment Component of the California New Teacher Project: Evaluation of Existing Teacher Assessment Practices. INSTITUTION Far West L '. for Educational Research and Developmen.., San Francisco, Calif. SPONS AGENCY California Commission of Teacher Credentialing, Sacramento.; California State Dept. of Education, Sacramento. PUB DATE Feb 92 CONTRACT TCC-0O21 NOTE 246p.; For related documents, see ED 323 197, ED 342 761, and SP 034 278-282. PUB TYPE Reports Research/Technical (143) EDRS PRICE MF01/PC10 Plus Postage. DESCRIPTORS Beginning Teachers; *Board of Education Policy; *College Outcomes Assessment; Elementary Secondary Education; Evaluation Criteria; Evaluation Methods; Evaluation Research; Higher Education; Inservice Teacher Education; *Needs Assessment; Policy Formation; Preservice Teacher Education; Research Design; *State Standards; Teacher Certification; *Teacher Evaluation IDENTIFIERS *Assessments of Performance in Teaching; *California New Teacher Project; Knowledge Base for Teaching; Reform Efforts ABSTRACT This document reports on the third year of the assessment component of the California New Teacher Project (CNTP). In the first 2 years, research focused on pilot testing innovative forms of new teacher assessment. Part of the thirdyear study was a careful review of the state of assessment practices in California. Thestudy was designed to yield information about existing assessment practices at the college and university level, the local school district level, and the State level within the context of the legal, social, and economic situation facing California educational systems andto provide information that would lead to the conceptual base and design of a new system of support and assessment for beginning teachers in California. This report, organized into six chapters, begins withan introduction describing the assessment component of the CNTP,the current context for assessment of California teachers, the essential design features of the study, guiding constructs foran ideal statewide assessment system, and developing interview protocols.The remainder of the report is organized into chapters covering institutions of higher education assessment practices,local education agency assessment practices, the State assessment practices substudy, experiences of beginning teachers, anda summary of findings across substudies and conclusions. Two appendicesare attached: Sample Items from the California Basic Educational Skills Test (CBEST); and Comparisons with Phase II Assessments. (LL) 4 2 LEST COPY AVAILABLE U.S. DEPARTMENT Or EDUCATION Otlyce of Educational Researchand improvement "PERMISSION TO REPRODUCE THIS EDUCATIONAL RESOURCES INFORMATION MATERIAL HAS BEEN GRANTED BY CENTER (ERIC) C This document has been reproducedas received from the person or organization I originating a Minor changes have been made to improve reproduction quality Points of new Or opinions stated 'II this docu- ment do not necessarily represent otttcrat OEM positron or policy TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC)." This report is based upon work performed pursuant to Contract TCC-0O21 withthe California Interagency Task Force of the California Commissionon Teacher Credentialing and the California State Department of Education New Teacher Project.The contents do not necessarily reflect the views or policies of these agencies and theaccuracy of the information is the sole responsibility of the Far West Laboratory. Assessment Component of the California New Teacher Project: Evaluation of Existing Teacher Assessment Practices Jo Ann Izu Claudia Long Kendyll Stansbury Dennis Tierney February, 1992 Gary Estes, Project Director Far West Laboratory for Educational Research and Development San Francisco, California 4 TABLE OF CONTENTS CHAPTER 1: INTRODUCTION 1.1 Assessment Component of the CNTP: Phase II 1.2 Current Context for Assessment of California Teachers 1.4 Essential Design Features of this Study 1.7 Guiding Constructs for an Ideal Statewide Assessment System 1.7 Development of Interview Protocols 110 CHAPTER 2: INSTITUTIONS OF HIGHER EDUCATION ASSESSMENT PRACTICES 2.1 Description of Sample and Methods 2.1 Sample Description 2.1 Methodology 2.3 Interview protocols 2.3 Documents collected 2.5 Interview teams 2.9 Institutions of Higher Education 210 Overview of Major Assessment Points 212 Admission to Credential Program 212 Range of methods used 212 Purposes and uses of admissions assessments 217 Technical quality of admissions assessments 220 Retention in Credential Programs 221 Range of methods used 221 Purposes and uses of retention assessments 222 Technical quality of assessments 2 23 Formal Student Teaching 2 24 Range of methods used 2 24 Purposes and uses of assessments 2 25 Technical quality 2 26 5 Recommendation for the Credential 2 29 Range of methods used 2 29 Purposes and uses of assessments 2 29 Technical quality 2 29 Post Credential Support 2 30 Range of methods used 2 30 Purposes and uses of assessments 2 30 Technical quality 2 30 Focus of IHE Assessments 2 30 Subject-matter 2 31 General Pedagogy 2 31 Content Pedagogy 2 32 Other Constructs 2 32 Basic skills proficiency 2 32 Personal attributes 2 33 Programmatic needs 233 Differential Impact of Assessment Practices 234 Costs of IHE Assessments 2 37 Admission Costs 2 37 Contributions of the IHE Assessments to Credentialing Recommendations 2 39 Summary and Conclusions 2 40 Key Admissions Elements 2 40 Key Retention and Recommendation Elements 2 41 CHAPTER 3: LOCAL EDUCATION AGENCY ASSESSMENT PRACTICES 3.1 Description of Sample and Methods 3.1 Sample Description 3.1 Methodology 3.4 Interview Protocols 3.4 Documents Collected 3.6 Interviewers 3.6 LEA Context 3.8 2 Overview of Mgjor Assessment Points 3.9 Hiring Teachers 3.9 Capacity for selectivity 310 school autonomy 312 Range of assessment methods used 313 Purposes and uses of assessments 316 Technical quality 317 Continued Employment: Retention and Tenure of Beginning Teachers 318 Range of methods used 320 Purposes and uses of assessments 323 Technical quality 325 Support 325 Range of methods used 326 Purposes and uses of assessments 328 Technical quality 329 Focus of Assessments 329 Subject-matter Competence 330 Pedagogy 330 Content Pedagogy 331 Other Constructs 331 Differential Impact of Assessment Practices 333 Costs of Assessments 333 Hiring Assessment Costs 334 Continued Employment Assessment Costs 335 Support Assessment Costs 336 Contributions of LEA Assessments to Staff Development Decision 336 Summary and Conclusions 337 Summary of Major Findings 337 Conclusion 340 3 CHAPTER 4: THE STATE ASSESSMENT PRACTICES SUBSTUDY 4.1 Description of Sample and Methods 4.1 Sample Description 4.1 Methodology 4.2 State Context 4.2 Basic Skills Proficiencies Requirement 4.3 Subject-matter Knowledge Requirement 4.3 Multiple subjects credential 4.4 Single subject credentials 4.5 Trends in the use of examinations to demonstrate subject-matter knowledge 4.6 Review of Credential Applications 4.6 Overview of the Major Assessment Point 4.8 Range of Methods Used 4.8 Purposes and Uses of Assessments 4 10 Technical Quality 410 Document review 410 Standardized tests 411 Focus of Assessments 41.2 Subject-matter Knowledge 412 General Pedagogy and Content Pedagogy 414 Basic Skills Proficiencies 416 Differential Impact of Assessment Practices 417 Ethnic groups 418 Gender groups 420 Type of credential 420 Review of studies of potential sources of differential performance 420 Costs 423 4 Contribution of State Assessments to IHE and LEA Decisions 423 IHE Decisions 423 LEA Decisions 425 Summary and Conclusions 425 Assessment Context 425 Methods of Mr assment 425 Focus of Assessments 425 Technical Quality of Assessments 426 Assessment and Support 426 Conclusions 427 CHAPTER 5:EXPERIENCES OF BEGINNING TEACHERS 5.1 Sample Description 5.1 Teachers 5.1 Range of IHEs and LEAs Represented 5.4 Traversing the Credentialing Assessment System 5.4 Subject Matter Preparation in College 5.5 Admission to a Teacher Education Program 5.6 Education Courses Prior to and During Student Teaching 5.7 Admission to Student Teaching 5.8 Student Teaching 5.9 Assessment by university supervisors 511 Assessment by master teachers E.13 Overall assessment experience 516 Employment Assessments 5 21 First-year Assessments 522 Assessment Content 5 30 General Pedagogy 5 30 Content Pedagogy 5 32 Working with Diverse Students 534 Curriculum Frameworks 5 36 5 7 Summary and Conclusions 5 38 The Forms of Assessment 538 Characteristics of Assessors 5 38 Content of Assessments 538 Rigor of Assessments 540 Thoroughness of Assessments 540 Redundancy of Assessments 541 Consistency of Assessments 541 Fairness of Assessments 542 Helpfulness of Assessments 542 Assessments and Support 543 CHAPTER 6: SUMMARY OF FINDINGS ACROSS SUBSTUDIES AND CONCLUSIONS 6.1 Focus of Assessments 6.1 Subject-matter Knowledge 6.1 General Pedagogy 6.2 Content Pedagogy 6.2 Knowledge of Students and Student Diversity 6.3 General Academic Ability and Personal Attributes 6.3 Forms of Assessments 6.3 Technical Quality of Assessments 6.4 Extent to which ERE, LEA, and State Assessments Constitute an Assessment System 6.5 Rigor 6.5 Thoroughness 6.7 Fairness 6.9 Consistency 6.9 Helpfulness 6 10 Conclusions 6 11 BIBLIOGRAPHY B.1 6 10 CHAPTER 1: INTRODUCTION [1 CHAPTER 1: INTRODUCTION The need for better preparation of beginning teachers is botha major national and State policy issue. As part of broader discussions about professionalizing teachers,