An Empirical Study of a District Wide K-2 Performance Assessment Program: Teacher Practices, Information Gained, and Use of Assessment Results
Total Page:16
File Type:pdf, Size:1020Kb
DOCUMENT RESUME ED 452 216 TM 032 501 AUTHOR Lanting, Ashley TITLE An Empirical Study of a District Wide K-2 Performance Assessment Program: Teacher Practices, Information Gained, and Use of Assessment Results. PUB DATE 2001-04-00 NOTE 297p.; Paper presented at the Annual Meeting of the American Educational Research Association (Seattle, WA, April 10-14, 2001). PUB TYPE Reports Research (143) Speeches/Meeting Papers (150) EDRS PRICE MF01/PC12 Plus Postage. DESCRIPTORS *Elementary School Teachers; *Inservice Teacher Education; *Literacy; *Performance Based Assessment; Primary Education; Qualitative Research; Scoring; *Test Use IDENTIFIERS Grounded Theory ABSTRACT This study used a qualitative research methodology to examine how four primary teachers used a district literacy performance assessment. Data were collected through observations, interviews, and documents. Grounded theory and NUD*IST software were used for text analysis and theory building. Findings show that a theory-grounded teacher-empowered K-2 performance assessment program accompanied by the school district's low level of interference could interact very well with teachers' high ethical standards on assessments. When the in-service training was voluntary, teachers did not spontaneously practice comprehended portfolio use or students self-assessment, but rather relied on observation and interview. Rubrics seemed to work as conceptual frameworks for data collection and evaluation, and teachers usually grounded their evaluations on evidence. Dimensional scoring and flexible marking across proficiency levels were implemented, and teachers appeared to focus on student strengths. The information obtained from the literacy assessment was criterion referenced and individualized. Teachers did not use normative language when commenting on student performance. Assessment results were generally used to keep track of student performance and to provide remedial teaching, but there seemed to be a gap between assessment results and corresponding pedagogical strategies. It is recommended that performance assessment programs be accompanied by teacher in-service education on repertories of instructional strategies, but it is not clear whether teacher involvement in rubric development and peer discussions would make assessment results more instructionally useful. Eight appendixes contain supplemental material and some of the documents used in the study. (Contains 3 tables and 79 references.) (SLD) .11:1711' POOR PRINT QUALITY Pgs224 2.89 Reproductions supplied by EDRS are the best that can be made from the original document. U.S. DEPARTMENT OF EDUCATION Off ice of Educational Research and Improvement PERMISSION TO REPRODUCEAND EDUCATIONAL RESOURCES INFORMATION DISSEMINATE THIS MATERIALHAS CENTER (ERIC) BEEN GRANTED BY This document has been reproduced as received from the person or organization originating it. Minor changes have been made to improve reproduction quality. Points of view or opinions stated in this TO THE EDUCATIONALRESOURCES document do not necessarily represent INFORMATION CENTER (ERIC) official OERI position or policy. 1 AN EMPIRICAL STUDY OF A DISTRICT WIDE K-2 PERFORMANCE ASSESSMENT PROGRAM: TEACHER PRACTICES, INFORMATION GAINED, AND USE OF ASSESSMENT RESULTS BY ASHLEY LANTING, PH.D. INDIANA CENTER FOR EVALUATION Submitted for presentation at the American Educational Research Association annual meeting in Seattle, Washington, 2001 Bloomington, Indiana BESTCOPYAVAILABLE 2 iii ABSTRACT This study employs a qualitative research methodology to examine how four primary teachers used a district literacy performance assessment. Data were collected through observations, interviews, and documents, which consist of procedures and rationale of teacher use of the assessment. Grounded theory and NUD*IST software were used for text analysis and theory building. The study shows that a theory-grounded teacher-empowered K-2 performance assessment program, accompanied with the District's low level of interference could interact very well with teachers' high ethical standards on assessments. Specifically, when in-service was voluntary, teachers did not spontaneously practice comprehensive portfolio or student self-assessment but mainly relied on observation and interviews; discussion among teachers on assessment was also limited because of time constraints. Rubrics seemed to work as conceptual frameworks for data collection and evaluation (Dorfman, 1997, Khattri et al., 1998, and Falk 1994) and teachers usually grounded their evaluation on evidence (Falk, 1998). Dimensional scoring and flexible marking across proficiency levels were implemented, which did not lead teachers to focus on student weakness as warned by Pearson, De Stefano, and Garcia (1998); in fact, teachers appeared to focus on strength. Information obtained from the assessment was criteria-referenced and individualized. Teachers did not use normative language (Pearson, De Stefano, & Garcia, 1998) when commenting on student performance. Still, in-service on direct questioning technique is strongly recommended (Resnick and Resnick, 1996) to explore higher-order thinking processes and to diagnose learning problems. iv The assessment results were used mainly to keep track of student performance and to provide remedial teaching. There seemed to a gap between assessment results and corresponding pedagogical strategies. It is recommended that performance assessment programs be accompanied with in-service on extensive repertories of instructional strategies (Darling-Hammond & Ancess, 1996; and Wolf & Reardon, 1996). It is not conclusive that teacher involvement in rubric development and peer discussion (Pearson, De Stefano, and Garcia, 1998) and teacher experiences on assessments, task types, and integration (Khattri et al. 1998) would make assessment results more instructionally useful. The present study suggests further research on low-performing schools. It is also important to explore the impact of assessment results on teaching where in-service on direct questioning, portfolio, student self-assessments, and/or repertories of pedagogy are provided. 4 To my father and mother. 5 vi ACKNOWLEDGMENTS I would like to acknowledge my appreciation to the teachers and district administrators who were involved in this study. I am grateful that they allowed me into their classrooms/offices. They answered my questions, read my drafts, and answered more questions. I admire their professional practices and their generosity. I would also like to thank their students whose normal leaning continued with my presence in their classroom. Special thanks to the four members of my committee: Drs. Fred Davidson, Lizanne De Stefano, Georgia Garcia, and Liora Bresler. They have tolerated my struggle and nurtured my growth throughout this study.I appreciate their professional guidance and feedback. In particular, I would like to give thanks to my advisor and thesis director Dr. Fred Davidson for his willingness to direct this long distance dissertationreading my drafts, providing prompt responses (and words of encouragement), and advising strategic work plans. Dr. Davidson serves as a wonderful model for guiding a dissertation. Collectively, the committee pushed me to reach for the highest standards. Thank you. 8 vii TABLE OF CONTENTS LIST OF TABLES xii CHAPTER 1 INTRODUCTION 1 2 LITERATURE REVIEW 8 Performance Assessment 8 Introduction 8 Formats 10 Effects on Teaching 11 Information Gained 14 Factors Affecting Information Gathering 15 Teachers' Uses of Information Obtained 16 Empirical Studies of Performance Assessment Programs For Young Children 19 The Work Sampling System 20 The New Standards Project 23 The Early Literacy Portfolio 26 Primary Language Records and Primary Learning Records 30 Summary 34 3 METHODOLOGY 36 Naturalistic Inquiry 36 Field Entry 38 The Site 40 The District 40 The School 41 Participants 41 The K-2 Performance Assessment Program 41 Data Collection 43 Documentary Data 44 Observation and Audio Taped Data 45 Interview Data 48 7 viii CHAPTER PAGE 3 METHODOLOGY (continued) Data Analysis 51 Data Coding 52 Memos 55 Building Trustworthiness 56 Audit Trail 58 Prolonged Engagement 58 Multiple Data Sources/Methods Triangulation 59 Debriefings to Project Supervisors 60 Member Checking 60 4 THE K-2 PEFORMANCE ASSESSEMNT PRORAM AT THE DISTRICT LEVEL: DESIGN AND EXPECTATIONS 62 Introduction 62 Design 62 Name(s) 62 Context 64 District's Vision of K-2 Curriculum 64 District's Vision of K-2 Instruction 64 District's Vision of K-2 Assessment 64 Needs for the K-2 Performance Assessment 65 Content 66 Outcomes and Criteria 66 Reporting Forms 72 Expectations 73 Assessment Practices 74 Information Gained. 79 Use of Assessment Results 80 8 ix CHAPTER PAGE 5 THE K-2 PERFORMANCE ASSESSMENT PROGRAM AT THE CLASSROOM LEVEL: FOUR CASE STUDIES OF TEACHER PRACTICES 82 Introduction 82 Ms. Nixon 83 Profile 83 Assessment Practices 84 Integration 84 Data Sources 85 Assessment Methods 86 Communication 95 Information Gained from Rubrics 97 Use of Information/Assessment Results 100 Ms. Carter 104 Profile 104 Assessment Practices 105 Integration 105 Data Sources 106 Assessment Methods 108 Communication 119 Information Gained from Rubrics 120 Use of Information/Assessment Results 125 Ms. Ford 131 Profile 131 Assessment Practices 132 Integration 132 Data Sources 134 Assessment Methods 136 Communication 146 Information Gained from Rubrics 148 Use of Information/Assessment Results 151 a CHAPTER PAGE 5 THE K-2 PERFORMANCE ASSESSMENT PROGRAM AT THE CLASSROOM LEVEL: