Willi As, Frederick Carrascolendas: Evaluation of a Bilingual Television
Total Page:16
File Type:pdf, Size:1020Kb
DOCUMENT RESUME ED 054 612 24 EM 009 194 AUTHOR Natalicio, Diana S.; Willi_as, Frederick TITLE Carrascolendas: Evaluation of a Bilingual Television Series. Final Report. INSTITUTION Texas Univ., Austin. Center for ComAunication Research. SPONS AGENCY Office of Education (DHEW) Washington, D.C. PUB DATE Jun 71 GRANT OEG-0-9-530094-4239-(280) NOTE 204p. EDRS PRICE MF-$0.65 HC-$9.87 DESCRIPTORS *Bilingual Education; Comparative Analysis; Conventional Instruction; *Elementary Grades; Feedback; *Instructional Television; *Mexican Americans; Program Descriptions; Program Evaluation ABSTRACT uCarrascolendas" was a thirty-program television series designed to aid in the bilingual instruction of Mexican-American children in the first and second grades. A systematic evaluation of the production and the effect of the series is presented here. Evaluation of the process of program development noted that the series was completed and did reflect the intended instructional objectives. some suggestions for improvement included: Modification of the time schedule to allow for more feedback and revision of the programs, an improved definition of the responsibilities of supervisory staff members, and a closer working relationship between the curriculum and production supervisors. A field experiment involving children from the target audience population and a survey of schools that used the programs showed statistically significant learning gains among television viewers in English tests of multicultural social environment, English language skills, physical environment, and cognitive development. The survey of schools, although indicating a major use of the program, did reveal a possible shortcoming in that a significant number of schools, even in predominantly Mexican-American areas, had no knowledge of the program's availability. (JY) CCRCENTER FOR COMMUNICATION RESEARCH School of Communication / The University of Texas at Austin / Austin, Texas78712 CARRASCOLENDAS: EVALUATION OF A BILINGUAL TELEVISIONSERIES CARRASCOLENDAS: EVALUACION DE UNA SERIE BILINGUETELEVISADA Diana S. Natalicio Frederick Williams June, 1971 a-.. U.S. DEPARTMENT OF HEALTH. MI EDUCATION & WELFARE OFEICE.OF EDUCATION r.... THIS DOCUMENT HAS BEEN REPRO- DUCED EXACTLY AS RECEIVED FROM %ID THE PERSON OH ORGANIZATION ORIG. INATING IT. POINTS OF VIEW OR OPIN- w* iONS STATED DO NOT NECESSARILY FIEPRESCNT OFFICIAL OFFICE OF EDU. LC% CATION POSITION OR POLICY CLI LW Final Report U.S. Office of Education Grant #0-9-530094-4239-(280) CARRASCOLENDAS: EVALUATION OF A BILINGUAL TELEVISION SERIES CARRASCOLENDAS: EVALUACION DE UNA SERIE B1LINGUE TELEVISADA Diana S. Natalicio Frederick Williams Center for Communication Research School of Communication The University of Texas at Austin Austin, Texas June, 1971 This research was conducted by the Center for CommunicationResearch, The University of Texas at Austin, under sub-contract withthe Education Service Center, Region XIII, which in turnwas supported by contra t with the U.S. Office of Education Grant #0-9-530094-4239. The research reported herein was performed pursuant toa grant with the Office of Education, U.S. Department of Health, Edixation,and Welfare. Contractors undertaking such projects uner Government sponsorship are encouraged to express freely their professional judgment in the conduct of the project. Points of view or opinions stated do not, therefore, necessarily represent official Officeof Education position or policy. U.S. DEPARTMENT OF HEALTH, EDUCATION, AND WELFARE Office of Education TABLE OF CONTENTS ACKNOWLEDGMENTS ABSTRACT SECTION I SECTION 2 2.1. What were the major steps in the overall process? 3 2.1.1. Scheduling: The time line 4 2.1.2. Recommendations 4 2.2. What is the relationship between proposed instructional objectives and those realized in program segments?. .. 6 2.2.1. Recommendations 6 2.3. Relative to jeb descriptions, whatwas the performance assessment in carrying out the responsibilities of the following positions?. 6 2.3.1. Television Coordinatcr 7 2.3.1.1. Recommendations 9 2.3.2. Curriculum and Staff Development Coordinator. 9 2.3.2.1. Recommendations 10 2.3.3. Executive Producer.... 11 2.3.3.1. Recommendations 11 2.3.4. Advisory Committee 13 2.3.4.1. Recommendations 14 2.3.5. Evaluation teamls contribution to process . 14 2.3.5.1. Recommendations 15 SECTION 3 17 3.1. Research Design 17 3.2. Subjects 17 3.3. The Interview Instruments 19 3.3.1. Piloting of test instruments 20 3.4. Test Administration 20 3.4.1. Reliability of interviewers. ... 21 3.4.2. Final versions of the test instruments 21 3.4.3. Interview procedure . ... 21 3.5. Alternative Activity for the Control (Non-watcher) Group . ... 24 3.6. Data Analysis 25 3.6.1. Scoring of responses to interviews 25 3.6.2. Subtest scores 25 Table of Contents (continued) 3.7. Pre-test Results 26 3.8. Post-test Interviews 28 3.9. Post-test Data Analysis 28 3.10. Results of the Experiment . c ... 29 3.10.1. English testing results 29 3.10.2. Spanish testing results 31 3.11. Summary 32 SECTION 4 35 4.1. Overall Utilization of Carrascolendasin the KERN Viewing Area 35 4.1.1. Procedures 36 4.1.2. Results of survey 36 4.1.3. Discussion 36 4.2. Teacher Attitude Questionnaire. 39 4.2.1. Development of questionnaire . 39 4.2.2. Results ...... 40 SECTION 5 67 5.1. Procedures 67 5.2. Summary of Teacherst Notations 67 5.3. General Remarks 77 APPENDIXA .. i 6 . 6 .. .. * 6 78 APPENDIX B 121 APPENDIX C 127 APPENDIX D 131 APPENDIX E . .. ...... 135 APPENDIX F 341 APPENDIX G = 151 APPENDIX H 155 APPENDIX ' 159 APPENDIX J . = . 161 APPENDIX K .. .......... 179 APPENDIX L 185 . APpEND1X M . .......... 191 REFERENCES 209 ACKNOWLEDGMENTS The success of a project as diverse as this one dependsupon the dedicated efforts of a large number of individuals. This project was very fortunate in having unusually dedicated and flexibiJ participants throughout its duration. The Zavala School children made the evaluation possible. The staff at Zavala School, Miss Hermelinda Rodriguez (principal), Mrs. Judy Peterson, Mrs. Yolanda PeWa, Mrs. Ruth Burnet, Mrs. Theresa McLain, Mrs. Emily Cole, Miss Irene Fernandez, Miss Betty Schwobel, Mrs. Jewline Selene (teachers), and the teacher aides provided the cooeration and assistance neces- sary to conduct the many interviews and the photography project. Miss Maria Barrera, Director of Bilingual Programs at the Education Service Center, Region XIII, supported our efforts at the school and provided necessary equipment for the photography project. Consul- tation on visual stimuli for the test instrument waS provided by Miss Mabel Jean Schmer. Mr. Dick Calkins provided consultation in the area of computer programming and data analysis. Mr. Robert Finnell assisted in the transcription of Spanish-language materials. Members of the evaluation team, whose individual and collective contributions touched nearly every aspect of the project at one time or another included: Miss Diana Alvarado, Mrs. Cecilia Berki, Mr. Eduardo Esparza, Miss Susan McRae, Miss Sally Morgan, M s. Christina Thomas, Mr. Andre Thumas, and Mr. Nicholas Valenzuela. iv ABSTRACT This research represented the systematic evaluationof the production and effects of the television'series "Carrascolendas," a program designed to facilitate the bilingual education of Mexican- American children. Evaluation of the process of program development noted first that the series was completed and did reflect the in- tended instructional objectives. Suggestions for improvement of the process aspects includel: (I) modification of the time schedule to allow for more feedback and revision of programs during the production schedule, (2) an improved definition of the responsibili- ties of supervisory staff members, and (3) a closer 1,,orking relation- ship between the curriculum and production supervisors. Product evaluation, or assessment of the effects of the series, was conducted by means of a field experiment involving children from the target au,.!ience population and bya survey of schools which used 1,3 program. Results of the field experiment indicated satistically significant learning gains of television viewers (as against a control group) in English tests of multicul- tural social environment, English language skills, physical environ- ment, and cognitive development. Viewers also gained significantly in measures of overall language performance in both Spanish and English. These results had generality across first and second grades (eXcept multicultural knowledge which increased only for second oraders) and across classrooms either engaged or not engaged in bilingual ih- struction programs. The school surveys, although indicating a major use of the program, did reveal a possible shortcoming in that a significant number of schools, even in heavy populated Mexican-American areas, had no knowledge of the program's availability. Teachers' altitudes were substantially positive concerning the overall program and its use in the classroom. Detailed responses of teachers provided a basis for suggesting program revisions. SECTION: Introduction This is a report of the evaluation of the first year's production and use of "Carrascolendas," a thirty program television series designed to facilitate the bilingual instruction of Mexican- American children, primarily in grades ono and two. Two types of evaluation were undertaken. One, process evaluation, was an in- quiry into the development of the program. Three main questions guided this phase of the evaluation: What were the major steps in the overall process