• Abstract and Figures
• Public Full-text

IDENTIFYING CHILDREN AT RISK: THE PREDICTIVE VALIDITY OF KINDERGARTEN SCREENING MEASURES By S. Suzanne Jacobsen B.Sc, California State Polytechnic University, 1973 M.A., California State Polytechnic University, 1978 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS OF THE DEGREE OF DOCTOR OF EDUCATION in THE FACULTY OF EDUCATION Department of Educational Psychology and Special Education We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA August 1990 ® S. Suzanne Jacobsen In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. The University of British Columbia Vancouver, Canada DE-6 (2/88) Abstract Identifying Children at Risk: The Predictive Validity of Kindergarten Screening Measures The early identification of children who are "at risk" of experiencing learning problems is of interest to educators and policy• makers. Conflicting evidence exists regarding the efficacy of screening measures for identifying children "at risk". The rationale for screening programs is that early identification of problems allows for treatment which may eliminate more severe problems from developing. If a student is identified as "at risk", school personnel may intervene with remedial programs. Subsequently, if the student succeeds, the earlier prediction is no longer valid. The identification of "at risk" would appear inaccurate because the intervention was successful in improving skills. Researchers often measure the prediction of "at risk" with a correlation coefficient. To the extent that the intervention is successful, the correlation of the identification of "at risk" with later measures of achievement is lowered. One of the problems with research on early prediction has been failure to control for the effects of the interventions which were implemented as a consequence of screening. An evaluation of "at risk" prediction is important because results of screening procedures are used to make decisions about retentions and the allocation of special services. The purpose of this study is to investigate the relationship between kindergarten screening measures and grade three achievement for two entire cohorts enrolled in 30 schools in one school district. The ii analysis employs a two-level hierarchical linear regression model to estimate the average within-school relationship between kindergarten screening measures and grade three achievement in basic skills, and determine whether this relationship varies significantly across schools. The model allows for the estimation of the relationship with control for individual pupil characteristics such as age, gender and physical problems. The study examines the extent to which the relationship between kindergarten screening and grade three achievement is mediated by children receiving learning assistance or attending extended (4-year) primary schooling. The study also examines differences among schools in the kindergarten screen/achievement relationships and the achievement of "at risk" pupils by including school characteristics in the analysis. The results of this study indicate positive relationships between kindergarten screening measures and achievement outcomes, even after controlling for age, gender and physical conditions. The kindergarten screen/achievement relationship did not vary among schools. The study failed to demonstrate that controlling for interventions would improve the kindergarten screen/achievement relationship; in fact the effects were in the opposite direction. Levels of adjusted achievement of pupils who obtained scores at the cut-off point for risk status varied significantly among schools. The "at risk" pupils performed better on all four achievement measures in schools with high school mean-ability than similar pupils in schools with low school mean-ability. These results show that progress in the study of the predictive validity of screening measures can be made through the use of iii hierarchical regression techniques. Researchers need to give consideration to the effects of educational interventions and the contextual effects of schools. iv Table of Contents Page Abstract ii Table of Contents v List of Tables viii List of Figures ix Acknowledgements x Chapter 1: Introduction 1 Background of the Problem 1 Purpose of the Study 6 Definition of Terms 6 Research Questions 8 Rationale 9 Limitations of the Study 10 Justification for the Study 11 Organization of the Study . 12 Chapter 2: Review of Literature 13 Part 1: Early Identification of Risk 13 Underlying Assumptions of Risk 14 At Risk On Screening Measures 15 Advantages of Early Identification 17 Disadvantages of Early Identification 18 Status on Criterion Measures 20 Problems in Identification 23 Kindergarten Screening 24 Screening Instruments 28 Human Figure Drawing Test 29 Copying of Geometric Shapes 31 Tests of Language Development 33 Knowledge of Letters and Numbers 35 Summary of Screening Tests 37 Part 2: Factors Which May Affect Prediction Research 38 Age at Entry 38 Gender Differences 39 Health Problems and Physical Problems 41 Educational Interventions 43 Remedial assistance and its effect on achievement. 44 Retention and its effect on achievement 47 Contextual Effects 50 v Part 3: Prediction Studies 55 Reliability and Validity 55 Methodological Paradox of Prediction-Performance Research . 58 Prediction-Performance Research 59 Correlation Analysis 59 T-tests and ANOVA 60 Discriminant Analysis 62 Prediction-Performance Matrices 63 Multiple Regression 68 Multilevel Modelling 69 Prediction Studies Summary 71 Summary 73 Chapter 3: Research Methodology 76 Introduction 76 Subjects 78 Procedures 79 Instruments 82 Draw-a-Person Test 83 Mann-Suiter Visual Motor Screen 83 Kindergarten Language Screening Test 84 Deverell Test of Letter and Numbers 84 Canadian Test of Basic Skills 85 Research Questions 85 Analysis of the Data 89 Data 89 Analyses 91 Preliminary Analyses 96 HLM Analysis 96 Threats to Validity 97 Summary 99 Chapter 4: Findings 100 Introduction 100 Correlation Matrix 100 Format of the Tables 102 Model I: Kindergarten Screening Measure/Achievement Relationships 103 Model II: Controlling for Pupil Characteristics 108 Model III: Controlling for Educational Interventions 112 Model IV: School-Level Variables 115 Parameter Variance Explained 118 Model V: Four Kindergarten Screening Measures in the Model . 120 Model VI: Simplified Models Including Only Significant Variables 124 Summary 124 vi Chapter 5: Summary and Conclusions 125 Overview of the Study 125 Principal Findings of the Study 126 Limitations of the Study 140 Implications of the Study 141 Recommendations for Future Research Research 144 References 148 Appendix A: Descriptors of Physical Problems 175 Appendix B: Number of Pupils Identified with Physical Problems and Number Receiving Interventions 176 Appendix C: Technical Information - Draw-A-Person 177 Appendix D: Technical Information - Mann-Suiter Visual Motor Screen 178 Appendix E: Technical Information - Kindergarten Language Screening Test 179 Appendix F: Technical Information - Deverell Test of Letters and Numbers 180 Appendix G: Technical Information - Canadian Tests of Basic Skills (CTBS) 182 Appendix H: Characteristics of Schools in the Study 183 Appendix I: Use of Grade Equivalent Scores 184 Appendix J: Data Plots Reflecting Interventions 187 Appendix K: Graphic Representation of Predictive Utility 191 Appendix Table 1: Summary of Selected Prediction Performance Studies 195 Appendix Tables 2: HLM Results for Attrition 201 Appendix Tables 3-20: HLM Results 202 Appendix Table 21: Means and Standard Deviations of Outcome Measures For Four Samples 220 Appendix Table 22: Prediction-Performance Matrix Analysis 221 vii List of Tables Page Table 1: Means, Standard Deviations, and Correlations of Student-Level Variables 101 Table 2: District and Sample Means and Intercepts for Pupils at the Risk Cut-off Score 103 Table 3: Estimates of the Effects on Grade Three Achievement of One Point (or one SD) Kindergarten Screening Measure Score 106 Table 4: Estimated Residual Parameter Variance of Mean Achievement for Pupils at the Cut-Off Score for Risk Status Ill Table 5: Estimated Coefficients for Kindergarten Screen/ Achievement Relationships 114 Table 6: "Null-Model" Estimates of Variance in Grade Three Achievement 119 Table 7: Average Achievement Scores for Pupils Who Scored at the Cut-off Point for "Risk" Status 122 viii List of Figures Page Figure 1: Effect of Changing the Cut-off Score of the Predictor or Criterion Measure 22 Figure 2: Prediction-Performance Comparison Matrix 64 Figure 3: Numerical Example of Prediction-Performance Matrix ... 66 Figure 4: Administration of Kindergarten Screening Measures ... 82 ix Acknowledgements I would like to thank the members of the committee for their support and participation, Dr. D. Willms, Dr. P. Leslie, Dr. J. Conry and Dr. O.A. Oldridge. I wish to extend my appreciation and gratitude to Doug Willms. He provided the opportunity to conduct the study. He patiently guided me through the hierarchical linear modelling analysis. He contributed ideas to the manuscript

Load more

  232

Copy Link