University of Massachusetts Amherst ScholarWorks@UMass Amherst Doctoral Dissertations 1896 - February 2014 1-1-2007 Promoting the early identification of internalizing problems in preliterate children : development of the Watkins early self-report of internalizing problems. Maren L. Watkins University of Massachusetts Amherst Follow this and additional works at: https://scholarworks.umass.edu/dissertations_1 Recommended Citation Watkins, Maren L., "Promoting the early identification of internalizing problems in preliterate children : development of the Watkins early self-report of internalizing problems." (2007). Doctoral Dissertations 1896 - February 2014. 5800. https://scholarworks.umass.edu/dissertations_1/5800 This Open Access Dissertation is brought to you for free and open access by ScholarWorks@UMass Amherst. It has been accepted for inclusion in Doctoral Dissertations 1896 - February 2014 by an authorized administrator of ScholarWorks@UMass Amherst. For more information, please contact [email protected]. PROMOTING THE EARLY IDENTIFICATION OF INTERNALIZING PROBLEMS IN PRELITERATE CHILDREN: DEVELOPMENT OF THE WATKINS EARLY SELF-REPORT OF INTERNALIZING PROBLEMS A Dissertation Presented by MAREN L. WATKINS Submitted to the Graduate School of the University of Massachusetts Amherst in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY September 2007 SCHOOL PSYCHOLOGY © Copyright by Maren L. Watkins 2007 All Rights Reserved PROMOTING THE EARLY IDENTIFICATION OF INTERNALIZING PROBLEMS IN PRELITERATE CHILDREN: DEVELOPMENT OF THE WATKINS EARLY SELF-REPORT OF INTERNALIZING PROBLEMS A Dissertation Presented by MAREN L. WATKINS CraigS. Wells, Member Christine McCormick, Department Head Dean of the School of Education DEDICATION To Melissa Gould, Tom Gould and Kevin Calzia... for your love, encouragement and patience. ACKNOWLEDGEMENTS I would like to express my sincere appreciation to the people who have helped me along the way since I started graduate school, and throughout the dissertation process. First, I would like to thank my advisor, dissertation committee chair, and friend, Gary Stoner for his advice, support and reassuring calm composure throughout my academic career. Thank you to John Hintze for inspiring my interest in assessment through his enthusiasm and knowledge. To Craig Wells, who helped me make sense of my data. Thanks to Chris Overtree, for his support and for serving on my dissertation committee. Also, thank you to Bill Matthews. I would also like to extend thanks to Jane Riley, Julie Clarfield, Pamela Plumer and Amanda Marcotte, my support system and distraction during graduate school. In addition, Geri O’Brien and Scott Methe who showed me the dissertation “ropes” while providing supervision during my advanced practicum. Thank you to my amazing team of research assistants: Tara Clark, Kristin Ezbicki, Andrew Koemer, Lisa Mirabito, Hadley Moore, JoAnne Morgan, Amanda Nnachetam, Pamela Plumer, Jane Riley, Jessie Tailing and Jamie Udaka. Without your assistance I could not have managed this project. I hope you all find such helpful research assistants when your time comes to collect data. Finally, to the principals, teachers, and support staff who allowed me into their schools, and believed that my research was valuable. In particular, thank you to Michelle Bienvenue, Catherine Brandon, Lisa Buchinski, Ann Marie Carpenter, and Elizabeth Neale. v ABSTRACT PROMOTING THE EARLY IDENTIFICATION OF INTERNALIZING PROBLEMS IN PRELITERATE CHILDREN: DEVELOPMENT OF THE WATKINS EARLY SELF REPORT OF INTERNALIZING PROBLEMS SEPTEMBER 2007 MAREN L. WATKINS, BA, DREW UNIVERSITY M.Ed., UNIVERSITY OF MASSASHUSETTS AMHERST Ph.D., UNIVERSITY OF MASSACHUSETTS AMHERST Directed by: Professor Gary Stoner A variety of academic, behavioral and social problems are first identified when children begin school as schools have an obligation to identify a child’s needs. However, internalizing problems such as anxiety and depression often go unnoticed due to their covert and subjective symptoms. Not readily apparent to observers, internalizing problems may be best identified through self-report. However, without an instrument to identify internalizing problems, children may suffer silently. Due to developmental limitations of young children to self-report options are limited to individual interviews, which are impractical for screening large groups of children. Thus, the purpose of this study was to develop an instrument that could enable children in Kindergarten and first grade to self-report their experiences of internalizing problems, in a wide-scale screening procedure. The Watkins Early Self-Report of Internalizing Problems (WESRIP) was created using pictures and oral administration to enable children to independently self-report their symptoms of internalizing problems, thus allowing large groups of children to be screened simultaneously. Kindergarten and first grade children (n = 235) consented to the study. Three forms vi of the WESRIP containing separate test items were administered to groups of students. The 26 items with the highest item-total correlations were chosen for the revised WESRIP that was administered in a pre-test (n = 207) and post-test (n= 197). Teachers completed a modified version of the Systematic Screening for Behavior Disorders (SSBD) for concurrent validity analysis. The WESRIP was found to have adequate internal consistency for screening decisions, and moderate test-retest reliability for this age group. Through factor analysis, two distinct factors were identified, “Physical and Emotional Manifestations of Internalizing Problems,” and “Self-Appraisal.” However there was no relationship between the self-report and teacher rankings, limiting concurrent validity. Without a comparable criterion instrument, diagnostic accuracy was not feasible. Further research is still needed in order to make the WESRIP a technically sound and useful tool. The WESRIP may one day serve to validly and reliably screen children who could benefit from further assessment and ultimately benefit children through early identification of internalizing problems and improved outcomes. Vll TABLE OF CONTENTS Page ACKNOWLEDGEMENTS.v ABSTRACT.vi LIST OF TABLES.xi LIST OF FIGURES.xii CHAPTER 1. STATEMENT OF THE PROBLEM.1 Schools as Screening Agents.1 Internalizing Disorders in Children. 6 Depression.10 Anxiety.12 Somatization.14 Other Characteristics.14 Early Identification.15 Available Tools and Their Limitations.17 Purpose of Study.22 2. LITERATURE REVIEW.25 Evaluation Procedures for Literature Review on Children as Informants.26 Children as Informants of Their Feelings and Behaviors: Creating the Mosaic Picture of a Child.27 Informant Reports.28 Inter-Rater Agreement.30 Child Self-Report.36 Developmental Differences and Self-Report.39 Evaluation Procedures for Literature Review of Available Instruments.42 Assessing Available Instruments.54 Broadband Rating Scales.57 Narrowband Rating Scales.64 Syndrome-Specific Rating Scales.66 Anxiety.67 Depression.68 3. METHOD.72 Instrument..72 Instrument Development Process.73 Initial Scale Development: Phase 1.73 Initial Scale Development: Teacher Survey.74 Initial Scale Development: Teacher Feedback Modifications.76 Participants and Setting.79 Research Design.80 Procedure.80 Procedure Phase 1.81 Procedure Phase II.81 Criterion Measure.82 Systematic Screening for Behavior Disorders (SSBD).82 Social Validity.84 Teacher Feedback Survey.84 4. RESULTS.85 Phase 1.85 Descriptive Statistics.89 Research Question 1: Internal Consistency.96 Research Question 2: Factor Analysis.97 Research Question 3: Test-Retest Reliability.103 Research Question 4: Concurrent Validity.106 Research Question 5: Accuracy/Predictive Power.107 5. DISCUSSION.112 Results of the Present Study Compared to Previous Research in this Area.114 Internal Consistency.115 Factor Structure.116 Test-Retest Reliability.118 Concurrent Validity.121 Accuracy.123 Teacher Feedback Survey.127 Limitations of the WES RIP Research.128 Internal Consistency.129 Factor Structure.129 Test-Retest Reliability.130 Concurrent Validity.130 ix Diagnostic Accuracy.131 Sample Size.131 Future Research.132 Implications, Summary and Conclusions.135 APPENDICES A. INITIAL TEACHER SURVEY.139 B. READABILITY AND VOCABULARY.143 C. WESRIP TEST FORMS.146 D. FINAL WESRIP ITEMS.149 E. MODIFIED SYSTEMATIC SCREENING FOR BEHAVIOR DISORDERS.150 F. ELECTRONIC TEACHER FEEDBACK SURVEY.152 BIBLIOGRAPHY.154 x LIST OF TABLES Table Page 1.1 Symptoms of Depression in Children.11 2.1 Behavior Assessments and Personality Tests for Children, with Anxiety and/or Depression Components.44 3.1 Words and Phrases Considered Difficult for Kindergarten and First Grade Students According to Teachers Surveyed.76 3.2 Words and Phrases Considered Understandable for Kindergarten and First Grade Students, According to Teachers Surveyed. 78 4.1 Scale Score Statistics.86 4.2 Initial Test Development Item-Total Statistics.87 4.3 Descriptive Statistics for WESRIP Pre-test and Post-test Administration.90 4.4 Pre-Test Total Score Frequency Table.91 4.5 Post-Test Total Score Frequency Table.94 4.6 Reliability Statistics.97 4.7 Pre-Test and Post-Test Factor Loadings.100 4.8 Pre-Test and Post-Test Item and Item-Total Statistics.105 4.9 Biserial Correlation.107 4.10 Pre-Test and Teacher Ranking ROC Curve.108 4.11 Post-Test and Teacher Ranking ROC Curve.109 LIST OF FIGURES FIGURE Page 4.1 Pre-Test Total Score Frequency Histogram...92 4.2 Post-Test Total Score Frequency Histogram.96