AEIXXX10.1177/1534508418771739Assessment for Effective InterventionReddy et al. 771739research-article2018 Special Series Article Assessment for Effective Intervention 2019, Vol. 44(2) 104 –119 Assessing the Effectiveness and © Hammill Institute on Disabilities 2018 Article reuse guidelines: sagepub.com/journals-permissions Interactions of Instructional Coaches: DOI:https://doi.org/10.1177/1534508418771739 10.1177/1534508418771739 Initial Psychometric Evidence for aei.sagepub.com the Instructional Coaching Assessments–Teacher Forms Linda A. Reddy, PhD1, Todd Glover, PhD1 , Alexander Kurz, PhD2, and Stephen N. Elliott, PhD2 Abstract The conceptual foundation and initial psychometric evidence are provided for the Instructional Coaching Rating Scales and Interaction Style Scales–Teacher Forms. These forms are part of a multicomponent online assessment system designed to evaluate the effectiveness of coaching skills and interactions that support the needs of teachers and students. Specifically, the article presents the theory, evidence, and measurement framework for the system. Findings indicate that the Rating Scales and Interaction Style Scales–Teacher Forms have very good internal structures based on multiple fit statistics for confirmatory factor analyses, high internal consistency, good item-to-scale total correlations, and freedom from item bias. Collectively, this promising statistical evidence is supportive of valid score inferences. Study limitations and directions for research are discussed. Keywords formative assessment, instructional coaching, formative feedback for coaches, educator improvement The implementation of Every Student Succeeds Act (ESSA, & Hasbrouck, 2009; Glover & Reddy, 2017). Although 2015) has placed greater responsibility on state education approaches to school-based coaching vary in scope and departments for effective educator evaluation and profes- foci, many assert coaching models share a focus on priori- sional development (PD). As a result, states are implement- tizing instructional needs, established goal-driven plans of ing changes to enhance PD that can improve classroom support, modeling, facilitating teacher practice, and provid- instruction and result in greater student achievement and ing ongoing regular feedback to promote high-fidelity overall school improvement (Mead, 2012; Reddy, Kettler, instructional practices (Denton & Hasbrouck, 2009; Erchul, & Kurz, 2015; Sawchuk, 2016). 2015; Kurz, Reddy, & Glover, 2017; Showers, Joyce, & Instructional coaching, a job-embedded PD approach, is a Bennett, 1987). Coaching is distinct from consultation in means of overcoming the limitations of workshop-based PD that coaching is primarily designed to support teachers’ with respect to the transfer of knowledge and skills into class- continuous PD. Many school-based coaches were former room practices (Gulamhussein, 2013). Despite the increased effective classroom teachers, whereas consultants tend to popularity of instructional coaching in schools, research is have specialized training and experiences in psychology, needed to better operationalize, assess, and evaluate the business, and/or other allied fields (e.g., occupational ther- implementation of essential coaching components (Denton apy, speech and language, and behavior interventions). & Hasbrouck, 2009; Glover & Reddy, 2017). The purpose of Often coaches reside within the district or school with the research reported is to provide initial psychometric evi- supervision from an administrative leader (i.e., curriculum dence for a multicomponent system for assessing research- based skills and interactions of instructional coaching. 1Rutgers University, Piscataway, NJ, USA 2Arizona State University, Tempe, USA Instructional Coaching to Enhance PD Corresponding Author: and Improve Student Achievement Linda A. Reddy, Graduate School of Applied and Professional Psychology, Rutgers University, 152 Frelinghuysen Road, Piscataway, NJ Most scholars agree that coaching is a job-embedded, indi- 08854, USA. vidualized, data-driven, and sustained practice (e.g., Denton Email: [email protected] Reddy et al. 105 director and assistant superintendent). Given the distinct published article that described the content validity of two role of coaches in schools, ongoing evaluation of key coach- Literacy Coach Appraisal Instruments (Lane, Robbins, & ing competencies via technically sound assessments is Price, 2013). Specifically, these assessments utilized a essential to enhancing effective practices. 4-point Likert-type scale measuring eight broad domains of Although, as outlined in the ESSA (2015), instructional coaching (e.g., Curriculum, Teachers; Staff Development; coaching is important for maximizing effective instruction, Technology; Liaison; Assessment; Home and Family there are few research-based coaching models and even Education; Resource Management; Professional fewer coaching measures and resources available to assess Disposition). The Literacy Coach Appraisal Instruments, and develop coaching skills and interactions. Most coach- however, do not have evidence of reliability and construct ing approaches and assessments are also narrow in scope, validity and do not offer multirater assessment. In summary, focusing on content-specific practices such as literacy/read- none of the 26 assessments measure content-neutral, core ing, mathematics, and science (e.g., American Institutes for process-focused coaching skills and interactions, nor do they Research [AIR], 2005; APQC Education Advanced offer multirater, online applications. As with teachers, there Working Group, 2011). The effectiveness of coaching also is an ongoing need to evaluate coaches in a variety of con- varies widely, often due to a lack of adherence to a defined texts supporting teachers in various instructional areas coaching process and a clear focus on research-based (Denton & Hasbrouck, 2009; Glover & Reddy, 2017; Kurz coaching actions and outcomes that guide PD (e.g., Denton et al., 2017; Showers et al., 1987). Despite the range of & Hasbrouck, 2009; Glover & Reddy, 2017). For example, coaching practices, the literature in education, sports, and without explicit coaching guidelines and assessment-driven executive coaching business points to core process-focused performance feedback, coaches in the Reading First skills and interactions that drive coaching effectiveness (e.g., Program from the No Child Left Behind Act of 2001 imple- Denton & Hasbrouck, 2009; Kurz et al., 2017; Showers mented a range of loosely defined activities to support et al., 1987). These core process-focused skills and interac- teachers across the nation, resulting in suboptimal teacher tions are important for advancing teacher practice and stu- and student outcomes (AIR, 2005; Deussen, Coskie, dent outcomes and can be evaluated via a content-neutral Robinson, & Autio, 2007). Thus, when the process of assessment approach. Teacher assessments of coaches’ per- coaching is poorly defined, assessed, and monitored over formance are essential for improving the key skills and inter- time, the consequences often are not positive, resulting in actions of coaches and the overall process of coaching for missed opportunities to improve instruction that enhances the PD of teachers. This study was the initial investigation of student achievement and school effectiveness. the utility and validity of the Instructional Coaching Current evaluation practices of instructional coaches Assessments–Teacher Forms. Subsequent research is under- rely on tools and standards meant for classroom teachers. way with the two remaining Assessment forms—supervi- This is a significant problem because the process, actions, sors and coaches themselves—in the 360 or comprehensive and intended outcomes for classroom teaching versus multirater instructional coaching system. instructional coaching are different. School leaders often charged with evaluating coaching staff must decide what The Development of an Assessment teacher standards to use when making effectiveness deci- Designed to Improve Instructional sions and PD recommendations. This can lead to inconsis- tent coaching evaluation practices and poorly defined Coaching feedback for PD (Killion, Harrison, Bryan, & Clifton, The Instructional Coaching Assessment (Reddy, Glover, 2012). Thus, providing instructional coaches skill-focused Kurz, & Elliott, 2017) is an online, multirater assessment and valid assessment-based performance feedback is criti- system that provides feedback reports to support the eval- cal to their development and effectiveness in fostering uation and development of instructional coaching talent. teacher PD ( Reddy, Dudek, & Lekwa 2017). The assessment approach involves conducting a 360° Scholars have reported that the provision of formative assessment, completed by the coach, teachers served by feedback can effectively modify learners’ knowledge, think- the coach, and/or the coach’s supervisor. A 360° assess- ing, or behavior that lead to improved learning (e.g., Brophy, ment offers a comprehensive assessment of coaching 1981; Hagermoser Sanetti, Luiselli, & Handler, 2007; Noell, effectiveness and interactions by capturing feedback from Witt, Gilbertson, Ranier, & Freeland, 1997; Schwartz & key stakeholders involved in the coaching process. Each White, 2000; Shute, 2007). Unfortunately, the availability of stakeholder provides unique and complimentary perspec- coaching assessments is sparse. A comprehensive literature tives on the coaching process that is valuable