University of Florida Thesis Or Dissertation Formatting

EXPLORING COACH-TEACHER INTERACTIONS WITHIN A PRACTICE-BASED COACHING PARTNERSHIP

DARBIANNE SHANNON

A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA

2017

To seeking important growth opportunities: “Do the best you can until you know better. Then when you know better, do better.” ~Maya Angelou

ACKNOWLEDGMENTS

I would first like to thank the members of my doctoral committee. I am grateful for my chair, Dr. Elizabeth Bondy, for her passion to learn both with and from her students.

Her support and thoughtful questions have made me consider the precision and meaning of my words more deeply. I am also immensely thankful for my co-chair, Dr.

Patricia Snyder, for her wisdom, constructive feedback, guidance, and confidence in me and my work. Her model of scholarship, service, and advocacy for teachers, young children, and their families has helped me to stay focused on why the integrity of our work is of the utmost importance. I appreciate Dr. Nancy Dana for sharing her insight and experience in the field of professional development, with a particular focus on supporting teachers’ self-inquiry regarding current practice and integration of new knowledge and skills into their teaching repertoire. I thank Dr. Mary McLean for her ongoing encouragement, knowledge, and dedication to making recommended and evidence-based practices accessible to all stakeholders who support families of young children with disabilities.

I would like to express my gratitude to the Institute of Education Sciences for their continued support of the Impact of Professional Development on Preschool

Teachers’ Use of Embedded Instruction Practices work (R324A07008 and

R324A150076) and to Dr. Patricia Snyder, the principal investigator, for allowing me to access a portion of the coaching database for my dissertation. Without the support of the partner school districts, teachers, children, and families who participated in the study, this research would not be possible.

I am sincerely grateful for the opportunity to work at the Anita Zucker Center for

Excellence in Early Childhood Studies at the University of Florida. Anita Zucker’s vision

and commitment to interdisciplinary scholarship with the capacity to impact research, policy, practice, and the lives of young children and their families is exceptional. The mentorship provided by each of the faculty and the applied research opportunities I experienced through the Center have shaped my knowledge, beliefs, and professional goals.

I would also like to thank my current and former colleagues in the Snyder research lab at the University of Florida for their friendship and encouragement. My doctoral program would not have been the same without the opportunity to collaborate with and learn alongside each of them. I owe a special thanks to my secondary coder

Lauren Buroker for her conscientious questions and dedication to learning the coding system. I am also truly appreciative for the embedded instruction coaching team at the

University of Florida and Vanderbilt University, for their collaboration, thoughts, resources, feedback, and dedication to supporting our coaching team, teachers, and the children in their classrooms.

Finally, I am most grateful for the love, support, and encouragement of my family and close friends. They have always believed in me, nurtured my passion for education, and demonstrated for me how informed actions, collaboration, quiet dedication, and perseverance can make a difference in the community and the lives of others.

TABLE OF CONTENTS

page

ACKNOWLEDGMENTS ...... 4

LIST OF TABLES ...... 10

LIST OF FIGURES ...... 12

LIST OF ABBREVIATIONS ...... 13

ABSTRACT ...... 14

CHAPTER

1 INTRODUCTION ...... 16

Rationale for the Study ...... 20 Dimensions of Follow-Up Support ...... 21 Empirical Efforts to Report the Process Dimension ...... 22 Practice-based Coaching ...... 23 Measuring Fidelity of Implementation of PBC ...... 27 Teacher and Child Outcomes Associated with PBC ...... 28 Embedded Instruction Practices and PBC ...... 29 Purpose Statement ...... 30 Significance ...... 32 Delimitations ...... 34 Limitations ...... 35 Summary ...... 36

2 REVIEW OF THE LITERATURE ...... 38

Search Procedures ...... 39 Early Childhood Professional Development ...... 40 Defining Professional Development ...... 42 Aligning Professional Development Inputs and Outcomes ...... 43 Features of Quality Professional Development ...... 44 Trends in Early Childhood Professional Development Research ...... 45 Use of systematic follow-up support ...... 46 Trends in studies classified as coaching ...... 46 Studies with PBC components ...... 48 Reporting limitations in systematic follow-up implementation support ...... 49 Recommendations for enhanced reporting ...... 49 Summary of Early Childhood Professional Development ...... 50 Implementing PBC in Early Childhood Contexts ...... 50 Evidence for the PBC Components ...... 51 Teaching practices ...... 51

Collaborative partnership ...... 52 Strengths and needs assessment to inform shared goal setting and action planning ...... 52 Focused observation ...... 53 Reflection and feedback ...... 53 Intervention Studies Implementing Practice-based Coaching ...... 54 Teacher outcomes ...... 54 Child outcomes ...... 56 Summary of Practice-based Coaching Literature ...... 56 Coaches as Mediators of Teacher’s Coaching Experiences ...... 56 Using Observational Coding Systems to Describe Coach Behavior ...... 57 Empirical Approach of Studies Using Observational Coding Systems ...... 57 Gaps and Recommendations for Future Studies ...... 60 Summary of Observational Coding Systems ...... 60

3 METHODOLOGY ...... 73

Research Questions ...... 74 Context for the Present Study ...... 75 Introduction of the Teaching Practices through Workshops ...... 76 On-site Practice-based Coaching ...... 78 Across coaching sessions ...... 78 Within each coaching session ...... 80 The Coaching Practices Observation Tool ...... 81 Step 1–Reviewing the Coaching Literature ...... 82 Step 2–Operationally Defining Coaching Process ...... 82 Step 3–Applying the Codes to Video ...... 83 Step 4–Conducting a Pilot Study of the CPOT-PV2 ...... 84 Step 5–Piloting the Revised CPOT-PV2 System Using Observational Coding Software ...... 86 Measures ...... 87 Coaching Practices Observation Tool ...... 87 Demographic Questionnaires ...... 88 Description of the classroom and teacher profile ...... 88 Description of the project staff demographic form ...... 89 PBC Weekly Coaching Session Data ...... 89 Coaching dose ...... 89 Coaching fidelity ...... 90 Action plans ...... 91 Teaching practices matrix ...... 91 Participants ...... 91 Coach Training and Support Procedures ...... 92 Training ...... 92 On-going implementation support for coaches ...... 92 Fidelity feedback ...... 93 Teacher Recruitment ...... 93 Procedures ...... 94

Video Coder Training Procedures ...... 94 Selection of Coaching Debrief Videos ...... 95 Coding Procedures ...... 96 Counting coach verbal behavior ...... 97 Duration of conversation focus and initiation...... 97 Interobserver Agreement ...... 98 Data Extraction and Analyses ...... 99 Conversation focus ...... 99 Initiations ...... 100 Overall verbal behavior ...... 101 Verbal behavior by conversation focus ...... 102 Coach differences ...... 102

4 RESULTS ...... 116

Participant Demographics ...... 117 Coaching Implementation Variables ...... 118 Interobserver Agreement ...... 119 Conversation Focus ...... 119 Research Question 1: Proportion of Time Spent in Each Conversation Focus ...... 120 Research Question 1a: Coach Differences in Percent of Time Spent in Each Conversation Focus ...... 120 Research Question 2: Proportion of Time Spent in Each Conversation Focus Across Occasions ...... 121 Research Question 2a: Coach Differences in Proportion of Time Spent in Each Conversation Focus Across Occasions ...... 121 Research Question 3: Coach versus Teacher Initiations ...... 122 Research Question 3a: Proportion of Coach and Teacher Initiations by Conversation Focus ...... 123 Research Question 3b: Coach Differences in the Proportion of Coach and Teacher Initiations by Conversation Focus ...... 123 Research Question 4: Proportion of Coach and Teacher Initiations Across Occasions ...... 124 Research Question 4a: Proportion of Coach and Teacher Initiations Across Time by Conversation Focus ...... 124 Research Question 4b: Coach Differences in the Proportion of Coach and Teacher Initiations Across Occasions ...... 125 Coach Verbal Behavior ...... 125 Research Question 5: Rate of Coach Verbal Behavior ...... 126 Research Question 5a: Coach Differences in Rate of Verbal Behavior ...... 126 Research Question 6: Rate of Coach Verbal Behavior Across Occasions ..... 127 Research Question 6a: Coach Differences in Rate of Verbal Behavior Across Occasions ...... 128 Research Question 7: Rate of Coach Verbal Behavior by Conversation Focus ...... 128

Research Question 7b: Coach Differences in Rate of Verbal Behavior by Conversation Focus ...... 129 Coach-Teacher Dyads ...... 129 Coach and Teacher Dyad 1 ...... 130 Coach and Teacher Dyad 2 ...... 131 Coach and Teacher Dyad 3 ...... 133 Coach and Teacher Dyad 4 ...... 135 Coach and Teacher Dyad 5 ...... 137 Coach and Teacher Dyad 6 ...... 139 Coach and Teacher Dyad 7 ...... 141

5 DISCUSSION ...... 175

Interpretation of Findings ...... 175 Fidelity of Implementation ...... 175 Coaching Implementation Quality ...... 179 Conversation foci ...... 180 Verbal behavior ...... 183 The Influence of Demographics on Conversation Foci and Verbal Behavior . 193 Dyadic Transactions Across the Collaborative Partnership ...... 195 Conversation foci ...... 197 Coach-teacher initiations ...... 198 Building Teacher’s Confidence and Competence Towards Sustainability ...... 200 Implications from the Present Study ...... 201 Implications for Coach-based Professional Development ...... 201 Methodological Implications for Defining and Describing “Coaching” ...... 204 Recommendations for Future Research ...... 206 Overall Summary ...... 209

LIST OF REFERENCES ...... 213

BIOGRAPHICAL SKETCH ...... 224

LIST OF TABLES

Table page

2-1 The “who”: recipients of systematic follow-up...... 62

2-2 The “who”: providers of systematic follow-up...... 63

2-3 The “what” of follow-up...... 64

2-4 The “how” of systematic follow-up...... 65

2-5 Coaching studies aligned with PBC coaching components or strategies...... 66

2-6 Empirical evidence for practice-based coaching...... 67

3-1 Percentage of total time in each instructional learning format for cohort 1 fall modules across sites...... 104

3-2 Coaching practices observation tool-pilot version 1...... 105

3-3 Coaching practice observation tool-pilot version 2...... 106

3-4 Coaching practices observation tool-research version 1.0 operational definitions...... 107

3-5 Cohort 1 Coaching fidelity by debrief session type for larger study...... 109

3-6 Available video footage from larger study for included dyads...... 110

4-1 Coach demographics...... 145

4-2 Teacher demographics...... 146

4-3 Comparison of implementation of select coaching protocol variables for all teachers coached by each coach and for each teacher coached in the present study ...... 147

4-4 Duration of conversation focus with initiation modifier occurrence agreement by code...... 148

4-5 Verbal behavior occurrence agreement by code...... 149

4-6 Percent occurrence agreement and Cohen’s kappa by coaching debrief meeting video...... 150

4-7 Percent of time in each conversation focus...... 151

4-8 Percent of time in each conversation focus by dyad...... 152

4-9 Percent of time in each conversation focus for each dyad across occasions. .. 153

4-10 Percent initiations by coaches and teachers by conversation foci...... 155

4-11 Percent of coach and teacher initiations by conversation focus...... 156

4-12 Percent of coach and teacher initiations across occasions...... 157

4-13 Percent of coach and teacher initiations across time...... 158

4-14 Rate of coach verbal behavior per 30 minutes...... 159

4-15 Coach differences in rate of coach verbal behavior per 30 minutes...... 160

4-16 Rate of coach verbal behavior per 30 minutes across occasions...... 161

4-17 Rate of coach verbal behavior within each conversation focus per 30 minutes...... 162

4-18 Coach 1 rate and type of verbal behavior per 30 min by conversation focus. .. 163

4-19 Coach 2 rate and type of verbal behavior per 30 min by conversation focus. .. 164

4-20 Coach 3 rate and type of verbal behavior per 30 min by conversation focus. .. 165

4-21 Coach 4 rate and type of verbal behavior per 30 min by conversation focus. .. 166

4-22 Coach 5 rate and type of verbal behavior per 30 min by conversation focus. .. 167

4-23 Coach 6 rate and type of verbal behavior per 30 min by conversation focus. .. 168

4-24 Coach 7 rate and type of verbal behavior per 30 min by conversation focus. .. 169

LIST OF FIGURES

Figure page

1-1 Components of the practice-based coaching framework...... 37

3-1 Phases of the coaching partnership...... 111

3-2 Coaching protocol components by session...... 112

3-3 A page from the coaching practices observation tool-pilot version 1...... 113

3-4 Coded transcript of a coaching debrief meeting...... 114

3-5 Video sampling for present study...... 115

4-1 Percent of total time in each conversation foci...... 170

4-2 Percent of time within each conversation foci by coach...... 171

4-3 Percent of total time in each conversation foci across time...... 172

4-4 Rate of coach verbal behavior per 30 minutes across time...... 173

4-5 Rate of coach verbal behavior per 30 minutes within each conversation foci. .. 174

LIST OF ABBREVIATIONS

CPOT-RVI Coaching Practices Observation Tool Research Version 1.0

EBP Evidence-based Practice

ECPDD Early Childhood Professional Development Database

EI Early Intervention

FGRBI Family Guided Routines Based Intervention

IES Institute of Education Sciences

IFSP Individualized Family Support Plan

NICHD National Institute of Child Health and Human Development

NPDCI National Professional Development Center on Inclusion

NRC National Research Council

PBC Practice-based Coaching

PD Professional Development

PRISMA Preferred Reporting Items for Systematic Reviews and Meta Analyses

TfT Institute of Education Sciences Goal 3 efficacy trial of Tools for Teachers

Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy

EXPLORING COACH-TEACHER INTERACTIONS WITHIN A PRACTICE-BASED COACHING PARTNERSHIP

Darbianne Shannon

August 2017

Chair: Elizabeth Bondy Co-chair: Patricia Snyder Major: Curriculum and Instruction

Professional development (PD) that includes coaching has demonstrated the capacity to effect teachers’ use of evidence-based practices (EBP) in their classrooms.

Not all studies involving coaching have reported positive outcomes, however.

Researchers have called for additional investigations that define and describe the

“active ingredients” of coaching, including content, structure, dose, and coaching processes that facilitate teachers’ acquisition of knowledge and skills.

The present study explored how coaches facilitated conversations during the debrief meeting component of an onsite Practice-based Coaching (PBC) partnership. A direct behavioral observation system was developed to investigate (a) the proportion of time spent in different conversational foci, including who initiated the conversation; (b) the verbal behaviors used by coaches; and (c) whether the conversation foci, initiations, and coach verbal behavior changed across three occasions for seven coach-teacher dyads. Data for the present study were collected as a part of a larger efficacy trial examining Tools for Teachers, a PD intervention designed to support classroom

teachers’ use of embedded instruction practices with young children with disabilities

(Snyder, Algina, Hemmeter, & McLean, 2015).

Results from the present study indicate coaches (a) spent the largest proportion of time in reflection and feedback and goal setting and action planning conversation foci and (b) used the study required verbal behaviors (supportive and constructive feedback and clarifying questions) across all sampled occasions. These findings are consistent with the PBC framework and associated protocols that guided the debrief meeting.

Dyads exhibited variability in the proportion of time spent in each conversation foci, coach versus teacher initiations, and the rate of coach verbal behavior. Some variability was related to participant characteristics. Change in coach and teacher behavior across three sampled occasions was evident.

Findings from the present study support the importance of operationally defining the components of coaching and the actions or behaviors of coaches related to these components. Further exploration of coach-teacher transactional interactions and their contributions to the collaborative coaching partnership are needed to advance the science of coach-based professional development and to make decisions about (a) accommodations and adaptations for individual teachers, and (b) thresholds of acceptable modification within structured coaching frameworks.

CHAPTER 1 INTRODUCTION

Interdisciplinary advances in neuroscience, economic, and education research have demonstrated high-quality early childhood experiences benefit the individual child and the community (Yoshikawa et al., 2013). High-quality early childhood experiences are associated with improved health, cognition, and social-emotional development later in life, providing the greatest benefit to children who are at-risk due to environmental or biological risk factors (Shonkoff, 2010). Preschool teachers1 are critical mediators of children’s early experiences. Yet, the entry-level and professional development (PD) requirements for teaching in state-funded preschool programs vary widely. In the 2015-

2016 school year, the National Institute for Early Education Research (NIEER) reported preschool programs existed in 44 of the United States, but only 25 states required a bachelor’s degree for lead teachers (Barnett et al., 2017). NIEER also found 37 states required inservice training of 15 hours or more per year; however, the quality of these training experiences has been described as inconsistent and rarely evaluated by the state for the impact it has on classroom practice (Barnett et al., 2017).

A survey of 3,750 early childhood educators, conducted by the National

Association for the Education of Young Children (NAEYC) in 2015, found 40% of teachers reported a lack of mentoring and guidance to be a critical barrier to their success in the profession, while 32% reported difficulty accessing relevant PD opportunities (NAEYC, 2015). The confidence and competence of teachers to provide

1 Teacher is used here to represent the adult who is responsible for providing interactional and instructional support to the child. In the early childhood context, this person may or may not hold a formal degree or teaching certificate.

high-quality preschool learning experiences for young children is critical for ensuring that federal and state funding allocated to increasing access to preschool education for more than 1.4 million children enrolled in preschool programs, as of the 2015-2016 school year, produces optimal child outcomes.

When seeking to support teachers’ implementation of evidence-based practices

(EBP), multi-component interventions, or curricula associated with high-quality early learning experiences, the PD supports provided to teachers should be informed by theory and research. Teacher PD is typically characterized as occurring along a continuum from preservice to inservice. Preservice PD experiences occur prior to the teacher’s entry into the profession or classroom and often result in a degree or certification from an institution of higher education. In contrast, inservice PD is designed to enhance the knowledge and practice of teachers who are already working with children and families in classrooms or other settings. Teachers working with young children birth to age five in the United States often enter the profession with little or no preservice training, making inservice training critical for increasing the quality of education and care (Gomez, Kagan, & Fox, 2015). For the purpose of the present study, PD is defined as “facilitated teaching and learning experiences designed to enhance practitioners’ knowledge, skills, and dispositions as well as their capacity to provide high quality early learning experiences for young children” (Snyder et al., 2012, p.188). Furthermore, the present study and literature presented is primarily focused on inservice PD, the context where the study data were collected.

Recent reviews of the PD literature have identified key features most likely to support teachers’ dispositions towards, and knowledge and implementation of, EBP in

the early childhood context (e.g., Artman-Meeker, Fettig, Barton, Penney, & Zeng, 2015;

Diamond, Justice, Siegler, & Snyder, 2013; Schacter, 2015; Sheridan, Edwards, Marvin,

& Knoche, 2009; Snyder et al., 2012; Trivette, Dunst, Hamby, & O’Herin, 2009; Winton,

Snyder, & Goffin, 2015; Zaslow, Tout, Halle, Whittaker, & Lavelle, 2010). These features include (a) the application of adult learning principles; (b) an explicit statement of the learning objective(s); (c) a focus on teaching practices that link research, theory, and practice; (d) opportunities for active participation including planning, practice with feedback, and self-evaluation; (e) content that is aligned with teachers’ professional values and context standards; and (f) the intensity of the PD, including the alignment between follow-up support and the desired PD outcomes.

The PD literature has also established that episodic and disjointed workshops generally are not effective for facilitating sustained learning or development in teachers’ knowledge, practices, or dispositions (Snyder, Hemmeter, & McLaughlin, 2011). A meta-analysis of PD studies delivered in K-12 settings, conducted by Joyce and

Showers (2002), suggested that theory and discussion, plus demonstrations in training, plus practice and feedback in training, plus coaching in the classroom are most likely to lead to teachers’ use of practices in classrooms. An estimated 95% of participants in the studies analyzed achieved the desired outcome of practice use in the classroom when these components were used in combination. These findings and more contemporary applications of this research suggest workshops may continue to be a valuable component of PD packages for delivering content and achieving knowledge- and application-based outcomes, when followed by more individualized, job-embedded

supports such as mentoring, consultation, and coaching to enhance implementation of knowledge and skills in practice settings (cf. Trivette et. al, 2009).

A number of studies conducted with preschool teachers have demonstrated that teachers who receive inservice learning experiences plus job-embedded support in their classroom are more likely to improve their practice implementation when compared to

(a) teachers who received materials, workshops, or coursework in the absence of job- embedded support (e.g., Neuman & Cunningham, 2009; Neuman & Wright, 2010;

Pianta, Mashburn, Downer, Hamre, & Justice, 2008), and (b) teachers who participated in more traditional PD provided by their district, grantee, agency, or program (e.g.

Greenwood, Abbot, Beecher, Atwater, & Petersen, 2017; Landry, Swank, Anthony &

Assel, 2009; Neuman & Cunningham, 2009; Powell, Diamond, Burchinal, & Koehler,

2010; Powell, Steed, & Diamond, 2010; Sutherland, Conroy, Vo, & Ladwig, 2015;

Hemmeter, Snyder, Fox, & Algina, 2016).

The authors in these studies classified job-embedded support using various terms including mentoring, consultation, and coaching; however, the activities associated with the support provided were inconsistent across terms. As contemporary researchers investigate components of early childhood PD most likely to promote teachers’ implementation of EBP and child outcomes, it is critically important to move beyond vague terminology toward a more explicit description of (a) what the focus and desired outcomes of the PD experience were, (b) who the participants are and the context in which they work, and (c) how participants were supported to achieve the desired outcomes (National Professional Development Center on Inclusion; NPDCI,

2008). The purpose of the present study was to explore and describe how coaches

facilitated coach-teacher interactions within a Practice-based Coaching (PBC) framework to enhance teachers’ knowledge of and capacity to implement EBPs in public preschool classrooms.

Rationale for the Study

Substantial evidence suggests high-quality early learning experiences are contingent on teachers who are well prepared at entry and supported by ongoing PD.

Furthermore, intentional alignment between the desired outcomes of PD and the type and intensity of support provided is necessary for programs to use limited resources efficiently and for improving teacher quality (Harris, 1980; McCollum & Cattlet, 1997;

Snyder & Wolfe, 2008; Winton et al., 2015; Zaslow et al., 2010). Recent reviews of the

PD literature provide evidence that workshops in combination with follow-up implementation support are increasingly prevalent, where enhanced knowledge and classroom practice is the desired outcome. For example, Snyder et al. (2012) found in a literature review of early childhood PD studies that 74% of the 256 studies included some type of follow-up support, with a larger proportion of studies using follow-up support occurring in the last 10 years of the review. Early childhood PD research has made an important shift toward more empirically sound approaches to scaffolding teachers’ learning using job-embedded follow-up implementation supports. However, follow-up supports are often not well defined in the available literature, with important details about content, structure, and process dimensions being underreported (Powell &

Diamond, 2013, Snyder et al. 2011).

The absence of detailed reporting presents two significant challenges: (1) replication and meta-analytic studies cannot be conducted, preventing the accumulation of knowledge regarding how to effectively and efficiently use PD and coaching

resources; and (2) readers might be misled about the fiscal and human resources required to achieve similar outcomes, preventing the successful installation and scaling up of promising PD interventions. The argument for more comprehensive and thorough reporting of each dimension of the PD intervention variable, including workshops and follow-up supports, is well documented as the field seeks to advance the science of PD implementation (Halle, Metz, & Martinez-Beck, 2013, Snyder et al., 2011), including implementation supports such as coaching (Artman-Meeker et al., 2015; Isner et al.,

2011; Snyder et al., 2012).

Dimensions of Follow-Up Support

Systematic follow-up to support the implementation of coaching is composed of three dimensions — content, structure, and process (Powell & Diamond, 2013b). The content dimension represents the knowledge, practices, or dispositions that are the focus of coaching. The structural dimension characterizes the number, duration, frequency, interval, and delivery format of coaching sessions, which together make up the dose and intensity of support provided. The process dimension of coaching includes the coaching behaviors, strategies, and materials used to promote the teacher’s confidence, competence, and use of an identified set of teaching practices.

The content of coaching is established through reviews of the empirical literature prior to initiating a PD intervention and often includes information about (a) what children need to know and be able to do, and (b) behaviors or practices the teacher can use to support child learning. Researchers have documented this dimension through the use of PD scripts, fidelity checklists, and printed manuals and guides for both PD trainers and teachers (e.g., Conroy et al., 2015; Neuman & Wright, 2010; Powell &

Diamond, 2013; Snyder, Hemmeter, & Fox, 2015). Similarly, the documentation of

structural coaching features can be accomplished through the use of coaching logs and protocols in combination with training for coaches on how to record their frequency, duration, and type of contact with teachers after it occurs (Durlak & Dupre, 2008;

Wolery, 2011; Snyder et al., 2015). In contrast, process features are guided by training on the use of coaching protocols and materials, but the composition of behaviors, strategies, and materials are often by design intended for flexible and responsive use with teachers to meet their unique classroom context, preferences, motivations, and individual learning goals (Diamond & Powell, 2013; Snyder et al., 2015). This differentiation aligns with the principles of adult learning, but presents a challenge for the accurate documentation of how various operationalized components are implemented during follow-up support interactions between coaches and teachers.

Empirical Efforts to Report the Process Dimension

Coach self-report of protocol implementation can be unreliable and ineffective, in the absence of (a) clear operational definitions for the behaviors coaches are supposed to implement, (b) opportunities to practice implementation, and (c) ongoing performance-based feedback on implementation fidelity. Researchers have begun to use direct behavioral observation systems to gain more accurate information about the process dimension of coaching. Studies employing direct behavioral observation have focused primarily on two outcomes (1) defining and describing the verbal behaviors of coaches as they interact with teachers’ and families during classroom and home-based coaching sessions and (2) examining the accuracy of coaches’ self-report data. For example, Jayaraman, Marvin, Knoche, and Bainter (2015) developed a system to describe the topography of coach verbal and gestural behaviors across home and school-based context. While Salisbury, Cambray-Engstrom, and Woods (2012) used

direct observation to explore the agreement between coaches’ self-documented use of coaching strategies and those observed by a trained project staff member who reviewed video-recorded coaching sessions. Salisbury and colleagues (2012) found that coaches under-reported their use of coaching strategies. In fact, no significant relationship was found between coaches’ self-report and the trained observer for five of the seven possible coaching strategies.

The use of video-based direct behavioral observation coding systems to characterize the behaviors of coaches and those being coached is somewhat new, but it has the potential to advance research focused on coaching processes by helping to define and describe the “active ingredients” of coaching. In addition, direct observation data about coaching processes could be used to explore why some forms of coaching might or might not be efficacious. Data from these systems also could be used to inform the development of initial training and the provision of ongoing supports for coaches to implement and accurately record their coaching behaviors. The data could help decision makers and program leaders to install coach-based PD systems that, when implemented with fidelity, will lead to positive teacher and child outcomes. Accurately documenting the use of coaching behaviors and correlating them with teacher and child outcomes in future studies is critical for understanding when coaching should be used, for whom, and under what conditions.

Practice-based Coaching

Practice-based Coaching (PBC) is a coaching framework that has operationally defined components and coach behaviors that when implemented with fidelity have demonstrated efficacy in supporting preschool teachers to implement and sustain their use of select EBP (Snyder, Hemmeter, & Fox, 2015). Several coaching frameworks

have been described in the extant literature (e.g., Knight, 2007, Rush & Shelden, 2011) but there have been few rigorous empirical studies that have systematically examined relationships among the fidelity of implementation of these coaching frameworks, teachers’ implementation of practices that are the focus of coaching, child outcomes.

Thus, studies demonstrating positive outcomes for teachers and children linked to the use of PBC are needed.

PBC is unique among approaches to coaching in the degree to which it was designed with both theoretical and empirical support. The PBC framework was developed by reviewing and aligning its components with principles from how people learn (Donovan, Bransford, & Pellegrino, 1999), organizational behavior management, applied behavior analysis (Crow & Snyder, 1998), and transactional theory (Sameroff,

2009). The empirical evidence for PBC includes a number of multi-site randomized controlled trials (e.g. Conroy et al., 2015; Hemmeter et al., 2016), quasi-experimental studies (e.g., Artman-Meeker, Hemmeter, & Snyder, 2014), and single case design studies (e.g. Fox, Hemmeter, Snyder, Binder, & Clarke, 2011; Hemmeter, Hardy,

Schnitz, Adams, & Kinder, 2015). Each of these studies have systematically measured and evaluated the degree to which coaches implemented the PBC components with fidelity and corollary relationships with teachers’ fidelity of practice implementation in the classroom setting using direct observation.

Snyder, Hemmeter, and Fox (2015) define PBC as “a cyclical process for supporting preschool practitioners’ use of effective teaching practices that lead to positive outcomes for children” (p. 2). PBC is composed of three cyclical components

(a) strengths and needs assessment to inform the development of shared goals and

action planning, (b) focused observation, and (c) reflection and feedback. The PBC components occur within the context of a collaborative partnership and focus on a set of interrelated teaching practices (Figure 1-1). The PBC framework is distinct from other approaches to job-embedded support and coaching in three additional ways: (1) an emphasis on a targeted set of evidence-based interactional or instructional teaching practices, (2) the use of supportive and constructive performance-based feedback, and

(3) the transactional collaborative partnership that is structured and goal-oriented while also allowing for individual adaptations and accommodations.

Teaching practices. Teaching practices are observable and measureable actions or behaviors teachers use in the classroom to support children’s development and learning. Effective teaching practices describe how the teacher will plan for, implement, and evaluate his or her teaching. PBC is a practice-ready coaching framework that can be (and has been) used to support high quality early learning environments, interactions, and instruction across a variety of developmental domains.

The teaching practices that are the focus of coaching set the occasion for the teacher to self-assess his or her strengths within a focused aspect of teaching and guide the development of the shared goal and subsequent action plan (Snyder & Wolfe, 2008).

Performance-based feedback. PBC coaches collect data that are focused on the teacher’s implementation of the identified teaching practices and how children are responding to the teacher’s practices, when they observe in the teacher’s classroom.

This narrow goal-oriented focus makes the coach’s role transparent and primes the teacher to receive performance-based feedback helping her or him to identify what they are doing well and how they can enhance their teaching to be better aligned with the

evidence-based teaching practices that are the focus of coaching (Snyder, Hemmeter,

& Fox, 2015). This is referred to as supportive and constructive performance-feedback.

This type of feedback is well aligned to principles of adult learning theory (National

Research Council, 2000) and applied behavior analysis (e.g., Bijou, 1993) and has an evidence-base demonstrating that it is likely a critical “active ingredient” contributing to the efficacy of PBC (Casey & McWilliam, 2011; Kretlow & Bartholomew, 2010; Snyder,

Hemmeter, & Fox, 2015; Thurlings, 2013).

Transactional collaborative partnership. The collaborative partnership is dyadic and transactional. It is developed over time and is shaped and maintained by the interactions, knowledge, skills, dispositions, and motivations of both the coach and the teacher. The coach and teacher may develop a relationship overtime (Shannon,

Snyder, & McLaughlin, 2015); but, the partnership is characterized from the onset by both relational coaching practices (e.g., respect for individual learning histories, beliefs, and culture) and participatory coaching practices (e.g., active listening, questioning, shared goals; Dunst & Trivette, 2009). It is the use of these coaching practices that facilitate the transactional nature of the collaborative partnership. Coaching practices in combination with the operationally defined PBC components, provide structures and processes that becomes routine. Coaching practices allow the coach to create intentional and systematic opportunities for the teacher to gain fluency in contributing to the coaching debrief meeting. These transactional exchanges increase the coach’s capacity to make accommodations and modifications to the coaching process and teaching practices that are the focus of coaching, and to better to meet the needs of the

individual teacher, his or her context, and the children who are learning and developing under his or her care (Sameroff, 2009).

Supporting pre-school teachers to gain confidence and competence with implementing EBPs with fidelity is a primary goal of PBC. When the teachers achieve this level of fluency they are more likely to experiment with generalization of teaching practices to new children and novel situations, increasing the likelihood the teaching practices that are the focus of PBC will be sustained over time. The theoretical and empirical data have already demonstrated PBC is effective for supporting positive teacher and child outcomes related to select teaching practices. Additional research is needed however, to better understand which of these active ingredients, at what intensity, and in which combination, are efficacious for increasing the efficiency with which PBC is applied and the likelihood that desired outcomes are reliably achieved.

Measuring Fidelity of Implementation of PBC

The operationally defined components of PBC are a mechanism for guiding coaches’ consistent implementation of evidence-informed coaching practices, which have demonstrated the capacity to enhance and support the development of teacher practice when implemented with fidelity (Snyder, Hemmeter, & Fox, 2015). Fidelity of coaching implementation in PBC is measured through coach self-report data recorded on project developed coaching logs and the ratings of trained project staff or “raters”.

Fidelity data provide information about the degree to which coaching was implemented as intended and to what extent observed outcomes can be reliably associated with coaching activities (Artman-Meeker et al., 2015; Isner et al., 2011; Snyder et al., 2011,

Snyder et al., 2015). Furthermore, qualitative social-validity data regarding the teachers’ experience of participating in a PBC collaborative partnership illustrates how each

component of the PBC framework was identified as an essential part of the coaching experience from acquisition, to fluency, to sustained implementation of effective teaching practices in preschool classrooms (Shannon et al., 2015).

Teacher and Child Outcomes Associated with PBC

Practice-based coaching is designed to promote teachers’ implementation of effective teaching practices, which, in turn should positively impact the early experiences and developmental trajectories of young children. PD studies implementing

PBC have demonstrated noteworthy effects on teachers’ implementation of practices, which were the focus of coaching, across a variety of delivery formats (e.g., expert, self- coaching, peer-coaching, onsite, web-mediated) and in diverse early childhood settings

(e.g., Head Start, public and private preschool, and early childhood special education).

In addition, PBC has been successfully adapted to support the implementation of diverse evidence-informed teaching practices including (a) Embedded Instruction for

Early Learning (e.g., Snyder, Hemmeter, McLean, Sandall, McLaughlin, & Algina), (b)

The Pyramid Model for Promoting Social-Emotional Competence in Young Children

(e.g., Hemmeter et al., 2016), (c) BEST in CLASS, a set of 10 teaching practices focused on preventing young children’s challenging behavior to increase their engagement and learning (e.g., Conroy et al., 2015), and (d) emergent language and literacy practices to increase children’s communication and pre-reading skills (e.g.,

Greenwood et al., 2017). Some studies using PBC have also begun to identify positive outcomes for children enrolled in classrooms where teachers have received coaching

(e.g., Conroy et al., 2015; Greenwood et al., 2017; Hemmeter et al., 2016).

Embedded Instruction Practices and PBC

Developmentally appropriate practices in early childhood education (National

Association for the Education of Young Children, 2009) and recommended practices in early intervention/early childhood special education (Division for Early Childhood, 2014) promote the inclusion of young children with or at risk for disabilities in environments where they can access and participate in early childhood environments with peers without disabilities or delays. Embedded instruction is an evidence-based approach for supporting the development and learning of young children with or at risk for disabilities in inclusive environments. It involves providing planned, systematic, and intentional instruction within the ongoing activities and routines of high-quality early learning environments (Snyder et al., 2013).

Embedded instruction teaching practices were the focus of PBC in the present study. Preschool teachers enrolled in a larger efficacy trial from which data for the present study were obtained were introduce to the 14 key embedded instruction practices organized under four components (Snyder, Algina, Hemmeter, & McLean,

2015). A brief description of each components is provided below (Snyder et al., 2013).

 “What to Teach” practices support teachers to identify target behaviors or skills, which build on children’s strengths and are aligned with the general preschool curriculum and the child’s individualized education program (IEP). These target behaviors or skills are designed to increase access to and participation in classroom activities.

 “When to Teach” practices include the identification of natural and logical times of day to provide embedded learning opportunities for the child to practice the target behaviors or skills, with sufficient intensity to make measurable growth.

 “How to Teach” practices guide the selection of systematic instructional strategies that support the child to demonstrate the behaviors or skills under supportive conditions during meaningful classroom activities.

 “How to Evaluate” practices foster the teachers’ ability to evaluate their implementation of embedded instruction and the child’s progress, to make data- informed decisions about instruction.

Purpose Statement

The purpose of the present study was to systematically describe select aspects of the process dimension of coach-teacher interactions during the coaching debrief component of on-site, expert-coach facilitated PBC collaborative partnerships. The

Coaching Practices Observation Tool research version 1.0 (CPOT-RVI, Shannon &

Snyder, 2016), a direct behavioral observation system, was designed for the present study. The CPOT-RVI quantifies what is occurring during the coaching debrief meeting including (a) the proportion of time allocated to different conversational foci, including who initiated the conversation focus; and (b) the type of verbal behavior used by coaches to support teachers’ active participation in planning for, implementing, and evaluating the teaching practices that are the focus of coaching.

Data used to conduct the present study were collected within year one of an

Institute of Education Sciences (IES) Goal 3 efficacy trial of Tools for Teachers (TfT), a

PD intervention designed to support classroom teachers’ use of embedded instruction with preschool children with disabilities (Snyder, Algina et al., 2015). The PD intervention package has the following components: (a) four integrated high-quality workshops, (b) implementation guides and materials, (c) a comprehensive web-based multimedia tool kit, and (d) either expert onsite coaching or self-coaching via a website.

Onsite coaching was the focus of the present study. Data sources for the present study were 21 video-recorded coaching debrief meetings representing seven coach-teacher dyads sampled on three occasions across 15 weeks of on-site coaching. The videos were used to address the primary aims of this study—to describe how coaches facilitate

PBC debrief conversations with preschool teachers by quantifying duration of conversation foci, including coach versus teacher initiations, coach verbal behavior, and whether changes are observed across time related to these variables. The following research questions guided the present study:

1. What proportion of the coaching debrief time was spent in each conversation focus across all coaching debrief meetings?

1a. Did differences exist between dyads in the proportion of coaching debrief time spent in each conversation focus?

2. Did the proportion of coaching debrief time, spent in each conversation focus, change over time?

2a. Did differences exist between dyads in the proportion of coaching debrief time spent in each conversation focus over time?

3. Of the total number of conversation initiations, what proportion were initiated by the coach versus the teacher?

3a. What proportion of each conversation foci was initiated by the coach versus the teacher?

3b. Did differences exist between dyads in the proportion of conversation foci initiations by the coach versus the teacher?

4. Did the proportion of coach versus teacher initiations change over time?

4a. Did differences exist between dyads in the proportion of conversation foci initiations over time?

5. At what rate was each coach verbal behavior used across all coaching debrief meetings?

5a. Did differences exist between coaches in the rate of verbal behaviors used?

6. At what rate were coach verbal behaviors used within each conversation foci?

6a. Did differences exist between coaches in the rate of verbal behaviors used within each conversation foci?

7. Did the rate of coach verbal behavior change over time?

7a. Did differences exist between coaches in the rate of verbal behaviors used over time?

8. What were the demographic characteristics of each dyad and were there differences in the coaching implementation, proportion of time spent in each conversation foci, coach and teacher initiations, and rate of coach verbal behavior relative to the overall sample?

Significance

Coaching is a core competency driver in the installation and sustained use of innovations and EBPs known to improve teacher and child outcomes (Halle et al.,

2013). Yet, much remains unknown about the PD (Sheridan et al., 2009, Snyder et al.,

2011) and coaching (Powell & Diamond, 2013b) processes that facilitate teacher learning about and implementation of the EBPs. Research is needed that focuses on factors like such as content, form, and dose of PD or coaching and how these factors interact with the characteristics of individual learns to influence the extent to which innovations and EBP are used in the classroom. First generation PD studies have helped to define and describe the features of PD that are efficacious. Second generation studies are needed to disentangle the “active ingredients” of multi- component PD approaches and to identify what works, for whom, and under what conditions (Snyder et al., 2011).

One way to advance the science and application of coach-based professional development is to improve the documentation and reporting practices associated with independent variables and learner characteristics (Artman-Meeker et al., 2015; Isner et al., 2011; Schachter, 2015; Sheridan et al., 2009; Snyder et al., 2011, Snyder et al.,

2012) so that relationships can be explored between these variables. The present study applies a finer grained analysis to one aspect of TfT a multi-component PD innovation, that includes PBC. Building on the work of Snyder, Hemmeter, and Fox (2015), the

present study explored the coach-teacher interactions within PBC debrief meetings using a video-based observational coding system.

The Coaching Practices Observation Tool-Research Version 1.0 (CPOT-RV1;

Shannon & Snyder, 2016) was developed, piloted, and used in the present study. The development of the CPOT-RVI was significantly enhanced by the clearly articulated and operationally defined components of the PBC framework and the established theoretical and empirical foundation of PBC. These features of PBC justified the development of the CPOT-RVI and the present study, because PBC has demonstrated efficacy for supporting positive teacher and child outcomes on average, across several teaching practices, content domains, and instructional contexts. There is a significant need, however, to explore the parameters of adaptation and individualization that the PBC framework can tolerate while maintaining promising outcomes (Hulleman, Rimm-

Kaufman, & Abry, 2013). Developing a system to further define and describe what aspects of the PBC framework and coaching practices are exhibited by coaches, for how long, and in what combinations is an initial step towards better understanding how coaches modify and adapt the PBC structure in a transactional and responsive ways to support individual teacher learning.

Furthermore, exploring PBC coaching practices has the potential to enhance the preparation and ongoing systems of support provided to coaches by professional development providers and program leaders. Most fidelity of implementation data for

PBC reported to date has focused on adherence to the PBC framework and coaching protocol, not the quality of implementation. Performance-based feedback for coaches

about both the adherence and quality of coaching practices implementation is needed.

The present study will help address this need.

Delimitations

The present study was conducted using an existing data set. These data focus on the use of on-site expert coaches’ implementation of PBC, using project-developed protocols, to facilitate public preschool teachers’ knowledge and implementation of embedded instruction practices with children ages 3-5 years with disabilities.

The present study involved coaches who were affiliated with university-based research teams at two implementation sites. Although coaches were diverse in their professional experiences, project staff trained all coaches on the PBC framework and the associated coaching manual, protocols, and materials. The present study does not examine the use of different coaching frameworks or models. Furthermore, different delivery formats (i.e., on-site vs. web-mediated, expert vs. self) are examined within the larger efficacy trial; however, only the onsite expert coach data were included in the present study.

The PBC framework is composed of interrelated components and the present study focuses primarily on the coaching debrief meeting where action planning, goal setting, and reflection and feedback are observed. The focused observation influences the debrief interactions, but is not directly included in the present study. Furthermore, the present study uses a single analytic technique to explore the coaching debrief meeting. There are several approaches to defining and describing the dyadic transactional exchanges between coaches and teachers and the scope of this study is intentionally focused on the use of direct behavioral observation. Furthermore, the observation codes emphasize positive coaching practices and were not designed to

provide information about practices not aligned with the theoretical foundations of the

PBC framework.

Limitations

The present study has several limitations. First, the study data were collected within a larger efficacy trial, and the sample of coaches who provided expert onsite coaching was limited to university-based, project trained coaches. It must also be noted that the teachers volunteered to participate in the study, representing a degree of interest and motivation in being coached. The findings of the present study represent a small sample of teachers who met specific inclusion criteria (a) taught in the public- school system, (b) held a state teaching certificate, (c) worked with children ages 3-5,

(d) engaged in coaching around embedded instruction practices, and (e) volunteered for the study. Finding from this sample of teachers might not generalize to teachers in diverse early learning settings, particularly locations where teaching certification and credentials may be less rigorous.

Only three randomly selected videos were observed and coded for each teacher, which may over or underestimate what actually occurred across the 15-weeks of coaching. Furthermore, the practices which were the focus of coaching (i.e., embedded instruction) could influence the observed conversational foci and use of coach verbal behaviors and materials. Given the emergent state of the CPOT-RVI and studies exploring the process dimension of coaching, associations between practices which are the focus of coaching, and the use of various coach behaviors and conversation foci have not yet been established.

Finally, the CPOT-RVI was designed to align with aspects of the PBC framework.

This alignment between the PBC framework and the practices captured by the CPOT-

RVI increases the likelihood coaches will engage in the specified behaviors. It is likely the CPOT-RVI would have some degree of application to any dyadic coaching conversation (e.g., use of clarifying questions, personal pleasantries), but it may miss what are considered to be important aspects of other coaching frameworks or models.

For example, it would not capture the use of vision-based coaching for leadership, which is grounded in intentional change theory. In this approach, the coachee focuses on articulating their “deepest aspirations, hopes, and dreams for the future, as well as positive aspects of one’s core identity” (Passarelli, 2015, p.1), rather than focusing on short term goals and action steps toward those goals that are part of PBC.

Summary

High quality early experiences, which in turn, lead to optimal developmental outcomes for young children, have been identified as an important outcome of teacher

PD. Enhanced understanding of the process dimension of coach-based PD could (a) inform policy and program decisions regarding efficient and effective approaches to developing and installing an adaptive system of PD; (b) strengthen the association between intervention components and child and teacher outcomes; and (c) inform the ongoing training and technical assistance provided to coaches who work with preschool teachers. The present study explored how coaches facilitate conversations with preschool teachers within a PBC partnership. The emphasis of the exploration was the conversational focus, including who initiated the conversation foci, and the coaches’ use of different verbal behaviors.

Figure 1-1. Components of the practice-based coaching framework (Snyder, Hemmeter, & Fox, 2015).

CHAPTER 2 REVIEW OF THE LITERATURE

The purpose of the present study was to explore select aspects of the process dimension of coaching by exploring how coaches facilitated conversations during the coaching debrief meeting of an on-site Practice-based Coaching (PBC) partnership. A direct behavioral observation system was developed to investigate (a) the proportion of time allocated to different conversational foci, including who initiated the conversation focus; and (b) a description of the verbal behaviors used by coaches to support teachers’ active participation in planning for, implementing, and evaluating the embedded instruction practices presented through coaching. A review of the literature is presented in this chapter. It addresses (a) features of quality in early childhood professional development (PD), (b) systematic follow-up support as a component of early childhood PD, (c) empirical evidence demonstrating the efficacy of the PBC framework to facilitate teacher learning and child outcomes, and (d) extant literature applying observational coding systems to measure coach behaviors within collaborative coaching partnerships. These topics provide a complementary rationale for the present study.

First, the extant PD literature is reviewed, with particular emphasis on the use of systematic follow-up supports (e.g., coaching, mentoring, consultation) to aid teachers’ use of new or refined skills when interacting with young children. Second, characteristics of empirical studies examining the relationships between teachers’ participation in PBC with (a) fidelity of practice implementation in the classroom and (b) child outcomes are described to illustrate what is known and unknown about the content, structure, and process dimensions of this coaching framework. Third, existing

observational coding systems, that record coach-coachee verbal behavior, facilitation strategies and practices, and, interactions are discussed.

Search Procedures

Literature reviewed in this chapter was identified using the Early Childhood

Professional Development Database (ECPDD) housed in the Anita Zucker Center for

Excellence in Early Childhood Studies. The ECPDD is updated annually through a systematic search procedure, which is informed by the Preferred Reporting Items for

Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Moher, Liberati, Tetzlaff,

& Altman, 2009). The student investigator has participated in the development and maintenance of the ECPDD since 2013.

First, the following electronic databases were used: Education Full Text, ERIC,

PsychINFO, and the Social Sciences Citation Index. Key words included natural language and controlled vocabulary related to PD delivery formats (e.g., workshop, preservice, coaching), filters included terms related to early childhood and early childhood age and settings (e.g., infant, preschool, Head Start). Limiters were peer-reviewed journals, settings serving birth-to-five, and studies published in English. Studies obtained were exported to reference management software where duplicates were removed. All remaining articles were reviewed by title and abstract, and full text when applicable, to determine if they met the inclusion criteria. Inclusion criteria specified the study (a) included a form of teacher PD, (b) represented children age birth to five, (c) included empirical data, and (d) occurred in the United States.

The identification process yielded 364 studies2 published prior to December 31,

2015. These articles were coded for 77 indicators across 10 areas (location, setting, child age, PD focus, PD content, PD model, PD strategies, follow-up strategies, research design, and independent variables) using the operational definitions developed by Snyder et al. (2012). Two hundred and twenty-seven (62%) of the 364 studies included systematic follow-up in the form of (a) coaching/performance feedback, (b) consultation, (c) mentoring, (d) peer support group, (e) communities of practice/shared inquiry. The 227 systematic follow-up articles were coded for 122 additional indicators across 13 areas (follow-up agent, agent qualifications, follow-up recipient, targeted of follow-up support, follow-up frequency, duration of coaching, duration of feedback session, follow-up agent strategies, feedback format, focus of follow-up, follow-up protocol, follow-up fidelity, and outcomes measured, follow-up agent’s training and support within the study). Relevant reviews and book chapters addressing PD, PBC, and observational coding systems in the early childhood context were also identified.

Early Childhood Professional Development

The education of young children is garnering unprecedented economic, policy, and research attention in the United States. Access to publicly funded preschool education services through statewide universal and voluntary pre-kindergarten and the concurrent expansion of Early Head Start slots reflect the changing national discourse around the education of young children (Barnett et al., 2017); one which emphasizes the potential to shift children’s developmental trajectories in powerful ways during the

2 Two of the articles included two studies and one of the articles reported five studies three of which met the criteria of this review, so the 364 studies represent 360 unique articles.

first five years of life (Shonkoff, 2010). Similarly, a number of national policy initiatives have begun to address the important work of developing cross-sector training and technical assistance and PD systems, which ensure teachers who work with young children have the capacity to provide high-quality education and care (NPDCI, 2008).

Contemporary policy initiatives, which advocate for and allocate capital toward a more cohesive system of support for early childhood teachers include: The Race to the Top

Early Learning Challenge; State Advisory Councils for Care within the Head Start reauthorization Act; the Individuals with Disabilities Education Act, provisions for knowledge and skill development of Part B/619 and Part C providers; and the option to use Title I funds to support preschool initiatives under the Every Student Succeeds Act.

Opportunities and challenges for research in the early childhood sector are a product of this social policy attention. Dedicated funding from the National Institute of

Child Health and Human Development (NICHD) and the Institute of Education Sciences

(IES) National Center for Education Research and National Center for Special

Education Research have supported the development of promising interventions which, when implemented with fidelity, are associated with important social and academic outcomes for young children. Many of these interventions have also explored the role of the classroom teacher in mediating child outcomes (Diamond et al., 2013). NICHD’s child development and behavior branch has program areas specifically focused on (a) promising interventions for children ages 3-5 years and (b) establishing causal linkages between teacher behavior and child outcomes. IES funded research studies have shown (a) PD in early childhood classrooms can improve the global classroom quality and the implementation of instructional practices, (b) enhanced teacher practice can

impact child learning, and (c) PD can be delivered using a variety of onsite and distance formats (Diamond et al., 2013). Grant-funded research in early childhood has also illustrated that although there are some common features of effective PD, the early childhood context is unique from K-12 and it is important to tailor PD supports to meet these unique needs (NPDCI, 2008; Zaslow et al., 2010, Snyder et al., 2011).

Defining Professional Development

For the purpose of the present study, PD is defined as “facilitated teaching and learning experiences designed to enhance practitioners’ knowledge, skills, and dispositions as well as their capacity to provide high quality early learning experiences for young children” (Snyder et al., 2012, p.188). Professional development experiences occur along the continuum of an early childhood teachers’ career and often include a variety of delivery formats. For most early childhood teachers, PD is experienced as a combination of (a) formal credentials, certifications, and degrees; (b) PD events such as conferences, workshops, seminars, in-services, and trainings; (c) embedded learning opportunities, which involve context-based learning such as mentoring, coaching, consultation, and action research; and (d) informal interactions with colleagues or literature. Contemporary theory and research has demonstrated, however, that finding the “fit” between the PD experience and the desired outcomes of that experience on teachers’ individual knowledge, skills, and dispositions is critical (Harris, 1980;

McCollum & Catlett, 1997; NPDCI, 2008; Shannon et al., 2015; Snyder et al., 2011;

Snyder & Wolfe, 2008).

Aligning Professional Development Inputs and Outcomes

The NPDCI (2008) conceptual framework for PD in early childhood was developed to facilitate consensus building around how to define and enact PD. It includes three key components:

(a) the characteristics and contexts of the learners (i.e., the “who” of professional development, including the characteristics and contexts of the learners and the children and families they serve); (b) content (i.e., the “what” of professional development; what professionals should know and be able to do; generally defined by professional competencies, standards, and credentials); and (c) the organization and facilitation of learning experiences (i.e., the “how” of professional development; the approaches, models, or methods used to support self-directed, experientially-oriented learning that is highly relevant to practice.)(p.3).

In addition, programs should consider the intended outcomes of the PD (i.e., the “why” of PD) to make decisions about who should attend, what content can and should be covered, and how learners will engage with the content.

These components in combination with local contexts provide a guide for individuals and programs who plan for, implement, and evaluate early childhood PD to develop experiences that “fit” an established learner or program need. Two studies, which exemplify finding the alignment “fit,” are presented here as an example. Anderson

(2014) reported that informational content and interactive videos within a 2-hour webinar

(how) about environmentally sound integrated pest management in early learning environments (what) was effective for increasing childcare administrators’ (who) knowledge of safe and effective pest-management (why). Furthermore, most participants found the delivery format to be cost efficient and of sufficient intensity to increase knowledge and compliance with their local policy context. In contrast,

Clements, Sarama, Wolfe, and Spitler (2015) reported on a study that provided preschool teachers (who) with 13 days of workshops and bi-weekly coaching over a

period of 2 years (how) to support the installation and sustained fidelity of implementation (why) of an early childhood math curriculum (what). The intensity of this support was, the authors argue, important for enhancing the way teachers thought about and presented math content, and produced high levels of sustained fidelity of implementation 2-years post intervention. Anderson (2014) and Clements et al. (2015) depict stark contrasts, illustrating an empirical application of how the NPDCI framework components can support the alignment of specified outcomes or the “why” of PD (i.e., knowledge vs. implementation) with the intensity or “how” of PD (i.e., 2-hour webinar vs.

2-years of workshops and coaching) to meet the needs of individual learners and programs. By considering each of the NPDCI components individually and in combination, program leaders can better support early childhood teachers and staff to increase the efficiency and effectiveness of PD.

Features of Quality Professional Development

In addition to aligning PD inputs and outcomes, there is a corpus of empirical research to guide providers and programs in selecting PD features which represent the extant knowledge on how people learn (Donovan, Bransford, & Pelligrinol,1999; Trivette et al., 2009) and findings from studies on what makes PD effective (Darling-Hammond

& Richardson, 2009; Garet, Porter, Desimone, Birman, & Yoon, 2001; Guskey, 2003;

Joyce & Showers, 2002; McCollum & Catlett, 1997; Snyder & Wolfe, 2008; Snyder et al., 2011, Zaslow, 2009). Many of the features can be applied across a number of PD delivery formats and experiences. The features include(a) a content focus with clear objectives; (b) integrated theory and practice; (c) opportunities for active engagement

(i.e., discussion, modeling, demonstration, role-play) and feedback on implementation;

(d) alignment with program goals; (e) matched intensity and desired outcomes; (f) the

provision of a framework for self-evaluation, and, when appropriate; (g) follow-up support to apply practices in context. Researchers who seek to advance and clarify knowledge regarding these PD features are now investigating what combination of these features is required, and if intensity (i.e., frequency, duration) and delivery format

(live vs. distance, group vs. individual, expert vs. peer vs. self), or individual characteristics mediate or moderate the efficiency and effectiveness of these features

(Isner et al, 2011; Snyder et al., 2011; Snyder et al., 2012; Zaslow et al, 2010).

Trends in Early Childhood Professional Development Research

The extant early childhood PD literature provides insight into who is currently being served, what content is being addressed, and how PD is being delivered. Across the four databases searched, 364 articles addressing early childhood (birth to five) PD in the United States since 1970 were identified. Nearly a third (29%) of these articles were published between February 2011 and December 31, 2015. These data further illustrate the exponential growth in the nation’s investment in early childhood and the increased availability of PD to teachers who work with young children— a traditionally underserved workforce (LeMoine, 2015). Across the identified articles, the majority of teachers participating in PD were located in preschool (40%) or Head Start (44%) and worked primarily with children age 3-5 years (89%), while fewer worked with infants and toddlers birth to 24 months (30%). Inservice/offsite (34%) and staff development/onsite

(28%) workshops are the most prevalent PD delivery format. The content of available

PD was most often focused on the social-emotional (45%), literacy (34%), and language development (38%) domains.

Use of systematic follow-up support

Systematic follow-up strategies can support teachers to translate new or refined knowledge and skills gained through PD experiences into their classroom context.

Follow-up strategies range in intensity (Snyder & Wolfe, 2008). Low intensity follow-up support includes the provision of reading materials or websites to review at home or job- aides like visual cues that can be used in the classroom. More intensive and systematic follow-up support can take a number of forms including coaching, mentoring, consultation, communities of practice, and peer support groups (Snyder et al., 2012).

The majority of articles (62%, n = 227) included in the review described the use of one or more systematic follow-up strategies. Of the studies that reported the use of systematic follow-up strategies, coaching was the most prevalent (82%, n = 186), followed by consultation (10%, n = 23). Mentoring, communities of practice, and peer support groups were less prevalent in the empirical literature.

Trends in studies classified as coaching

Eighty-two percent (n = 186) of the 227 studies that included systematic follow-up support were classified as involving “coaching” as a form of PD. This section provides additional information about the characteristics of the who, what, why and how for the studies that were classified as involving coaching.

The “who”. The majority of the recipients of the coaching support were lead teachers (72%). Most recipients worked in classrooms classified as Head Start (44%), preschool (42%), and childcare (26%). Ninety percent of the programs served preschool-aged children while only 29% served infants and toddlers. Nearly half of the programs served children with identified disabilities (46%) and children who were at-risk

for adverse childhood experiences (41%). Additional information about the recipients of coaching and other forms of systematic follow-up supports can be found in Table 2-1.

The majority of the coaches were associated with research projects (54%), and some held a master’s degree (22%) and had teaching experience (34%). Twenty-three percent of coaches received training and support to be a coach and 17% had training on the content that was the focus of coaching. Twenty-five percent of the coaches participated in delivering training experiences for the recipients beyond coaching (e.g., workshops, webinar). Table 2-2 shows additional information about the people responsible for delivering the coaching and other forms of systematic follow-up support.

The “what”. Similar to the broader PD literature the majority of the coaching studies had a social-emotional (44%), language development (43%), and literacy (37%) content foci. Typically coaching was focused on supporting teachers to implement a curriculum (23%) or group of practices (20%) and it was the only intensive form of systematic follow-up support used to address a single practice (6%). Thirty-two of the coaching studies were to support the teacher to implement class-wide teaching practices, but coaching was also used to support the teacher to work with one child

(11%). More information about the content focus of the systematic follow-up support can be found in Table 2-3.

The “how”. Many of the PD studies did not adequately report information about how systematic follow-up support was provided. Eighty-two percent of the studies did not report the duration of the coaching collaboration; however, available data indicate coaching can occur for as little as 1 week or up to several years. Coaching sessions

most often occurred weekly (31%) and monthly (24%). In the few studies that reported the length of coaching sessions, 29% lasted more than 30 minutes.

Seventy-five percent of the coaching studies did not report the use of coaching guides or implementation supports, but when these tools were reported they typically included manuals (11%) and scripts or protocols (8%). Eighty-one percent of coaching studies did not provide a measure of coaching implementation fidelity; however, 8% of the studies reported the use of a checklist and 5% reported the use of a project developed fidelity measure. More information about the coaching and other forms of systematic follow-up support provided is shown in Table 2-4.

Studies with PBC components

Eighty-two percent (n = 186) of the 227 studies that included follow-up support were classified as coaching and 29% (n = 54) of the coaching studies were classified as focusing on a set of interactional or instructional teaching practices. This section provides additional information about the studies which were classified as coaching around teaching practice that included (a) components of PBC (Snyder, Hemmeter, &

Fox, 2015) or (b) coaching strategies described in the PBC coaching manuals (Snyder,

Hemmeter, Bishop, Shannon, & McLean, 2015). The majority of studies that were classified as coaching on a set of teaching practices reported the use of verbal performance feedback (70%) and observation (65%).

Strengths and needs assessment is consider to be a required component of implementing PBC with fidelity. Nineteen percent of all coaching studies reported the use of a strengths and needs assessment, but only 6% of the coaching studies that also focused on teaching practices used a strengths and needs assessment. In contrast,

19% of the coaching studies that reported the use of coaching to support teacher’s use

of teaching practices reported the use of goal setting and graphic performance-based feedback (13%), a higher percentage than all the coaching studies reviewed (3% and

8% respectively). Action Planning was under-utilized among both groups at 3-4%.

These data suggest that although coaching in some studies is focused explicitly on the implementation of teaching practices, the PBC framework provides a systematic and intentional way to integrate and consistently implement effective coaching practices like those described in Table 2-5 to support teachers acquisition of those teaching practices.

Reporting limitations in systematic follow-up implementation support

Despite the use of a systematic procedure for locating and reviewing articles, much remains unknown about the dose of follow-up support and what occurs between participants during follow-up support because this information is underreported. Of the coaching studies reviewed, 82% did not report information about the duration of systematic follow-up, 29% did not report the frequency, and 54% did not report the length of follow-up sessions. In addition, these studies frequently did not report information about who the follow-up agent or interventionist was, including qualifications and training for his or her role (46%). Furthermore, only 25% of the studies reported the use of materials such as a manual or guide to inform the delivery of the systematic follow-up intervention, while only 19% reported conducting fidelity checks to ensure the systematic follow-up was delivered as intended.

Recommendations for enhanced reporting

The Zaslow et al. (2009) review of early childhood PD and Isner et al. (2011) and

Artman-Meeker et al., (2015) reviews of early childhood coaching provide recommendations for reporting PD activities and systematic follow-up, which explicitly address the reporting limitations described above. Their recommendations emphasize

the need to describe the dose, follow-up strategies, and intervention or follow-up agent characteristics in detail. Although not yet prevalent, it appears these recommendations are making their way into the early childhood PD research community. Abell, Ariswalla,

Putnam, and Miller (2014) explicitly references these recommendations stating,

By explicitly describing the details of the Family Childcare Partnerships (FCCP) professional development model, it’s underlying activities, and the processes FCCP follows to assist providers to improve their caregiving quality, we offer information others may find useful for the development and testing of practice-based professional development initiatives. (p. 588).

Increased specificity regarding the “who” (i.e., a description of personnel delivering PD and how they are trained/supported) and “how” (i.e., structure and processes including delivery format and dose follow-up support) is more prevalent in contemporary research

(e.g., Landry et al., 2011; Pianta et al., 2014; Wasik & Hindman, 2011; Zan, 2014).

Summary of Early Childhood Professional Development

The number of rigorous empirical PD studies being conducted in early childhood is growing, but the quality and efficacy of that PD warrants additional research.

Advancement of the science of early childhood PD cannot be achieved in the absence of a commitment by researchers to provide complete and accurate documentation of the

PD features, including the content, structure, and processes, which led to the reported teacher and child outcomes (Sheridan et al., 2009; Snyder et al., 2011; Snyder et al.,

2012; Zaslow et al., 2010).

Implementing PBC in Early Childhood Contexts

PBC is one form of coach-based professional development designed to enhance the confidence and competence of early childhood teachers to implement evidence- based teaching practices (Snyder et al., 2015). It is one form of systematic, follow-up

implementation support. PBC is often delivered in combination with workshops or other

PD experiences, and can occur in a variety of delivery formats (e.g., live or web- mediated; expert, peer, or self-facilitated). This section will present the theoretical and empirical support for the components of the PBC framework followed by studies demonstrating the efficacy of this approach.

Evidence for the PBC Components

PBC is a cyclical process which includes (a) strengths and needs assessments that inform shared goals and action planning, (b) focused observation, and (c) reflection and feedback. The components occur within the context of a collaborative partnership and focus on a set of interrelated teaching practices (Figure 1-1).

Teaching practices

Teaching practices represent observable and measureable actions or behaviors teachers can use in the classroom to facilitate child learning. Isner et al.’s (2011) review of 44 early childhood coach-based professional development studies found 31 of the studies reported outcomes related to observable quality or teaching and interactional practices, while 21 of these studies linked teacher practices to child development and behavior outcomes. Implementation science frameworks suggest coaching is a key implementation driver for translating evidence-based interventions, innovations, and teaching practices into real world contexts (Fixsen, Naoom, Blasé, Friedman, &

Wallace, 2005; Martinez-Beck, 2013). All applications of the PBC framework occur around a set of identified teaching practices. In the present study, the identified teaching practices are related to embedded instruction.

Collaborative partnership

PBC is transactional and requires both the coach and teacher to contribute time, effort, and expertise. The collaborative partnership is grounded in a shared goal focused on practice implementation and mutually agreed upon expectations regarding the role of the coach and teacher. Each partnership is individualized and continually evolves to meet the motivations, preferences, strengths, and needs of each teacher. The dyadic collaborative partnership is the primary focus of early coaching interactions and continues to develop throughout the coaching cycle. The dyadic transactional collaborative partnership is the context in which all other components of the PBC framework are implemented (Snyder et al., 2015)

Strengths and needs assessment to inform shared goal setting and action planning

The coach uses a strengths and needs assessment specifying the observable and measureable teaching practices, which are the focus of PBC, to elicit the teacher’s perspectives and priorities for the coaching partnership (Snyder & Wolfe, 2008). The strengths and needs assessment informs the initial goal development and can provide ongoing insight for the coach and the teacher regarding the teachers’ goals throughout the coaching cycle. Identified goals include one or more teaching practices, which the coach and teacher have mutually agreed upon as the focus for coaching over the next

2-5 weeks. Goals are recorded on an action plan, which specifies the materials and resources needed to achieve the goal, a timeline, and a criterion for the coach and teacher to know when the goal has been achieved. The strengths and needs assessment, goal, and action plan provide a “road map” for how the coach will support the teacher to achieve a goal he or she has identified as important for increasing

confidence and competence with the use of evidence-based or evidence-informed practices to support child outcomes.

Focused observation

Observations are “focused” when the coach records information related to the teacher’s identified goal and action steps. Focused observations facilitate the coaching partnership by increasing transparency around the coach’s role in the classroom. The observation promotes fidelity of implementation by providing a setting event for the teacher to use a teaching practice (Kretlow & Bartholomew, 2010). In addition, the focused observation may also include participatory or on-the-spot coaching support aligned to the teacher’s preferences. This support is designed to enhance or refine the teacher’s implementation of practices through coaching strategies (e.g., modeling, verbal or gestural cues, feedback; Snyder, Hemmeter, Bishop et al., 2015). In situ demonstrations and interactions promote knowledge, and skill acquisition and fluency, through immediate feedback, which lets teachers know when, where, how, and with whom to use (or not use) particular teaching practices (Metz, Halle, Bartley, & Blasberg,

2013).

Reflection and feedback

Reflection and feedback occurs within a post-observation coaching debrief meeting. Reflective conversations prompt the teacher to think about his or her implementation of the teaching practices, identify strengths, solve problems, ask questions, and plan for future implementation. Feedback within the PBC framework is performance-based and guided by the teacher’s current and former goals and action steps. Performance feedback is designed to provide the teacher with specific verbal, written, graphic, or video-based information about his or her implementation of identified

teaching practices; it is both supportive and constructive (Snyder, Hemmeter, & Fox,

2015). Repeated performance-feedback opportunities related to a specified goal(s) can also support the development of error detection skills, which build the capacity to provide self-feedback aimed at reaching and sustaining the desired behavior (Hattie &

Timperley, 2007; Kluger & DeNisis, 1996; Thurlings, Vermeulen, Bastiaens, & Stijnen,

2013). Teachers who have participated in PBC report performance feedback is useful and acceptable (Shannon et al., 2015).

Intervention Studies Implementing Practice-based Coaching

A number of studies have been identified from the ECPDD, which explicitly describe the use of PBC or the PBC components. These studies are summarized in

Table 2-6. The identified studies illustrate the efficacy of PBC for supporting teachers’ implementation of teaching practices designed to (a) increase global classroom quality and (b) facilitate the development of children’s social-emotional, language and literacy, pre-academic skills, and reduced challenging behavior through positive behavior intervention supports.

Teacher outcomes

The studies summarized in Table 2-1 include studies conducted with early childhood practitioners that explicitly indicated the use of PBC and those that included the PBC components (i.e., teaching practices, strengths and needs assessment, goal setting and action planning, focused observation, reflection, and performance-based feedback). The teacher and child outcomes shown in Table 2-1 suggest most teachers successfully acquired, gained fluency with, and maintained targeted teaching practices through participation in onsite or blended (i.e., onsite and web-mediated) delivery formats when PBC or the components of the PBC framework were used. Statistically

significant differences in teachers’ classroom implementation of identified teaching practices, when compared to the control condition, were reported in all eight of the randomized controlled trials; significant effects were reported in three of four pre-post designs, and implementation of teaching practices above baseline were reported in four of the five single-subject studies.

The web-mediated delivery format, however, showed greater variability in teacher outcomes. For example, Artman-Meeker et al. (2014) found participation in 12- weeks of web-mediated coaching did not increase teachers’ implementation of Pyramid

Model practices above those who participated in workshops alone. This finding suggests web-mediated support might not be the most effective delivery format for

Pyramid Model practice supports, because onsite coaching around the Pyramid Model has been shown to be more effective than workshops alone (Hemmeter et al., 2016). In contrast, Powell and Diamond (2013) found no difference between onsite and remote coaching conditions on teacher practice implementation in their literacy intervention.

The analyses of coach feedback across onsite versus web-mediated conditions found both remote and onsite conditions had significant effects on teachers’ implementation of identified literacy practices. It is important to note, however, that coaches provided more unique feedback statements when meeting with teachers’ onsite, suggesting the quality of coaching interactions might vary across formats. Additional analyses of the structural and process components of coaching in future studies will likely be required to discern if an onsite versus web-mediated delivery format is more efficacious, for whom, and under what conditions.

Child outcomes

The majority of PBC studies reported working with children ages 3-5 in Head

Start or public Pre-K classrooms. Positive child outcomes were reported in 9 of the 17 studies. Statistically significant effects on child learning and development were reported in five studies related to (a) language and literacy and (b) reduced challenging behavior and increased engagement and social skills. One study reported positive descriptive statistics. Three single-subject studies reported mixed results related to decreases in children’s challenging behavior in the classroom.

Summary of Practice-based Coaching Literature

The literature reviewed suggests PBC is effective for supporting preschool teachers’ competence and confidence to implement evidence-based and evidence- informed practices in the classroom setting. Furthermore, teachers’ fidelity of implementation has demonstrated corollary child outcomes in some studies. Additional research is needed to further explore the relationship between the coaching content, structure, and process dimensions of PBC and teacher and child outcomes to apply this framework effectively and efficiently across additional teaching practices, contexts, and learners.

Coaches as Mediators of Teacher’s Coaching Experiences

The effectiveness of coach-based approaches to PD, including PBC, relies on the coaches’ knowledge of the practices that are the focus of coaching and their ability to deliver the components or “active ingredients” as intended. There is a small, but growing body of research that has examined how attributes and behaviors of the coach influence coaching processes. Both qualitative (e.g., Lofthouse, Leat, Towler, Hallet, &

Cummings, 2010) and quantitative (e.g., Downer, Locasale-Crouch, Hamre, & Pianta,

2009; Diamond & Powell, 2010, 2013; Conroy et al., 2015) analyses of coach behavior have demonstrated the potential for individual coaches to mediate the teachers’ experience of coach-based PD. Lofthouse et al. (2010) identified differences in the quality of constructive feedback, while Downer et al. (2009) and Powell and Diamond

(2013) identified statistically significant differences between the quantity and type of feedback teachers received from coaches. In addition, the coach appears to influence the number of goal cycles completed (Downer et al.,2009) and the distribution of practices addressed across those goals (Powell, Steed et al., 2010). In a collaborative partnership, however, the teacher also plays a transactional role in these observed differences. Beyond reporting teacher and child outcomes, it appears studies specifically exploring the role of coaches and the integrity with which they implement coaching are important for better understanding how and why coaching supports (do or do not support) the implementation of EBPs in classrooms.

Using Observational Coding Systems to Describe Coach Behavior

Direct behavioral observation is one method with the potential to provide exploratory and descriptive insights into the actions and behaviors used by coaches to promote teacher participation in coaching conversations that occur as part of the collaborative partnership.

Empirical Approach of Studies Using Observational Coding Systems

Direct behavioral observation has been used in six early childhood professional development studies. Four studies used the Routine and Instructional Strategy Coding

Protocol-IL (RISCP-IL; Salisbury, Cambray-Engstrom, Woods, & Friedman, 2008) to examine provider’s implementation of the Family Guided Routines Based Intervention

(Woods, 2005). The fifth study used an unnamed set of operationally defined behaviors

(Campbell & Coletti, 2013), and the sixth study used the Early Childhood Coaching

Conversations (ECCC) coding system (Knoche & Bainter, 2012).

Each of the four studies using the RISCP-IL were designed to examine coaching practices of early interventionists providing home-based services related to the child’s individual family service plan (IFSP; Friedman, Woods, & Salisbury, 2012; Marturana &

Woods, 2012; Oborn & Johnson, 2015; Salisbury et al., 2012). Across all four studies, providers included licensed speech language pathologists, occupational therapists, social workers, and early childhood special educators. Providers spoke English or

Spanish. Coded videos included typical early intervention (EI) sessions with families on the providers existing caseload. The RISCP-IL has seven codes, which represent evidence-based EI practices. When coding with the RISCP-IL, coders used the procedures developed by Salisbury and colleagues (2012). The entire EI session video is coded using a 30-second interval recoding procedure. The codes were mutually exclusive and exhaustive (ME & E) ensuring that each interval received one code. In the event that two practices were observed, the code, which comprised the majority of the interval, was selected.

Salisbury and colleagues (2012) used the RISCP-IL to determine the extent to which providers implemented the evidence-based coaching practice and recorded the use of them on the intervention fidelity log. Ninety videos across six providers were included in the sample. Friedman et al. (2012) used the RISCP-IL to describe the influence of a training followed by weekly reflective supervision on providers’ use of evidence-based coaching practice over 18 weeks, and to compare coaching practices used by different programs. Thirty-six videos across 12 providers were included in the

sample. Marturana and Woods (2012) provided monthly, web-mediated, video-chat performance feedback to 34 EI providers for 1 year, which resulted in a significant increase, on average, in one EBP and a decrease in one undesirable practice. Osborn and Johnson (2015) also used the RISCP-IL in conjunction with web-mediated coaching to send weekly written and graphic performance feedback to three EI providers over a period of 6 weeks with minimal changes observed using a multiple-baseline design.

Campbell and Coletti (2013) examined the agreement between EI providers and a blind coder as to when coaching strategies did or did not occur. Seventy-five providers submitted between one and three video segments of their regularly scheduled sessions.

Each segment represented what they believed to be one of five strategies introduced during 13 hours of training held over 6 days. Video segments were coded for length, primary functional skill, severity of disability, and one of five coaching strategies.

Agreement between the independent raters and coaches was 85%, indicating providers could identify and implement the strategies when a single strategy was the focus of each video segment.

Jayaraman and colleagues (2015) conducted an exploratory study of 24 coach- teacher dyads’ verbal and gestural behavior after a 3-day workshop on characteristics of effective coaching using the ECCC coding system. Each coach submitted one video of a self-selected coaching meeting focused on an early intervention topic with an existing client. These videos were coded using a discrete 2-minute interval coding system. Findings suggest there was variation in the frequency of behaviors demonstrated; however, there were behaviors used more frequently across all 24

participants. Verbal acknowledgment, nonverbal acknowledgment, and clarifies intent had rate per minute data greater than other coded behaviors.

Gaps and Recommendations for Future Studies

Gaps in the extant early childhood direct behavioral observation literature focusing on coaching reflect a need for further research that mitigates the absence of

(a) a coding system that looks specifically at behaviors occurring during a dedicated coaching debrief meeting between the participant and the provider in the absence of child care responsibilities; (b) a common set of instructional provider practices around which coaching occurred (i.e., what the teacher is learning), including any necessary modification of strategies used by coaches; (c) information about the duration of the coaching partnership (i.e., session 2 versus session 15); and (d) the systematic coding of materials used to facilitate the interaction. In addition, the existing systems described above used interval coding procedures that may under or overestimate the occurrence of behavior dependent on the length of the interval and the coding decision rules (Yoder

& Symons, 2010). Each of these is an area for future research to better understand the coaching processes.

Summary of Observational Coding Systems

Direct behavioral observation has a well-established place in studies of young children’s behavior (Pellegrini, Symons, & Hoch, 2014). The use of this method to examine coach-based professional development in early childhood contexts is relatively new. This promising method might serve as an important tool for better understanding the role of the coach in PBC and other coach-based PD initiatives. From this level of specificity, important information can be obtained about the coaching process including

(a) which verbal behaviors are used at higher rates and how they are used to facilitate

different coaching foci, and (b) how coaches and teachers make adaptations and accommodations over time, changing their roles and the duration and frequency of coaching processes as the partnership develops. These data are important for understanding how differences in the coaching experience for individual teachers may be attributable, in part, to the individual coach and his or her knowledge, skills, and interactions.

Chapter 3 describes the methods of the present study. It includes information about the development and piloting of the direct behavioral observation system designed for the present study to examine the conversation foci, coach versus teacher initiations, coach verbal behaviors, and use of materials in PBC debrief conversations

61 Table 2-1. The “who”: recipients of systematic follow-up. Systematic Peer Follow-Up Community of Coaching Consultation Mentoring Support Support Practice Group Studies (n = 227) (n = 186) (n = 6) (n = 22) (n = 14) (n = 8) Work Setting Head Start 44% 44% 17% 68% 43% 25% Early Head Start 3% 2% - 9% 7% 13% Child Care 26% 29% - 27% 14% - Special Education 9% 1% - 9% - 13% Early Intervention 7% 5% 17% 5% 14% 25% Family Childcare Home 4% 5% - - - - Pre-School 40% 42% 50% 27% 36% 38% Kindergarten 2% 2% - 9% - - Other 7% 7% 33% 5% 7% - Not Reported > 1% 1% - - - -

Children Served With Disabilities 43% 46% 17% 18% 29% 63% At Risk 43% 41% 50% 77% 36% 25% None with Disabilities 1% 1% - - 7% - Not Reported 12% 11% 33% 5% 29% 13%

Age Infants-Toddler 30% 29% 50% 27% 29% 25% Preschool 89% 90% 83% 96% 79% 75%

Follow-Up Recipient Lead Teacher 72% 72% 67% 19% 79% 50% Paraprofessional 14% 15% 17% 9% 14% 13% Home Care Provider 6% 6% - - 7% 25% Team 11% 12% - 14% 7% 25% Undergraduate/Student 8% 10% - - - - Other 7% 5% 50% 9% 7% 13%

Table 2-2. The “who”: providers of systematic follow-up. Systematic Community of Peer Support Follow-Up Coaching Consultation Mentoring Practice Group Support Studies (n = 227) (n = 186) (n = 6) (n = 22) (n = 14) (n = 8) Follow-Up Agent Self 12% 4% 50% 14% - - Peer 11% 7% 50% 9% 36% 50% Supervisor 11% 9% 17% 9% 29% 25% Consultant 22% 18% - 59% 14% 13% Research 48% 54% 50% 27% 21% 25% Other 4% 5% - 5% - 13% Not Reported 9% 10% 17% - - 13%

Agent Qualifications Less than Bachelors 2% 2% - - 7% 25% Bachelors 13% 13% - 18% 7% 25% Masters 23% 22% 33% 32% 29% 13% Higher than Masters 9% 8% 17% 27% - 13% Teaching Experience 39% 34% 50% 18% 43% 25% Experience with Adults 19% 17% 17% 40% 21% - Other 5% 5% - 9% - - Not Described 43% 46% - 27% 29% 38%

Agent Training/Support Study Provided Agent Training/Support 26% 23% 33% 55% 43% 50% One Day Training 2% 1% - - 14% - Multi Day Training 9% 8% - 23% 7% 13% Ongoing Support 15% 14% 33% 27% 14% 13% Periodic Support 5% 4% - 9% 14% 13% Content Focus 19% 17% - 41% 21% 50% Adult Learning Focus 14% 13% - 55% 29% 13% Agent Attended/ Presented Recipient Training 24% 25% 67% 27% 7% 13%

63 Table 2-3. The “what” of follow-up. System atic Peer Follow- Community Coaching Consultation Mentoring Support Up of Practice Group Support Studies (n = 227) (n = 186) (n = 6) (n = 22) (n = 14) (n = 8) Content Area Approaches to 5% 5% - 9% 7% - Learning Language 38% 43% 33% 32% 21% - Development English Language 1% 1% 33% 5% - - Development Literacy 34% 37% 50% 32% 14% 13% Logic and Reasoning >1% >1% - - - - Mathematics 8% 9% - 14% 7% 25% Science 2% 3% - - - - Social Studies >1% >1% - - - - Physical Development 5% 5% - 5% 7% - Social/Emotional 45% 44% - 72% 36% 50% Creative Arts 1% 2% - 5% - - Expression Other 10% 10% - 9% 14% - Not Reported 21% 19% 50% 5% 43% 25%

Instructional Practices Single practice 4% 6% - - - - Group of practices 19% 20% 17% 23% 7% 2% Practices from a multi- 6% 7% - 5% - - component approach Multi-component 14% 17% 17% 5% - - approach Targeted Curriculum 17% 17% 16% >1% 29% 13% Curriculum 23% 23% 33% 23% 29% - Other 14% 15% 33% 5% 7% 25% Not Reported 17% 12% - 41% 57% 25%

Focus of Instruction One Target Child 9% 11% - 9% - - Target Children 5% 6% - - - Small Group 3% 4% - - - - Whole Class 30% 32% 33% 27% 29% 38% Program 4% 3% - 18% - - Other 1% 2% - - - - Not Reported 44% 43% 67% 37% 71% 38%

Table 2-4. The “how” of systematic follow-up. Systematic Peer Follow-Up Community Coaching Consultation Mentoring Support Support of Practice Group Studies (n = 227) (n = 186) (n = 6) (n = 22) (n = 14) (n = 8) Duration of Collaboration <1 Week ------1 Week >1% 1% - - - - 1 Month 6% 8% - - 7% - 1 Quarter 9% 10% 33% 9% - - 1 Semester 13% 13% 17% 18% 7% 13% 1 Year 18% 19% - 23% 14% 38% > 1 Year 13% 12% 17% 9% 29% 38% Not Reported 38% 82% 2% 10% 7% 1%

Frequency of Session > 1 per Week 12% 13% - 5% 21% - Weekly 31% 34% 50% 23% - 13% Monthly 23% 24% - 18% 7% 50% Infrequently 4% 5% - 5% - 38% Not Reported 32% 29% 50% 50% 29% 25%

Session Length 0-15 8% 10% - 5% - 13% 15-30 8% 10% - 5% - 13% 30+ 27% 29% 33% 9% 7% 50% Not Reported 58% 54% 67% 82% 93% 25%

Follow-Up Agent Guides Manual 9% 11% 17% 5% 7% - Rubric 1% >1% 17% 9% - - Script/Protocol 7% 8% - 5% - - Other 4% 5% - 5% - - Not Reported 76% 75% 67% 77% 93% -

Fidelity Checklist 8% 8% - 13% 7% - Observation Measure 5% 5% - 14% - - Rating Scale >1% - - - - - Other 9% 10% 16% 18% - 13% Not Reported 80% 81% 83% 59% 93% 87%

Table 2-5. Coaching studies aligned with PBC coaching components or strategies. Systematic Coaching plus Follow-Up Coaching + teaching Support practices Studies (n = 227) (n = 186) (n = 54)

Focus on Teaching Practices 28% 29% N/A Needs Assessment 8% 19% 6% Goal Setting 17% 3% 19% Action Plan 2% 3% 4% Observation 61% 65% 65% Performance Feedback Verbal 57% 64% 70% Performance Feedback Written 22% 27% 28% Performance Feedback Graphical 6% 8% 13% Feedback Reflective Journal >1% 1% 2%

66 Table 2-6. Empirical evidence for practice-based coaching. Teaching Study Reference Participants Coaching Dose Teacher Outcomes Child Outcomes Practices Design

Abell, Family Childcare Global Quality Format: Onsite Pre-Post Global quality as Not Reported Ariswalla, Homes (N = 365) Duration: Provider measured by the Putnam, and preference FDCRS3 Miller (2014) Frequency: 1-2/week Length: 2-2.5 hours PBC

Components

Artman- University-based Pyramid Model Format: Video Multiple Team 1: Clear Team 1 child: Meeker & Childcare Teams Practices: Observation and Email baseline effects on social- Lower levels of Hemmeter, (N = 4); Children Transitions, Feedback Single emotional challenging 2012 age 3-4 years (N Rules, Social- Duration: ~7 weeks Subject strategies, behavior; Team 2 = 2) emotional Frequency: ~2.2/week Transitions and child: behavior was PBC Length: Observation = Rules decreased variable across Not reported; Feedback post-feedback; strategies with = ~15 min Team 2: Clear transitions being effects on all 3 most effective strategies

Artman- Head Start (N Global Quality; Format: Web-mediate Randomized Teachers with the Not Reported Meeker, =33) across an Pyramid Model Duration: 12 weeks Group highest levels of Hemmeter, & intervention and Practices Frequency: 1/ two Comparison participation (i.e., Snyder, 2014 BAU condition weeks more videos Length: Observation submitted for PBC =30-60, Debrief =NA feedback and visits to the website) maintained or increased their use of practices; Global quality as measured by the CLASS4 was inconsistent

3 Family Day Care Rating Scale (Harms & Clifford, 1989) 4 Classroom Assessment Scoring System (Pianta, La Paro, & Hamre, 2008)

Table 2-1. Continued Teaching Study Reference Participants Coaching Dose Teacher Outcomes Child Outcomes Practices Design

Conroy et al., Head Start and BEST in Class Format: Onsite Randomized Significant increase Significant 2014 Public Pre-K (N = practices Duration: 14 weeks Controlled in use of rules, pre- increase in the 53) across a targeting the Frequency: 1/week Trial correction, behavior overall percent of PBC intervention and prevention of Length: 2 hours specific praise, child engagement BAU condition; emotional instructive feedback, and positive Children ages 3- behavioral corrective feedback, interactions; 5 years (N = 130) disorders and and opportunities to Significant increase child respond decrease in engagement overall percentage of disruptive, aggressive and defiant behavior and negative interactions Conroy, Head Start, BEST in Class Format: Onsite Pre-Post Teacher’s use of the Children’s Sutherland, Public Pre-K and practices Duration: 14 weeks strategies increased problem behaviors Vo, Carr & University based targeting the Frequency: 1/week and maintained after decreased over Ogston, 2014 Childcare (N = prevention of Length: 60 min the intervention time and their 10); Children emotional engagement PBC ages 3-5 years behavioral slightly increased (N = 19) disorders and increase child engagement Fettig & Childcare (N=6); Pyramid Model Format: Onsite/Group Pre-Post Teachers increased Not Reported Artman- Children ages 2- Practices Duration: 6 months in the mean Meeker, 2016 5 years Frequency: 2-3/month percentage of Length 60-90 min indicator PBC implemented as measured by the TPOT5 and a reduction in red flag items

5 Teaching Pyramid Observation Tool (Hemmeter, Fox, & Snyder, 2013)

Table 2-1. Continued Teaching Study Reference Participants Coaching Dose Teacher Outcomes Child Outcomes Practices Design

Fox, Early Childhood Pyramid Model Format: Onsite Multiple Teachers increased Not Reported Hemmeter, Special Practices Duration: 60 days Probe their use of practices Snyder, Education Frequency: 1-2/week Single- above baseline levels Binder, & (N = 3) Length: 60-90 min subject and 2 teachers met Clarke, 2011 criterion levels

PBC

Greenwood, Early Childhood Literacy 3D: Oral Format: Onsite Waitlist Teachers in year 1 did Children in year 2 Abbott, Special language, Duration: 26 weeks Randomized not have significant made significant Beecher, Education alphabet Frequency: 1/week Controlled increases in their use linear growth in Atwater, & Year 1 (N= 20), knowledge, and Length: 60 min Trial of practices at the end the PELI Peterson Year 1 (N =10) phonological of the year compared Composite. The (2017) awareness to BAU. Teachers in PELI ™6 year 2 had significant vocabulary and PBC increases in their use comprehension of teacher literacy had significant focus practices and slopes favoring child literacy the children with engagement IEPs compared to their Year 1 BAU scores.

Hemmeter, Public Pre-K Pyramid Model Format: Onsite and Multiple All teachers acquired Class-wide Hardy, (N = 3); Children Practices Web-mediated Probe the practice and challenging Schnitz, ages 3-5 years Duration: Not Reported Single- maintained the behavior was Adams, & Frequency: 2-3/week subject practices above reduced in 2 of Kinder, 2015 Length: Not Reported baseline the 3 classrooms

PBC

6 Preschool Early Literacy Indicators (Abbott, Kaminski, Aguayo, & Latimer, 2014)

Table 2-1. Continued Teaching Study Reference Participants Coaching Dose Teacher Outcomes Child Outcomes Practices Design

Hemmeter, Public Pre-K and Pyramid Model Format: Onsite Randomized Significant increase Children in the Snyder, Fox, Early Childhood Practices Duration: 12-16 weeks Controlled in the use of the intervention group & Algina Special Frequency: 1/week Trial practices as had higher scores (2016) Education Length: 10-305 min measured by the than the control (N = 40) across TPOT in comparison group on the PBC intervention and to the BAU condition, SSIS, differences BAU condition; Higher levels of were statistically Children ages 3- emotional support significant 5 years and behavior management on the CLASS for intervention teachers compared to BAU

Hemmeter, Head Start (N = Descriptive Format: Onsite Multiple All teachers Increased Snyder, 43) and praise Observation and Email Probe increased their use of engagement in all Kinder, & Childcare (N = statements to Feedback Single- descriptive praise 4 classrooms; Artman, 2011 1); Children ages comment on Duration: 1.5 to 3 weeks subject and maintained their reduction in 3-5 years children’s Frequency: 2-3/week use of descriptive challenging PBC positive behavior Length: Not reported praise above behavior for 3 of baseline levels the 4 teachers

Hsieh, Childcare (N = Oral language, Format: Onsite Multiple All teachers Paired t-tests Hemmeter, 3), Public Pre-K comprehension Duration: 5.5-7.5 weeks baseline increased their use of revealed that the McCollum & (N = 2); Children of text, Frequency: 1-2/week Single- the strategies in all children Ostrosky, ages 3-5 years phonological Length: 4-5 hours subject areas and maintained demonstrated 2009 awareness and their use of the significantly higher alphabetic strategies above scores on picture PBC principle, and baseline levels naming, print concepts alliteration, and written rhyming, and book language handling after the practices intervention

Table 2-1. Continued

Teaching Study Reference Participants Coaching Dose Teacher Outcomes Child Outcomes Practices Design

Johnson, Head Start (N = Pyramid Model Format: Onsite Pre-Post Significant increase Not Reported 2017 30), Early Practices Duration: 1 year in the mean Childhood Frequency: Monthly percentage of PBC Special Length: Not reported indicators Education implemented from collaborative (N = fall to spring as 26), State school measured by the readiness TPOT programs (N = 23), Early Childhood Special Education self-contained (N = 7), Childcare (N = 7); Children ages 3-5

McCollum, Public Pre-K (N = Vocabulary, Format: Onsite Randomized Significant post Not Reported Hemmeter & 12) Phonological Duration: 30 weeks Group intervention group Hsieh, 2011 Awareness, and Frequency: bi-weekly Comparison difference for Print Concepts Length: Feedback 15 Phonological PBC min, Observation not Awareness, and reported Print Concepts; Significant post intervention group difference for 3 areas of the ELLCO7; Most teachers reached 80% criterion in most areas

7 Early Language and Literacy Classroom Observation (Smith & Dickinson, 2002)

Table 2-1. Continued Teaching Study Reference Participants Coaching Dose Teacher Outcomes Child Outcomes Practices Design

Neuman & Center based Global Quality; Format: Onsite Randomized Statistically significant Not Reported Cunningham, care (N = 177), Language and Duration: 32 weeks Controlled improvements in 2009 Family Childcare Literacy Frequency: 1/week Trial language and literacy (FCC) Homes (N Practices Length: 60-90 min practices for teachers PBC = 114), across 3 who received Components conditions coursework plus college course, coaching, with on-site coaching, substantial effect BAU sizes for center and FCC providers as measured by the ELLCO and CHELLO8

Neuman & Community Global Quality; Format: Onsite Randomized Statistically significant Not Reported Wright, 2010 center or public Language and Duration: 10 weeks Controlled improvements in the preschool Literacy Frequency: 1/week Trial structural environment PBC teachers (N = Practices Length: 3 hours as measured by the Components 148) across 3 ELLCO, when conditions compared to the other college course, conditions on-site coaching, BAU

Powell, Head Start (N = Language and Format: Onsite or Web- Randomized Statistically significant Statistically Diamond, 88) across on- Literacy mediated Controlled effects were identified significant effects Burchinal, & site and remote Practices Duration: 15 weeks Trial in the areas of were identified in Kohler, 2010 coaching Frequency: biweekly classroom the areas of letter conditions; Length: Onsite = 2 environment, knowledge, PBC Children ages 3- hours, Web-mediated = classroom supports blending skills, Components 5 years (N = 759) 30 min for early literacy and writing, and language concepts about development print

8 Child/Home Environmental Language and Literacy Classroom Observation (Neuman, Dwyer, & Koh, 2007)

CHAPTER 3 METHODOLOGY

Systematically describing aspects of the process dimension of coach-based professional development (PD) is critical for understanding how coaches promote teachers’ learning and use of evidence-based practices (EBP) in the classroom. The process dimension of coaching includes the coaching behaviors, strategies, and materials used to promote the teacher’s confidence, competence, and use of an identified set of evidence-based teaching practices. The present study explored how coaches facilitated conversations during the coaching debrief meeting of an on-site

Practice-based Coaching (PBC) partnership. A direct behavioral observation system was developed to investigate (a) the proportion of time allocated to different conversational foci, including who initiated the conversation focus; (b) a description of the verbal behaviors used by coaches to support teachers’ active participation in planning for, implementing, and evaluating the embedded instruction practices that were the practice-focus of the coaching partnership; and (c) an exploration of whether the conversation foci, coach and teacher initiations, and verbal behaviors changed over the course of the partnership (across three occasions). Descriptive methods were used to examine these aspects of the debrief conversation across seven coaches and within each coach-teacher dyad across three occasions (occasion 1 = sessions 1-5, occasion

2 = sessions 6-10, occasion 3 = sessions 11-15). Data for the present study were collected as a part of year one of a larger efficacy trial examining Tools for Teachers

(TfT), a professional development (PD) intervention designed to support classroom teachers to use embedded instruction with preschool children with disabilities (Snyder,

Algina et al., 2015).

The purpose of this chapter is to describe the methods used to conduct the present study. The research questions are presented below, followed by a description of the context, measures, participants, procedures, and analytic techniques employed.

Research Questions

The primary aim of the present study was to describe the PBC debrief meeting processes including how the coach and teacher initiated and participated in six conversation foci across three occasions and how coaches facilitated those conversation foci using eight verbal behaviors. The following research questions guided the analyses conducted in the present study: