Moving from Fair to Good to Great: Human Capital Leaders in DC Public Schools Reflect and Look Ahead

Case Study September 2015

In March 2015, President Obama – during a conversation with urban school district leaders from across the United States – held up Washington, D.C.’s school system as an example of what was most promising in public education.1 It was a shining moment for a district with a longstanding reputation as one of the nation’s worst.

For years, Washington D.C. Public Schools (DCPS) had spent more money per student than the vast majority of other districts but had little to show for it. In 2006, for example, only 12 percent of DCPS eighth graders were proficient readers, according to the National Assessment of Educational Progress (NAEP), and less than half of its students completed high school. Even so, more than 95 percent of DCPS teachers received positive evaluations.

Beginning in mid-2007, however, a series of major changes were launched in DCPS by new leaders who were determined to provide a high-quality education to the district’s 46,000 students, three-quarters of whom were economically disadvantaged. (See Attachment A.) And they were willing to challenge vested interests and uproot entrenched policies and practices in order to do so. Over the following years, their efforts – in particular, initiatives aimed at instilling accountability and improving principal and teacher quality – reaped results. From 2007 to 2014, math proficiency scores in DCPS rose 23 percentage points, and reading scores rose 13. (See Attachment B.) In fact, the district’s students made larger performance gains than peers in any other state in the NAEP assessments.

Yet despite the progress, DCPS leaders like Kaya Henderson (Chancellor), Jason Kamras (Chief of the Office of Human Capital), Scott Thompson (Deputy Chief for Teacher Effectiveness), and Hilary Darilek (Deputy Chief for Principal Effectiveness) were keenly aware in the spring of 2015 that there was still much work to be done. Student proficiency in the district still lagged far behind other urban school systems, and large achievement gaps persisted. Moreover, DCPS’s lowest performing schools were still struggling to attract and retain great teachers. And so, even as they basked in the president’s praise, the district’s human capital leaders were taking stock and considering what their next steps should be.

Part I: 2007-2010

Creating a Culture of Accountability DCPS’s reform journey began in 2006 when Washington, D.C. residents elected a new mayor named who was determined to fix the city’s woefully underperforming school system. Soon after his election victory, Fenty crafted a reform plan that stripped the elected board of education of its decision making authority and gave unprecedented control over DCPS to the mayor and his newly appointed chancellor, Michelle Rhee. Known for being a tenacious leader, Rhee had founded – and, for the past decade, led – The New Teacher Project (TNTP), where her work had focused on strengthening the teacher

1 Michael Alison Chandler, “Meeting with Obama highlights D.C. Public Schools as Leader in Reforms,” Washington Post, March 18, 2015.

This case study was written by Lynn Jenkins for the Urban Schools Human Capital Academy. Copyright 2015. Please do not quote or distribute without permission from Urban Schools Human Capital Academy. workforce in urban districts. Rhee chose TNTP vice president Kaya Henderson as her deputy, whose role would include human capital leadership.

Rhee and Henderson were keen to tackle their top priority: improving student achievement. But before they could make much headway in strategic work such as improving teacher and principal effectiveness, they knew they had to do something about central office. During her initial weeks in DCPS, Rhee met with principals, teachers, and parents across the district who provided countless examples of unresponsiveness and ineptness among central office staff. Repair work orders were frequently ignored, many schools lacked textbooks and supplies, teachers’ paychecks were delivered late, and more.

When Rhee tried to fire central office employees who were performing poorly, though, she found that this was almost impossible because they had never been evaluated. Determined to end this untenable situation, she drafted new legislation giving her the authority to terminate central office staff for performance reasons, and the city council passed it despite union opposition.2 With her newly granted authority, Rhee fired 10 percent of the central office employees in her first year, but she emphasized that this was not going to solve the district’s problems. “What’s going to solve the problems,” she said, “is creating a culture of accountability in central office first and then everywhere in the district.”3

To that end, Rhee and her senior team moved promptly to establish a new accountability and performance management system. From now on, every department in central office would have specific goals, and progress toward them would be closely monitored. Expectations for central office employees were also defined, and a system for evaluating their performance was created. Other early central office initiatives, such as establishing a robust information management system, would take longer – but observers were encouraged to see that severe problems, which had festered for years were finally being addressed.4

Seizing a Window of Opportunity Rhee and Henderson knew from the outset that human capital reforms would be vital to their efforts to dramatically improve student achievement and eradicate achievement gaps throughout the school system. As Deputy Chancellor Henderson noted at a city council hearing in January 2009:

This administration believes, in no uncertain terms, that the key lever to change in any organization is the quality of its people. . . . With that in mind, we are working diligently to ensure that we attract and select high-caliber candidates to DCPS, create the conditions in which great people can be successful, reward those who excel, and fairly transition-out those who are ineffective or unqualified. [Our] goal is to have the highest-performing, highest-paid, best supported, most satisfied, and most honored educator force in America. Our students deserve nothing less.5

Efforts to make good on these promises began during Rhee’s first year, when she and Henderson started to shake up the existing school leadership based on evidence that many campuses lacked strong leaders. At the end of the 2007-08 school year, 39 percent of DCPS’s 131 principals left the district. The next year, 22 percent left. (Years later, a Mathematica study would validate these changes, finding that the principals who replaced those who left generated significant gains in reading and math.)6

Keenly aware of the research evidence that high-quality teachers are the most important school-based factor for improving student achievement, Rhee and Henderson were also intent on improving teacher

2 Non-union staff could choose between being reclassified as “at-will” or resigning and getting a severance payment. Union employees were not affected. 3 PBS, Interview with John Merrow, Air Date November 19, 2007, p. 4. 4 Mike DeBonis, “The Office,” Washington City Paper, November 15, 2007. 5 District of Columbia Public Schools, Testimony of Kaya Henderson, Deputy Chancellor, Oversight Hearing of the Council of the District of Columbia, Human Resources and Human Capital Initiatives, January 16, 2009. 6 Mathematica Policy Research, The Impact of Replacing Principals on Student Achievement in DC Public Schools, December 2014. DCPS Case Study 2 quality in DCPS. A recent study conducted by The Parthenon Group documented the severe challenges they faced, describing “a lack of systemic emphasis on basic teaching and learning” in DCPS. “Quality of teaching is the leading pain point,” the report stated, “and adult job-saving is prioritized over student achievement.”7

To provide the backbone for DCPS’s teacher quality efforts, Rhee and Henderson believed that a robust evaluation system was urgently needed in order to accurately assess performance among the district’s teachers. At the time, principals rated teachers based on infrequent classroom observations that were largely seen as meaningless. Teachers complained that principals lacked the time and subject matter expertise needed to assess their practice. Moreover, implementation was inconsistent, so the results lacked credibility – especially since nearly all teachers were rated as meeting expectations, even as student achievement remained very low.

To lead the development of a new evaluation system, Rhee tapped Jason Kamras, a seasoned DCPS teacher who had been named the 2005 National Teacher of the Year. (See Attachment C.) Kamras became the Director of Human Capital for Teachers, a new role within the newly formed human capital department. As Kamras later reflected, the convergence of various forces provided a window of opportunity to make major changes:

We had strong mayoral support and the district’s contract with the teachers union had recently expired, so we began by negotiating with the union to shift from a seniority- based system to a performance-based one. But when you stop using years of service as your primary indicator of who gets hired, who loses their position during tight budget times,8 and who gets paid more – and instead use performance – you need an evaluation system. These things go together. That’s why tackling the evaluation system was so important.9

Eventually, the new IMPACT evaluation system would cover all 7,000 school-based staff, from custodians and aides to teachers and principals.10 A new era of accountability in Washington, D.C.’s public schools had begun.

Developing IMPACT From 2007 through 2009, Kamras led an intensive research and design process to develop a new teacher evaluation system for DCPS informed by input from educators. To ensure that IMPACT focused on what mattered most, Kamras and his team spearheaded the creation of a new Teaching and Learning Framework for DCPS, consisting of nine instructional practice standards. From now on, every teacher in the district would be observed five times a year, three times by his or her school administrator and twice by a “master educator,” an expert teacher who was content-aligned but external to the school. Most of the master educators would be recruited from outside of DCPS.

Based on their observations of teachers’ performance, the principals and master educators would score teachers on various areas linked to the Framework, ranging from classroom management to the use of specific instructional strategies.11 After each observation, the master educator or principal would provide feedback and create a growth plan identifying support or professional development that the teacher

7 The Parthenon Group, “Fact-Base for DCPS Reform” [Powerpoint], December 2006. 8 This issue was timely, since DCPS was facing a budget crisis and reductions in force (RIFs) were likely. At the time, district leaders had to make layoff decisions based on teachers’ seniority; the most recent hires were the first to go, regardless of their effectiveness. 9 Interview with Jason Kamras, May 14, 2015. Unless otherwise indicated, all quotations from Kamras are from this interview. 10 IMPACT divided employees into 20 functional groups, including teachers with and without value-added data, special education teachers, ELL teachers, student support professionals, librarians, counselors, social workers/psychologists, program coordinators/deans, instructional coaches, aides, office staff, custodial staff, etc. 11 DCPS leaders conducted rigorous training to ensure inter-rater consistency in how teachers were observed and rated; in fact, an Align team developed an innovative online platform to norm the evaluators. DCPS Case Study 3 needed. Then, based on the observations, student achievement data,12 and other information, all teachers would receive one of four ratings: Highly Effective, Effective, Minimally Effective, or Ineffective.13 (See Attachment D.)

All in all, hundreds of teachers participated in the development of IMPACT. Nevertheless, union leaders still criticized it. Some complained that the process was too complicated, while others called the rating system simplistic. “It takes the art of teaching and turns it into bean counting,” one union leader said.14 But national experts had a different view. Brad Jupp, who had led the development of Denver’s ProComp teacher incentive pay program, called IMPACT “an excellent blend of precisely described excellent teaching behaviors and fair administrative procedures for making judgments about teaching performance.”15 Kate Walsh, president of the National Council on Teacher Quality, called it “light years ahead of what’s available in most districts.”16 They and many others continued to watch closely as DCPS embarked on implementation.

Reaching a Groundbreaking Agreement For Rhee, Henderson, and Kamras, the new evaluation system was an important step, but it was not the end goal. Rather, it was part of a broader set of strategies intended to make DCPS “smarter” and more intentional about teacher recruitment, selection, evaluation, development, compensation, and retention with the ultimate goal of increasing student achievement. Kamras captured district leaders’ human capital theory of action in a diagram (Figure 1).

Figure 1: DCPS’s Human Capital Theory of Action

Retain the best and extend their reach Provide INCREASE Recruit and select the Set new hires up for Rigorously evaluate STUDENT best possible talent success performance targeted ACHIEVEMENT support Transion out lowest performers

Source: DCPS presentation.

Some of the major elements of this theory of action had to be negotiated with the Washington Teachers Union (WTU). Throughout 2008 and 2009, Henderson and Kamras were immersed in contract talks with union leaders, trying to work out new provisions related to performance-based pay and termination, among other issues. After a lengthy bargaining process, DCPS reached a tentative five-year agreement with the WTU and AFT in April 2010 that would give teachers a 21.6 percent pay increase, compounded over the five-year period, with three years of back pay. In return, teachers would give up job security based on seniority. Retaining a job in DCPS would now depend on performance as measured by IMPACT.

The new agreement also included performance-based compensation for the first time. Teachers rated Highly Effective would be eligible for higher compensation as well as significant bonuses, while those

12 Value-added data were included in IMPACT for teachers in tested subjects but were not available for other staff. DCPS worked with Mathematica Policy Research, Inc. to design the value-added model. 13 A fifth rating category, Developing, was added later based on evidence that the Effective category was too broad. 14 Bill Turque, “Schools Chancellor Working on Overhaul of Teacher Evaluations,” Washington Post, April 7, 2009. 15 As quoted in Richard Whitmire, The Bee Eater, 2011, p. 212. 16 Bill Turque, “D.C. Launches Rigorous Teacher Evaluation System,” Washington Post, October 1, 2009. DCPS Case Study 4 rated Effective would receive their normal salary increase. Teachers receiving a Minimally Effective rating would receive extra professional development but no salary increase. And those rated Ineffective, as well as those receiving Minimally Effective ratings for two consecutive years, would be terminated.

In addition, a new “mutual consent” clause would end the forced placement of teachers in DCPS schools. From now on, educators who were displaced from a school for budgetary or other reasons would not be guaranteed a position in another school; they would need to find a principal who wanted to select them. Those with Effective or higher ratings who could not find a new position in DCPS would have one year with full pay to search for a new spot. All educators who were not selected would be eligible for a $25,000 buyout, and those with 20+ years of service could retire early with full benefits.

The agreement was viewed as a breakthrough agreement nationally, and on June 2, 2010, the WTU membership approved it by a vote of 1,412 to 425. Rhee expressed her satisfaction with the new deal. “It strikes a great balance between making teachers understand that we very much value and support the work they do every day and on the administrative side giving us the tools we need to staff schools effectively,” she said.17

Using the IMPACT Results That same summer, the 2010 DCPS assessment results were released, and they showed evidence of progress: 45 percent of fourth graders were now proficient in reading (up from 36 percent in 2007), and 46 percent were proficient in math (up from 34 percent). Across the grades, more students were performing at the highest levels of achievement and fewer were at the lowest levels.

The release of the achievement data also made it possible to calculate value-added scores and finalize the first round of IMPACT ratings. In July 2010, Henderson foreshadowed that a “sizable” number of teachers and other school staff with low ratings would be dismissed but did not give specific numbers. A few days later, the 2009-10 IMPACT results were released: • Highly Effective: 16 percent • Effective: 66 percent • Minimally Effective: 16 percent • Ineffective: 2 percent18

With the evaluation results in hand, DCPS was also able to implement its new performance-based compensation program, IMPACTplus. Based on their ratings, Highly Effective teachers were eligible for bonuses ranging from $5,000 to $25,000, and those who had attained the top rating for two years would also be eligible for a base salary increase of $10,000 to $20,000.19 As a result of IMPACTplus, starting DCPS teachers could now earn more than $74,000 (vs. $42,000 previously), and the best teachers could now earn as much $131,000 annually if they stayed in the system over time.

Kamras and his colleagues saw a wide variety of other ways in which the evaluation results could be used, such as comparing the quality of various recruitment pipelines, assessing the impact of professional development efforts, and understanding which factors were correlated with retaining high performers. “There are hundreds of human capital questions you need to answer to effectively run a school district,” Kamras said, “and now, for the first time, we have the data to answer them.”20

17 Bill Turque, “D.C. Teachers’ Union Ratifies Contract, Basing Pay on Results, Not Seniority,” Washington Post, June 3, 2010. 18 District of Columbia, 2010-2011 IMPACT Results (Powerpoint presentation), July 15, 2011. 19 Highly effective teachers who taught in high-poverty schools were eligible for the largest increases. 20 Stephanie McCrummen, “D.C. Schools to Use Data from Teacher Evaluation System in New Ways,” Washington Post, February 14, 2011. DCPS Case Study 5

Growing Resistance DCPS’s human capital initiatives were not the only changes underway in the district. By the summer of 2010, Rhee and Fenty were also moving forward with their reform plan, Renew-Revitalize-Reorganize, which called for closing or restructuring numerous schools.

Key members of city council along with many community leaders were upset by the proposed changes, however, and by the lack of community input into them. They urged Rhee and Fenty to slow down and adopt a more collaborative and deliberative approach. Yet the chancellor remained adamant about the need to press forward, repeatedly emphasizing that the needs of students had to be put before the needs of adults. She told a group of public policy officials in a quote that was often repeated, “If there is one thing I’ve learned, it is that cooperation, collaboration, and consensus-building are way overrated.”21

As 2010 continued, the criticisms of Rhee and Fenty intensified. One of their most vocal critics was City Council Chair Vincent Gray, and in the September 2010 mayoral primary, Gray defeated Fenty in a stunning upset. Seeing the writing on the wall, Rhee resigned as chancellor, and Gray chose Henderson to replace her – first as an interim and then, in July 2011, permanently. Kamras, who had been instrumental in leading the district’s teacher quality initiatives, became the new chief of human capital. Other members of the senior team also remained, and DCPS’s human capital initiatives persisted despite the changes in leadership.

Part II: 2011-2015

From mid-2011, when Henderson officially assumed the chancellor role, through early 2015, DCPS’s human capital leaders moved ahead on numerous fronts. As Scott Thompson, the Deputy Chief for Teacher Effectiveness, described:

After starting off with the evaluation system, the union contract, and performance pay, we took on other pieces of the human capital puzzle such as teacher recruitment and selection, development, and retention. Then we began focusing more intently on teacher leadership and on non-teaching staff roles and responsibilities. So over time, the picture began coming together in a comprehensive way.22

Unlike the Rhee era, which had often been tumultuous, much of the work now underway was less controversial. It was seldom featured in dramatic headlines, and from Henderson’s point of view, this was a good thing. As she noted in a Washington Post editorial in early 2013:

Many people think that if we are not angering the community, clashing with unions, creating discord in our schools, and making headlines, we must not be making change. When I began working for DCPS in 2007, the school district was broken, and we spent tense and contentious years fixing the most immediate problems. We made changes that were critical to improving our school district. But they were not the only changes needed to ensure that any parent would be proud to send his or her child to D.C. schools. . . . At this stage in our work, I do not believe that we need to create more friction to get better results. We will continue to invest in our educators, academic content, and student and family engagement because we know that these efforts will pay off.23

Recruiting and Selecting Great Teachers Once the new Teaching and Learning Framework and the IMPACT system were fully operational, DCPS leaders set their sights on changing how the district recruited and selected teachers. Historically, the key

21 Bill Turque, “D.C. Has a Long History of Troubled Education Reform,” Washington Post, October 31, 2009. 22 Interview with Scott Thompson, May 14, 2015. Unless otherwise indicated, all quotations from Thompson are from this interview. 23 Kaya Henderson, “Quiet, Critical School Reform at D.C. Public Schools,” Washington Post, April 28, 2013. DCPS Case Study 6 measure of success used by the recruitment and selection team had been the ratio of candidates for each teacher vacancy (the target was 1.5 or more). They decided to move to a very different metric: the percentage of entering teachers who were effective in their first year. As a result, the focus was no longer on simply having a large pool of prospects; it was on having a robust pool as well as a rigorous process for choosing and hiring high-caliber candidates.

To pursue this new measure of success, the recruitment and selection team developed and launched a broad marketing and outreach effort aimed at increasing the number of outstanding candidates. In an effort to attract more teachers of color, they also expanded outreach to students and faculty at historically Black colleges and Hispanic-serving institutions, invested in targeted marketing, and began featuring more teachers of color in DCPS’s marketing materials.

The team also developed a new selection model linked to the Teaching and Learning Framework. Applicants were now required to undergo a multi-step process designed to assess their skills and fit. The first stage consisted of an online application with essays; the second was a telephone interview; and the third was an “audition” in which applicants uploaded a teaching video for the selection team to review. Those who moved through these stages successfully were placed in the “approved pool” to be considered for open slots in schools.

These efforts were demonstrably successful: the number of teacher applicants rose by 38 percent from 2012 to 2015, while the percentage of candidates who identified as people of color increased from 42 percent to 54 percent of the applicant pool.24 The changes also led to earlier hiring. In 2012, DCPS had hired only 32 teachers by June 1. In 2015, it hired more than 400 teachers by that date. Because later hires tended to be less effective than early hires, these changes – along with other efforts, such as improving induction and establishing a new teacher residency program25 – meant that DCPS was far more likely to achieve its goal of simultaneously expanding and improving the pipeline of entering teachers.

Retaining Effective Teachers The flip side of recruitment and selection was retention. Like other urban districts, DCPS had long struggled with high rates of teacher attrition. For example, 15 percent of DCPS teachers left at the end of the 2009-10 school year, and a similar percentage left after the following year.26

One of the major concerns raised during the development of IMPACT was that imposing a rigorous evaluation system would worsen attrition among the best educators by exacerbating pressures upon them. Therefore, once IMPACT and the new contract gave DCPS leaders the ability to shed the least effective teachers, one of the next big priorities was to figure out how to keep the best. “That is ultimately what we think is important and necessary for improving student achievement,” said Michelle Hudacsko, the Deputy Chief for IMPACT, “along with helping all teachers improve.” 27

The new compensation and bonus system was a vital part of the equation, but as Thompson noted:

We see performance pay as less about trying to incentivize people to work harder and more about paying teachers a salary that enables them to stay in the profession, stay in the classroom, and stay in DC, which is an expensive place to live. We also believe that performance pay is an important step toward giving our teachers the recognition they

24 Internal data provided by DCPS recruitment and selection staff, June 2015. 25 The DC Teacher Residency prepares recent college graduates, career changers, and paraprofessionals to be effective teachers in DCPS’s highest-need schools. Residents train in classrooms with high-performing mentor teachers for an academic year in preparation for assuming a lead teaching role in the second through fourth years of the program. At the end of the residency, participants receive teaching certification and a master’s degree with a dual focus in general and special education. 26 The National Academies Press, op cit. 27 Interview with Michelle Hudacsko, May 13, 2015. Unless otherwise indicated, all quotations from Hudacsko are from this interview. DCPS Case Study 7

deserve, attracts more people to the profession, and raises the prestige and status of the profession to where it belongs.

Instructional superintendent LaKimbre Brown agreed that although compensation was important, it was not the only factor in motivating teachers to stay. What teachers want, she said, “is to be recognized, to be involved in leadership decisions, to have a say in the school, and to be given opportunities to grow.”28

As part of its concerted efforts to retain the best teachers, one initiative that DCPS launched was a new career ladder program called The Leadership Initiative for Teachers (LIFT). Designed with broad teacher input during the 2011-12 school year and launched in 2012-13, LIFT made it possible for teachers to advance in their careers without leaving the classroom altogether. A five-stage career ladder provided high-performing teachers with opportunities for advancement inside the classroom as well as additional responsibility and compensation.

Another new pilot program called Teacher Leadership Innovation, or TLI – fueled by a $62 million Teacher Incentive Fund (TIF) grant – enabled participating teachers to assume “hybrid” roles in which they spent part of their time teaching and the rest in a variety of leadership roles in their school, such as observing and coaching peers, or leading teams or initiatives.29

Thompson and his team also continued to explore other innovative ways to keep effective teachers. For example, he said:

We started reaching out twice a year to different subsets of effective teachers – for example, highly effective teachers in our highest-poverty schools, as well as those in their second and third and fourth year, when a lot of teachers leave – to check in and say “we want to keep you” and to ask what they are thinking and see if there is anything we can do for them. It’s a tough line to walk, because we don’t want to be encouraging teachers to leave their school. At the same time, we want to identify people who want to stay in DCPS but don’t have a good fit with the school they’re in so that we can support them in transferring within the district. In the past, those people often left DCPS and moved to a charter, because if they went to an internal job fair, they might run into their principal. Last year, we talked with 25 teachers who said “Yes, this is something I want to do,” and about half ended up transferring to another school. The other half decided to stay where they were, and only one left DCPS. So that’s a success.

Developing Better Teachers In addition to these efforts aimed at improving teacher recruitment, selection, and retention were strategies aimed at developing teachers and improving their effectiveness. Instructional coaches began working with teachers in all DCPS schools, providing on-the-job feedback and training. Professional development resources were also significantly expanded; for example, a video library of effective teaching practices was made available to educators throughout the district.

At the same time, DCPS leaders decided to overhaul the district’s curriculum to increase its rigor and bring it into alignment with the Common Core standards. With extensive teacher involvement, DCPS developed and rolled out a curriculum “rich in science, history, and literature, beginning in kindergarten,” as well as six-week units of study that included recommended texts and activities.

28 Jonas Chartock and Chong-Hao Fu, “A Superintendent’s Perspective on Teacher Leadership,” Education Week, Straight Up (Blog), May 7, 2015. 29 The roles were defined by the teachers and principal in each school based on campus needs and goals. In 2013-14, 21 teacher leaders in seven schools participated in the program; by 2015-2016, 108 teacher leaders in 30 schools were participating. Leading Educators provided training and coaching for teacher leaders. DCPS Case Study 8

These new modules were designed to reduce school-by-school disparities in students’ access to high- quality learning opportunities.30 But because teachers were not required to use the materials, implementation varied widely. “If I go to one side of town,” Henderson observed, “I might see the same lesson being taught very differently than on another side of town.”31 As a result, many low-income students in DCPS continued to lack access to high-quality teaching and kept falling further and further behind their peers.

Accordingly, with educational equity in mind, district leaders decided to develop a set of “cornerstone assignments” – rigorous, in-depth lesson plans in various subject areas. Starting in the 2015-16 school year, every DCPS teacher would be required to use them. In this way, the district’s efforts at improving how teachers were teaching and how principals were leading proceeded alongside strategies aimed at improving what was being taught.

Refining IMPACT Over Time Because Kamras and his colleagues were committed to making DCPS’s evaluation system the best that it could be for improving instructional practice, they continued to refine IMPACT over time, even as they turned their attention to other aspects of their human capital theory of action.32 Some of the changes to IMPACT were initiated to ensure that it remained rigorous. Others were developed in response to feedback from teachers and principals on ways to improve the system and reduce the anxiety it caused, especially for the most effective educators.

In 2012, for example, based on evidence that the score range for the Effective level was too wide, DCPS leaders decided to add another level to the evaluation rating scale. As one leader recalled:

We knew anecdotally that a teacher at 250 was different from a teacher at 350, but then our work with researchers showed that it amounted to something like six months in reading and eight months in math, so it really did not make sense to lump them together. By adding a new Developing category and moving the lower boundary for the Effective level up to 300, we were able to break the distribution a bit and make it clear who needed additional support, which is important. Teachers appreciated the distinction, too.

Other mid-course modifications to IMPACT included lowering the weight of the value-added component for tested teachers, eliminating “outlier” scores,33 reducing the number of required observations for the most effective teachers, and adding informal observations for teachers who were new to the profession or new to the district so that they could receive additional “no-stakes” feedback.

Whenever DCPS’s human capital leaders considered changing IMPACT in some way, they thought carefully about reasons and ramifications. Said Hudacsko, “We don’t want people to feel like, ‘I just got this and now you’re changing it again.’ We have to balance the desire to keep improving the system against the risk of creating more anxiety and instability.”

Investing in School Leadership As the district’s teacher quality work continued to move forward, DCPS leaders were also making headway on another front: changing how school leaders were recruited, chosen, evaluated, compensated,

30 Natalie Wexler, “A New Plan to Narrow D.C.’s Achievement Gap,” Washington Post, June 12, 2015. 31 Michael Alison Chandler, “D.C. Schools to Introduce More Challenging ‘Cornerstone’ Lessons,” Washington Post, May 28, 2015. 32 This work was led by DCPS’s Chief for Teaching and Learning, Brian Pick. 33 Teachers expressed anxiety that if they were observed on the occasional “bad day,” it could inappropriately skew their rating; furthermore, the fear of receiving a low observation score discouraged them from trying new teaching approaches that might ultimately be very helpful for their students. DCPS leaders therefore decided to drop any observation scores that were substantially lower than the average score for a teacher’s other observations. The change affected a small number of teachers, but according to Thompson, “it was important because it showed that we were being responsive to teachers’ feedback on ways to make the system better.” DCPS Case Study 9 and developed. This was seen as pivotal due to their role in providing strong instructional leadership and also, Thompson said, because “the principal is often the top why teachers want to leave or stay.”

The district’s principal quality work was led by Hilary Darilek, who came to the district from New Leaders. When she had arrived at DCPS in mid-2009, there were 20 principal vacancies for the upcoming school year – and no candidates. “Everything was reactive,” Darilek recalled.34 After she and her team scrambled to fill the open positions, they began to develop a rigorous recruitment and selection process so that DCPS could be more strategic about school leadership and prevent a last-minute rush in the future. This was especially important because the attrition rate among district principals was often 20 percent or higher, so there were many positions to fill.

To identify and woo external principal candidates, Darilek’s team developed a multifaceted recruitment strategy that included marketing and advertising (via school administration association websites and a new recruitment website), outreach to existing school leaders, and partnerships with other organizations. Integral to the recruitment process was the creation of a detailed database to capture and track information about prospects. “It’s been a lot of work,” Darilek said, “but we finally reached the point where we have a robust database that allows you to answer any query.”

In addition to expanding and improving the candidate pool, Darilek’s team instituted a rigorous, competency-based selection process that included an online application, followed by an interview with a community panel, and culminating in appointment to a school by the chancellor. For the 2013-14 school year, 300 candidates applied for school leader positions, and 26 made it all the way through the process.35

Later, Darilek and her team also launched a new initiative that she and Henderson had envisioned years before: a rigorous 18-month program for DCPS educators who aspired to become school leaders. The first group of eight Fellows entered the Mary Jane Patterson Fellowship in early 2013, and after completing the program, they assumed new school leadership roles in mid-2014, augmenting the pool of leaders recruited from outside the district.36 The program, Darilek said, was “an investment in our own talent.”

Revamping Principal Evaluation and Compensation As it was with the district’s teacher quality work, the development of a new evaluation system was an integral part of DCPS’s principal quality work. As Darilek explained:

When I came in 2009, principals were evaluated using a checklist and the results were not captured or reviewed centrally, so nobody really knew – or looked at – how principals were performing. First we started collecting the forms centrally. Then we developed a Leadership Framework to clarify the system’s expectations. To develop the Framework, we did a bunch of research and talked to people from New Leaders, The New Teacher Project, and other places. Our core question was, what are the best practices in great schools? That became the foundation for a new evaluation system, in addition to becoming the core of the selection process.

The new evaluation system, School Leader IMPACT, merged supervisor observations tied to the new DCPS Leadership Framework, test scores, and other measures of student progress. Based on this combined information, school leaders would be rated as Highly Effective, Effective, Developing, or Ineffective, and the ratings would be tied to consequences. Those in the top category would be eligible for

34 Interview with Hilary Darilek, May 14, 2015. Unless otherwise indicated, all quotations from Darilek are from this interview. 35 District of Columbia Public Schools, Office of Human Capital, Principal Recruitment and Selection, 2013. 36 The fellowship was open to teachers, administrators, and instructional coaches with strong instructional expertise and other leadership qualities. Fellows worked with leadership coaches and professors from Georgetown University and served as “resident principals” in two schools, where they were mentored by high-caliber principals. DCPS Case Study 10 bonuses and extra leadership opportunities, while those with lower ratings would receive more support and would not be eligible for annual salary increases.37

In September 2013, DCPS released the first round of ratings from the School Leader IMPACT principal evaluations. Eleven percent of the principals were rated Highly Effective, 32 percent were rated Effective, 49 percent were rated Developing, and 8 percent were rated as Ineffective. Among the 14 principals receiving the highest ratings, nine worked in schools that were predominantly low-income. These leaders were eligible for $30,000 merit bonuses – twice the amount given to peers leading lower-needs campuses.

District officials touted the new evaluation system as “an important tool for recognizing and rewarding the best leaders and targeting areas for improvement,” but some principals complained that the system’s complexity made it hard to understand exactly how the ratings had been determined. Many were also unhappy that so many principals were rated below the Effective level. “If you have 50 percent in ‘Developing,’ you know something is wrong with the evaluation tool,” said Aona Jefferson, president of the principals union. Some principals called the new ratings “bad for morale.” 38

Kamras initially defended the evaluation system, explaining that being in the Developing category did not mean that a principal was ineffective – only that he or she had room to grow. But when the pushback persisted, he and Darilek convened a task force of principals, assistant principals, and instructional superintendents to provide input on ways to improve the system.

Based on this feedback, DCPS leaders unveiled a revised version of IMPACT for principals in January 2014. School leaders would continue to be assessed based on their school’s progress toward student achievement goals as well as supervisors’ observations. But DCPS agreed to hold school leaders harmless, and the plan to freeze the pay of principals rated below the Effective level was suspended. “This was an issue of particular concern to our principals,” Kamras said, “so we decided to press pause.”39

He stressed, however, that DCPS would continue to reward principals who earned top ratings. The performance-based bonuses that the district now offered were in addition to salaries that were significantly higher across the board than they had been a few years earlier as a result of a new salary schedule. Starting principals in DCPS now earned an average salary of $123,000 – dramatically more than the $106,000 they had earned in 2010. “This significant increase in compensation not only signaled principals’ value,” Darilek noted, “but was also essential to our efforts to attract and retain high-quality school leaders.”

Developing Better Principals With a robust principal evaluation system and better recruitment and selection processes in place, DCPS’s human capital leaders began intensifying their efforts to develop principals’ skills. They decided to double the number of instructional superintendents and revamp their responsibilities to provide better professional support for school leaders. As Darilek explained, “The goal was to move the superintendent role from a compliance-based position to one where the superintendent could observe and support principals and have a consistent and significant presence in schools.”40

In 2013, DCPS also began partnering with Georgetown University to offer an Executive Master’s in Leadership program for its principals. District leaders began to explore more innovative ways to boost principals’ effectiveness, too, including streamlining their roles. One strategy borrowed from the charter

37 The new ratings did not change principals’ job security, because DCPS principals had always been on one-year contracts, but did cause some to lose their positions. 38 Emma Brown, “More than Half of DC Public Principals Rated Just Below ‘Effective’ on Revised Evaluations,” Washington Post, September 29, 2013. 39 Emma Brown, “DC Schools Change IMPACT Evaluations for Principals,” Washington Post, January 21, 2014. 40 Lee Alvoid and Watt Lesley Black, Jr., The Changing Role of the Principal: How High-Achieving Districts Are Recalibrating School Leadership, Center for American Progress, July 2014. DCPS Case Study 11 sector proved to be both very popular and exceptionally productive. As Kamras explained, “We started a pilot in nine schools to try something that the best charter schools often do: they have a principal who is an instructional leader and a different individual who handles operations. This helps the principal really focus on the core academic leadership work that only he or she can do.”

The Strategic School Operations pilot41 had an immediate impact on how principals used their time. Before adding the new operations leader role, participating principals reported spending approximately half of their time, on average, dealing with facilities, security, custodians, and other operational issues. After, the figure dropped to about 20 percent. As one school leader said, “I didn’t have time to do any informal teacher observations last year, but this year I did 300 because I’m not doing boilers and buses.” As a result of the pilot’s popularity and success, almost 40 principals signed up the following year.

From Thompson’s perspective, these results were exceptional not only because of their impact but also because they had not required any extra funding. Moreover, he said, “we have a comparatively high number of operations staff working in our schools, and in the past, we haven’t been able to provide many growth or career path opportunities for them. So along with principals, teachers, and students, they benefit as well. It’s a win for everyone.”

Part III: Reflecting on Results and Next Steps

As the 2014-15 school year came to an end, Henderson, Kamras, and the deputy chiefs were taking stock of what had been accomplished thus far and contemplating what to do next.

A recent study conducted by researchers at the University of Virginia and Stanford had provided powerful evidence of the influence of the district’s teacher evaluation system. First, researchers found that IMPACT had resulted in substantial improvements in teacher practice for lower-performing as well as higher-performing teachers.42 Second, they found that DCPS was not only retaining more of its teachers overall (80 percent, compared to 71 percent before IMPACT); it was retaining its best teachers at very high rates (92 percent of Highly Effective teachers, versus 59 percent of Minimally Effective teachers). Last but not least, DCPS was replacing low-performing teachers who left the system with significantly more effective educators.43

These findings, along with the latest student achievement data, provided compelling evidence that DCPS’s teacher and principal effectiveness strategies over the past several years – along with the curriculum overhaul and other initiatives – were paying off.

In 2013, for example, DCPS students made larger gains in reading and math than their peers in any other state on the National Assessment of Educational Progress (NAEP). For example, 25 percent of the district’s fourth graders were proficient in reading, up from just 14 percent in 2007.44 District assessment results showed progress, too. In 2014, DCPS students attained their highest proficiency rates ever on the local Comprehensive Assessment System (DC CAS). Almost half of DCPS’s students were now proficient in reading, up from 34 percent in 2007; and 51 percent were proficient in math, up from 28

41 The name of the program was later changed to the School Strategy & Logistics (SSL) pilot. 42 For example, teachers rated Minimally Effective improved an average of 12.6 points on the 100-400 IMPACT scale in one year; this was more than half the amount that new teachers typically improve over their first three years. At the other end of the spectrum, Highly Effective teachers improved by an average of 10.9 points as a result of IMPACT. Additional unpublished research showed that minimally effective teachers on the cusp of reaching the next higher performance level were not only improving from a pedagogical standpoint, as measured through the Teaching and Learning framework; their students were also making significant achievement gains. 43 Thomas Dee and James Wyckoff, Incentives, Selection, and Teacher Performance: Evidence from IMPACT, Journal of Policy Analysis and Management, Vol. 34(2), 2015. 44 The Nation’s Report Card, 2013 Trial Urban District Snapshot Report, District of Columbia Public Schools: Reading, 2013. DCPS Case Study 12 percent. (See Attachment B.) The 2014 DC CAS results also showed gains at the high school level.45 As Hudacsko summed up, “the feedback that we give about pedagogy is translating into results.”

Yet despite the signs of progress, large achievement gaps persisted in the district. For example, only 41 percent of DCPS’s economically disadvantaged students were proficient or advanced in math and just 37 percent were proficient readers.46 Moreover, as noted in a long-awaited evaluation of the DCPS reforms conducted by the National Research Council and released in June 2015, graduation rates remained “disturbingly low.” This was especially true for minority and low-income students, special education students, and English language learners. In 2014, the graduation rate for economically disadvantaged students in DCPS was a mere 53 percent.47

Large disparities in student achievement also remained starkly evident across the city’s wards. From 2007 to 2014, the reading proficiency of students attending schools in Ward 3 (the most affluent area of the city) had risen by almost 9 percentage points, compared to just 3 to 4 percentage points for students in Wards 7 and 8, the most economically challenged communities. In math, the proficiency of students in Ward 3 had increased by 21 percentage points since 2007, in contrast to only 12 percent in Ward 8.48

As he reflected on DCPS’s improvement trajectory and the challenges that remained, Thompson sometimes thought about Jim Collins’ influential book Good to Great.49 He perceived that the district had made the leap from “poor to fair” during the years of Michelle Rhee’s leadership, then another step from “fair to good” from 2011 through the middle of 2015. “I think the key levers we pulled early on were improving all of the basic operational functions that weren’t necessarily working well before, and then focusing on the human capital levers,” Thompson said. “We’ve gotten a lot out of those.”

But he believed that moving from “good to great” would require a different set of levers. The key would be deciding which levers to pull and which order to pull them in. As Thompson continued:

We have an extraordinary number of talented, hard-working people in each part of central office, and everyone is trying to get a lot of good work done. People want to collaborate, and we have good conversations about how to be more aligned. But while we’ve made some progress in this regard, at the end of the day, all of these different efforts still play themselves out at the school level. At a leadership session last summer, one principal said she feels like each part of central office is launching missiles that all converge at the school level. Likewise, teachers consistently tell us two things: “I’m being bombarded with new expectations” and “I feel like I’m not being developed.” These things are juxtaposed. Every year, there are more expectations, layer upon layer, so the challenge is how to prioritize. There are many things we still need to do to improve student outcomes, but we can’t do them all at once.

By the summer of 2015, as DCPS’s human capital leaders thought about how to close the persistent achievement gaps and accelerate the district’s academic progress, one of the most urgent priorities on their minds was how to ensure that the district’s most challenged schools and students had more equitable access to high-quality teaching. 50 Recent data showed, for example, that only 19 percent of the teachers in the city’s most economically disadvantaged area (Ward 8) were rated as Highly Effective, compared with half of the teachers in the wealthiest area (Ward 3).51 (See Attachment E.)

45 District of Columbia Public Schools, DC CAS 2014 Results, July 2014. 46 Ibid. 47 National Research Council, An Evaluation of the Public Schools of the District of Columbia: Reform in a Changing Landscape, June 2015 (Advance Copy). 48 Office of the State Superintendent of Education, 2014 District of Columbia Comprehensive Assessment System Results (DC CAS), July 31, 2014. 49 Jim Collins, Good to Great: Why Some Companies Make the Leap…And Others Don’t, 2001. 50 Equity concerns were fueled in mid-2014 when the Obama administration issued a mandate requiring all states (and districts) to submit plans for getting better teachers into high-poverty schools and classrooms. 51 Michael Alison Chandler, “IMPACT Scores Ward-by-Ward in DC,” Washington Post, February 24, 2015. DCPS Case Study 13

Because district leaders believed strongly in mutual consent, forcing highly effective teachers to transfer to high-needs schools was not viewed as a desirable option. While they had hoped the performance-based bonuses and salary increases in IMPACTplus would motivate many more outstanding teachers to voluntarily move to more challenged schools, it was not yet clear whether this had occurred. For one thing, Hudacsko pointed out, “Teachers get committed to their communities and often don’t want to leave.” There was evidence, however, that IMPACTplus was influencing where people chose to stay – and as Kamras observed, “that’s just as important, if not more so.”

With these complexities on his mind, Kamras decided to use one of the human capital team’s meetings in the spring of 2015 to take “a big step-back” on teacher quality and ponder some fundamental questions:

Where are we now? Are we headed in the right direction? What does the teacher quality distribution look like across the system? What should we be doing about that? It was an opportunity to surface some things that we had been talking about for a couple of years.

In short, there are two big stories. One is that teacher quality is up. It’s one of the greatest success stories in public education: over the past five years, we’ve doubled the percentage of highly effective teachers, from 16 to 32 percent. With that kind of increase, you’d expect significant gains in student achievement, and that is what we’ve seen. According to the gold standard of national assessments, the NAEP, we’re the fastest improving school system in America.

But the second story is that the distribution of teacher quality is still not equal. A student in a high poverty school is still 2.5 times less likely to have a highly effective teacher than a student in a low poverty school. That’s unacceptable. So the core question we now face is, how do we close that gap? I don’t believe it’s a matter of giving great teachers more money to move schools. It’s a far more complex problem than that. I think it ultimately comes down to the very hard and very complicated work of helping “good” teachers become “great” – particularly in our highest-poverty schools.

To that end, Chancellor Henderson announced in July 2015 that she was preparing to reorganize central office. One of the changes would be to place Kamras in charge of a new office tasked with developing the district’s teachers and principals. Instructional coaching, teacher development, the principal residency, and related work would move into this new office. A team of instructional superintendents would also be redirected to focus more intensively on the district’s lowest performing schools.

As Henderson explained in her announcement of the impending changes:

We have done amazing things in the past five years. We have graduated more students than ever before. We have increased student satisfaction to all-time highs. We have improved outcomes for students across the district. I am proud of our accomplishments, but I am not satisfied. . . . I am still frustrated by the inequities that I see in our district. I know that we can do much more for all of our students.

“If we are going to dramatically improve outcomes,” Henderson continued, we can’t keep doing the same work. We must recognize and build on our successes while adjusting our organization to address new challenges. I have total confidence that our team can achieve things that no-one thinks are possible.”52

52 Chancellor Kaya Henderson, internal email regarding upcoming reorganization plans, July 17, 2015. DCPS Case Study 14

Attachment A: DCPS District Snapshot, 2014-15

Number of Schools: 111

Budget: $701,344,360

Cost Per Pupil: $14,750

Enrollment: 47,548

Enrollment Trend:

Source: Friends of Choice in Urban Schools (FOCUS), 2015

Market Share: 56% of DC public school students attended traditional public schools (DCPS) and 44% attended charter schools

Student Demographics/Characteristics: 69% African American, 16% Hispanic, 11% White, 4% Other 77% Eligible for Free or Reduced-Price Lunch 10% English Language Learners 16% Special Education

Staff: Approximately 11,000 (4,000 teachers and 7,000 total school-based staff)

Teacher Demographics: 46% African American, 5% Hispanic, 30% White, 3% Asian (15% had no race reported)

Teacher Sources (i.e., for new teachers): 3% from , 8% from DC Teaching Fellows program (affiliated with TNTP), 89% from other pathways

DCPS Case Study 15

Attachment B: Academic Proficiency Data

DC CAS Trend Results

Changes in Percentage of Students Scoring Proficient or Advanced on DC CAS, by Subgroup, 2007-2014 2007 2014 Change (in percentage points) Reading All Students 34% 48% +14 White 87% 92% +5 African American 29% 39% +10 Hispanic 40% 49% +10 Economically Disadvantaged 28% 37% +9 English Language Learners 35% 37% +2

Math All Students 28% 51% +23 White 82% 92% +10 African American 23% 41% +18 Hispanic 36% 58% +22 Economically Disadvantaged 22% 41% +19 English Language Learners 35% 50% +15 Source: Friends of Choice in Urban Schools (FOCUS) data tool

Percentages of Students Scoring Proficient or Advanced on DC CAS, by Ward, 2014 Reading Math Reading All Wards 48% 51% Ward 1 43% 51% Ward 2 69% 72% Ward 3 83% 84% Ward 4 51% 58% Ward 5 42% 41% Ward 6 48% 49% Ward 7 31% 40% Ward 8 24% 26% Source: Friends of Choice in Urban Schools (FOCUS) data tool

DCPS Case Study 16

Attachment C: Bios for Selected DCPS Leaders

Jason Kamras • Chief of the Office of Human Capital, DCPS – 2010 to present • Director, Human Capital Strategy for Teachers, DCPS – 2007 to 2010 • National Teacher of the Year – 2005 • Teacher, Middle School, DCPS – 1996 to 2005 • Teach For America,1996 DC Corps • Education: o Master’s degree, Harvard Graduate School of Education, 2000 o Bachelor of arts, public policy, Princeton University, 1995

Scott Thompson • Deputy Chief of Teacher Effectiveness Strategy, DCPS – 2012 to present • Director, Teacher Effectiveness Strategy, DCPS – 2010 to 2012 • Manager, IMPACT Design, DCPS – 2010 • Assistant Director, Master Educator Program, DCPS – 2009 to 2010 • Teacher, Middle School and High School, NYC (Teach For America) – 2006 to 2008 • Education: o Master of Science in Social Policy, Oxford University, 2009 o Master of Science in Teaching, Secondary Social Studies, Pace University, 2008 o Rhodes Scholar, 2008 o Bachelor of Arts, Psychology and Political Science, Stanford University, 2006

Michelle Hudacsko • Deputy Chief of IMPACT, DCPS – 2011 to present • Partner, The New Teacher Project – 2009 to 2011 • Site Manager, The New Teacher Project – 2007 to 2009 • Manager and Program Specialist, Reading Is Fundamental – 2001 to 2007 • Teacher, Elementary and Middle Schools (Teach For America and other), multiple districts – 1996 to 2000 • Education: o Master’s degree, curriculum and instruction, 2005 o Bachelor of arts, government and politics, University of Maryland at College Park, 1996

Hilary Darilek • Deputy Chief of Principal Effectiveness, DCPS - 2009 to present • Managing Director, New Leaders for New Schools DC Program – 2005 to 2009 • Associate Quantitative Analyst, RAND Corporation – 2002 to 2005 • Teacher, City Public Schools – 1999 to 2001 • Education: o Master of Science in Organizational Leadership, Georgetown University, 2013 o Master of Science in Operations Research, London School of Economics, 2002 o Master of Arts in Secondary Education and Teaching, Johns Hopkins University, 2001 o Bachelor of Arts in Economics and Mathematics, College of William and Mary, 1999

DCPS Case Study 17

Attachment D: DCPS Teaching and Learning Framework

DCPS Case Study 18

Attachment D (continued): Overview of IMPACT

Example of IMPACT Evaluation System, 2014-15: General Education Teachers (Group 1)

Source: DCPS, IMPACT 2014-15: The District of Columbia Public Schools Effectiveness Assessment System for School-Based Personnel, Group1--General Education Teachers with Individual Value-Added Student Achievement Data.

DCPS Case Study 19

Attachment D (continued): The LIFT (Career Ladder) Program, Excerpts from Guidebook, 2014-15

Source: DCPS, LIFT Guidebook, 2014-15

DCPS Case Study 20

Attachment D (continued)

Source: DCPS, LIFT Guidebook, 2014-15

DCPS Case Study 21

Attachment E: IMPACT Ratings by City Ward, 2013-14

100%

18% 19% 26% 32% 31% 33% 80% 49% 50%

60% 48% 50% 47% 42% 47% 42% 40% 35% 41%

20% 25% 22% 17% 20% 18% 17% 12% 6% 8% 5% 5% 7% 6% 4% 4% 0% 3% 3% Ward 1 Ward 2 Ward 3 Ward 4 Ward 5 Ward 6 Ward 7 Ward 8

Ineffective Minimally Effective Developing Effective Highly Effective

Additional Statistics, by Ward Ward 1 Ward 2 Ward 3 Ward 4 Ward 5 Ward 6 Ward 7 Ward 8 Poverty Rate 22% 19% 8% 12% 20% 21% 25% 36% Number of 405 183 460 510 352 500 395 514 Teachers % of teachers rated Highly 74% 84% 91% 78% 73% 75% 66% 69% Effective or Effective % of teachers rated Ineffective 9% 4% 1% 5% 7% 7% 8% 9% or Minimally Effective

Source: District data, as reported in Michael Alison Chandler, “IMPACT Scores Ward-by-Ward in DC,” Washington Post, February 24, 2015.

DCPS Case Study 22