Rethinking K–12 Education Procurement

RETHINKING K–12 EDUCATION PROCUREMENT

Why Promising Programs, Practices, and Products Seem to Rarely Get Adopted, Implemented, or Used

Edited by Frederick M. Hess

AMERICAN ENTERPRISE INSTITUTE RETHINKING K–12 EDUCATION PROCUREMENT Why Promising Programs, Practices, and Products Seem to Rarely Get Adopted, Implemented, or Used

Essays by Thomas Arnett • Lauren Dachille • David DeSchryver • Mike Goldstein Melissa Junge • Eric Kalenze • Julia Kaufman • Rebecca Kockler Sheara Krvaric • Joel Rose

Edited by Frederick M. Hess

The American Enterprise Institute (AEI) is a nonpartisan, nonprofit, 501(c)(3) educational organization and does not take institutional posi- tions on any issues. The views expressed here are those of the author(s).

American Enterprise Institute 1789 Massachusetts Avenue, NW Washington, DC 20036 www.aei.org Contents

Foreword...... 1 Frederick M. Hess Why Aren’t Teachers Using the Resources Companies Sell to Their Districts?...... 5 Thomas Arnett If Education Procurement Is Broken, Is Teacher Choice the Answer?...... 20 Mike Goldstein What It Will Take to Improve Evidence-Informed Decision-Making in Schools...... 32 Eric Kalenze How Confusion over Federal Rules Can Get in the Way of Smart School Spending...... 50 Melissa Junge and Sheara Krvaric Helping Education Entrepreneurs and School Leaders Navigate the Procurement Maze...... 68 David DeSchryver Lessons from Louisiana’s Efforts to Create a New Marketplace for High-Quality K–12 Curricula and Professional Development...... 81 Julia Kaufman and Rebecca Kockler Overcoming the Challenges Facing Innovative Learning Models in K–12 Education: Lessons from Teach to One...... 97 Joel Rose Procurement Practices That Inhibit Innovation: A Case Study from the Edtech Startup Perspective...... 113 Lauren Dachille

iii iv RETHINKING K–12 EDUCATION PROCUREMENT

Conclusion...... 124 Frederick M. Hess About the Authors...... 126 Foreword

–12 schooling is awash with new learning technologies, professional K development, reading programs, and any number of things that schools and school systems have to decide whether to buy. Some programs, practices, and products deliver on their promise. But more typically they’re ignored, misused, or abandoned shortly after adoption. For instance, while school districts annually spend more than $8 billion on education technology, tech firms estimate that about two-thirds of all educational software licenses go unused each year.1 Meanwhile, as I’ve observed time and again over many years of working with teachers, educators end up eschewing costly materials their districts purchase in favor of using home-brewed resources or free online materials. Even as district leaders routinely call for more funding, untold billions of K–12 dollars are wasted each year through problematic procurement processes. What explains the disconnect between what gets purchased and what ultimately gets adopted, implemented, and used? Why do districts spend money on the same ineffective programs year after year, despite poor results? How should districts evaluate whether a new learning model or instructional resource will live up to marketing claims? These are the questions the contributors tackle in the pages that follow. The goal is to provide an accessible exploration of these questions while offering practical guidance on what might be done. Along the way, the contributors draw from an array of experiences and backgrounds, offering insights from the classroom, schoolhouse, statehouse, and boardroom. The chapters broadly fall into three categories. Chapters 1–3 look at education procurement from inside schools and classrooms. Chapters 4 and 5 examine how policy and bureaucracy inhibit smart procurement practices. Chapters 6–8 delve into specific case studies to identify lessons learned from the practical work of improving education procurement. In Chapter 1, Thomas Arnett, a one-time middle school math teacher and now a senior research fellow at the Clayton Christensen Institute,

1 2 RETHINKING K–12 EDUCATION PROCUREMENT tackles the disconnect between districts’ purchases and teachers’ use of instructional resources through the lens of the “Jobs to Be Done” business theory. Arnett concludes that the reason purchased resources are often left to collect dust is that teachers decide the materials won’t help them accomplish essential classroom tasks—“jobs” that need to be done. In Chapter 2, Mike Goldstein, Match Education founder and Bridge International Academies chief academic officer, argues that one way to align resources with what teachers need is to give teachers themselves control of the funds districts currently spend on them. Such a move, he con- tends, would help ensure that teachers receive products and services they actually want and need, significantly increasing the likelihood of effective implementation and maximizing return on investment. In Chapter 3, Eric Kalenze, director of researchEd US, looks at what it will take to improve evidence-informed decision-making in schools. He offers three suggestions: Construct (or upgrade) bridges between education researchers and practitioners, enhance school-improvement planning with various evidence-review processes, and create designated “learning engineers” to direct districts’ evidence-based improvement work. Of course, not all of procurement’s ills can be remedied at the school or district level. Bureaucracy and outdated policies at the state and federal levels often leave principals and district administrators feeling like their hands are tied with the programs and materials they’re allowed to purchase. In Chapter 4, veteran education attorneys Melissa Junge and Sheara Krvaric point out that, regarding resources purchased with federal funds, the rules designed to ensure that funds are used as intended have evolved into a maze of paperwork and confusing legalese. The result is that funds are not necessarily spent on the materials that administrators believe will help teachers and students the most, but on those that are considered legally “safe”; new, potentially more effective programs are often shunned by risk-averse administrators in favor of the familiar. In Chapter 5, Whiteboard Advisors’ David DeSchryver opines that one of the keys to overcoming the confusion often inherent in federal and state regulations is getting policymakers, vendors, and district administrators on the same page regarding the meaning and intent of statutes and guidance. Often, district administrators’ perceptions of what counts as a FREDERICK M. HESS 3

“permissible” use of funds are narrower than what’s actually allowed, due to overly conservative interpretations of key statutory terms or definitions. What does the practical work of overcoming education procurement challenges look like? In Chapter 6, Julia Kaufman, a senior policy researcher at the RAND Corporation, and Rebecca Kockler, a former assistant superintendent at the Louisiana Department of Education, look to Louisiana for a case study on how states can assist with the procurement of high-quality materials and training programs that are actually aligned with state standards (irrespective of marketing claims about “standard alignment”). Kaufman and Kockler argue that states can follow Louisiana’s lead by identifying, incentivizing the procurement of, and monitoring the quality of standard- aligned materials. In Chapter 7, New Classrooms Founder and CEO Joel Rose reflects on his experience working to find partner schools for New Classroom’s “Teach to One” learning model. Rose posits that K–12 innovation has a fundamental paradox: Meaningful improvements to student outcomes require learning models that challenge the basic constructs of how classrooms operate—yet, the more new education models do so, the harder they become for schools to adopt. Inflexible accountability policies, bureaucratic inertia, and balky procurement processes make administrators reluctant to purchase programs that don’t map neatly onto century-old grade-based models of instruction. Finally, in Chapter 8, Lauren Dachille, founder and CEO of the education technology company Nimble, observes that “incentivizing innovation is not just about changing districts’ perspectives on products and companies” but also “about changing procurement processes in a way that allows district leadership to be more open to innovation while still adequately de-risking the process.” She suggests that districts should be open to working closely with vendors to create open-ended solutions and advises that administrators should dispense with the notion that early-stage companies are necessarily “riskier” than established vendors are. In the latter days of a devastating pandemic, as districts face tough budget decisions and explore opportunities to reinvent the way they serve students, smarter procurement may be more essential than ever. Districts cannot afford to continue purchasing resources that never make it off the 4 RETHINKING K–12 EDUCATION PROCUREMENT classroom bookshelf or curricula that teachers ignore. Education procurement is a broad and complex topic. While this volume covers only some of the myriad issues at work, my hope is that the insights and guidance offered by its contributors will be taken up by a few determined administrators seeking every opportunity to do better by the students they’re privileged to serve. —Frederick M. Hess

Notes

1. Michelle R. Davis, “K–12 Districts Wasting Millions by Not Using Purchased Soft- ware, New Analysis Finds,” Education Week, May 14, 2019, https://marketbrief.edweek. org/marketplace-k-12/unused-educational-software-major-source-wasted-k-12- spending-new-analysis-finds/. Why Aren’t Teachers Using the Resources Companies Sell to Their Districts?

THOMAS ARNETT

nstructional resources matter to a school’s success. As many teachers I will tell you, good resources—such as textbooks, reference materials, computers, and software—are crucial to delivering effective instruction. Teaching without good resources is like trying to build a house without power tools or scaling a mountain without good hiking boots. It isn’t impossible. But having the right resources makes a huge difference. During my time as a middle school math teacher, I experienced firsthand the stress and frustration of resource depravation. I spent hours of my mornings, afternoons, and weekends scouring Google, educational websites, and various textbooks and workbooks to mash together instructional materials for my class. I had no expertise as a curriculum and assessment developer, and the hodgepodge I created was a far cry from a cohesive, well-scaffolded, standards-aligned curriculum. But it was the best I could do under the circumstances. The options from my district just didn’t meet my needs or those of my students. It would be nice if my experience could be chalked up to an anomaly. But unfortunately, how districts procure resources often leaves teachers disconnected from what gets purchased, what’s needed, and what’s used. My experiences and realization that the problem was systemic led me to graduate school and then to my current work as a senior research fellow at an innovation-focused think tank called the Clayton Christensen Insti- tute. My goal in changing my career was to find system-level solutions to the challenges I faced as a teacher. Yet sadly, as I’ve scanned the field from this vantage point for the past six years, these problems still loom large and continue to play out in schools and districts across the country. To illustrate, consider a few examples.

5 6 RETHINKING K–12 EDUCATION PROCUREMENT

Example 1. A state adopts new college- and career-ready standards and undertakes a major effort to help its schools with the transition. As part of that effort, the US Department of Education and partner foundations invest millions of dollars to help districts pay for new curriculum and professional development. Yet three years into the effort, an independent report finds that not much has changed in local schools. Many teachers continue using phased-out textbooks, and their teaching strategies reflect old pedagogical paradigms that don’t line up with the new standards.

Example 2. A district’s leadership team takes on a new initiative to improve test scores. As part of the initiative, the district invests in supplemental resources for supporting English language learners. To ensure that they choose the best resources for their needs, district leaders convene a committee of teachers and site-based curriculum coaches to vet and then pilot the most promising proposals. At the end of the vetting period, the committee members identify a clear winner and seem excited to roll it out. But a year into adoption, district leaders notice that teachers are turning to Google, Pinterest, and Teachers Pay Teachers for supplemental resources, while the paid-for resources sit idle in classroom closets.

Example 3. An educational technology company wants to ensure its products meet teachers’ needs. To that end, it hires a few former teachers to be part of its product development and sales teams. It also makes extensive efforts to solicit feedback from teachers and adjust its products according to their input. Despite these efforts, the company’s software is used by just 15 percent of the students who have access paid for by their districts. Problems like these are not new to education, yet they persist. They frustrate taxpayers who see millions of dollars wasted on unused resources. They frustrate state officials, district leaders, and foundations that invest in these resources only to see lackluster return on their investment. And they frustrate teachers who feel swamped with resources they don’t want and deprived of those they need. At the Clayton Christensen Institute, our mission is to use innovation theories to shed new light on persistent challenges such as these. Recently, we completed a project using a theory and methodology called “Jobs to Be Done” to find out what causes teachers to change how they teach. From this THOMAS ARNETT 7 project, we now understand common causes of the disconnects between resource purchases and resource use. In short, teachers often turn down the resources their districts provide because developers and districts often fail to really understand teachers’ circumstances and struggles.

What Is the Jobs to Be Done Theory?

Before I get carried away with our findings and their practical implications, it’s important to explain the underlying theoretical basis of our research— the Jobs to Be Done theory.1 Jobs to Be Done is a theory of customer demand. It starts with a simple premise: All people—teachers included—are internally motivated to make changes in their lives that help them progress in their particular life circumstances. When you understand people’s life circumstances and the progress they are trying to make, you understand the criteria that drive their decisions. To find Jobs to Be Done, we start by identifying people who have made recent changes in their lives—for example, teachers who recently switched to new lesson resources. We then interview those people to reveal the causal narratives underlying their decisions. The interview questions are not scripted. Rather, we start by establishing rapport and then asking people to help us shoot the documentary about their adoption experience. As the interviews proceed, we don’t ask people why they like what they chose or why they chose one over another. The rational explanations rarely reveal the true reasons behind their choices. Instead, we aim to capture the events and circumstances that caused their decisions: from the first fleet- ing thought that the status quo wasn’t cutting it; to the twists and turns of struggling with their old options, learning about new alternatives, and con- fronting the inevitable anxiety that comes with the uncertainty of change; and finally to when they decide to invest in something new. Once we complete a set of interviews, we code the events in people’s stories and then use cluster analysis to look for similarities across the interviews. As the clusters emerge, they reveal common sets of circumstances people struggle through that shape the decision criteria for their choices. Each cluster of stories connected by similar circumstances constitutes 8 RETHINKING K–12 EDUCATION PROCUREMENT what we call a “job.” The theory labels these clusters of circumstances as “jobs” because just as people hire contractors to help them build houses or lawyers to help them build a case, people search for something they can hire to help them when certain circumstances arise in their lives. It’s important to clarify what Jobs to Be Done is not. The term “jobs” does not refer to the roles people occupy in their professions, such as teacher, principal, or school counselor. Additionally, Jobs to Be Done does not represent a person’s professional responsibilities. A teacher may find his or her day filled with planning lessons, grading quizzes, taking atten- dance, or attending staff meetings, but these are responsibilities, not Jobs to Be Done. Lastly, the Jobs to Be Done theory explains the choices people actually make, not the choices they should make. For example, all people should exercise regularly and make healthy eating choices, but manifest behaviors reveal that for many people, “help me live a healthy lifestyle” is not a Job to Be Done. Understanding people’s Jobs to Be Done helps us understand their choices. For example, Harvard professor Clayton Christensen, the founder of the institute, conducted one of the first Jobs to Be Done studies to discover why people purchase milkshakes in the morning. Many morning milk- shake customers need something that mitigates hunger, stress, and boredom while driving their car to work. For these people, milkshakes don’t just satisfy a sweet tooth. They compete against coffee, bananas, donuts, granola bars, and talk radio to address a host of desires and circumstances that arise during a daily commute. In other words, it’s circumstances that define a Job to Be Done (being hungry and bored during a commute), not the attributes of the customer (age and gender) or the solution (sweet and thick). Jobs to Be Done also helps us see the key features of a product or service that will motivate people to bring it into their lives. As another example, Bob Moesta, the leading expert on Jobs to Be Done research and a key col- laborator with Clayton Christensen in developing the theory, used Jobs to Be Done to help a Detroit-area home builder market townhomes to empty nesters. The builder had a problem: It offered affordable homes with a host of customizable amenities that attracted many interested customers, but few potential customers signed purchase agreements. Using a Jobs to Be Done approach, our colleague discovered that motivation to close the contract on a new home happened only once people THOMAS ARNETT 9 navigated the emotional process of sorting through all their memory-laden possessions that they couldn’t take with them. The key to selling the new townhomes was less about improving their amenities and more about giving people free storage space where they could take time to sort through their belongings. It’s people’s life circumstances (such as having an old house full of nostalgic knickknacks) and their struggles for progress in those circumstances (downsizing while still keeping important memories) that define their Jobs to Be Done.

What Jobs to Be Done Motivate Teachers?

Teachers also face Jobs to Be Done in their classrooms. These Jobs to Be Done motivate teachers to adopt new resources and change their practices. To discover what some of these Jobs to Be Done might be, I worked with a research team in 2018 to conduct and analyze a set of Jobs to Be Done interviews with public school teachers from across the US about their experiences leading up to the adoption of a new instructional practice.2 The patterns that surfaced in these interviews revealed a number of Jobs to Be Done that describe the desires and circumstances often shap- ing teachers’ motivation. Two of those Jobs to Be Done are particularly insightful for explaining why teachers chose to adopt—or not adopt—the resources their districts purchase.

Job A: Helping Find Manageable Ways to Engage and Challenge Stu- dents. For many of the teachers we interviewed, there is a Job to Be Done that naturally motivates them to adopt new resources and practices. We describe this Job to Be Done as “help me find manageable ways to engage and challenge more of my students.” However, just because this Job to Be Done motivates adoption doesn’t mean adoption always happens. Here’s why. Teachers with this Job to Be Done generally felt comfortable and con- fident in their abilities as professional educators, but they also knew that in any given school year they would struggle to reach a few students and a few lessons would flop. As they strove to meet their students’ needs, these teachers believed that, by broadening their repertoire of teaching strategies 10 RETHINKING K–12 EDUCATION PROCUREMENT and activities, they could better engage more of their students in activities that promote learning. They were continually searching for how to adapt or enhance the curriculum so that more students would feel challenged. Importantly, however, the teachers we interviewed who experience this Job to Be Done were not interested in new resources or strategies that required them to dramatically change how they taught. If a 20-minute tutorial gave them what they needed to incorporate an attractive new resource or practice into the next day’s lesson, they would give it a shot. But if adoption required 12 hours of training and a complete overhaul of their unit plans, they weren’t interested. This lack of appetite for major change was a rational decision. They had spent years developing their own toolbox of strategies and resources that seemed to work well for them. If a new resource or strategy offered a practical addition to their current repertoire, they bought in. But if it required an extensive overhaul to their tried-and-true methods or went against their intuition about what is best for students, they weren’t interested. Additionally, the Job to Be Done for these teachers was about not only helping their students but also making teaching more enjoyable and taking pride in their own professional expertise. They were willing to spend time and effort creating or finding new activities and resources because their classrooms were more enjoyable to manage when students were excited to be there. But if the time and effort to adopt something new took them too far outside their normal planning and preparation routines, it wasn’t a practical proposition.

Job B: Helping Teachers Not Fall Behind on a School’s New Initiative. When teachers seem to drag their heels on adopting something new, it’s tempting for school leaders or product developers to just write them off as curmudgeonly laggards who are stuck in their ways. Yet in truth, many of these teachers will be fast adopters of a new resource or practice if it fulfills a relevant, circumstance-based Job to Be Done. The problem of dragging heels happens when these teachers don’t see the new resource as a solution to a Job to Be Done. Instead, the new thing seems like an added layer of unnecessary complexity on top of already demanding work. When this happens, pressure from school leaders creates a new Job to Be Done we call “help me not fall behind on my school’s new initiative.” THOMAS ARNETT 11

For teachers motivated by Job B, adoption happens because non- adoption carries consequences that aren’t worth the cost. For example, one kindergarten teacher we interviewed knew that if she didn’t start using laptops with her class, she would be blamed when her students reached first grade and struggled to use the technology. Other teachers voiced wor- ries that not adopting would mean falling behind their colleagues. Some teachers explained that they didn’t want to run the risk of getting trans- ferred to a new site for not being up with the new program. Others did not want to receive a negative teaching evaluation.

Debunking Common Myths About Adoption

Now that I’ve laid the groundwork of uncovering the Jobs to Be Done that motivate teachers, how can this research help improve the likelihood that the resources a district purchases are actually used? To answer that question, we first need to clear away some of the misconceptions that frequently lead to failed adoption strategies. As I illustrate below, districts and resource developers often fall flat in their efforts to encourage adoption because they don’t see the whole picture of struggles and circumstances that drive adoption.

Mandates Don’t Motivate Use. Job A and Job B described above can both produce adoption in the short term but lead to different outcomes in the long run. In essence, these two Jobs to Be Done lead to different “hires.” When analyzing adoption through the lens of the Jobs to Be Done theory, we consider two different types of adoption: “big hires” and “little hires.” Big hires are the moments when a person formally commits to a new solution—such as when you pay for a new article of clothing at a store’s checkout aisle. Big hires are easy to track and therefore often get used to measure adoption. In contrast, little hires are the moments when people use the solution they’ve committed to—such as when you take the tags off that new piece of clothing and repeatedly decide to wear it in public. If a solution truly addresses a Job to Be Done, many little hires will follow the big hire. 12 RETHINKING K–12 EDUCATION PROCUREMENT

The important difference between the two teacher Jobs to Be Done described above is that they lead to different hires. Both Jobs to Be Done can get teachers to make the big hire. But Job A (“help me find manageable ways to engage and challenge my students”) leads to many little hires, whereas Job B (“help me not fall behind on my school’s new initiative”) does not. When teachers find that a new resource truly provides them with manageable ways to engage and challenge their students (Job A), they will use that resource repeatedly. They will consistently turn to reliable solutions to this Job to Be Done. On the other hand, when they don’t see a resource as a good solution for Job A but feel pressure not to fall behind in their school’s initiative to use that resource (Job B), few little hires will follow. Their use of the resource happens only as long as the pressure for compliance remains. As soon as school leaders get busy and stop checking for compliance, little hires fade away. Leaders should avoid using mandates to force resource adoption. Man- dates are likely to push teachers into the circumstances of Job B (“help me not fall behind”) and thus lead to unmotivated adoption merely for adoption’s sake. Instead, allow teachers to opt in when possible and then pay close attention to what they choose to adopt as a signal of the design criteria that will meet their Jobs to Be Done. When opt in isn’t an option, be extra careful to design, select, and implement resources in a way that will align with teachers’ Jobs to Be Done.

Promoting Product Benefits Fails to Address Teachers’ Full Jobs to Be Done. Another reason for the common disconnect between district purchases and classroom use is the shortsighted belief that effective promotion drives adoption: Automakers run ads that paint a life of luxury or adventure from the driver’s seats of their cars; educational technology companies make upbeat YouTube videos showcasing how much students and teachers love their products. Yet these efforts fall flat if they fail to account for the other circumstances that define Jobs to Be Done. In reality, four types of circumstances shape the motivation that comes from Jobs to Be Done, as illustrated in Figure 1. Promotion, whether from a vendor or through positive word of mouth, generates the pull of a new idea. Pull represents the magnetism and allure of a new idea that leads people to envision how it can improve their lives. THOMAS ARNETT 13

Figure 1. Jobs to Be Done Forces of Progress

Source: Author’s design based on concepts from Clayton M. Christensen and Bob Moesta.

But pull is only one piece of the puzzle. The other force that moves people toward adoption is the push of the sit- uation. Push represents the moments of struggle that cause them to want to change—such as when a teacher notices that engagement, achievement, classroom management, or postgraduation success isn’t going as well as expected. Push and pull describe why a person desires change, but two circumstances opposing change are just as important for understanding the full picture of why change happens with varying degrees of success. First, habits of the present keep people invested in the status quo. “I’ve used these lesson plans for years” or “I don’t love this textbook, but at least I know what’s in it” are classic habits of the present. Second, the anxiety of the new solution deters people from adopting a new solution. Thoughts such as “Will technical glitches make me lose the attention of my class?” or “What if my students’ parents don’t like this?” reveal some of the real anxieties that hold teachers back from adopting something new. Together, these four categories of circumstances help us see the big picture regarding what drives adoption. They reveal the elements of context beyond the appeal of a new solution that determine whether someone will bring that solution into his or her life. For teachers to adopt something 14 RETHINKING K–12 EDUCATION PROCUREMENT new, the circumstances that push and pull them toward change must be strong enough to overcome the habits and anxieties that hold them back. When trying to encourage adoption, leaders should consider how it might influence all four categories of circumstances: pushes, pulls, anxieties, and habits. Sometimes, to effectively encourage adoption, we must eliminate habits and anxieties that hold teachers back. For example, key strategies for alleviating teachers’ anxieties include providing job-embedded professional development and giving teachers leeway to fail forward as they figure out how to make new resources work for them. As another example, leaders can overcome habits by being opportunistic when changes in teaching assignment occur. When teachers move to a new school, grade level, or course, these changes make them more likely to try new resources or practices because the resources and practices that worked in their prior teaching assignment are no longer relevant.

Getting Teacher Input Isn’t the Same as Uncovering a Job to Be Done. Another common pitfall is thinking that soliciting teachers’ input—while either designing resources or going through a resource adoption process— will ensure alignment with their Jobs to Be Done. Countless experts and books suggest we should listen to what people say and then build accordingly. Yet the counterintuitive reality is that, without meaning to, people frequently misrepresent what they want because they don’t actually know what they want. Consider past innovations in the textbook industry.3 During market research interviews, students and their teachers expressed enthusiasm for books that included online links to websites where they could learn more about topics covered at only a cursory level in the books. In response, textbook companies spent several billion dollars creating websites where students could explore topics more deeply. As it turns out, however, few students or teachers ever click on those links. Most students are really just trying to get through the course material, as evidenced by what they do, rather than what they say. When people are asked for feedback, they often feel a need to give answers they can justify. As a result, they give rational or socially acceptable responses rather than real insights into their actual Jobs to Be Done. Additionally, it’s hard to notice the factors defining your Jobs to Be Done THOMAS ARNETT 15 when you are not in the setting that causes you to experience those Jobs to Be Done. When trying to identify the resources that will fulfill teachers’ Jobs to Be Done, don’t ask teachers what they want. Instead, ask them to tell you the story of the circumstances that shaped their last adoption decision. You can uncover those stories with questions such as “When did you realize that there was a problem you needed to solve?” and “What concerns did you have to work out before you were ready to make a decision and move forward?”

“Evidence Based” and “Standards Aligned” Are Not Jobs to Be Done. One of the most frustrating procurement conundrums happens when high-quality curricula struggle to get uptake while poor-quality curricula get a pass. Foundations and policymakers have invested heavily in efforts to ensure that schools use standards-aligned and evidence-based resources. But the mantra “if you build it, they will come” doesn’t seem to hold. The reason is that neither evidence-based nor standards-aligned describe teacher Jobs to Be Done. It’s not that teachers don’t care about evidence or standards. It’s just that the day-to-day circumstances that define their Jobs to Be Done lead them to prioritize other resources ahead of evidence-based or standards-aligned resources. When a teacher’s Job to Be Done is “help me find manageable ways to engage and challenge my students,” evidence-based resources that are not manageable or engaging don’t make the cut. My teaching experience mentioned above illustrates this point. My district’s official curriculum included a textbook series that was developed through years of research at a major university-based school of education, with millions of dollars in funding from the National Science Foundation and pilot testing with hundreds of teachers and thousands of students across the country. It was a great curriculum, but I hardly ever used it. For one, it wasn’t manageable. The curriculum came in the form of a few dozen small student and teacher booklets for each unit and a number of other guides, resource books, and manipulatives. It was a nightmare trying to dig through my school’s resource room to find all the hands-on manipulatives, and I never figured out how all the books fit together. Another 16 RETHINKING K–12 EDUCATION PROCUREMENT challenge was that the curriculum was carefully scaffolded, which meant that figuring it out in bits and pieces was impossible. I suppose that had I been around at initial adoption, I might have received professional development on how to make the curriculum work. But absent that training, I resorted to making most of my lessons from scratch and turning to websites such as Mathalicous.com and Khan Acad- emy for inspiration. When the rubber hits the road, most teachers look for resources that (1) are easy to adopt and (2) students will find engaging (Job A). Unless a resource first meets these two criteria, all the evidence and alignment in the world is superfluous.

Designing and Adopting Resources with Jobs to Be Done in Mind

Debunking misconceptions about adoption helps leaders avoid bad adoption strategies, but it doesn’t point them to strategies that will work. If the sections above marked the end of this chapter, they would land in your life like the frustrating friend or relative who eagerly tells you what’s wrong with your approach but doesn’t actually help solve your problems. Fortunately, Jobs to Be Done isn’t that friend. The sections below offer examples of how to translate Jobs to Be Done research into practice.

Spend Time in Classrooms to Discover Teachers’ Jobs to Be Done. To effectively uncover Jobs to Be Done, resource developers and district leaders need to observe teachers wrestling through their moments of struggle. Set aside pilot, focus-group, and survey data. These data offer only an opaque lens into correlations between resource features and teacher behavior. Instead, spend time watching and listening to teachers. When observing teachers, don’t look for gaps between how they teach and how you think they should teach. Instead, observe the circumstances that motivate their actual choices. What problems are teachers trying to solve? What contextual factors set the parameters for a good solution? Don’t focus on how teachers use products from existing product categories. Instead, look for cobbled-together workarounds and compensating behavior as they struggle with Jobs to Be Done that don’t yet have good solutions. THOMAS ARNETT 17

When you interview teachers, don’t ask them to explain what they like about a particular resource or why they picked a particular product. Instead, ask them about the events and struggles that precipitated their actual decisions. Asking about product preferences will only get you socially acceptable and logically coherent answers. Asking for stories, in contrast, will reveal the circumstances that shape decisions.

Motivate Teachers by Changing Their Circumstances. Those who develop and sell resources have to take teachers’ Jobs to Be Done as a given and then design around them. But school system leaders can shape the circumstances of teachers’ lives to activate their Jobs to Be Done. They can influence teachers by not only creating pull for new solutions but also generating pushes and addressing habits and anxieties to activate teachers’ Jobs to Be Done. Education leaders can create pushes by arranging experiences that lead teachers to a sense that the status quo isn’t working. These experiences might include visiting schools out of the district, attending conferences, visiting students in their homes, or reviewing test score data. These experiences become the precursors to change when they lead teachers to conclude for themselves that students aren’t as challenged or engaged as they should be. Administrators must be careful, however, to cultivate these experiences in ways that feel authentic to teachers. If the experiences don’t resonate, teachers will feel coerced, and the pushes will activate a compliance-oriented Jobs to Be Done (Job B). Education leaders can also address habits by being opportunistic with changes in teaching assignments. We found in our interviews that teachers who moved to a new school, course, or grade level were more open to trying new resources and practices. When the resources teachers rely on become less relevant, teachers automatically start looking for new resources to help with their Jobs to Be Done. Finally, sometimes the biggest barriers to adoption come from anxieties. Despite compelling reasons to abandon the status quo, sometimes the devil you know seems better than the devil you don’t know. Teachers know that change is hardly ever easy. Therefore, education leaders need to create systems and supports to make change as smooth as possible and make it safe to work through initial setbacks. 18 RETHINKING K–12 EDUCATION PROCUREMENT

Focus Professional Development on Fulfilling Jobs to Be Done, Not Covering Content. Professional development (PD) inevitably goes hand in hand with successful resource adoption: Resources fail almost inevitably if teachers don’t get support on how to adopt and use them. Yet, even with corresponding PD, many resources have poor uptake because the PD still fails to address teachers’ Jobs to Be Done. This happens because of how district administrators often frame the problem. They know that teachers need PD, so they ask, “What PD needs to go with this resource?” The better approach asks, “What experiences do teachers need for this resource to meet their Jobs to Be Done of finding manageable ways to engage and challenge their students (Job A)?” With this lens, it’s easy to see that upfront technical training on how to use a resource is a necessary but not sufficient condition. Ongoing,job-embedded support is just as important as upfront training is because in-the-moment support is key for making adoption a manageable process. If teachers can’t find colleagues or other supports to turn to when they run into hiccups, they’ll predictably turn back to the outdated resources that worked for them. Think of this kind of PD as more like robust customer support than like technical training.

Conclusion

In education, money isn’t easy to come by, which makes it especially frustrating when districts spend money on resources that go unused. The district staff members who make procurement decisions surely don’t intend for their purchases to go to waste, yet this continues to happen. A good sales pitch may get a product through the district office’s front door. But only by helping teachers fulfill their Jobs to Be Done canhigh-quality educational resources make it through the classroom door and into students’ hands. Vendors need to design their products and supports, and district staff need to consider purchasing decisions, with teachers’ Jobs to Be Done in mind. THOMAS ARNETT 19

Notes

1. Clayton M. Christensen et al., Competing Against Luck: The Story of Innovation and Customer Choice (New York: HarperCollins, 2016). 2. For the full report on this research, see Thomas Arnett, Michael B. Horn, and Bob Moesta, “The Teacher’s Quest for Progress: How School Leaders Can Motivate Instruc- tional Innovation,” Clayton Christensen Institute, September 2018, https://www. christenseninstitute.org/wp-content/uploads/2018/08/JTBD.pdf. 3. This example comes from Clayton M. Christensen and Michael E. Raynor, The Innovator’s Solution: Creating and Sustaining Successful Growth (Boston, MA: Harvard Business Review Press, 2003). If Education Procurement Is Broken, Is Teacher Choice the Answer?

MIKE GOLDSTEIN

et’s say our government bought exercise bikes, ergonomic chairs, and L weekly spinach for all its citizens. After all, they are “good for you,” and the government would receive some great “bulk” prices. Is this a good or bad idea? It’s a bad idea, of course. Most individuals would choose not to use them. For example, in my home the ergonomic chair would have a fighting chance of being used if comfortable. But the bike would be banished to the garage (next to the one already there). The spinach would be thrown away if it were the cheap kind sold in a bag. Even if it were the high-end organic kind sold in a plastic tub, it’d end up on our plates only occasionally. That would be my story. Your mileage would vary. Instead, fortunately, we have a more proper market. Thank goodness. I have a personal trainer, you jog, she does Pilates, your cousin drinks many protein shakes, and your uncle doesn’t exercise. Buying us all bikes in hopes that your uncle finally exercises is a bad idea. Moreover, entrepreneurs flourish when allowed to invent things for small tribes. The Peloton is a new online biking experience that some people really love. “Ninja gyms” are a niche product nobody could have con- ceived of a decade ago. Meanwhile, other entrepreneurs try to scale steeper cliffs: newmass-market products that many people will love. Impossible Burger has its name because we all thought veggie burgers would always taste terrible. But it’s flying off the grill at Burger King. All this is possible because the end user is the buyer. Our K–12 system is quite different. The individual teacher is neither in control nor the customer. He or she is “bought for”—by a school leader, superintendent, or other administrator. This, I argue, creates many

20 MIKE GOLDSTEIN 21 distortions that lead to the sad stories contained in this volume. And importantly, entrepreneurs are mostly sidelined and constrained. I’ve been on both sides of the K–12 market. I founded Match Edu- cation. It’s a small Boston nonprofit that, over the years, created services purchased by charter schools and districts (e.g., $100 curriculum, $2,000 one-on-one teacher coaching, $3,000 high-dosage tutoring, etc.). We even created, to the best of my knowledge, the only graduate school of education funded primarily by K–12 schools paying $8,000 placement fees for the right to hire our graduates. Meanwhile, I was the chief academic officer of one of the largest “districts” in the world, Bridge International Academies, a network of over 900 schools in Africa. In that role, I was the central decider who made various top-down decisions on behalf of teachers. And as a consultant, I’ve worked both on behalf of the largest charter networks and districts (the buyers) and for some of the biggest curriculum providers in the nation (the sellers). From those 20 years of experience, I’d like to offer what I see as the three largest hurdles to effective education procurement.

Half-Baked Implementation of Top-Down Anything

Earlier in 2019 I visited a middle school in an extremely polite midwestern city. (I repeat myself.) The teachers had a new Common Core math curriculum. It was rated “good” by EdReports and backed by philanthropists. So let’s call it a “promising program” for this chapter. The laudable idea behind this math curriculum is “rigor.” It contains many multistep word problems. Most teachers weren’t embracing the new curriculum. But only a few were fully rejecting it—leaving it on the shelf and using their preferred materials. Instead, many chose a middle ground. They did half-baked implementation to keep administrators off their backs. For example, teachers would cover some of what was in a typical lesson but would assign homework that eliminated all the more challenging problems. Essentially, they made the new curriculum into something quite similar to the old curriculum. 22 RETHINKING K–12 EDUCATION PROCUREMENT

The curriculum implementation was supposed to be helped along by an outside consultant. The consultant was exasperated. She identified the problem as teachers having low expectations of the students. She was perhaps partially correct. Yet I was sympathetic to the teachers. The kids arrived to their classrooms in September struggling with the basics. So the new “challenge” problems frequently created cognitive overload for students. You could notice moments when kids would palpably give up during class, even put their heads down on the desks. Sure, if these were elite teachers, they could make this curriculum work and perhaps even shine. They would somehow concurrently remediate the worst-off kids and explain the complex problems with such clarity that all kids would understand. But that’s a big “if.” If I were an elite basketball player, I could make left-handed layups through contact. I’m not elite, so I can’t. And these teachers weren’t elite; they were good, hardworking people, but they were not elite. So whether it was their “fault” or not, any observer would agree that the new curriculum was not “working.” This “promising program” was getting a half-baked implementation because of a reasonable-if-not-ideal reaction by teachers who were seeing their students fail and give up. When we think of programs, practices, and products that are bought for teachers—whether software such as Dreambox, pedagogies such as “no excuses” data-driven instruction, progressive “advisory” programs to begin each school day, or one-on-one teacher coaching—there are three possible outcomes:

1. Full implementation,

2. Total avoidance, or

3. Half-baked implementation.

It’s unlikely that something bought for “all teachers” is precisely what a typical individual teacher wants. Often, he or she doesn’t do a full implementation. MIKE GOLDSTEIN 23

Meanwhile, teachers probably can’t overtly reject it. They can’t openly say, “I am never taking these laptops out of the box” or “I’ll just skip that staff meeting, because I can tell you right now I will never use this response to intervention approach to math remediation.” So “total avoidance” doesn’t happen so often either. What remains? That last one, half-baked implementation, is, I contend, the dominant response out in the real world.

Underestimated Teacher Time

Most schools are good at budgeting cash. School spending is typically within 2 percent of what was budgeted for a given year. But schools are abysmal at budgeting teacher time. “Promising new practices” typically have large teacher time costs that schools refuse to model for. Moreover, an expert is often needed to show teachers the new way. When a corporation adopts a new software system, it tries to budget one-time costs: time for employees to train and cash to pay experts to explain and troubleshoot. Schools don’t typically do that. They refuse to model implementation costs. Picture a school with a new “laptop-for-all” initiative. There are 30 teachers and 400 kids. The school buys 400 laptops at $1,000 each. Let’s say the school needs a minimum of 50 hours of August training for teachers. First, teachers must master basic laptop operations: charging, cleaning, securing, learning software, and so forth. Second, they have to nail the policy layer: privacy settings, how to borrow for home use, and what to do when kids play Fortnite. Only then can teachers work on the education purposes and fold new lessons into their existing day-to-day routines. Instead, the school arranges two hours of August training from the laptop vendor, two days before school starts. They underestimate the needed time by 25 times. The district refused to honestly estimate the implementation time needed because of teacher contracts. District officials perceived the choice as either buying the laptops and having teachers “wing it” without meaningful training or not buying the laptops in the first place. As I understand it, they could have offered to pay teachers $40 per hour to attend August training, so long as it was optional. If all 30 teachers 24 RETHINKING K–12 EDUCATION PROCUREMENT participated, that would have added $60,000 to the price tag. But they didn’t even consider that. Moreover, another option would be hiring a teacher coach for the whole school year. He or she could help teachers one-on-one, showing each one how to use the laptops in a way that fit the teachers’ preferences, grade level, style, strengths, and weaknesses. Let’s call that a possible $90,000 expenditure. Any “normal” results-driven observer would think this is crazy. Instead, they would point out that the question we should ask is about return on investment. For example, how much student gain would we get for each dollar spent? Option 1 is $400,000 for computers plus a “free” two-hour training session. This would result in laptops mostly distributed during study hall or in the last 10 minutes of class, mostly used to play games or communicate with students’ friends, and rarely integrated into core classes. That’s low return on investment—arguably zero “return” on the investment. Option 2 is $400,000 for computers plus about $150,000 in one- time costs for meaningful teacher training and the full-time coach. The absolute cost would be higher, but you might have much higher return on investment—more kids actually using the laptops productively. Option 3 is $400,000 to buy 250 laptops instead of 400 and include the training and the coach. Perhaps the 10 teachers who least wanted the laptops would have their wishes granted and be able to opt out. That would leave more of a “coalition of the willing” for the 20 teachers who did get them. But that’s not the way the superintendent thought about spending. His goal was to tell the school board about an exciting technology program called “laptops for all.” He cared about optics. Option 3 was “laptops for some.” Option 2 cost more cash. Let me give another example. Restorative Justice is a controversial “promising new program.” I interviewed a number of teachers about it for a consulting client. I want to sidestep for a moment the debate on whether it “works” or if it is indeed “promising.” I want to instead point out that this program has a huge cost that is not often addressed by its adherents: the opportunity costs of teachers who now solve discipline problems with long after-school MIKE GOLDSTEIN 25 or planning-period conversations with misbehaving students (time teachers might otherwise use for grading, planning, parent communication, tutoring, or “life”). A teacher at a Boston charter school commented to me:

I think in theory, Restorative Justice [RJ] is awesome. Logisti- cally, however, for a teacher’s daily life, it’s really tough and not really feasible unless some tasks are alleviated from our work- loads. I’m not sure when in the school day these restorative justice conversations are supposed to happen . . . implied is that they happen during my already booked prep time, which makes my eyes roll. I’d be more on board with RJ if someone explicitly modeled the time costs for me as a teacher to do this extra work, instead of just piling it on top of everything.1

A former colleague now teaching at a Vermont public school explained to me:

My school recently adopted a Restorative Justice curriculum. Our experience has been mixed. It’s working for me. With a few colleagues, I took a three-credit course, that my school paid for. But we do not have all the staff on board. Many seasoned teachers are hesitant to embrace new ideas and trends, simply because they have tried new ideas so many times in the past, only to be asked to change again after putting significant effort in. They opted out of training, are making their classrooms worse, and seething with frustration.2

Let me also say that I have personally been guilty of this refusal to honestly account for limited teacher time. For example, I believe an excellent teacher practice is proactively phoning parents to build one-on-one relationships. I wrote a book about it, we developed a few videos about it, and we even commissioned a randomized control trial on teacher-parent phone calls. The data showed large academic gains for students whose parents received such phone calls. 26 RETHINKING K–12 EDUCATION PROCUREMENT

But in asking teachers to make these calls, I was never sufficiently willing to explicitly acknowledge the time trade-offs. I was unwilling to tell teachers what they should cut from their already-full workload or to suggest that while I was watching football on Sunday afternoons, they should be working their phones.

The Two-Headed Monster of Class Size and Specialists

In the first years of my charter school, we had a bookstore program. Every student would get $100 per year, and staff would take kids to Barnes & Noble (remember it?) once a month to findpleasure-reading books. It was a fairly small expense. There was no research per se behind it. But it seemed to be good. Many kids were reading for pleasure, which they’d never done before. We also had a program with many full-time tutors. It grew to become a national standout and was replicated not just in some charter schools but also in large districts such as Houston, Chicago, and New York City. Ran- domized trials showed huge gains for kids. Both programs met the same fate: They were significantly diminished over time. That’s because there is a politically attractive alternative to buying promising programs, practices, and products. It’s lowering class size and hiring more full-time specialists. That’s the two-headed monster. It’s the most direct response to the never-ending teacher cry for “more help.” The monster gobbles up most of the discretionary cash, leaving only a little to invest in promising programs, practices, and products. Teachers like smaller class sizes because they make the job seem more doable. They like the sound of “more help”: another counselor, dean, literacy specialist, math coach, social worker, paraprofessional, librarian, art teacher, or nurse. It’s hard to overstate the pressure school leaders face, particularly from stressed-out teachers, to always cut class sizes and increase head count of specialists. “Everything else” has much less political protection. Anything out of the ordinary is the first item cut or diluted. Why should we give kids books when they can go to the library? Why should we give kids one-on-one tutors when six to one is still a small group? MIKE GOLDSTEIN 27

In healthy organizations, “what works” gets more money. It grows. But in schools, “what works” doesn’t stand much chance against the voracious appetite around class size and head count. Few leaders will openly acknowledge this challenge. Eva Moskowitz is one who does. I love her fearlessness. Here is a small clip from an interview with NPR in 2011.

Neal Conan: But would you agree with Lisa that in middle school, there is a [class size] tipping point of around 28, 29, 30—which it is counterproductive? Eva Moskowitz: I don’t think it’s there. . . . I think that if I had to make the choice between 32 and 34 kids, it would depend on what I would get in the bargain. In other words, if I could take my kids to trips across the country, and I could hire a tutor to help them in math, those are the trade-offs that we have to be looking at. It’s not; if you think that we’re not making choices when we invest in small class sizes, you’re not being realistic. There are things we cannot do if we reduce class size.3 (Emphasis added.)

But Moskowitz is an outlier. Most leaders bend to the demand for more specialists and lower class size without honestly making (or narrating) the choices about which practices and programs are lost. Looking at all three of the largest hurdles to effective education procurement, we see that top-down deciders buy on the cheap for a fictitious average teacher, underestimating implementation costs and investments, which predictably results in half-baked implementation. Nobody wins. Teachers are hosed. They experience one new initiative after another, a perpetual hazing ritual. Meanwhile, social entrepreneurs can’t create services and products that individual teachers would love; those inven- tors are forced to sell bland products in bulk like everyone else or not exist at all. What keeps this stupid system propped up? The illusion by top-down deciders that this time, this time, they’ll get it right. 28 RETHINKING K–12 EDUCATION PROCUREMENT

A Radical Idea

Stanford historian Larry Cuban writes:

Too often wannabe reformers of an engineering mind-set see educational problems as complicated yet amenable to smart technological fixes—say a NASA-like “mission control,” too often overlooking that schools and districts are open systems where zillions of relationships and pieces have to be acknowl- edged and managed, where resources matter, and [individual] teachers are the central players in working through the complexity.4

That’s the key point here. Individual teachers are the central players working through the complexity. What if we asked teachers, individually, for permission? There are two ways to seek permission. The first way is a boutique practice in some charter schools. Permission is asked at the front door, during the recruitment and interview process. For example, “We believe X, Y, Z. That is different from other schools. We all try to row in the same direction, so your individual discretion would be limited. Are you sure you would want a job here?” I’d estimate this explicit permission seeking happens in less than 5 percent of schools. It isn’t scalable. The second way to seek permission from teachers would be to hand them all the cash.

Teacher-Controlled Funds. What if individual teachers controlled the funding that is currently “spent on them”? All the curriculum, training, coaching, education technology, field trips, and so forth would be bought by individual teachers. This isn’t unprecedented: New college professors receive a lump sum “package” to spend however they want—equipment for their lab, conferences, and so forth. Left-leaning reformers want teacher status, power, and freedom. Right- leaning reformers like choice and markets. This addresses both. Everything purchased would happen on a genuine opt-in basis. And, importantly, this creates needed opt out. We’ve all seen what happens MIKE GOLDSTEIN 29 when we require a new pedagogy, technology, or curriculum. Some teachers want it; some loudly complain and resist; many do a half-baked embrace. What if the only category were true opt in? I know what you’re thinking. Your reaction reminds me of the Passover seder’s “four types of sons.” Yes, sure, let’s help the wise teacher and give him or her discretion. But what about the wicked one who rips off the system, the simple one who makes bad choices, and the one who does not even know how to imagine new ways to spend money? Donors Choose already does a remarkable job of protecting against wicked teachers who try to enrich themselves. They have the highest possible ratings from Charity Watch, Charity Navigator, and Guidestar, an honor earned by less than 1 percent of charities. They vet field trips, professional development, and even passion projects for individual kids (e.g., giving 10-year-old Joe drumming lessons). They have accounting controls strong enough to review even “questionable” asks they think are in fact legitimate but might look sketchy to the public. They don’t allow overtime pay to teachers or any other financial benefit, and they prohibit religious materials. Next there is the simple teacher who makes bad decisions. Sure that would happen sometimes. In theory, if choice allows many simple teachers to now make bad decisions, who otherwise would have used “good curriculum” and pedagogy with fidelity, it might outweigh all the gains from good teachers with their newly gained freedoms. To believe this, you have to believe not only that the number of good teachers is quite low but also that the current system currently features many simple teachers implementing the top-down material with fidelity. That cuts against all the evidence. Finally, at the seder, there is the son who does not even know how to ask a question. I liken that to the fear of a new teacher. He or she has no orientation on how to make decisions, particularly around curriculum and “what is good.” Similarly, I think independent paid advisers would spring up to help teachers with these choices. Is this more than just a premise that “happy teachers are good teachers”? Definitely. The key idea here is that the current supply side—services and goods available to teachers—is not only inefficient but also not spe- cialized enough and not customer friendly enough. The winners (who provide products many teachers want and highly recommend to others, 30 RETHINKING K–12 EDUCATION PROCUREMENT sometimes “just” because execution or service is excellent) typically don’t get rewarded. If we just gave $5,000 or $10,000 to each teacher but left the existing marketplace intact, I predict failure. There needs to be a parallel investment in developing new supplies of teacher coaches, curriculum and education technology curators, “plug and play” field trips, and so forth.

Individual Teacher Choice. You might ask: Hasn’t individual teacher choice been tried before? It’s only been tried in two small ways. One is giving teachers a couple hundred bucks. Of course they use that for school supplies. That’s boring, uninspired, and off by a factor of 30. The first $1,000 given to a teacher is invariably sucked up by teachers’ desire for supplies. They buy magic markers, tissues that don’t feel like sand- paper, and $10 headphones to replace the $1 break-on-contact earbuds the district information technology guy chose to go with the new $400 iPads. That’s not what I’m talking about here. The $10,000 per teacher individual teacher choice effort, by contrast, unleashes the chance for much larger expenditures. Individual teacher choice has been tried in a form that I call “coerced group teacher choice.” Imagine a second grade team with six teachers. Teachers A, B, and C are fairly traditional. Teachers D and E are progressive but have a fair share of disagreements. Teacher F is a mix regarding pedagogy. Currently, their grade team meetings accomplish little. They are faux collaborations. Anyone who has ever sat through these knows that they are often collegial but circular discussions. As a grade-level team, they control some of the budget, but only as a clunky decision-making body. Now imagine a new reality for these six teachers, with individual teacher choice. Teachers A, B, and F choose to collaborate—and A and B don’t mind that F sometimes pushes them a little on their traditional pedagogy (e.g., “maybe we could do fewer worksheets”). Teacher B also spends $4,000 on the science field trip he or she always wanted (but none of the other teachers do). Teacher F is the only teacher who loves education technology. Teacher F typically unleashes it for the strong kids, so that he or she is free to spend more small-group time with strugglers. That’s a big investment for Teacher F. MIKE GOLDSTEIN 31

Teacher C is an introvert who likes to work alone and intensely dislikes meetings. This is fine. Teachers D and E combine their resources to hire the same teacher coach, who works with them individually. The coach unleashes some great new classroom experiences. Teachers D and E still don’t “collaborate”— no coerced efforts to make them plan lessons together and so forth. One can imagine the net result is much better than the previous coerced meetings of all six, where Teacher C was visibly irritated and Teachers D and E were always outvoted. Individual teacher choice is admittedly a radical idea. And, of course, if it gains any traction, there will be studies to determine its effectiveness and the challenges of making it work. But that’s a story for another day. But for now, here’s the larger point: If done well, individual teacher choice could lead to higher teacher satisfaction, lower teacher attrition, and increased student engagement—all without any loss in student achievement and likely resulting in achievement gains in the subset of teachers actually trying to accomplish that. And just as importantly, this would offer one overlooked response to the education procurement challenges that schools face today.

Notes

1. Interview with teacher. 2. Interview with teacher. 3. Neal Conan, “School Founder Says Class Size Doesn’t Matter,” NPR, March 29, 2011, https://www.npr.org/2011/03/29/134956184/school-founder-says-class-size-doesnt- matter. 4. Larry Cuban, “Skeptic or Cynic? An Interview,” Larry Cuban on School Reform and Classroom Practice, May 31, 2018, https://larrycuban.wordpress.com/2018/05/31/ skeptic-or-cynic-an-interview/. What It Will Take to Improve Evidence- Informed Decision-Making in Schools

ERIC KALENZE

In recent decades, American education has thrown all kinds of stuff at its various stuck needles: advanced technologies, standardized measurements of performance, tightened accountabilities, expanded school options, and so forth. No matter what gets pitched at those needles, however, precious few even budge. I have spent the past 15 years or so of my career fairly obsessed with the question of what we are still not getting right. And based on my experience and study, the answer is simply evidence. For all that the education field buys and sets into motion, its improvements are still missing evidence. More specifically, the education enterprise struggles to get truly evidence-supported practices and programs operating in schools and classrooms. Until the field does more to account for this reality, we will likely not see much positive movement in student outcomes. Measuring student learning more accurately, holding professionals more accountable, and strategically planning improvements are, of course, important. None of those things, however, will move much closer to the vision of equitable student outcomes if education’s practitioners do not know about—or flat-out refuse to use—the instructional practices and programs research tells us are most likely to work. Make no mistake: I am fully aware of how touchy it can be to talk about what “works” in education. In over two decades working as a teacher, administrator, independent researcher, and consultant, I have been in many conversations with other education professionals about the idea of evidence-based practices. If there is a spirited take on research’s reliability, questionable methodologies, the importance of context, schools’ inability

32 ERIC KALENZE 33 to implement properly, or whatever else, I have probably heard it. I understand how messy it is. The plain fact, though, is that some instructional practices and programs have more evidence to support their implementation than others do. Instructional practices and programs based on evidence from cognitive science, for instance, simply have higher chances of driving successful learning. They are designed expressly according to what decades of clinical trials have shown us about how humans learn and develop. Relatedly, some practices and programs have more evidence of having worked elsewhere to support them than others do, at both micro levels (practices and programs used in classrooms where students are making the largest gains) and macro levels (achievement data across entire systems). Still, I will not over-reduce the matter by saying that just “plugging in” practices and programs recommended by these evidence bases will guarantee huge student gains. Effective implementation is crucial, and successful implementation in schools could be covered in another whole report. But just as stock market investors study the best possible evidence before knowing where to put their money, so, too, should educators use sound evidence about how people learn and what has worked elsewhere as they select which practices and programs to implement. And that is education’s evidence problem: The enterprise’s practitioners and decision makers rarely and ineffectively consult cognitive- scientific or precedential evidence bases when choosing which practices and programs to invest in. Indeed, the education field tends toward practices and programs that are intuitively pleasing (e.g., differentiat- ing instruction according to individuals’ learning styles) and innovative (e.g., leveraging technology to fully “personalize” learning) but that have little or no evidence—from either cognitive science or sheer precedence—to justify their selection.1 To put it another way (and to borrow from renowned educators Doug- las Carnine and the late Jeanne Chall, respectively): That the education field has “less respect for evidence than for opinion and ideology” too often causes educators to choose practices and programs that move “in a direction opposite from the existing research evidence.”2 It is not a particularly responsible tendency, either, considering that the enterprise’s time, money, and angst are finite—and, of course, that each student passing 34 RETHINKING K–12 EDUCATION PROCUREMENT through the K–12 system gets only one chance to receive an education that will prepare him or her for the world. All moments spent getting better at things that might work, or at things that feel intuitively right, are not moments spent getting better at what evidence suggests will have a higher likelihood of working. Again, think of an investor who follows his or her gut and does not research the market before putting forward some amount of cash: When it is lost, you cannot get it back.3 With all this in mind, getting evidence-supported practices and programs into classrooms will not be easy. US education has a long way to go on this score and many challenges to work through. The remainder of this chapter will discuss some specific issues US education must address to build research and evidence more productively into its decision-making processes and will suggest actions to get us started. To give a better idea of just where decision-making breaks down when not informed by research and evidence, I start with the steps an actual school today would likely use to plan its improvement strategies. This brief look, informed by my own work with US schools, will shed some light on (1) how schools commonly determine their improvement priorities and (2) how schools commonly use evidence ineffectively (or neglect it altogether) when planning improvement actions. And it will provide an illustrative landmark I can return to when discussing challenges and recommendations.

Inside Current Schools’ Improvement Planning

As the work of improvement scientists and efficiency experts moves into education from fields such as health care and manufacturing, schools and districts are becoming considerably more strategic and focused with their achievement data.4 Increasingly, in fact, districts are requiring their schools to conduct constructed continuous school improvement– planning processes. A large majority of US states seems primed to make such a standard operation: 2018’s initial review of Every Student Succeeds Act (ESSA) improvement plans showed states everywhere pledging (if vaguely, in some cases) to become better data-driven decision makers.5 While these improvement-planning processes differ from one to the next, they commonly begin with some sort of needs assessment: teams ERIC KALENZE 35

Figure 1. The Public Education Leadership Project’s Problem-Solving Approach to Strategy Design and Implementation

1. Identify local needs

2. 5. Select relevant, Examine and evidence-based reﬂect interventions

4. 3. Implement Plan for implementation

Source: Stacey Childress, “A Problem-Solving Approach to Designing a Strategy to Improve Perfor- mance,” Harvard University, Public Education Leadership Project, April 24, 2017, pelp.fas.harvard.edu/ files/hbs-test/files/pel083p2.pdf.

of school stakeholders sifting through various performance and climate data and, after structured consideration and discussion, landing on “root causes”—the fundamental, underlying reasons driving unacceptable data points on the surface. (Example root causes might include “too many of our students miss too many school days per year to stay on grade level” or “too few of our students read to grade-level standard by the third grade.”) Leadership teams then build upward from their determined root cause to construct a “theory of action”—an “if-then” statement aiming appropriate actions directly at the problematic root cause.6 (For a sample of this type of decision-making process, see Figure 1.) 36 RETHINKING K–12 EDUCATION PROCUREMENT

And it is here, in the decisions about theories of action, that evidence- poor decision-making knocks productive school-improvement processes fully off their rails. Both up close and at a distance, I have seen schools (and indeed, entire districts) identify their root causes fairly well and then choose actions that have little positive evidence to support them. While piles of sound research exist about how humans develop and which actions have the best chances of systematically supporting such development, schools do not often consult the research when building their theories of action. Rather, they enact practices that are innovative, trendy, and just intuitively pleasing. Just a quick flyover of the past decade or so of American education reveals a long list of such decisions: solutions such as interactive whiteboards, standards-based grading, restorative practices, one-to-one digital devices, growth-mindset interventions, and others are accepted as promising investments practically out of hand, whether they have been proven to transform student learning elsewhere or not. Even worse, absent strong attention to evidence, such investments set an inevitable cycle into motion: Millions of dollars are spent on professional development, infrastructure, and materials to mobilize chosen improvements; the promised great shifts in student results do not materialize; theoreticians blame the low return on investment on all manners of “flawed implementation,” such as recalcitrant staff and inadequate funding; and decision makers move to still the next innovative or intuitively attractive solution. Rinse, wash, repeat.

Three Issues Limiting Informed Decision-Making by Educators

This evidence-unbothered tendency—which can elude even the most deliberate and careful improvement-planning processes—is most certainly frustrating, but it is far from educators’ fault alone. Multiple issues overlap, actually, to block research and evidence from better informing schools’ selection of effective programs, practices, and products. The issues have built up over generations of American education, and, while a full analysis is likely too complex for this chapter, three issues are particularly crucial to acknowledge and address here. ERIC KALENZE 37

Figure 2. Steps for Promoting Continuous Improvement

Adapt and modify for continuous improvement

Assess progress Analyze the problem and diagnose the causes

Implement the Develop a Identify the Strategy theory of Problem action

Plan for Design the implementation strategy

Source: US Department of Education, “Non-Regulatory Guidance: Using Evidence to Strengthen Educa- tion Investments,” September 16, 2016, https://www2.ed.gov/policy/elsec/leg/essa/guidanceuseseinvestment.pdf.

Issue #1: Limited Understanding of Evidence-Supported Practices at the Practitioner Level. When ESSA’s “Using Evidence to Strengthen Educational Investments” guidance was released in 2016, it explicitly brought evidence into the school-improvement process. In the guidance document’s steps to promote continuous improvement, in fact, step two is “Select Relevant, Evidence-Based Interventions.”7 (See Figure 2.) Compared to other commonly used continuous improvement processes (such as the Public Education Leadership Project’s problem-solving process, in which theories of action are built from “what do we think” questions over “what does the evidence say” questions), the ESSA guidance should be considered a giant leap forward for evidence. 38 RETHINKING K–12 EDUCATION PROCUREMENT

It will not be as simple, however, as instructing all continuous- improvement planners to “add evidence here.” It cannot be that simple, actually, as US educators just do not know enough about what works and why to just add evidence. Due to the field’s initial and continuing professional training, education’s practitioner level has a remarkably thin understanding of both the science of learning and evidence-supported practices.8 An interesting illustration of this assertion appeared in the past year thanks to an audio documentary by American Public Media’s Emily Han- ford and the hubbub that followed. In Why Aren’t Kids Being Taught to Read?, Hanford clearly explained how reading tends to be taught unscientifically across US elementary schools, including why teachers prefer the flawed methods they do.9 This was not news to many, as teacher-prep critics and reading researchers have been saying the same for years.10 The incredu- lous and even embarrassed reactions from career teachers on social media and elsewhere, however, confirmed that some fairly necessary stuff is not included in teachers’ initial and ongoing training.11 Lacking such crucial foundations (and at times even believing that scientifically unsupported approaches are idealbased on their training), the improvement strategies educators choose often lack sound evidence. Cur- rent instructional materials and approaches are too rarely interrogated, as they match educators’ trained-in ideals and senses of quality. And in instances when results do not materialize as promised, educators seeking new strategies default toward the sentimental, innovative, and more highly student-centered approaches. These student-centered ideas constitute most of educators’ continuing professional development, and they have hardened into a loop that has become fairly impenetrable over time. Indeed, ideas with strong cognitive-scientific evidence bases (e.g., the importance of explicit pho- nics instruction for developing readers, effective reading comprehen- sion’s reliance on background knowledge, and the role of deliberate and repeated practice in building expertise) have a tough time getting in front of teachers, much less properly taught and moved into practice, simply because they run counter to truths long considered self-evident by the education establishment. ERIC KALENZE 39

Issue #2: Centralized Selection of (Sometimes Evidence-Weak) Improvement Strategies. Accepting the issue above, imagine a school where it is not actually an issue—where all classrooms are staffed with the most research-literate and evidence-informed practitioners imaginable. Because they know their research, they will build perfect school improvements, right? Not necessarily. Schools operate within larger entities, after all. And if the strategies preferred and required by a district or state lack a strong justifying evidence base or are not well-thought-out, individual schools’ strategies will be evidence weak or disorganized by extension. Worse, the district- and state- imposed strategies take up so much infrastructure that site-determined improvements may not receive the maintenance they require. In other words: If a school’s district goes all in on a one-to-one iPad initiative or an entire state adopts a proficiency-based grading system, each school in those jurisdictions is responsible for site-level implementation. Whether the iPads or grading systems best address those schools’ actual student needs or not, additional staffing may be necessary to coordinate implementation, professional development time will be necessary to train staff, and administrative time will be necessary to keep things on track. This all leaves little time, funding, and energy for other improvements. To illustrate, take a look at Minneapolis Public Schools, a district I have worked a lot with in recent years. In the portfolio of schools I worked with (which, due to their chronic low performance, were state-designated “focus” and “priority” schools), it was quite common for school leaders to assign improvement actions toward literacy instruction. In general, though, this did not mean they were free to audit and possi- bly replace the practices and resources that clearly had not been moving their kids. Rather, their school-improvement plans explicitly cordoned off professional development time and space for the district to help them better execute the district-preferred literacy approach. (In early grades, this was—and is—the balanced literacy approach.) Although trend data over nearly a decade should call into question balanced literacy as the best approach for all early readers in Minneapolis (see Figure 3), the district’s leadership was firm: Schools are not to deviate from this approach; they are supposed to get better at doing it.12 In effect, the district denied its most struggling schools the opportunity to explore 40 RETHINKING K–12 EDUCATION PROCUREMENT

Figure 3. Proficiency Rates of Minneapolis Public Schools’ Third Graders on the Minnesota Comprehensive Assessment of Reading

100

80 Grade 3 MPS igh SES

60 Grade 3 M verall 50 oficiency Rate

Pr 40

20 Grade 3 MPS Lo SES

0 2011 2012 2013 2014 2015 2016 2017 2018

Source: Author’s calculations. effective replacement practices—even in some schools that, thanks to federal School Improvement Grant dollars, could have afforded to do so. Every hour and dollar they put toward improving the district’s preferred (but evidence-shaky) approach was an hour or dollar they could have spent learning and refining moreevidence-sturdy ones. This is one district’s example, and it is limited to a specific academic category. All US schools in “transformational, system-changing” districts or state education departments, however, give up large parts of their improvement infrastructure and decision-making to enable their districts’ and states’ preferred programs and practices, even if those preferred programs and practices are themselves weakly supported by multiple sources. On the question of why promising practices and programs are not adopted, top-down imposition of practices and programs is a large part of the answer. ERIC KALENZE 41

Issue #3: Research on Islands, Practitioners on Mainland. As I said earlier, it is a good thing that US education’s policy level has picked up on evidence-informed decision-making. Positively, the US Department of Education’s Institute of Education Sciences (IES) and its Regional Educa- tion Laboratory Program (and other nongovernment-affiliated providers) are producing solid—if not yet perfect—resources for helping educators access research more quickly and conveniently.13 Not so positively, however, these educational research and evidence resources are still too far removed from the world of practice. It is rather like they are concentrated on remote islands, and all education practitioners are on the mainland. (Interestingly, teachers around the country pretty much told IES the same thing in recent listening sessions: Eye-popping numbers of teachers had never heard of resources such as IES’s What Works Clearinghouse and Regional Educational Laboratories.14) As I have said, getting education to make better evidence-informed decisions is going to take a lot more than dropping “add evidence here” directions into improvement-planning processes, sitting back, and wait- ing. Indeed, active guidance and assistance will be needed to move educators toward the resources. No matter how conveniently the education research community pack- ages its research and evidence resources, education’s practice community will continue defaulting to existing knowledge bases and preferences and the sometimes flawed directives of their districts and states without protest if it is not made much clearer how and where to find the convenient resources. So researchers, you are on the clock. Strong, highly visible bridges— including lots of guides to help practitioners across—are needed and fast. If these bridges and assistance cannot be provided fairly soon, it is probably just a matter of time until practitioners stop looking out to sea altogether, turning instead back to the mainland.

Toward More Evidence-Informed Educational Decisions: The Big Aspiration and Three Actions

The big aspiration and three actions that conclude this chapter are presented as places to start. While many actions could be set in motion to 42 RETHINKING K–12 EDUCATION PROCUREMENT improve schools’ and systems’ evidence-supported decision-making, it is crucial that they all aim to strengthen the research literacy of the field’s practitioners and leaders.

The Big Aspiration: Strengthen Research Literacy of All Education Practitioners and Leaders. Flatly, strengthening research literacy across the education field has to be a primary target and guide. Some may shake this aspiration off, arguing that schools and districts are not designed to retrain their entire teaching and leadership forces in the relevant cognitive-scientific and “what works” practical literature. Others may challenge that building research literacy across the field’s practitioners is even the right aspiration: Is it better to just find practices and programs that are proven to work and then ensure that practitioners are making the methods come to life in their classrooms? Still others may believe that strengthening research literacy falls in the domain of preprofessional training and accordingly turn to reforming schools of education, the gateway through which almost all educators must pass. They are all fair points, for sure, but all seem to neglect what the term “research literacy” even is. Elevating research literacy, after all, does not necessarily mean making all educators experts with the educational research literature. More so, it means building practitioners’ and leaders’ skills to evaluate claims made by vendors, education “futurists,” other educators, or whomever so they can make the best instructional decisions possible for their kids’ needs. While retraining the entire education field to become deeply expert in the available literature would indeed be next to impossible, training the field to be more effective and evidence-guided claim evaluators is not. Functional grasps of these claim-evaluating skills can absolutely be taught and practiced, as the principles are quite straightforward. (The more challenging part, of course, is where and how these skills can be taught and practiced. Read on for some specific actions to meet these challenges.) And if our ultimate goal is to get evidence-supported practices and programs fully working in schools, we will need all the research-literate skill we can get. These elevated levels of research literacy will help teachers, schools, and districts choose practices and programs that genuinely account for their kids’ learning needs (where their specific kids are now), ERIC KALENZE 43 kids’ preparatory needs (what they will need to succeed in the future), and what science has told us about how people learn (the common things that make us all work). Additionally, elevated research literacy across the field will help all education decision makers distinguish good science from bad when setting priorities, selecting actions, and monitoring progress at system levels. Imagine how much time, money, angst, and student potential would be saved if even a few more voices, informed by evidence, rose to challenge a district on its literacy programming, its intent to roll out an as-yet-unproven (but cool) digital learning initiative, or its requirements that teachers differentiate lessons according to students’ learning styles. Again, when resources are finite (as education’s are), we must go with options that give us the best odds of success when aiming at a particular improvement target. Only evidence can tell us what those options are, and only we can parse the available evidence. Therefore, we must strengthen our ability to parse evidence, as widely across the enterprise as we can. Plainly, too much opportunity is being lost. We cannot just task schools of education with producing new teachers who are more research literate, as newly minted teachers make up only a tiny fraction of the overall practitioner force.15 Nor can we hold out hope that more practitioners will find their own ways to becoming more research literate. Rather, we should find systematic ways to accelerate research literacy across the field. On that note, let us segue from this overarching aspiration to consider some actions that can aid this acceleration. While many more beyond these can certainly be imagined and brought to life, the three suggested here would provide considerable positive momentum.

Action #1: Upgrade—or Construct—Bridges Between Research and Practice. In my earlier discussion of issues that must be addressed if truly evidence-supported practices and programs are to make their way into schools, I mentioned that bridges are effectively out between education’s research and practice communities. Even federally provided research resources, such as IES (and its What Works Clearinghouse, Regional Education Labs, and National Center for Education Statistics), are all but invisible to most educators, indicating that either their formats are 44 RETHINKING K–12 EDUCATION PROCUREMENT not sufficientlyuser-friendly , their perceived value is not high enough, or both.16 Education’s research community (and policymakers) can improve this by better understanding practitioners’ realities and needs, continuing to adjust their resources accordingly, and marketing their services more effectively. One way forward may be to partner with organizations such as researchED, Deans for Impact, and the Reading League. These entities and others like them have emerged in recent years to help educators make more evidence-supported instructional decisions, connecting them to sound research and a worldwide network of fellow learners.17 The networks of research-interested educators growing around each organization could help IES get in touch with the practice community more quickly than they ever could if they were starting from scratch.

Action #2: Strengthening Continuous-Improvement Processes with Evidence. If the pledges written into dozens of states’ ESSA plans are any indication, schools and districts across the US will be working hard to become better data-driven decision makers in the years to come.18 Influ- ential supporters, such as the Bill & Melinda Gates Foundation, are even pitching in to see how these processes can be maximized moving ahead.19 This sweeping commitment to carrying out sound continuous- improvement processes presents a massive opportunity to elevate scores of educators’ research literacy. (As a bonus, of course, the current continuous-improvement moment carries great potential for bringing evidence analysis into schools’ improvement-planning processes—the place where practices and programming are determined.) More specifically, problem-solving processes could be enhanced to make them useful for not just strategically planning improvement but also teaching research literacy to practitioners and leaders. Guidance, templates, and protocols could walk leaders through the process of using evidence to vet prospective improvement actions, just as current continuous-improvement processes walk leaders through needs assessments and root-cause analyses.20 While ESSA’s “Using Evidence” guidance (see endnote 7 and Figure 3) is a laudable step forward on this score, it is hardly an applaudable one yet. This guidance effectively tells leaders to “add evidence here” when ERIC KALENZE 45 significantly more facilitation is needed. If current documentation were expanded, however, and schools’ teams could receive training, technical support (perhaps from dedicated personnel such as the “learning engineers” or “research leads” recommended below) and fresh ideas about looking at research in school-improvement planning could be dissemi- nated and internalized quite rapidly. As so many schools, districts, and states have committed to carrying out thorough continuous-improvement processes, US education has a huge opportunity to immediately usher leaders toward better evidence- informed choices for their schools and permanently build those leaders’ research literacy skills and sensibilities.

Action #3: Formalizing Research Capacity in Schools and Districts. Whereas “data people” are common in US school districts (usually in departments such as research, evaluation, and assessment), the time is now to make genuinely “educational research people” formal parts of central administrative—and better yet, school-level administrative—fabrics. Rather than continuing as education has for so long (i.e., deciding based on “what is available” as much as “what is best,” making “instructional leadership” a small percentage of already overburdened administrators’ responsibilities, etc.), individuals with expertise and dedicated time could well be more productive. Bror Saxberg, the Chan Zuckerberg Initiative’s vice president of learning science, has for several years expressed a similar need for such individuals closer to education’s front lines:

learning engineers [in other words, people] who are deliberately trained and focused on designing and systematically improving learning environments at scale in measurable ways . . . [who] make use of the current and new science of how learning and motivation work, and . . . [who] collect careful measurements, but the focus is on improving success and impact at scale, within constraints (economic, regulatory, practical), not research per se.21

More than just curriculum and materials shoppers, these dedicated “learning engineers” or “research leads”22 could be tasked with all manners 46 RETHINKING K–12 EDUCATION PROCUREMENT of schools’ and districts’ research- and evidence-based improvement work. A position description might include the following:

• Audit current programming and practices;

• Recommend changes to programming and practices based on audit and sound evidence from research literature;

• Observe teachers and provide evidence-sound improvements to practices;

• Coach teachers as necessary;

• Design and deliver professional development;

• Coach teachers’ job-embedded, professional development teams on effective use of data, research, and evidence in instructional planning; and

• Inform school-wide continuous improvement processes and decisions.

This is a rough list, but it raises the question: Who currently does these things in schools or at district levels? How many dollars per year are spent on ensuring schools and districts are staying within evidence-informed guardrails? If the answers are “no one” and “no dollars per year” (and I am pretty sure they are), well, you get what you pay for.

Rescuing Opportunity

To repeat one final time, getting evidence-supported practices and programs into classrooms certainly will not be easy. Many systemic factors— from educators’ initial training to districts’ top-down influence—are standing in the way, and they are difficult to move. Luckily for US education, the need to be more evidence informed in our choices of practices and programs has been recognized, and interest for the idea is most certainly swelling. While we should always aspire to ERIC KALENZE 47 a consistently research-literate field, we certainly do not have to wait or wish for it. Rather, we can seize on this moment—and usefully bridge research-provided evidence straight into educators’ daily realities— with some rather straightforward operational steps. Dropping sturdy bridges between education’s research and practice sectors, enhancing school-improvement planning with various evidence-review processes, and minting “learning engineers” offer just a few practical, feasible ways to do so. We should get right on bringing these ideas and others like them to life in American schools. The education enterprise’s tendency away from truly evidence-supported practices and programs has cost us enough opportunity as it is.

Notes

1. Regarding the two examples provided above: Learning styles have never been shown to exist in any meaningful or actionable way, and personalized learning, despite all the recent enthusiasm surrounding it, has no serious record of transforming results. For more on learning styles, see Harold Pashler et al., “Learning Styles: Concepts and Evidence,” Psychological Science in the Public Interest 9, no. 3 (2008): 105–19, https:// journals.sagepub.com/doi/full/10.1111/j.1539-6053.2009.01038.x. For more on the lack of evidence to support personalized learning, see Benjamin Herold, “Personalized Learn- ing: Modest Gains, Big Challenges, RAND Study Finds,” Education Week, July 11, 2017, blogs.edweek.org/edweek/DigitalEducation/2017/07/personalized_learning_research_ implementation_RAND.html. 2. Douglas Carnine, “Why Education Experts Resist Effective Practices (and What It Would Take to Make Education More Like Medicine),” Thomas B. Fordham Foun- dation, April 2000, 10, www.wrightslaw.com/info/teach.profession.carnine.pdf; and Jeanne S. Chall, The Academic Achievement Challenge: What Really Works in the Classroom? (New York: Guilford Press, 2000), 180. 3. This is, of course, the concept of opportunity cost—a crucial idea to keep in mind, considering the highly finite nature of schooling. For a series of thoughtful pieces on opportunity cost in education, see David Didau, “Opportunity Cost,” Learning Spy, learningspy.co.uk/tag/opportunity-cost/. 4. Carnegie Foundation for the Advancement of Teaching has emerged as a leader in educational improvement science. For more on its basic tenets, see Carnegie Foun- dation for the Advancement of Teaching, “The Six Core Principles of Improvement,” www.carnegiefoundation.org/our-ideas/six-core-principles-improvement/. 5. Alyson Klein, “‘Continuous Improvement’ Model Woven into State ESSA Plans,” 48 RETHINKING K–12 EDUCATION PROCUREMENT

Education Week, May 8, 2018, www.edweek.org/ew/articles/2018/05/07/continuous- improvement-model-woven-into-state-essa.html. 6. The terms used here are from the Public Education Leadership Project’s Problem-Solving Approach to Strategy Design and Implementation, an improvement- planning process I have seen used in several schools I have worked with. For more, see Stacey Childress, “A Problem-Solving Approach to Designing a Strategy to Improve Performance,” Harvard University, Public Education Leadership Project, April 24, 2017, pelp.fas.harvard.edu/files/hbs-test/files/pel083p2.pdf. As defined by the Public Educa- tion Leadership Project, a theory of change (action) is “the organization’s belief about the relationships between certain actions and desired outcomes, often phrased as an ‘if . . . then . . .’ statement. This theory links the mission of increased performance for all students to the strategy the organization will use to achieve that goal.” 7. US Department of Education, “Non-Regulatory Guidance: Using Evidence to Strengthen Education Investments,” September 16, 2016, ed.gov/policy/elsec/leg/essa/ guidanceuseseinvestment.pdf. 8. For a useful starting point on this idea, see Kate Walsh, “What Teacher Candidates Won’t Find in Their Textbooks,” National Council on Teacher Quality, January 2016, https://www.nctq.org/blog/What-teacher-candidates-wont-find-in-their-textbooks----. 9. Emily Hanford, “Hard Words: Why Aren’t Kids Being Taught to Read,” APM Reports, September 10, 2018, https://www.apmreports.org/story/2018/09/10/hard- words-why-american-kids-arent-being-taught-to-read. 10. The National Council on Teacher Quality (NCTQ) issues periodic reports that point out what is taught and where across US teacher-preparation programs. For NCTQ’s most recent Teacher Prep Review, in 2018, see Robert Rickenbrode et al., 2018 Teacher Prep Review, National Council on Teacher Quality, April 2018, https://www. nctq.org/dmsView/2018_Teacher_Prep_Review_733174. For information on early reading instruction, see Rickenbrode, 2018 Teacher Prep Review, 18–20. While several reading researchers going back decades could be cited here, a particularly thorough and valuable example appears here: Mark S. Seidenberg, “The Two Cultures of Science and Education,” in Language at the Speed of Sight: How We Read, Why So Many Can’t, and What Can Be Done About It (New York: Basic Books, 2017), 247–81. 11. Madeline Will, “Teachers Criticize Their Colleges of Ed. for Not Preparing Them to Teach Reading,” Education Week, October 24, 2018, blogs.edweek.org/edweek/ teacherbeat/2018/10/teacher_prep_programs_reading.html. 12. Again, decades of work from reading researchers should also call this into question. As I have already referred to reading researchers’ take on balanced literacy in this chapter, I will leave them aside for now and assume that Minneapolis’ (and many others’) teaching and learning specialists are not familiar with or interested in their viewpoints. Results from Minneapolis Public Schools’ third graders on the state reading test between 2011 and 2018 are grouped by the district’s five highest and lowest socioeconomic status schools. Demographic information accessed at Minneapolis Pub- lic Schools, “Meal Eligibility at MN DOE,” October 1, 2018, https://studentaccounting. mpls.k12.mn.us/uploads/fall_2018_meal_eligiblity_official.pdf. 13. Regional Educational Laboratory Program, “The Regional Educational Labora- tory Program: About Us,” https://ies.ed.gov/ncee/edlabs/about/; and David B. Malouf ERIC KALENZE 49 and Juliana M. Taymans, “Anatomy of an Evidence Base,” Educational Researcher 45, no. 8 (2016): 454–59, https://journals.sagepub.com/doi/abs/10.3102/0013189X16678417?jour- nalCode=edra. 14. Sarah D. Sparks, “Teachers Want Education Research. The Feds Spend Mil- lions on It. So Why Can’t It Get to the Classroom?,” Education Week, November 27, 2018, http://blogs.edweek.org/edweek/inside-school-research/2018/11/get_teachers_ research_on_education.html. 15. That said, this is absolutely an improvement worth pursuing. As its impact is more incremental and long term, however, discussion should be saved for another report altogether. 16. Sparks, “Teachers Want Education Research.” 17. I am researchED’s US ambassador, arranging all its US conferences and operations. ResearchED is a grassroots education-improvement movement seeking to strengthen the research literacy of educators. It began in the UK in 2013. 18. Klein, “‘Continuous Improvement’ Model Woven into State ESSA Plans.” 19. Carolyn Phenicie, “Fixing Struggling Schools Is Hard. Moving Past Quick Fixes and Focusing on ‘Continuous Improvement’ Is Key, Gates Grantees Say,” 74, July 16, 2019, http://www.the74million.org/article/fixing-struggling-schools-is-hard-moving- past-quick-fixes-and-focusing-on-continuous-improvement-is-key-gates-grantees-say/. 20. For a model of matters to consider when choosing educational actions and interventions, see cognitive scientist Daniel Willingham’s step-by-step process for evaluating vendors’ claims in Daniel Willingham, When Can You Trust the Experts: How to Tell Good Science from Bad in Education (San Francisco, CA: Jossey-Bass, 2012), 135–222. 21. Bror Saxberg, “For White House Symposium on the Future of Education R&D and Digital Learning: Learning Engineers in Education,” Bror’s Blog, October 11, 2016, https://brorsblog.typepad.com/brors-blog/2016/10/for-white-house-symposium-on- the-future-of-education-rd-and-digital-learning-learning-engineers-in-e.html. 22. For more on the role of the school research lead, a position that has grown in the UK alongside the popularity of ResearchED, see Tom Bennett, The School Research Lead, Education Development Trust and ResearchED, http://www. educationdevelopmenttrust.Com/Educationdevelopmenttrust/Files/93/93c332a4-40df- 41ac-8a9b-f803c6573d10.pdf. How Confusion over Federal Rules Can Get in the Way of Smart School Spending

MELISSA JUNGE AND SHEARA KRVARIC

f the many factors that affect what school districts buy for their O students, an often-overlooked issue is the influence of federal education grant programs. Nearly every school district in the country receives funding from the US Department of Education (ED) through grant programs to support elementary and secondary education.1 While this funding represents a relatively small share of education spending overall,2 the rules governing how it can be spent strongly influence local decisions about student services. This influence is hard to see, but it works as follows. ED programs are governed by complex rules. In this chapter, we are not referring to accountability rules, such as how to identify and intervene in low-performing schools. Instead, we are discussing program implementation and spending rules such as who is eligible for services, what activities ED funds can support, the time frame in which districts can spend ED funds, paperwork requirements, and other day-to-day issues that receive little attention. The complexity of these rules can make it hard to spend ED funds and breeds confusion about what the funds can pay for. Compli- ance with the rules is then enforced by various oversight entities, and since there can be significant consequences for noncompliance, districts tend to be cautious in their spending decisions. Together, the complexity of the rules governing ED programs, and the way they are enforced, can unintentionally incentivize spending on the same activities from year to year, which limits opportunities for improvement. It can also incentivize a fragmented approach to education that favors unaligned interventions over systemic activities. These incentives affect more than just what districts do with their federal funds but also how they spend state and local funds.

50 MELISSA JUNGE AND SHEARA KRVARIC 51

As lawyers who help states, school districts, and other educational organizations navigate the federal rules governing ED programs, we see directly how important federal funds are for schools and students. We have helped districts use federal funds to improve the reading curriculum in struggling elementary schools, establish counseling and mentoring programs in low-income high schools, help students arrive to school safely, and engage parents in their children’s school through community school approaches. Yet, spending federal funds on these kinds of systemic activities is uncommon for reasons we will discuss throughout this chapter. To mitigate some of the complexities that limit ED grant spending, this chapter suggests ways stakeholders across the education sector could make it easier for schools and districts to understand their spending options. Specifically:

• Congress could revise program implementation rules to reduce confusion and make them easier to implement.

• ED and other federal agencies could use flexibilities that are already on the books to ease spending barriers.

• ED, state educational agencies, and entities that support states and school districts could provide state and district leaders with tools, such as simple, clear explanations of the rules and implementation- oriented guidance with concrete examples of effective practices.

• State and local education leaders can engage with the rules directly so they better understand their options and obligations.

• Education advocates, policymakers, and others who care about education can study how federal rules work in practice to understand their effect on students and schools.

Helping local leaders recognize that federal requirements have a wide-ranging influence on student services, not just those that are federally funded, and helping them better understand and more easily navigate federal compliance requirements could result in more effective spending and improved outcomes for students. 52 RETHINKING K–12 EDUCATION PROCUREMENT

Spending ED Funds Is Complicated

Each year ED grants more than $40 billion to states and school districts to support elementary and secondary education. Most of this money is intended to provide extra funding to benefit specific groups of historically underserved students3 and is governed by hundreds of rules,4 such as the kinds of activities the money can support and the type of paperwork states, districts, and schools must keep to prove they spent the money appropriately.5 To underscore the volume and complexity of these federal rules, here is a list of some requirements districts must consider when they spend ED money.

• Eligibility rules that limit who can participate in, or benefit from, grant-funded activities;

• Use of funds rules that limit the kinds of services grant funds can support (including earmarks that require spending on particular activities or caps that limit spending on particular activities);

• Planning requirements that oblige districts and schools to develop written plans describing how they will carry out certain aspects of a grant program, which usually must be developed in prescribed ways and address prescribed issues;

• Financial tests districts must pass to show ED funds add to, and do not replace, state and local education funding;

• Reporting requirements that oblige states, districts, and schools to gather and submit certain information to ED; and

• Spending time frames that set the start and end dates for spending ED funds, which can vary from program to program and year to year.6

Districts must also follow federal paperwork rules, procurement rules (which might differ from state or local rules), inventory management rules, and accounting and other financial management standards.7 MELISSA JUNGE AND SHEARA KRVARIC 53

This list is by no means exhaustive, but each rule affects student services in specific ways. For example, earmark requirements limit choices over how grant money is spent.8 Planning requirements vary between and in ED programs, which can make coordinating and aligning approaches hard. Paperwork rules require employees paid with federal funds to report the time they spend on individual programs, which can result in students receiving services in program silos.9 These are some examples of how federal rules affect student services; there are many more. And, importantly, federal rules are not the only rules governing ED programs. All of ED’s major elementary and secondary education programs are “state administered,” meaning ED gives money to states that are then responsible for giving money to districts and ensuring districts comply with federal requirements. Given their oversight responsibilities, states may, and typically do, impose their own rules for ED grants on top of the federal rules. Sometimes these state rules are based on state laws or regulations,10 and sometimes they are designed to promote state policies.11 Sometimes, however, state rules are based on misperceptions of federal rules, which can be hard for districts to parse out.12 This layering of state rules on top of federal rules can intensify the complexity of managing federal education programs, and since districts get ED funds through their states, state rules carry a lot of weight.13 In practice, all this complexity makes spending ED funds and understanding what the funds can pay for hard. For example, many states and districts believe Title I spending must be limited to reading and math instruction and that services funded by the Individuals with Disabilities Education Act (IDEA) for students with disabilities must be completely separate from a student’s regular education program. Neither is true, and in fact Congress, ED, and education advocates have tried various ways to dispel these myths. But, the rules are so complicated to unpack that the myths persist. Even when spending options are well understood, districts must follow complex procedures to make and account for purchases, which disincentivize spending in new ways. 54 RETHINKING K–12 EDUCATION PROCUREMENT

Enforcement Pressures Motivate Caution

If it seems that districts are being overly cautious in the face of complexity, understanding the significant compliance pressures they face is important.14 Local compliance with grant-related rules is overseen by states, ED, ED’s Office of Inspector General, independent auditors, and other federal agencies. The oversight occurs throughout the year in many different ways, including state review of district applications for ED funds, state review of district payment requests, annual audits required by federal law,15 state monitoring of district activities, ED monitoring, and audits by ED’s Office of Inspector General. This means districts must always be prepared to document and justify their grant-related activities. There are genuine consequences for noncompliance, including repay- ment,16 additional oversight, additional rules, and directives to change practices. In addition, noncompliance is often reported publicly, which can lead to local news stories, questions from school boards, and distrust from the community. The complexity of the rules, coupled with enforcement pressures, affects districts in two important ways. First, districts are often reluctant to change how they spend ED funds once they have passed compliance scrutiny. This incentivizes spending on the same activities from year to year even if those activities do not improve student outcomes. Second, districts tend to spend ED funds on activities that are easy to justify from a compliance standpoint, which favors fragmented, add-on educational services as opposed to systemic activities. We discuss both issues below.

Spending on the Same Activities from Year to Year

Districts tend to spend ED funds on the same things from year to year because it feels safe.17 Once spending is approved by a state and passes audit scrutiny, there is an incentive to maintain the status quo. Change means taking a risk that the new spending will be found to be unallowable, which has consequences for districts. For example, proposing to spend money in a new way might delay approval of a district’s application for funds, which disrupts programming and creates financial pressures. Or, MELISSA JUNGE AND SHEARA KRVARIC 55 proposing to spend money on a cost a state ultimately finds unallowable might increase state concern about a district leading to more intensive oversight. And of course, if a cost is found to be unallowable after the fact, the district might have to repay funds. This makes it hard for education advocates to convince districts to spend federal funds in new ways and for education service providers to convince districts to invest federal funds in their goods or services. The incentive toward the status quo is so strong even states themselves, which have a substantial say in how districts spend ED funds, can have a hard time convincing their districts to change. We worked with a state that invested significant resources to support districts in improving literacy only to see districts spend their Title I funds on one particular reading program year after year. The state’s leadership did not believe the program was effective, but it had been recommended by the state’s Title I office and had never been flagged in an audit, which motivated districts to support that reading program over others.

Spending on Fragmented Services over Systemic Improvements

Most ED programs limit what districts can buy with program funds and who can participate in program activities. It can be easiest to demonstrate compliance with those limits if ED-funded services are different and separate from the services provided to students generally. In practice, this favors spending on interventions over systemic improvements such as improving curriculum, training teachers on effective instructional practices, implementing positive behavior or mentoring supports, or redesigning school schedules to better support students. Often, interventions paid for with federal funds are deliberately unaligned to the general curriculum to make it clear they are different and separate.18 This is not required by federal law, but in our experience, many state and local administrators believe it is. Take Title I as an example. Researchers Nora Gordon of Georgetown University and Sarah Reber of the University of California, Los Angeles, interviewed Title I administrators and found they often described their Title I activities as “supplemental” and “intervention,” avoiding the term “core” and clarifying that Title I materials are “separate from the regular 56 RETHINKING K–12 EDUCATION PROCUREMENT curricular materials.”19 This is consistent with research that shows districts mostly spend Title I funds on supplemental instruction for reading and math,20 a pattern that has been consistent for decades.21 Largely, this preference for “supplemental” activities has been driven by a federal fiscal rule known as “supplement not supplant.” The rule is intended to ensure that federal education funds add to, and do not replace, state and local funds. Historically, districts have complied by showing each federally funded service is separate and distinct from any other service the district provides. This made it hard to align federally funded services to the rest of a district’s educational program. Congress recently made changes to Title I’s supplement not supplant requirement to make it easier to spend Title I funds on systemic improvements in Title I schools, but the changes will take time to saturate, and supplement not supplant is not the only federal rule that incentivizes separated services. This issue is not limited to Title I, and we know from experience with other federal laws that changing fiscal rules does not always lead to changes in spending without robust support.22 For example, a number of states believe federal law prohibits districts from using IDEA funds for services for students with disabilities if other struggling students without disabilities receive the same services. In practice, this (incorrect) interpretation means a district could not use IDEA funds to deliver a reading intervention to a student identified for special education services if the district delivered the same intervention to nondisabled students with another funding source. This approach is not required by federal law and is even discouraged by ED, which recognizes struggling students with and without disabilities often benefit from the same services. But this misunderstanding persists based, in part, on old regulations that have been off the books for more than two decades. ED has tried to flag that the regulations changed, but its guidance has tended to be technical and complicated and has lacked examples of what the changes mean in practice for how districts can spend IDEA funds. Even when fiscal rules are well understood, other federal rules incentivize fragmentation. For instance, federal inventory management rules require districts to show that items purchased with ED grant funds were used exclusively by eligible participants. This is easiest to do when the MELISSA JUNGE AND SHEARA KRVARIC 57 items are kept separate. Many other rules, and even reporting requirements, have the same effect. All this tends to fragment student services in detrimental ways.23 For example, many districts use Title I funds to support one kind of reading intervention for Title I students, IDEA funds to support a different reading intervention for students with disabilities, and Title III funds to support yet a different reading intervention for English learners, all of which is separate from the core reading instruction delivered to all students. Fed- eral law does not require this, but it is considered a safe way to spend. And since decisions about how to spend federal funds tend to be made by federal program staff rather than instructional staff, compliance considerations tend to play a more prominent role than academic ones do. This kind of spending has implications for not only students but also district operations and state and local spending. One of our district clients realized it was using hundreds of different reading programs across 50 elementary schools largely because it believed it had to use federal funds to pay for different programs for different student groups. Having so many programs spread across so many different schools affected things such as how the district staffed its schools, provided professional development to staff, set up school schedules, and made other operational decisions. Spending on fragmented interventions has persisted despite decades of research demonstrating systemic approaches benefit vulnerable popula- tions, particularly in high-poverty schools.24 Congress and ED have made it increasingly clear federal funds can pay for systemic activities in many cases, but funds are rarely spent this way largely because states and districts are unaware of these options given the complexity of federal programs.25 We do not point any of this out to suggest a preference for systemic approaches over interventions. Interventions can be an important and effective support for struggling students, particularly when they align to a strong core instructional program.26 Rather, we point this out to show that districts often make choices about the kind of approach to take because of their perceptions about federal requirements, rather than instructional preferences.27 Once a district chooses how to approach federally funded services, it can shape the district’s whole educational model, in which struggling students are served primarily through interventions rather than through 58 RETHINKING K–12 EDUCATION PROCUREMENT effective core instruction and supports. This certainly has implications for students, and it ends up driving how districts spend their state and local dollars, how they set up their school schedules, and how they deploy staff and other resources, all of which has major implications for how districts purchase goods and services in general.

Supporting Districts to Navigate Compliance Complexity

Given the long-standing effect federal rules have had on local decision- making,28 which has not changed substantially over the years despite federal efforts to increase flexibility, what can be done now to help districts invest in effective programs? Stakeholders across the education sector can acknowledge how complex ED programs are and equip leaders to manage them effectively. First and foremost, this means having leaders who will engage with compliance as an essential part of their job and something necessary to accomplish goals for students. This means leaders should understand:

• The influence federal rules have over their organizations,

• The obligations that come with accepting federal funds, and

• The mechanisms available for achieving their goals in their regulated environments.

Yet, leaders in the education sector often defer decisions on federal compliance issues to others in their organizations. Such deference can minimize a leader’s effectiveness, particularly considering the pervasive misunderstandings about what federal rules actually require.29 Ultimately, understanding federal rules can lead to better programming for schools and students. For example, we worked with a school improvement leader who was routinely told she could not spend federal funds on comprehensive improvements in a struggling school.30 Skeptical, this leader dug into the rules herself, with our support, and found she could add collaborative planning time for teachers by adding music and art to the MELISSA JUNGE AND SHEARA KRVARIC 59 school’s schedule. To do so, however, she had to understand the rules and lanes she could run in. It certainly is not reasonable—or even ideal—to expect education leaders to master all the intricate ins and outs of the rules governing ED programs, but there is a lot education stakeholders can do to give leaders the tools they need to better navigate the compliance environment they work in. Below are some suggestions.

What Congress Can Do

Congress can support states and districts by ensuring technical rules support effective program implementation.

Consider Technical Changes to Federal Law to Support Program Implementation. When people think about the rules governing ED programs, they often think about things such as Title I’s accountability rules or IDEA’s rules for ensuring students with disabilities receive a free appropriate public education. But ED programs also have program implementation rules that govern what states and districts must do as part of the programs (such as develop certain plans or send certain noti- fications to parents) and what they can do with program funds. How these program implementation rules are written sometimes make coordinating services hard. For example, federal law requires two different types of plans for certain low-performing Title I schools—a school-developed plan that describes the school’s strategies for meeting its needs31 and a district-developed plan that describes the school’s strategies for improving student achievement.32 Although the Title I law encourages districts to align these plans, each plan must address different things, be developed by different entities (school versus district), and have slightly different stakeholder engagement requirements, all of which can make them hard to align in practice while still meeting compliance requirements. There are many other examples of these kinds of disconnects, which Congress can address through modest technical changes. 60 RETHINKING K–12 EDUCATION PROCUREMENT

What ED and the Office of Management and Budget (OMB) Can Do

ED and OMB can support states and districts by promoting existing flexibilities, providing clear guidance about federal requirements, and promoting clearer distinctions between federal and state rules.

Promote Existing Flexibilities. ED and OMB currently have authority to grant flexibility around rules that can inhibit effective spending. For example, ED can extend certain spending deadlines and relieve districts and schools from duplicative planning requirements that can complicate program implementation.33 OMB can exempt states and districts from certain paperwork requirements and promote and approve innovative funding models that make it easier to coordinate across programs.34 For the most part, neither agency, however, has proactively solicited these kinds of flexibilities in the past. Taking steps such as advertising flexibility options, giving examples of how flexibilities could improve programming for students and how states and districts could pursue these flexibilities, articulating a clear review and approval process for requests, and making a commitment to approve reasonable requests for flexibility unless there are compelling reasons not to could help make the most of these existing opportunities.

Provide Clear Guidance. Leaders need clearer explanations of federal rules. We and others have written extensively about the need for clearer federal guidance,35 but a few points stand out.

• Guidance should be written as plainly as possible with minimal use of technical terms.

• Guidance should be as user-friendly as possible.36

• Guidance should be targeted to a wide audience—the “full range of school building and central personnel”37—not just federal program administrators.

• Guidance should be easy to find and access. MELISSA JUNGE AND SHEARA KRVARIC 61

• Guidance should use examples, with particular importance on providing various permissible spending examples.

• Guidance should address and clarify common misinterpretations of federal rules.38

• Special effort is needed to make sure guidance is made available to, understood by, and used by auditors to enhance audit quality and reduce compliance fears.

Promote Clear Distinctions Between Federal and State Rules. ED’s two largest programs, Title I and IDEA, require states to identify state-imposed rules on those programs, yet ED rarely enforces this requirement.39 When ED monitors states, it would be helpful to flag areas where states impose stricter rules than federal law requires. This would help in three ways. First, states may not be aware they are being more restrictive. Second, it promotes transparency so districts understand where a rule is coming from. Third, it would help ED identify and address widespread misunderstandings about federal requirements.

What State Educational Agencies Can Do

State educational agencies can support districts by providing clear guidance, ensuring state oversight of federal funds is consistent with federal rules and aligned to state goals, and partnering with districts to address barriers.

Provide Clear, State-Developed Guidance, Distributed to Appropriate Local Audiences. States should also provide clear and accurate guidance consistent with the suggestions above. Not only can states help put rules into local context and disseminate guidance to district staff more directly than ED can, research shows school districts receive most of their information about federal rules from their state.40

Ensure State Oversight of Federal Funds Is Consistent with Federal Rules and Aligned to State Goals. States oversee local implementation 62 RETHINKING K–12 EDUCATION PROCUREMENT of ED programs in many different ways (e.g., technical assistance, reviewing and approving local applications for federal funds, paperwork reviews and other forms of monitoring, data reviews, and more). This oversight is often diffused across many different offices in a state, so ensuring -over sight aligns across offices and processes and is consistent with state goals is important.41 For example, if a state includes chronic absenteeism in its accountability system, its technical assistance, application approval procedures, and monitoring should support district spending on activities to reduce chronic absences.42 In our experience, misalignment is common.

Partner with Districts to Address Barriers. Districts are on the front lines of federal program administration and have keen insight into where barriers to effective spending exist. We have worked on several projects in which states and districts partnered to address those barriers and have found that these kinds of collaborative projects tend to be more sustainable because they account for real-world implementation challenges.

What School Districts Can Do

School districts can engage with federal rules and work together with other districts to navigate barriers.

Engage with the Rules. Often, districts tend to accept what they hear about federal rules from states or third parties that hold themselves out as compliance experts.43 While it is true federal rules are complicated, there is value in looking independently at federal laws, regulations, and guidance. For example, we worked with one district official who learned she could use Title I funds for school counselors from reading ED guidance, an option her state did not know about. When she showed ED’s guidance to her state, they allowed her district to spend on counselors.

Work Together with Other Districts. It can be daunting for a district to challenge a state’s interpretation of federal rules. Some districts we work with find it easier to coordinate with other districts to tackle shared challenges jointly. MELISSA JUNGE AND SHEARA KRVARIC 63

What Education Advocacy and Support Groups Can Do

Education advocacy and support groups can support states and districts by providing accurate and practical technical assistance about federal programs and assisting with access to lawyers when necessary.

Provide Accurate and Practical Technical Assistance About Federal Requirements. National organizations and education advocacy groups are in a unique position to explain federal rules and how they play out in practice. For example, the California Alliance for Arts Education, which advocates for high-quality arts education for all students, created websites to help districts understand how they can use Title I funds to support arts education.44 The websites include links to ED guidance, stories from districts that used Title I successfully, and, importantly, tips for how to implement programs effectively. Although many national organizations have put together resources, the resources do not always explain federal requirements correctly, which undermines their value. If organizations wish to effectively support education, they must also try to master the rules that states and districts must implement.

Assist with Access to Lawyers When Necessary. State and district leaders also need access to lawyers to help them understand their options, determine what federal law actually requires, assess risks, and challenge misinterpretations when necessary. We are not talking about litigation, though there is a role for that. Instead, we mean using lawyers as problem solvers to get around day-to-day roadblocks to accomplish what is needed for schools and students.45

Conclusion

We must recognize the influence ED grant programs, and the rules attached to them, have on local decision-making. The more we recognize this influence and support leaders in better navigating these rules, the better posi- tioned districts will be to spend on the goods and services that meet their schools’ and students’ needs. 64 RETHINKING K–12 EDUCATION PROCUREMENT

Notes

1. For example, more than 90 percent of school districts receive funding through Title I, Part A, the Department of Education’s (ED) largest elementary and secondary education program. See, for example, US Department of Education, Institute of Edu- cation Sciences, National Assessment of Title I Final Report, 2007, https://ies.ed.gov/ncee/ pubs/20084012/. 2. Federal funding makes up about 8 percent of elementary and secondary public school revenues nationally, which includes funding from ED and other agencies (such as the Department of Health and Human Services and the Department of Agriculture). See J. McFarland et al., The Condition of Education, National Center for Education Sta- tistics, May 2019, https://nces.ed.gov/ programs/coe/. Please note this is an average; the percentage is much higher in some districts. 3. For example, Title I, Part A of the Elementary and Secondary Education Act (ESEA)—ED’s largest K–12 program—supports educationally disadvantaged students in high-poverty schools; Title III, Part A of ESEA supports English learners; and Part B of the Individuals with Disabilities Education Act supports students with disabilities. 4. ED’s Office of Inspector General (OIG) once estimated that Title I, Part A alone contained more than 500 discrete compliance requirements. US Department of Educa- tion, Office of Inspector General, “Compliance Requirements Within Title I, Part A of the No Child Left Behind Act,” 2006. While that estimate was based on a prior version of the law, most of the requirements OIG counted are still part of the law today. 5. Not all these rules come from Congress. States and districts that receive ED funds must also follow rules set by the Office of Management and Budget (OMB) and rules set by ED. 6. These program-specific requirements are established by federal education laws, such as Title I, Part A of the ESEA; Part B of the Individuals with Disabilities Education Act; and the General Education Provisions Act. 7. These grants management requirements are established by a set of regulations that govern all federal grants, known as the Uniform Grant Guidance. 8. For example, most of ED’s major programs require districts to set aside some of their money to provide services to private school students (a requirement since the 1960s), which limits the amount available for public school programs and requires districts to devote sufficient administrative capacity to running private school programs, a responsibility federal law prohibits districts from delegating. Likewise, the new Title IV, Part A program requires districts receiving over $30,000 spend 20 percent on safe and healthy schools, 20 percent on well-rounded education, and “some” on technology, with a 15 percent cap on technology infrastructure, such as devices, equipment, and software. In other words, just because a district receives a certain amount of money through an ED grant does not mean it has discretion to spend all that money on its own priorities. 9. For more information about how time and effort requirements shape program services, see Melissa Junge and Sheara Krvaric, “‘Time and Effort’ Takes Too Much Time and Effort,”Education Week, October 26, 2011, http://blogs.edweek.org/edweek/ rick_hess_straight_up/2011/10/time_and_effort_takes_too_much_time_and_effort. html. The federal rules governing time and effort changed since we wrote this blog post MELISSA JUNGE AND SHEARA KRVARIC 65 to be, in theory, more flexible, but neither ED nor OMB has issued guidance on how to satisfy the new rules, so the vast majority of districts still keep the records described in this blog post. 10. For example, federal law typically requires districts to keep records related to federal grants for three years, but a state might require districts to keep records longer based on state law. 11. For example, a state might require districts to submit more information to justify spending on activities that do not align to state policy. 12. See, for example, US Government Accountability Office,Elementary and Second- ary Education: Flexibility Initiatives Do Not Address Districts’ Key Concerns About Federal Requirements, September 1998, 4, https://www.gao.gov/assets/160/156363.pdf. 13. For an overview of how states affect local spending choices, see Nora Gordon and Sarah Reber, “The Quest for a Targeted and Effective Title I ESEA: Challenges in Designing and Implementing Fiscal Compliance Rules,” Russell Sage Foundation Journal of the Social Sciences (2015), https://www.rsfjournal.org/content/rsfjss/1/3/129.full.pdf. 14. For more about compliance fears and how they affect districts, please see Melissa Junge and Sheara Krvaric, Federal Compliance Rules Can Work Against Education Policy Goals, Federal Education Group, 2011, http://www.fededgroup.com/uploads/7_2011_ Federal_Compliance_Rules_Junge_and_Krvaric_EDU.pdf. 15. This is required for any entity that spends more than $750,000 in federal funds in a year. 16. While researchers have found that ED rarely imposes financial penalties on districts that run afoul of federal rules, in our experience, states—that have authority to order repayments—often do. See, for example, Gordon and Reber, “The Quest for a Targeted and Effective Title I ESEA,” 3. 17. Kerstin Carlson Le Floch et al., Study of Title I Schoolwide and Targeted Assistance Programs: Final Report, US Department of Education, 2018, 38, https://www2.ed.gov/ rschstat/eval/title-i/schoolwide-program/report.pdf; and Beatrice F. Birman, The Cur- rent Operation of the Chapter 1 Program: Final Report from the National Assessment of Chap- ter 1, Office of Education Research and Improvement, 1987, 33, http://files.eric.ed.gov/ fulltext/ED289935.pdf. 18. This has been the case for decades. See, for example, Birman, The Current Oper- ation of the Chapter 1 Program, 87–91. 19. Gordon and Reber, “The Quest for a Targeted and Effective Title I ESEA,” 12. 20. Le Floch et al., Study of Title I Schoolwide and Targeted Assistance Programs, xi. 21. See, for example, Gordon and Reber, “The Quest for a Targeted and Effective Title I ESEA”; General Accountability Office,School Districts Have Used Title I Funds Primarily to Support Instruction, 2011, https://www.gao.gov/assets/330/321048.pdf; US Department of Education, Institute of Education Sciences, National Assessment of Title I Final Report, 2007, https://ies.ed.gov/ ncee/pubs/20084012/; and Birman, The Current Operation of the Chapter 1 Program. 22. See, for example, Birman, The Current Operation of the Chapter 1 Program, 9. 23. Le Floch et al., Study of Title I Schoolwide and Targeted Assistance Programs, 1. 24. Le Floch et al., Study of Title I Schoolwide and Targeted Assistance Programs, 1. 25. Le Floch et al., Study of Title I Schoolwide and Targeted Assistance Programs, xv. 66 RETHINKING K–12 EDUCATION PROCUREMENT

26. See, for example, Gordon and Reber, “The Quest for a Targeted and Effective Title I ESEA,” 11–12; and Kenneth L. Wong, Gail L. Sunderman, and Jaekyung Lee, “Redesigning the Federal Compensatory Education Program: Lessons from the Imple- mentation of Title I Schoolwide Projects,” in Implementing School Reform Practice and Policy Imperatives, ed. Margaret C. Wang and Kenneth K. Wong (Philadelphia, PA: Tem- ple University, 1997), http://files.eric.ed.gov/fulltext/ED433391.pdf. 27. Gordon and Reber, “The Quest for a Targeted and Effective Title I ESEA,” 3; and Birman, The Current Operation of the Chapter 1 Program, 56. 28. Researchers have highlighted the influence federal compliance rules have on district practices for decades. See, for example, Birman, The Current Operation of the Chapter 1 Program; and Gordon and Reber, “The Quest for a Targeted and Effective Title I ESEA.” 29. Gordon and Reber, “The Quest for a Targeted and Effective Title I ESEA.” 30. Melissa Junge and Sheara Krvaric, “Managing the Law in Education: Strate- gies for Education Leaders and the Organizations That Support Them,” American Enterprise Institute, October 23, 2014, https://www.aei.org/research-products/report/ managing-law-education-strategies-education-leaders-organizations-support/. 31. This is known as a “school-wide plan” and is required for any high-poverty Title I school that operates as school-wide program under Section 1114 of Every Student Succeed Act (ESSA). 32. This is known as a “comprehensive support and improvement plan” and is required for certain low-performing schools identified by the state through its accountability system under Section 1111(d) of ESSA. 33. ESEA Section 8401. 34. 2 CFR § 200.102. 35. Nora Gordon, Increasing Targeting, Flexibility, and Transparency in Title I of the Elementary and Secondary Education Act to Help Disadvantaged Students, Brookings Institution, March 2016, https://www.brookings.edu/research/increasing-targeting- flexibility-and-transparency-in-title-i-of-the-elementary-and-secondary-education- act-to-help-disadvantaged-students/; Junge and Krvaric, “Managing the Law in Educa- tion”; and Junge and Krvaric, Federal Compliance Rules Can Work Against Education Policy Goals. 36. For example, ED guidance on a single topic is often split across multiple documents. Guidance on related topics should be combined to the extent possible. 37. Gordon, Increase Targeting, Flexibility, and Transparency in Title I of the Elementary and Secondary Education Act, 12. 38. See, for example, US Department of Education, Supporting School Reform by Leveraging Federal Funds in a Schoolwide Program, September 2016, 5–7, https://www2. ed.gov/policy/elsec/leg/essa/essaswpguidance9192016.pdf. 39. ESEA Section 1603(a)(1), IDEA Section 608. 40. Gordon and Reber, “The Quest for a Targeted and Effective Title I ESEA,” 2. For more information about state-developed guidance, please see Melissa Junge and Sheara Krvaric, Developing Effective Guidance: A Handbook for State Educational Agen- cies Delivering Guidance on Federal Education Programs to Drive Success, Council of Chief State School Officers, January 2017,https://ccsso.org/ sites/default/files/2017-10/ MELISSA JUNGE AND SHEARA KRVARIC 67

CCSSODevelopingEffectiveGuidanceHandbook.pdf. 41. For more information about state oversight processes under ESSA, see Melissa Junge and Sheara Krvaric, A Guide to State Educational Agency Oversight Responsibilities Under ESSA: The Role of the State in the Local Implementation of ESSA Programs, Council of Chief State School Officers, March 2017, https://ccsso.org/sites/default/files/2017-10/ CCSSO_State_Authority_Over_ ESSA_Programs.pdf. 42. For more information about promoting alignment, see Melissa Junge and Sheara Krvaric, Decision Guide for Implementing ESSA: State Considerations for Effective Grant Pro- grams, Council of Chief State School Officers,https://ccsso.org/sites/default/ files/2017- 10/CCSSODecisionGuideForESSAImplementation.pdf. 43. There is a compliance industry that sells federal compliance-related materials to districts. Unfortunately, these materials are sometimes inaccurate. 44. See California Alliance for Arts Education, “Title I Initiative,” https://www artsed411.org/action_center/title1; and Title1arts, website, https://www.title1arts.org/. 45. For more specific suggestions about how education organizations and education schools can help state and district leaders better use lawyers and navigate legal requirements without lawyers, see Junge and Krvaric, “Managing the Law in Education,” 7–9. Helping Education Entrepreneurs and School Leaders Navigate the Procurement Maze

DAVID DESCHRYVER

ears ago, a scrappy startup created a diagnostic assessment that Y teachers could use in their classrooms. It was cutting-edge, and the company grew by leaps and bounds—except in one state in the Southeast. It didn’t matter how many agreeable conversations they had with district officials, state program officers would not allow districts to purchase the product. Without clear answers why its product didn’t fit with state practices and preferences, the company’s leadership began to suspect that state officials did not like them. That wasn’t the case. It turned out that the state’s decision (and, in turn, the district’s) was informed by a long-forgotten statute that prescribed specific requirements for diagnostic reading assessments. That statute, and the rules that followed, were drafted a decade before the startup came into being. It did not account for digital assessments that could be administered with handheld technology (at the time, a PalmPilot). The company’s technology and assessment didn’t square with the largely outdated regulations. District leaders understood that their choices were prescribed in ways that prevented using the company’s technology. But few appreciated that those decisions were informed by a statute passed when many of the district and state decision makers were still in college. Once presented with the incongruity among emergent technology, district preferences, and an outdated statute, policymakers were keen to update. The new technology was then quickly adopted in the state, helping the state achieve its accountability objectives while giving teachers diagnostic tools to inform instruction and improve outcomes. The barrier to procurement wasn’t personal, of course. But it required translating a

68 DAVID DESCHRYVER 69 product into terms that were cognizable to policymakers and analyzing the policy preconditions that shaped the market. For more than 20 years, Whiteboard Advisors has been performing this sort of analysis to bridge the divide between emergent technologies and legacy policies, creating flexibility for districts, schools, and educators to improve student outcomes with new tools and approaches. Along the way, Whiteboard Advisors has found that education entrepreneurs and investors’ perception that education procurement is “broken” is often misguided. Procurement often works exactly as intended. Ed Kirby notes that sometimes rules and requirements exist because advocates worked with legislators to create opportunities only for services or activities that meet defined criteria.1 School officials safeguard against waste, fraud, abuse of public funds, and noncompliance with program rules. This can seem like a drag, but it’s not that school officials aren’t against innovation and expediency; they are against risk, which is prevalent in a fast- growing sector. According to EdSurge, the online education technology journal, more than 2,500 digital products emerged over the past decade, including more than 640 curriculum resources alone, and every year new companies enter and exit what’s estimated to be a $10 billion market.2 Navigating and fil- tering what’s available doesn’t appeal to most school officials. The procurement of education technology is clouded by “the difficulty of sorting through the increasingly large number of products available,” and it’s complicated by the need to abide by layers of statutes, regulations, guidance, and prior practice.3 It’s fertile ground for confusion and wasteful spending—and a profound opportunity to establish common ground. In this chapter, I build on the story of the scrappy startup by sharing two in-depth examples of Whiteboard Advisors’ work to create a shared understanding of new ideas and material terms between education organizations and decision makers. In the first example, confusion arose from the relatively new online service “crowdfunding.” At the behest of the Office of the Washington State Auditor, school officials in Washington interpreted crowdfunding in a sweeping manner that restricted teacher access to all crowdfunding platforms. In the second example, I examine confusion about the application of the “evidence-based” research requirements in the Every Student Succeeds Act (ESSA). The absence of a shared 70 RETHINKING K–12 EDUCATION PROCUREMENT and commonly used framework to implement the requirements continues to cause grief for vendors and school officials alike. Both examples have something to offer school officials and entrepreneurs. Regulations and technology constantly reshape the education marketplace. This requires public officials to continually monitor how new services allow them to fulfill their responsibilities while improving how they go about their business. It also compels business officials to clearly anchor their work in the rules that bind procurement with public money. The most innovative ideas, no matter how smart, aren’t helpful if they don’t fit within a compliance framework, and it’s even worse if everyone is speaking past one another. But this shared understanding doesn’t just happen; it requires careful facilitation.

Case One: The Confusion of Crowdfunding

In the movie The Princess Bride, there’s a famous scene (among a certain aging demographic) in which the protagonist Inigo Montoya reacts to the persistent but inaccurate use of the word “inconceivable” by Vizzini the Sicilian, a man who holds his own intellect in high regard. After one such use, Montoya turns to him and says, “You keep using that word. I do not think it means what you think it means.”4 The statement is famous (and funny) for many reasons, but largely because people attach different and often inaccurate meanings to words. This is rife in public education policy, from the broad objectives of education funding to the use of technical terms of art that dictate whether and how school administrators use public money. Consider the word “equity.” It is both an aspiration and imperative that informs our work and a linchpin of education policy since the common school’s inception. Yet its technical meaning can be slippery. When asked to define the term, a chief financial officer (CFO) will likely provide a- dif ferent answer than a chief academic officer (CAO) or a parent would. The CFO likely focuses on the distribution of funding, the CAO on curriculum and access to courses, and the parent on a wider range of matters, such as a teacher’s demeanor, the school climate, and other aspects of their child’s day-to-day experience. DAVID DESCHRYVER 71

Now consider a new term such as crowdfunding, which generally involves fundraising through a third-party website. It is poorly understood and infrequently used, and it can take on many meanings, which is challenging when compliance requires a clear understanding.

Distinguishing Crowdfunding from School-Based Crowdfunding. A series of events in Washington state illustrates the importance of clarifying the application of this consumer term to the education market. In January 2018, the Office of the Washington State Auditor posted reminder guidance to “follow best practice, state law when using [a] third-party receipting vendor.”5 The auditor’s guidance drew the school leaders’ attention to existing cash receipting rules that restrict how school employees may accept and manage cash on behalf of the school district. The rules, in short, prohibit school staff from accepting cash on behalf of their agency unless they follow stringent cash management procedures.6 The reminder to clamp down on third-party receipting came, with good reason, as a response to the risk of popular crowdfunding sites that enable the easy flow of cash online between individuals and organizations. Teach- ers were tapping into Kickstarter, GoFundMe, and other services to help get materials for class projects. While these services are excellent in many regards, they are not designed to serve school officials. For example, Kickstarter and Indiegogo connect entrepreneurs to early investors. Causes, as the name suggests, supports social causes. GoFundMe, one of the best-known services, is often used in response to personal or community tragedies and emergencies. In most instances, the funds transfer from the donor to the recipient’s bank account. But as the Washington state auditor makes clear, this flow of dollars in a school setting is not allowed. District officials in Washington state also harbored concerns about crowdfunding. The potential liabilities seemed to outweigh the benefits. What happens to the interest earned on cash that’s improperly managed? Will the funds be used to purchase items that violate the district’s technology and security requirements? What happens if the activities detract from the educational objectives described in binding school improvement plans? As one official put it, “Too many things can go wrong.” The list of worrying questions—coupled with an attentive state auditor—led to a clampdown on crowdfunding services. Districts 72 RETHINKING K–12 EDUCATION PROCUREMENT adopted policies prohibiting creating fundraising campaigns that rely on third-party vendors. Officials scrutinized cash donation policies. School boards did not want teachers requesting materials and supplies outside of established district procedures. If teachers wanted to use a crowdfunding site, they could do so only as private individuals—not on behalf of the district. Using district emails, school letterhead, and so forth was a no go. Risk aversion prevailed.7 DonorsChoose, a nonprofit crowdfunding service built specifically for schools, was caught in the cross fire and received a flood of emails from teachers about prohibitive school board policies. It didn’t matter that the nonprofit was unlike the typical crowdfunding service in that it was created by a teacher familiar with the needs of local education agencies and responsible for more than $980 million in donations of supplies to classrooms across the country. Pointing out DonorsChoose’s advantages over other types of solutions was insufficient to allow for differentiation by policymakers or administrators. Brands are not policies, and policy is not the forum to pick winners and losers. What matters is how a service or product serves policy objectives, can be described in policy terms, and comports with legal requirements. School boards classified DonorsChoose as a prohibited online cash donation service. DonorsChoose, in turn, had to convince skeptical school officials that its service, unlike other crowdfunding services, not only was compliant with state cash management requirements but also could be used as a tool to advance the districts’ instructional objectives. This reframing—“I do not think [crowdfunding] means what you think it means”—required collaboration among often-disparate stakeholders to establish a meaning for the term “school-based crowdfunding service” and examples of how it works in other schools. But the organization couldn’t be the only messenger. Notwithstanding its dedication to public school teachers, leaders, and students, it was only one voice, and an ostensibly biased one at that. It needed to be part of the process of developing a lexicon for school-based crowdfunding to ensure that decision makers concerned with the practice were talking about the same thing. Simply clarifying the meaning and underlying assumptions of terms can be tricky. Consider what someone from the South calls a “Coke” may DAVID DESCHRYVER 73 be the same as a “soda” for someone from Connecticut. Needing a basic translation is so common that scholars created a dictionary, the Dictionary of American Regional English, to work through the complexity.8 Beyond basic meanings of terms, mutual understanding is about being regarded as a peer. The late David Foster Wallace put it this way:

People really do judge one another according to their usage of language. Constantly. Of course, people are constantly judging one another on the basis of all kinds of things . . . and . . . it’s all terribly complicated and occupies whole battalions of sociolin- guists. But it’s clear that at least one component of all this inter- personal semantic judging involves acceptance, meaning not some touchy-feely emotional affirmation but actual acceptance or rejection of someone’s bid to be regarded as a peer, a member of somebody else’s collective or community or Group.9

Fostering Shared Meaning Through Collaboration. To create a lexicon that better differentiated the types of solutions within an emergent category, DonorsChoose, with Whiteboard Advisors’ support, began to brief organizations about their work and listen to the concerns that could inform a new definition of school-based crowdfunding. The leadership and members of the School Superintendents Association (AASA), the Association of School Business Officials International (ASBO), the National School Boards Association (NSBA), and regional organizations such as NEOLA (formerly known as the North East Ohio Learning Associates) provided guidance about their needs and how the safeguards that were part of the DonorsChoose platform could be read into language and practices that were cognizable to school administrators. Unsurprisingly, we learned that there was a great deal of optimism for school-based crowdfunding once the connection was made. For example, administrators helped tighten the relation between their need for “internal controls” (a requirement of sound grant management) and the platform’s back-end technology. The technology could help administrators inventory and document materials and supplies, ensure compliance with fiscal rules, and provide transactional transparency with 74 RETHINKING K–12 EDUCATION PROCUREMENT detailed reporting—all necessary aspects of sound internal control practices. With this perspective, administrators began to recognize and speak about school-based crowdfunding as a practice they could deploy to mitigate risk. School officials were also happy to discuss the academic opportunities of school-based crowdfunding when they analogized it to targeted microgrants. Through this lens, the strategic benefits for both administrators and teachers became clearer. A school-based crowdfunding platform would help both advance a district’s academic objectives and animate teachers’ and students’ creative energy in their classrooms. The collaboration resulted in a model school board policy that cog- nizably defines school-based crowdfunding for administrators—and, ultimately, policymakers. This advances the educational objectives of a district and outlines implementation guidelines that mitigate risk. It clarifies when teachers need the prior approval of principals or superintendents and when, if proper internal controls are in place, they are authorized to request donations without prior approval. The model policy clearly states—in response to the concerns in Washington state and from other state auditors—that staff members may not receive cash payments and that the materials and supplies become the property of the agency, consistent with existing internal control practices for materials and supplies. Through the collaboration, AASA, ASBO, NSBA, NEOLA, and other organizations now had a way to differentiate school-based crowdfunding from other (riskier) services, and they built on the meaning. They made it theirs. AASA hosted webinars for its members, ASBO developed a tool kit resource for its members to learn more and build on the foundational work, and NEOLA (a consultancy for school boards across the nation) incorporated some of the language into its popular school board policy and administrative procedures resources. Each organization, in its own way, offered a clarifying model of theschool-based practice and elucidated how it can benefit staff, students, and its members. Within a year, crowdfunding evolved from a practice loaded with lia- bility and risk for school administrators to one that school officials could use to animate the creativity of their teachers and the generosity of donors across the nation (and around the globe). A common understanding of terms and practices emerged because stakeholders chose to establish DAVID DESCHRYVER 75 recognizable terms of art and definitions for the practice with the support of DonorsChoose and Whiteboard Advisors. As for Washington, the state auditor’s guidance is still good guidance. School staff may not accept cash on behalf of their agency. How- ever, more districts now recognize that school-based crowdfunding can flourish within the constraints of state rules and established internal control practices.

Lesson Learned: Shared Meaning Requires Translating Products and Services to Terms That Resonate with Educators and Policymakers. Over the past decade, crowdfunding has become an estimated $10 billion market. That may impress investors, but it’s more likely to cause some eye- brow raising by school officials who must be risk averse to protect public funds. Entrepreneurs must demonstrate not only that their programs can improve the way school officials carry out their work but also that they help officials with compliance and don’t impose unnecessary risk on the agency. The language describing the work is better understood and reso- nates more when it is developed in partnership with educators, administrators, and community peers.

Case Two: What Does “Evidence-Based” Mean, Anyway?

In the parable of the blind men and the elephant, the blind men try to deduce what an elephant looks like by touch. When they describe the animal to each other, their accounts differ, because each is touching a different part. One describes a hairy tail, another describes asnake-like trunk, and another has a totally different take based on an ear. Everyone is talking about an elephant, but they are not talking about it consistently or comprehensively. As in the parable, school officials and vendors have, in recent years, begun to fragmentarily describe evidence-based research requirements of ESSA. Many describe what’s right in front of them: whether programs and services have been substantiated by prior research studies. Few, however, capture the totality of the requirements’ objective: to ensure that grantees invest in programs that will benefit their students and that their investment 76 RETHINKING K–12 EDUCATION PROCUREMENT strategy is part of a program of continuous improvement. Without such a commonly held board perspective, the requirement can do (and has done) damage to long-standing partnerships and jeopardized the predictability of contracts between schools and vendors. But first, let’s clarify the requirements and how they frustrate vendors and school officials alike.

Patchy Interpretations of New Rules Such as the “Evidence-Based” Requirement Can Hinder Procurement. ESSA requires schools that continually fail to meet the state’s academic achievement standards, have per- sistently underperforming groups of students, or (for high schools) have a graduation rate of 67 percent or below to submit a plan for improvement to the state. If the state approves the plan, it gives the district a portion of the state’s share of ESSA funding, the School Improvement Funds (SIF), to spend on the improvement strategy. Schools that receive SIF may use those funds only for “evidence-based” activities. ESSA defines anevidence-based activity as an activity, strategy, or intervention that demonstrates a statistically significant effect on improving student academic or other relevant outcomes. That effect must be founded on evidence-based research that meets one of three standards: “strong,” “moderate,” or “promising.” Officials may not use SIF to invest in a program that lacks one of the three levels of evidence—even if they love the program and it works well for school staff, parents, and students. In some states, the restriction extends beyond ESSA. Some states incorporate the rule into their own state grants. In Nevada, for example, the state sets aside a portion of funding for schools with underperforming English learners and students who qualify for free and reduced-price lunch.10 Those funds may be used only for activities that meet the evidence-based research standards identified in ESSA. What’s more, many states encourage evidence-based investment practices for all federal and state funds—but don’t go so far as to technically require them. This really muddies the water and creates confusion in applying the requirement. According to a 2018 Whiteboard Advisors survey of 84 districts across 13 states, 63 percent of responding districts said their state agency applies ESSA’s evidence-based requirements to its state programs and related DAVID DESCHRYVER 77 training. Some districts report that using any federal funds (not just SIF) requires a vendor to demonstrate its evidence base. Other districts cited the evidence requirements as a condition of state funding. Still others claimed that they could only purchase programs listed on private third-party websites that review the programs’ evidence-based research or alignment with standards, even though standard alignment has nothing to do with evidence-based research.11 This mishmash of interpretation substantially affected procurement. School officials told vendors they could not renew contracts or do business with them unless they met the new research criteria. It didn’t matter if they could use nonfederal funds or if prior work and relationships were well regarded and good for students. It all depended on the myopic view of “compliance.” It depended on the part of the elephant the school official last touched. School officials also shared in the frustration.Evidence-based requirements assume there’s a wealth of applicable studies out there and officials have the time and capacity to use them well. Neither assumption is right. According to a 2018 RAND Corporation survey, educators are troubled by the requirement because existing research is limited and often not relevant.12 Where it does exist, there is insufficient guidance to help school officials apply it to their local context, and if they want a repository of useful research, districts have to build it themselves.13 As a result, teachers rely more frequently on their peers’ experience or third-party sites referred to them by state officials to review a program than on valid research. According to one public official, searching for a stamp of approval is replacing, not enabling, productive conversations about evidence-based investment strategies. “There are many strategies that are effective that have not received an evidence-based‘ ’ stamp,” points out a school official in a district that enrolls about 5,000 students. “Please be satisfied with the professional judgement of educators in choosing methods for increasing achievement/SEL.”14

Getting on the Same Page About the “Evidence-Based” Requirement. Bridging the divide between divergent interpretations of new policy started with clarifying the underlying purpose of the requirements. That meant working with clients to unpack the components of their service and 78 RETHINKING K–12 EDUCATION PROCUREMENT align them to existing research. In some cases, the vendor’s program ben- efited from prior research already published with partner districts. More often, the program could cite that research about an activity that aligned with a component of the program’s model. For example, if the program delivered a reading intervention and included tutoring, that tutoring component could align to well-studied, research-driven practices. If there was alignment, that component of the program could arguably comply with the technical requirements of the law. Armed with an understanding of the regulations’ intent and structure— and data to support their position—the conversation between the service provider and school officials could address the benefit of the work in a school’s setting and how they could engage in research and improvement together. The specific contextual benefits are the real value of the partnership, more so than distal academic research that does not likely speak to the unique needs of students and staff in the schools. When conversations moved from evidence-based research to the value for districts and implementing a program of continuous improvement as a key part of procurement, the partnership advanced. While working with individual vendors and districts, Whiteboard Advi- sors began facilitating structured conversations—not just a one-time event—in which school officials and researchers can come together to discuss a practical evidence-based education marketplace. The conversations aim to develop agreed-on guidelines for using evidence, spanning from small pilot project work to district-wide rollouts. With such guideposts, the market can respond accordingly, and there will be more predictable conditions for procurement and contract renewal, benefiting school officials and vendors alike. This work is necessary because new technology, research, and insights can always improve student learning and school operation. The challenge is to help put all stakeholders into a position to quickly translate the potential benefits to practice and improve enabling policies and rules along the way.

Lesson Learned: Defining a Term Isn’t Enough; Creating a Shared Understanding Requires Intentionality. The evidence-based research example demonstrates that creating mutual understanding is necessary even when there are explicit statutory language and definitions that DAVID DESCHRYVER 79 presumably clarify what it means to comply with and carry out legal and regulatory requirements. Compliance, however, isn’t just a paper exercise. It’s a behavioral science that demands care and attention. Left unattended, it can cause plenty of (unwelcome) surprises in purchasing decisions and practices. Whether new terms emerge from the consumer side or from regulation, the meaning must be constant.

There Is Always a New Thing

Districts get a new superintendent about every four years. The national framework for school accountability is up for reauthorization every five years. States buy and refresh their standards and instructional material every five to seven years. Experienced staff take on new roles or retire, and new staff, with new expectations about how things should work, come on board every year. Stability is uncommon. There is always a new thing. Sometimes, as with crowdfunding, a new idea comes from the marketplace and requires introduction and orientation to the rules-bound universe of school funding. Sometimes, as with the evidence-based requirements, a new rule changes expectations and contractual requirements. This transition requires mediation to get all parties in agreement about what’s necessary and efficient to carry out the new responsibility. In every case, the change demands attentive and careful mediation. As new ideas and regulations arise, entrepreneurs and school leaders must always work to create shared meaning of the terms and practices that drive compliance, procurement, and implementation. Change is constant, and the next few years will not allow us, the education community, to forget that. Consider, for example, blockchain in education. This mind bender will require serious reconciliation with current rules and practices. What are the common terms and shared expectations? What’s the delineation of responsibility for a technology built independent from government regulation? The answers are not yet clear. That’s the work before us. 80 RETHINKING K–12 EDUCATION PROCUREMENT

Notes

1. Ed Kirby, “Breaking Regulatory Barriers to Reform,” in The Future of Educational Entrepreneurship: Possibilities for School Reform, ed. Frederick M. Hess (Cambridge, MA: Harvard Education Press, 2008). 2. Learning Counsel, Market Spend Analysis & 2017 Digital Curriculum Strategy Sur- vey, 2017. 3. Jennifer R. Morrison et al., Fostering Market Efficiency in K–12 Ed-Tech Pro- curement, Johns Hopkins University, Center for Research and Reform in Education, September 22, 2014, https://digitalpromise.org/wp-content/uploads/2016/02/DP_ ImprovingEdTechPurchasing_FullReport.pdf. 4. Rob Reiner, The Princess Bride (Los Angeles: 20th Century Fox, 1987). 5. “Best practice—and state law (RCW 39.58.080)—require local governments to have all funds directly deposited into their Public Deposit Protection Commission (PDPC)-approved bank account.” Office of the Washington State Auditor, “Follow Best Practice, State Law When Using Third Party Receipting Vendor,” https://sao.wa.gov/ follow-best-practice-state-law-when-using-third-party-receipting-vendor/. 6. Wash. Rev. Code § 43.09.240 (2018). 7. Edmonds School District, “6102P Procedures for District Fundraising Activities,” http://agenda-hrtf.edmonds.wednet.edu:8085/docs/2018/SBA/20180109_380/5954_ Procedures%206102P%20District%20Fundraising%20Activities.pdf. 8. Dictionary of American Regional English, website, https://www.daredictionary. com. 9. David Foster Wallace, Consider the Lobster and Other Essays (New York: Little, Brown and Company, 2006). 10. For example, in Nevada, state lawmakers created an account that provides an additional $1,200 for eligible free and reduced-price lunch and English learner students in the state. At least 90 percent of those funds have to go to certain activities. If the district invests in an academic intervention, as defined in that state law, that intervention must meet one of the top three evidence-based standards, as defined in the Every Student Succeeds Act. 11. Whiteboard Advisors, “ESSA Inspired Evidence-Based Scrutiny of Investments,” December 2018, https://www.surveymonkey.com/stories/SM-XQKG7MM/. 12. Laura S. Hamilton and Gerald P. Hunter, “Where Do Educators Turn to Address Instructional and Behavior Challenges?,” RAND Corporation, 2020, https://doi. org/10.7249/RR2575.9-1. 13. Hamilton and Hunter, “Where Do Educators Turn to Address Instructional and Behavior Challenges?,” 3. 14. National Association of Federal Program Administrators et al., The 2019 ESSA Title IV-A Survey, July 2019, https://www.aasa.org/content.aspx?id=43779. Lessons from Louisiana’s Efforts to Create a New Marketplace for High-Quality K–12 Curricula and Professional Development

JULIA KAUFMAN AND REBECCA KOCKLER

y the late 1990s, nearly all states had adopted academic content B standards for mathematics and English language arts (ELA) for kindergarten through 12th grade in response to federal mandates. Those content standards have only become more challenging over time, with many states adopting Common Core and similar, more rigorous standards over the past decade and more challenging year-end assessments to measure mastery of standards.1 Raising the bar for what students should know is good. Too many children—especially poor children, black children, and children from various other minority groups—have been victim to low expectations that keep them from achieving their full potential.2 But, as many researchers have documented, standards alone have not transformed teaching and learning.3 Teachers do not appear to have a deep understanding of their state standards,4 and some analyses suggest that new standards adopted in most states have not raised student achievement.5 Teachers’ instruction and thus student learning do not appear to have improved because of high standards and assessments alone. Critics of the evolution of standards-based policies argue that state and district instructional systems, not teachers, must improve for standards to support educational improvement. These critics argue that for standards-based reforms to work as intended, those overseeing instructional systems must focus more on aligning all the messages teachers receive about their instruction and improvements in the tools and resources they are provided to help them make instructional choices every day.6

81 82 RETHINKING K–12 EDUCATION PROCUREMENT

Like lead actors in a play, teachers’ actions are both motivated and constrained by the tools and supports they receive in their instructional system, which includes the state in which they teach, their school system, and their school. Actors depend on a well-written script and clear, aligned messages from their producers, director, and cinematographers about their motivation and how they should perform each small scene. However, in most instructional systems across the US, teaching is not as straightforward as acting is. Unlike actors, teachers are often given several “scripts” all at once and are expected to provide their own direction and production to decide what to teach on a given day and for a given lesson. One script is the standards and learning objectives that often come from both their state and school system. Another script is the set of textbooks and learning materials they are given to use. A third script is given through the professional development and training they receive. Still another script is the assessments their students must master. If these scripts were aligned with one another, it might be a matter of just stitching scripts together or inserting the elements of one script into another. But the scripts states and school systems give their teachers typically do not provide commensurate information. Commonly used textbooks and learning materials are not clearly aligned with most state standards.7 Professional development vendors may claim standards alignment, but they typically focus on their own learning theories and products that may not be easily reconciled with standards and even more rarely with the curricula teachers use. Benchmark and end-of-year assessments may test for various things that are not present in standards, learning materials, or professional development. Often these benchmarks take into account a teacher’s evaluation, fueling the focus on yet another input for their instruction. Teachers are then left having to do it all—act in a play that they also direct and produce—using a lesson plan they have pieced together on their own from their many scripts. In this chapter, we will describe efforts in one state—Louisiana— to provide teachers with more coherent scripts and messages for their instruction. In particular, we’ll discuss the three main actions the Louisi- ana Department of Education (LDOE) has undertaken to attempt to scale standards-aligned instructional improvement: (1) identifying standards- aligned instructional materials and professional learning supports, JULIA KAUFMAN AND REBECCA KOCKLER 83

(2) incentivizing their adoption and effective implementation, and (3) monitoring their use. LDOE’s approach has led to some successes—and some challenges— and it may not be the right approach for all state contexts. Nonetheless, some early results of LDOE’s efforts suggest the possibility of a new role for states as innovators that create marketplaces for high-quality, standards-aligned instruction. In doing so, states may be able to develop a more coherent instructional system and, if not one script, a set of aligned scripts and messages to teachers about what to teach and how to teach it. This chapter draws on both Rebecca Kockler’s experience as assistant superintendent in Louisiana from 2012 to 2018 and RAND research Julia Kaufman and her colleagues conducted over the past several years. Since 2015, RAND has been investigating policies intended to support state standards implementation across the United States and in Louisiana, and RAND has published several studies that examine the implementation and outcomes of Louisiana’s education policies across the state’s school systems.8 The RAND studies have highlighted perceptible shifts in the curricula teachers are using across the state and the support they are receiving, although these studies also identified considerable implementation challenges that Louisiana’s policies presented to schools and teachers. That said, the latest results from the National Assessment for Education indicate that Louisiana’s rate of improvement since 2009 in all subjects is greater than that of the nation, and it ranks among the top 10 states for improvement over the past decade.9 This progress suggests that Lou- isiana’s policies may be worth learning more about, especially if those policies are potentially related to improvements in teaching and learning across the state.

Dilemmas for School Systems That Want to Be More Standards Aligned

Let’s start with the problem that LDOE strove to solve: how to help students master state standards by providing more coherent information to schools and teachers about what textbooks and training lead to more standards-aligned instruction. The roots of this problem are probably most vividly described from the viewpoint of school leaders in many 84 RETHINKING K–12 EDUCATION PROCUREMENT school systems across the US, where curriculum adoption laws typically have remained unchanged for decades.10 Here are five realities school systems and leaders that want to be more standards aligned often face.

Unclear Information About the Alignment Between Standards and Curricula. When standards in a state change, district and school leaders typically have little information or resources to help them determine which existing curricula align well with the new standards. Imagine you’re a school system leader in a medium-sized school district. Your current elementary school math curriculum is outdated, and you know it is not well aligned with the new math standards your state has adopted. As most school districts in your state do when thinking about adopting a new curriculum, you assemble a committee of educators from your school to provide a recommendation to your board about which new math curriculum series to adopt. Your committee starts with a scan of the most popular curricula among those on the list of state-approved textbooks. Most appear relatively well aligned to your state’s standards. At least, that is the claim the publishers make on their websites. Your committee doesn’t have time to do an in-depth review of textbooks, but it tries to assess a few top titles to see if they align with the standards. The committee concludes that all the titles it has reviewed provide at least some evidence of alignment with standards. However, the committee’s conclusions are not based on clear, dependable information about which curricula are aligned with state standards and which are not.

Local Marketing. To make curricula more appealing to a particular local market, publishers will sometimes hire individuals who were educators in a given region to help them sell textbooks. Imagine that a well-admired retired school principal from your district started working for one of those publishers and she met with your committee to walk you through their products. What she said made the curricula from her publisher look appealing, and the committee was impressed that she took the time to come to your school district to talk about their curricula. These local employees of publishers operate as a trusted resource that can convince a school system to choose one curriculum over another. JULIA KAUFMAN AND REBECCA KOCKLER 85

Outdated Curriculum Adoption Laws. In most states across the US, curriculum adoption laws have remained unchanged for decades, making it difficult for school systems to adopt untraditional or digital resources or texts. You might imagine that the curriculum adoption committee at your school has considered some curricula that are available for free online. One of those curricula has a reputation for being closely aligned with your state’s standards, and teachers can even look up specific standards and tie them to lessons. You like that idea. But, your state’s laws for textbook adoption are oriented around adopting physical textbooks, not online curricula. Your state has given you funds for textbook adoption, but the funds don’t cover printing online materials, which would be necessary because computers are not available for every student. Your textbook adoption funds also cover professional development through the publisher from whom you adopted the textbooks. But professional development providers for online materials are typically provided by a separate external vendor, so you can’t use your textbook funds for that professional development. Your school system cannot afford to spend extra money to print online materials and bring on an external vendor whose expenses cannot be paid through state funding. Thus, your committee decides against adopting an online textbook. Finally, your committee settles on a curriculum and recommends it to the school board, and the board approves it for adoption. The next challenge is getting teachers to use the materials that have been adopted.

No Standard Procedures for Implementing New Curricula. There is no manual that outlines the do’s and don’ts of implementing a new curriculum, despite that textbook adoption can be extremely demanding on students, teachers, and those supporting teacher instruction. Imagine that the math curriculum your school district has adopted is a significant shift in how math content is taught compared to your last math curriculum. A lot of the recommended strategies and pedagogical approaches in the curriculum are fairly different from what teachers used before, and some content is even a little different at each grade level. As in the past, your school district asks teachers to attend a five-day professional development session in the summer to learn more about the new materials and how to use them. After the training, a few teachers tell you 86 RETHINKING K–12 EDUCATION PROCUREMENT they are anxious about using the new curriculum because they don’t totally understand all the learning activities in it, and some ask what was wrong with the old textbooks. You have never told teachers before what textbooks to use in the classroom, and your district leaders have not encouraged you to do that now. So, you just ask teachers to use the new curriculum in whatever way they think makes sense for their students, so long as they are addressing state standards and their benchmark assessments—and their teacher evaluation rubrics. Even then, you are not totally sure what teachers will do with that advice, since the state-approved teacher evaluation rubric you use doesn’t focus at all on math standards or curricula, and you aren’t sure about the standards alignment of the benchmark assessments you’ve been using. The absence of existing guidelines for curriculum implementation makes it impossible to know if you’ve chosen the right path with the messages and supports you’ve given teachers.

Difficulty in Scaling Curriculum-Specific Professional Development. While adopting a few curricula presents a significant learning challenge for teachers, school systems do not have many options for bringing in intensive, curriculum-aligned professional learning. Imagine again that you’re a school leader in a school with a newly adopted math curriculum. When you walk through classrooms several months after school has started, you see that many teachers haven’t even unpacked the new curriculum; it is in boxes in the corners of classrooms. A few teachers are trying to use the new texts, but they tell you they are struggling to understand how to use them. One of your longtime instructional coaches for math says he is thinking of retiring next year because he doesn’t think he can learn to teach in the “new way” the new textbooks require and will not be able to provide the coaching teachers need. Further, last year, before you adopted a new curriculum, you hired a professional development vendor to provide sessions this year on how to deliver more standards-aligned mathematics instruction by integrat- ing many new student tasks. However, the vendor wasn’t planning to tie that professional development specifically to the textbook you have since adopted. Now you are worried that professional development will just confuse teachers who are already being asked to teach multiple new tasks JULIA KAUFMAN AND REBECCA KOCKLER 87 through their new curriculum. The textbook publisher has offered to come back a few times in the winter to provide some half-day sessions on the curriculum, but you aren’t sure whether such a small number of professional development sessions will help teachers learn to use the new materials and get comfortable with them. Furthermore, the sessions are expensive, so you don’t think you can afford them on top of the professional development you’ve already purchased. They also can’t come on the days you have already set aside for teacher training. You don’t know what else will encourage teachers to use the new materials. You tell yourself they just need time. Maybe another professional development from the publisher next summer will help, but you don’t have the resources you need now.

State Solutions for Identifying, Incentivizing, and Monitoring Quality

The hypothetical school leader mentioned above was in the same situa- tion as many school leaders in Louisiana were before 2012. Most popular textbooks were actually not closely aligned with state standards, despite their claims to the contrary. Big publishers often hired retired school system leaders to sell materials to school systems, which is a big advantage in a system in which school leaders tend to trust one another more than they trust outsiders. Old textbook laws were not created to accommodate newer online materials. There were no rules of thumb for how new textbook adoptions should be implemented or how much teachers should be expected to use a newly adopted textbook. Lastly, despite teachers’ need for intensive curriculum-specific professional development in the first year of adopting a new curriculum, little existing professional development was curriculum specific, andpublisher-provided professional development was not scalable at the level that could meet teachers’ needs. But the central problem—the problem that all these issues touched on—was that there was no marketplace explicitly oriented around quality. There were no clear criteria identifying high-quality materials or professional development, no clear incentives for adopting high-quality materials or training, and no systems for monitoring use of high-quality materials. Faced with all this incoherence in the system, the state decided the only 88 RETHINKING K–12 EDUCATION PROCUREMENT recourse was to create a marketplace oriented around quality for itself. We will describe the process LDOE undertook to do this work. Additionally, we’ll share data suggesting that Louisiana succeeded in reshaping the curriculum materials market. The market for professional development has been—in the words of one state official—the “hardest nut to crack.”11 But the state is making strides in that realm too. Then, we’ll consider what Louisiana’s experiences could offer other states—and school systems— that want to make standards-based instruction a reality.

Solution 1: Identify Quality Materials and Training. Before 2012, Lou- isiana maintained a list of approved textbooks from which school districts could choose. One Louisiana state official who was interviewed for RAND’s research described a process whereby approved texts were chosen by a committee that met every few years. During the committee meetings, they listened to publishers who came in to tout their materials, and then they recommended a list of those materials, although state officials could not say much about the rigor of that process. LDOE proposed a new process using rubrics adapted from those developed by Student Achievement Partners and Achieve to examine alignment of instructional materials with college- and career-ready state standards. With the new process, texts could be reviewed any time and would be rated according to tiers: Tier I meant that a textbook exemplified quality, Tier II meant that the text approached quality, and Tier III meant that the textbook didn’t represent quality. The state did not require school systems to pick a Tier I material for adoption. Instead, the state told school systems what they considered high quality based on their reviews and kept that list of reviewed materials up-to-date. The new process brought concern among some publishers that had shaped the previous legislative process for textbook adoption and felt like they may not fare as well under the new policies. To try keeping that concern at bay, the LDOE leveraged allies in school system leaders willing to defend the new process with the legislature. Specifically, LDOE conducted a set of pilots with individual districts that agreed to use—or had already been using—the materials the state deemed as Tier I. Districts that piloted those materials were seeing some good effects and spoke to other districts about them. LDOE also found allies in the state legislature by convincing JULIA KAUFMAN AND REBECCA KOCKLER 89 policymakers of the merits of its review process. Local educators made the case that the new process provided a real service that both empowered them as local decision makers and allowed them to more quickly and effi- ciently find and usehigh-quality materials. Finally, in 2015, the Louisiana legislature and the Board of Elementary and Secondary Education updated textbook adoption laws and policy regulations to match the review process LDOE had proposed, and LDOE began to regularly post textbook reviews from Louisiana teachers online. In the process of doing these reviews, the state saw a gap in high-quality textbook materials for ELA and decided to develop its own curriculum to fill that gap: Louisiana Guidebooks, which is an open online curriculum LDOE designed with LearnZillion to align closely with Louisiana’s ELA standards for third through 12th grade. Today, hundreds of reviews of mathematics, ELA, science and social studies, and early childhood textbooks are available on the LDOE website.12 Fewer than 20 of those textbook series have been reviewed as Tier I for ELA or mathematics. For professional development, identifying high-quality training has been more challenging than it has been for curriculum materials. Curric- ulum materials are immutable. Although they can be used various ways in the classroom, their content does not change. In contrast, good professional development depends on the people who provide it. While trainings may be scripted, typical documentation just includes an outline and materials provided to participants. Furthermore, facilitators can change how a training is provided and experienced in large and small ways. Despite these challenges, LDOE has created a catalog of vendors for recommended professional development. The catalog includes only vendors that intend to help teachers use Tier I curricula. Other criteria considered for including a vendor are whether the training is intended to enhance teachers’ content knowledge and whether it provides them with opportunities to practice their skills. The 2018–19 Vendor Professional Devel- opment Course Catalog included 19 vendors for ELA curricula, 26 for mathematics, and a much smaller number for science and social studies.13

Solution 2: Incentivize Quality Materials and Training. Just providing recommendations for good curricula and professional development may not compel schools or teachers to adopt that curriculum or participate in 90 RETHINKING K–12 EDUCATION PROCUREMENT that training. As one state official interviewed for RAND’s research put it, many districts were using Tier II or III curricula and “still eating at the one-star restaurant, maybe because they’re cheaper or they’re more familiar with them.”14 So, Louisiana took the next step and negotiated textbook pricing directly with Tier I publishers, completing the necessary paperwork for procurement itself. Districts that wished to purchase those curricula just had to request to be included in the state contracts. In an interview, one state official commented that the contracts encouraged more uptake of Tier I materials “immediately.”15 Other incentives the state provided were free regularly offered administrator and teacher training for using Tier I curricula. In particular, state-provided regional and statewide professional development sessions offered throughout the year included multiple sessions devoted to using Tier I curricula, particularly Eureka Math and the Louisiana ELA Guidebooks. In 2018–19, the state put in place even stronger incentives compelling use of Tier I curricula. To receive school improvement funding, schools have to submit an improvement plan that includes use of Tier I curricula and curriculum-aligned professional development. For 2019–20, any schools with one or more subgroups receiving an F letter grade for at least two years will be required to submit an improvement plan that includes Tier I curricula and training.16

Solution 3: Monitor Quality Materials and Training. The accountability incentives put in place over the past few years for schools needing improvement are one way of monitoring uptake of Tier I curricula and curriculum-aligned professional development. But while requiring struggling schools to adopt Tier I materials and training will support higher adoption of Tier I materials, it does not guarantee that those materials will actually be used. When school systems in Louisiana first began adopting Tier I materials, state officials who visited those school systems noticed that the adopted materials were frequently not used in the classrooms they observed. They even saw newly adopted materials still packed in boxes in the corner of a classroom after the school year started, although those materials had been brought into the school months before. Thus, to ensure the incentives to adopt high-quality materials were having the intended impact, the state JULIA KAUFMAN AND REBECCA KOCKLER 91 played a stronger monitoring role by physically going into schools and classrooms before school started each year to ensure that materials were ready to go. Another approach the state has taken is to build capacity for teachers to monitor and support one another from within. Specifically, the state has begun to provide teacher leaders with the tools to monitor uptake of standards-aligned curricula and support use of those curricula. Louisi- ana began cultivating a group of teacher leaders in 2013. These teachers, which the state asks the district to appoint, number in the thousands. They receive monthly newsletters, conduct curriculum reviews, participate in webinars, and attend quarterly trainings and an annual teacher leader summit. Louisiana has also recently begun offering a more intensive training for teachers called a “content leader training.” Teachers receive a stipend from their school to attend the nine-day content leader training, which focuses on Louisiana standards and curricula. Those teachers are then expected to return to their school systems to provide support and information to other teachers regarding what they learned. To date, over 1,000 teachers across Louisiana have completed a content leader training or are planning to complete the training during the 2019–20 academic year.

Successes Thus Far

By several measures, the work Louisiana has done to identify, incentivize, and monitor uptake of high-quality curricula and professional development has been successful. For the past several years, RAND has conducted several teacher surveys on use of curricula, professional development, and teachers’ instruction that included nationally representative samples of teachers and state-representative samples in Lou- isiana. In its preliminary analyses of those survey findings, RAND was surprised by how Louisiana teachers were standing out from the rest of the nation in impressive ways. First, in response to a 2017 teacher survey, nearly half of all Louisiana teachers reported using Eureka Math, a Tier I mathematics curriculum.17 More recently conducted surveys—based on data published in spring 92 RETHINKING K–12 EDUCATION PROCUREMENT

2020—suggest even higher uptake of Eureka Math, Louisiana’s ELA Guidebooks, and other Tier I curricula in Louisiana. Second, a 2016 survey asked teachers about how much their professional development focused on topics such as helping students make sense of mathematics problems, solve math problems in real-world contexts, develop a conceptual understanding of mathematics, use textual evidence to make inferences drawn from texts, and build reading skills through using complex texts. These are all topics emphasized a great deal in college- and career-ready standards most recently adopted across every state in the United States, including Louisiana.18 About two-thirds of Louisiana teachers reported a moderate or high focus on all these standards-aligned topics in their professional development, compared to less than half of other US teachers.19 This use of Tier I materials and standards-aligned professional development could be paying off. Our surveys have also demonstrated that Louisiana teachers appear to know more about their standards than other US teachers do, and Louisiana teachers also have reported more student engagement in some standards-aligned classroom practices than other teachers have. While data based on survey self-report can suffer from response biases and cannot tell us what is actually happening in classrooms, these differences in reported use of materials, training, knowledge, and instructional practice in Louisiana, compared to the rest of the US, are consistent and significant across numerous surveys.

What States and School Districts Across the US Can Learn from Louisiana

RAND’s findings suggest that states can reshape their markets so that teachers use more standards-aligned materials and participate in more standards-aligned professional learning opportunities. That said, differing state contexts could certainly play a role in what change is possible, and what has worked for Louisiana may not work elsewhere. When states do not have as much flexibility to act because of political stalemates and other issues, school systems themselves may be able to take on at least some of what Louisiana has done at a smaller scale. The following is a short list of JULIA KAUFMAN AND REBECCA KOCKLER 93 the possible actions for states and school systems to consider, based on Louisiana’s work.

Define and Identify High-Quality Instruction Through Curricula and Training. Definitions of high-quality“ instruction” might vary greatly from school system to school system and from teacher to teacher in each state. Yet, the Louisiana example suggests that states and school systems might be able to shift how educators think about high-quality instruction by defining what constitutes high-quality curricula and then organizing professional development to support productive use of those curricula. Even beyond curricula and professional development, states could also consider what other resources and tools—or “scripts” directing instruction—they make available in their state system and the alignment of those resources and tools with their definition of quality. Other ongoing work that Louisiana is doing to align the whole system to reflect its definitions for quality includes revising teacher evaluation rubrics to align better with use of standards-aligned curricula and providing a pool of benchmark assessment items that reflect LDOE’s definitions of quality.

Incentivize Uptake of Texts and Training Identified as High Qual- ity. States should consider what levers are present in their system that can encourage using curricula they have defined as high quality. State contracts for the curricula and training considered to be high quality is one potential action. In addition, if any training provided at the school or school district level focuses on such curricula, each training will serve as a suggestion and incentive to use those materials more. If teachers see how high-quality curricula can address the dilemmas they face in the classroom to support student learning and student needs, they will use those curricula more.

Monitor and Build Capacity to Monitor Use of High-Quality Curric- ula. Accountability can be one way to monitor adoption. But accountability regulations are a blunt instrument that cannot mandate how much or in what ways teachers use curricula. States and school systems should consider how to build capacity for educators to monitor and support one 94 RETHINKING K–12 EDUCATION PROCUREMENT another. Statewide trainings are the route that Louisiana has taken to do this. Teacher preparation institutions and micro-credentialing systems could be other ways to create this internal capacity at scale.

Conclusion

Louisiana still has work to do. To date, measures of student achievement have tracked some potential growth in student achievement but also suggest persistent achievement gaps among students of different ethnicities and incomes.20 To address those achievement gaps, Louisiana is now considering how to evaluate the quality of remediation and supports for struggling students within instructional materials. In addition, Louisiana is still puzzling over how to scale high-quality professional development from external vendors that typically lack the capacity and curriculum focus that Louisiana needs. Yet, by several measures, Louisiana appears to have succeeded in creating a marketplace for quality K–12 curricula and professional development, which is a big first step toward high-quality teaching and learning.

Notes

1. Andrew Porter et al., “Common Core Standards: The New U.S. Intended Cur- riculum,” Educational Researcher 40, no. 3 (2011): 103–16; and William H. Schmidt and Richard T. Houang, “Curricular Coherence and the Common Core State Standards for Mathematics,” Educational Researcher 41, no. 8 (2012): 294–308. 2. Ronald F. Ferguson, “Teachers’ Perceptions and Expectations and the Black- White Test Score Gap,” Urban Education 38, no. 4 (2003): 460–507; and David N. Figlio, “Names, Expectations and the Black-White Test Score Gap,” National Bureau of Eco- nomic Research, March 2005, https://www.nber.org/papers/w11195. 3. Cynthia E. Coburn, “Collective Sensemaking About Reading: How Teachers Mediate Reading Policy in Their Professional Communities,” Educational Evaluation and Policy Analysis 23, no. 2 (2001): 145–70; Cynthia E. Coburn, “Beyond Decoupling: Rethinking the Relationship Between the Institutional Environment and the Class- room,” Sociology of Education 77, no. 3 (2004): 211–44; and David K. Cohen and Deborah L. Ball, “Policy and Practice: An Overview,” Educational Evaluation and Policy Analysis 12, no. 3 (Fall 1990): 233–39. JULIA KAUFMAN AND REBECCA KOCKLER 95

4. Julia Kaufman et al., “Changes in What Teachers Know and Do in the Common Core Era: American Teacher Panel Findings from 2015 to 2017,” RAND Corporation, 2018, https://www.rand.org/pubs/research_reports/RR2658.html. 5. Zeya Xu and Kennan Cepa, “Getting College-Ready During State Transition Toward the Common Core State Standards,” Teachers College Record 120, no. 6 (2018): 1–36; Tom Loveless, “2014 Brown Center Report on American Education: How Well Are American Students Learning? Part III: A Progress Report on the Common Core,” Brookings Institution, March 18, 2014, https:// www.brookings.edu/research/a-progress- report-on-the-common-core/; and Tom Loveless, “2015 Brown Center Report on Amer- ican Education: How Well Are American Students Learning? Part II: Measuring Effects of the Common Core,” Brookings Institution, March 24, 2015, https://www.brookings. edu/research/measuring-effects-of-the-common-core/. 6. Laura S. Hamilton, Brian M. Stecher, and Kun Yuan, “Standards-Based Reform in the United States: History, Research, and Future Directions,” RAND Corporation, December 2008, https://www.rand.org/content/dam/rand/pubs/reprints/2009/ RAND_ RP1384.pdf; and Cynthia E. Coburn, Heather C. Hill, and James P. Spillane, “Alignment and Accountability in Policy Design and Implementation: The Common Core State Standards and Implementation Research,” Educational Researcher 45, no. 4 (2016): 243–51. 7. Kaufman et al., “Changes in What Teachers Know and Do in the Common Core Era.” 8. V. Darleen Opfer, Julia Kaufman, and Lindsey E. Thompson, “Implementation of K–12 State Standards for Mathematics and English Language Arts and Literacy: Find- ings from the American Teacher Panel,” RAND Corporation, 2016, http://www.rand.org/ pubs/research_reports/RR1529.html. 9. Louisiana Department of Education, “Louisiana 2019 NAEP Results Summary,” November 3, 2019, https://www. louisianabelieves.com/docs/default-source/links-for- newsletters/louisiana-2019-naep-results-summary.pdf?sfvrsn=bd999a1f_2. 10. Chiefs for Change, “Choosing Wisely: How States Can Help Districts Adopt High-Quality Instructional Materials,” April 2019, https://chiefsforchange.org/wp- content/uploads/2019/10/CFC-ChoosingWisely-FINAL-1.pdf. 11. Interview with a state official. 12. Louisiana Department of Education, “Instructional Materials Reviews,” https:// www.louisianabelieves.com/academics/ONLINE-INSTRUCTIONAL-MATERIALS- REVIEWS/curricular-resources-annotated-reviews. 13. Louisiana Department of Education, “Louisiana Academic Content PD Ven- dor Guide,” https://www.louisianabelieves.com/ docs/default-source/teacher-toolbox- resources/pd-vendor-guide.pdf?sfvrsn=26. 14. Julia Kaufman, Elizabeth D. Steiner, and Matthew D. Baird, Raising the Bar for K–12 Academics: Early Signals on How Louisiana’s Education Policy Strategies Are Working for Schools, Teachers, and Students, RAND Corporation, 2019, https://www.rand.org/pubs/ research_reports/RR2303z2.html. 15. Interview with a state official. 16. Louisiana Department of Education, 2019–20 School System Planning Guide, August 31, 2019. 96 RETHINKING K–12 EDUCATION PROCUREMENT

17. Kaufman, Steiner, and Baird, Raising the Bar for K–12 Academics. 18. Achieve, “Stronger Standards: A Review of State Standards Since the Common Core,” November 13, 2017, https://www. achieve.org/strong-standards. 19. Kaufman, Steiner, and Baird, Raising the Bar for K–12 Academics. 20. Kaufman, Steiner, and Baird, Raising the Bar for K–12 Academics. Overcoming the Challenges Facing Innovative Learning Models in K–12 Education: Lessons from Teach to One

JOEL ROSE

ver the past seven years, I’ve been leading a nonprofit organization O called New Classrooms Innovation Partners that works to develop a more modern and impactful classroom experience. We’ve developed an innovative learning model called “Teach to One: Math” that integrates live, collaborative, and independent learning in ways that enable personalized instruction for each student, each day. In developing this model, we’ve learned how crucial it is to integrate academic expertise, operational prowess, and technological know-how into the model’s design. We’ve also learned that no matter how strong the design, it’s equally essential to work with enthusiastic school partners that serve as early adopters. Finding partner schools interested in profound changes to the classroom experience has not always been easy. Our team has met with hundreds of administrators, principals, teachers, funders, policymakers, and key influencers all across the country. While many of those interactions led to highly successful implementations, more conversations ended with some version of “this is exactly the kind of innovation that schools need, but we aren’t quite ready for it.” Part of the reason some schools and districts are slow to adopt Teach to One is assuredly on us. We don’t spend much on marketing, have only a handful of representatives across the country, and have a set of staffing and scheduling parameters that, while crucial for driving outcomes, doesn’t necessarily fit within the existing structures of how some schools operate. Beyond that, “innovative learning models” usually aren’t one of the line items to check off when planning a school budget, so it’s often hard

97 98 RETHINKING K–12 EDUCATION PROCUREMENT for schools to allocate funding for an initiative as intensive as ours (particularly given the professional development required to retrain school-based staff). Innovation is often viewed as nice to have in a world where even the must-haves struggle getting the dollars they need. But let’s assume for a moment that we had the marketing budget of a textbook publisher, few operational barriers to implementation, and a bargain-basement price point. Even then, the forces for keeping school as is would—more often than not—still win the day. This chapter is about illuminating these forces and exploring how they might be addressed. My goal is to take you behind the scenes into the conversations we’ve had with state departments of education and district administrators, with school principals and teams of teachers, with leading technology companies and educational publishers, and with successful investors and prominent philanthropists. These experiences have helped us understand a fundamental paradox with K–12 innovation: Driv- ing dramatic gains in student outcomes will require new learning models that challenge some of the basic constructs for how classrooms operate, and yet the more new models challenge these constructs, the harder they become for schools to adopt. Our nation’s K–12 system is so resistant to novel approaches that redesign the classroom itself because of several underlying forces that have built up over time, including (1) an assessment and accountability system that demands a uniform instructional path for all students, (2) inadequate investment in the development of innovative learning models, (3) bureaucratic inertia that reinforces the status quo, and (4) stifling procurement processes. But before we get into all this, it’s important to understand what I mean by the term “innovative learning model” and how it really differs from a traditional learning model (fully acknowledging that reflexiveeye-rolling often accompanies new education terms that are often just dressed-up versions of past terms). Let’s start with how schools generally work today: If your daughter happened to be one of roughly four million US seventh graders who began school this fall, she most likely attended her first math class with 28 or so same-age students and her assigned math teacher. Her teacher, trained in the seventh grade curriculum (one would hope), assigned to JOEL ROSE 99 her a seventh grade textbook that included dozens of lessons that span your state’s seventh grade curriculum standards. Over the school year, your daughter will generally experience the same daily lessons as everyone else in her class, receive the same homework to complete, and take the same tests and quizzes. It’s probably not too different from how you learned seventh grade math. Now, here’s how your daughter would experience math if her school implemented Teach to One: At the beginning of the school year, she would take a diagnostic assessment that would ultimately help generate a personalized annual curriculum that’s right for her. It could include filling key learning gaps from prior grades, any of the seventh grade standards, and even more sophisticated eighth grade concepts. Over the school year, she would experience that curriculum through a variety of instructional approaches—from teachers, in collaboration with her peers, and independently. And each day, she would take a quick online exit slip that feeds a sophisticated scheduling engine, enabling her teachers to continually regroup students so they can work with similarly situated peers on lessons most likely to enable acceleration. It is important to distinguish innovative learning models, such as Teach to One, from the myriad of software products that schools and districts use. Those products can help, and we embed many of them in our model. But by themselves, many of these products are often used as tools that make the traditional classroom paradigm more efficient. Innovative learning models challenge the paradigm itself. They are an alternative to the standardized, age-based models of instruction that assume a one-size-fits-all curriculum is best for all students. Our innovative learning model is certainly not the only way of reimagin- ing the classroom. There are limitless ways of challenging what educational historians David Tyack and Larry Cuban termed the “grammar of school”— those aspects of our educational system so deeply embedded and accepted that one would hardly recognize a school as a “real school” without them.1 Its basic tenets—one teacher, an 800-square-foot room, and 28 or so same-age students all moving through a standardized curriculum—have been around longer than the telephone, automobile, and periodic table. Indeed, the current grammar of school serves some students well, but its shortcomings are abundantly clear. Age-based student cohorts are 100 RETHINKING K–12 EDUCATION PROCUREMENT administratively convenient, but they don’t reflect the reality that students begin the school year at significantly different academic starting points. Textbooks are relatively inexpensive and portable, but they can be an unengaging medium for students. Teachers are the most important factor that goes into a student’s academic success, but the job itself can be so taxing that many of the best teachers can easily burn out. Most importantly, the current grammar seems to work for only 30–40 percent of students across the country—those who graduate each year ready for college or a career. Given these systemic shortcomings, why aren’t more districts and schools aggressively looking for innovative solutions?

Reason #1: Assessment and Accountability Policies Demand a Uniform Instructional Path

Federal law requires states to administer annual tests to all students in third through eighth grade that reflect the standards from their enrolled grade level.2 That means all seventh graders take the seventh grade test, regardless of whether they began the year on a fourth or eighth grade level. Since teachers, school leaders, and districts are accountable for the results on the seventh grade test, they generally want their teachers to spend the year covering seventh grade material. While this approach can help mitigate the risks of teachers having low expectations of students, it fails to account for the fact that math itself is cumulative; a student who hasn’t quite conceptually grasped decimals, for example, is going to struggle with percentages. Our data have taught us that the best way to support students in mathematics who have unfinished learning from prior years is to provide them with a strategic mix of pre- and on-grade skills, all to ultimately get them back to grade level. Third-party research of our program provided further evidence that students are more likely to accelerate when their unique needs are prioritized over grade-level exposure.3 But that approach can make many district and school leaders nervous: More time spent on pre-grade skills means less time spent on skills that will appear on the annual state test.4 JOEL ROSE 101

The insistence on accountability for grade-level expectations has created many barriers to adoption. Some districts we’ve spoken to couldn’t move forward because the curriculum dollars they wanted to use were allowable only for grade-level-aligned curriculum materials (an approach reinforced with the introduction of EdReports and other ratings efforts that evaluate curriculum based on its alignment to grade-level standards). Another district got stuck when it learned that our approach to team teaching would make it harder to cleanly calculate each teacher’s unique evaluation score. One senior district official shared that her district could not move forward because she had to attest to the state that each student was taught all the grade-level skills during the school year. A fixed orientation aroundgrade-level standards and assessments can also make it harder to measure the true impact of innovative learning models. When schools have students who begin the school year multiple years behind and success is measured based on performance on annual grade-level assessments, any learning growth made on pre-grade skills can readily go undetected. These can leave decision makers with the mistaken assumption that innovations “didn’t work”—a useful buttress for the con- tinued reign of textbooks and the current grammar of school.5 The implicit assumption that underlies many of these policies is that all same-age students should be learning the same grade-level material each year—a hallmark of the current grammar of school. Approaches that diverge from this basic core assumption will be met with some resis- tance because they conflict with thegrade-level orientation embedded in federal and state education policies and practices assumed best by local decision makers.

Reason #2: Inadequate Investment in Developing Innovative Learning Models

The fact that state laws require the purchase of grade-level textbooks sends an unmistakable message to those who might otherwise have the capacity to design new learning models: Don’t waste your time. But even if states purged all references to textbooks from their education codes, where exactly would new models emerge from? The research and development 102 RETHINKING K–12 EDUCATION PROCUREMENT

(R&D) required to thoughtfully design new learning models requires mul- tiyear efforts from academic experts, instructional designers, software engineers, operational specialists, and educational researchers—all of whom come together to develop and iterate new models that teach students about world history, the basics of life science, or how best to write an expository essay. In our work around middle-grade math, we’ve meticulously inves- tigated the standards and underlying concepts reflected at each grade level, explored and tested the mathematical relationships among those concepts, reviewed tens of thousands of lessons that relate to those concepts, analyzed the results from over 100,000 third-party summative and formative assessments, administered over six million internally designed assessments, and partnered with universities and research firms to advance our collective understanding of how students learn math. We’ve even started to use machine-learning techniques to deepen our understanding. None of this R&D comes free. In other industries, the private sector has incentives to foot the bill for these efforts. In 2015, businesses invested $333 billion in R&D in sectors such as health care, transportation, energy, defense, and technology, fueling breakthrough discoveries and overall human progress.6 So why don’t prominent venture capitalists from Silicon Valley or elsewhere put those same investment dollars into driving trans- formation in our classrooms? I remember vividly a conversation with Rob Stavis, an adviser to our work and a seasoned venture capitalist. When I asked him about his firm’s private investments in K–12 companies, he said that with some rare excep- tions, his firm and others had a “big red X” on the K–12 sector. Like many investors, he finds the market too fragmented (15,000 school districts plus 40,000 charter and independent schools), slow-moving, and often resistant to change. As a result, it’s harder to scale a business in the K–12 sector and achieve a financial return than it is in other sectors. To be sure, some private capital still goes into K–12 education technology. But many investments are focused on either noninstructional tools (e.g., parent communications and district administrative services) or supplemental learning products for teachers to use as they see fit. Few are designed to encompass the beginning, middle, and end of a student’s JOEL ROSE 103 core academic experience given the systemic orientation around textbook adoption. Limited meaningful private investment in innovative K–12 learning models leaves only two other potential sources for the R&D required to develop innovative learning models: the government and philanthropy. Government is no stranger to supporting innovation. In 2015, it invested more than $100 billion in R&D to support innovations such as those that improve our nation’s renewable energy capacity, incorporate genetics into our health care system, and make warfare safer for soldiers.7 This is how federal policy drives innovation when markets are ill-equipped to do so. K–12 saw less than 0.10 percent of that amount. Next time you begin to wonder why so much progress has been made in the clean energy sector (a doubling of renewable energy capacity over the past 10 years) while our K–12 system has been relatively stagnant, look no further than the difference in R&D investment dollars.8 Without accessible and consistent capital to mount serious challenges to the current grammar of school, few organizations can actually create innovative learning models that enable far stronger outcomes. And without a vibrant ecosystem of organizations continually innovating to improve student outcomes, new ideas will be perceived by decision makers as fads.

Reason #3: Bureaucratic Inertia

We often meet district and school leaders who are true champions for new learning models. But we also meet those whose careers have oriented around the traditional school model and who view innovation with skepticism. Many of these people have explicit or implicit decision rights on moving forward, which can make it much harder to get to yes. I’ll never forget a meeting with a large city school district. Around the table were several of the district’s top leaders, many of whom were familiar with our work and eager to dig into the nuances of learning progressions, teacher supports, and space redesign. Some even proposed potential partner schools. Then the district’s director of math curriculum shared what he said was a philosophical difference: “I believe all students should do 104 RETHINKING K–12 EDUCATION PROCUREMENT the same thing on the same day, no matter where they are.” The room fell silent, and the meeting soon ended. There’s certainly nothing wrong with having a different underlying pedagogical philosophy. But I’ve also come to understand that these reservations are sometimes more about internal politics. Once a school wants to adopt an innovative learning model, what does that say about the chief academic officer who drove last year’s textbook adoption? Or the area superintendent whose handpicked math guru has other ideas about what should happen in the schools they oversee? Or the curriculum director who spearheaded an in-house effort to generate a standardized scope and sequence? As soon as one administrator shares reservations, the others must then calculate if it’s in their personal interest to wage an internal battle. Even in situations in which the central office is fully aligned, the challenge of what some call “asymmetric risk” remains. Simply put, district and school administrators have limited political capital to expend, and many fear they have more to lose if a new initiative fails than if it succeeds. Of course some skepticism around new projects and approaches is fair and appropriate given the underlying stakes for students and the core expectations of teachers and parents to ensure quality. However, just as 20 years ago a catchphrase in the technology industry was “no one ever got fired for hiring IBM,” the same holds true for many school administrators and their inclination to continue buying textbooks. Interestingly, the one group of stakeholders we’ve found the most open to new learning models is teachers. As part of our adoption process, we engage with teachers from each potential partner school to be sure they all have a voice in the decision-making process and are fully aware of what they’re signing up for. We walk them through all the details on how they get their schedules, how their role changes, and what supports we provide. The vast majority of the time, they are on board. But teachers are typically not the primary decision makers when adopting innovative learning models. Decisions more typically rest with administrators whose job it is to carefully consider new approaches or initiatives and balance the asymmetric risk inherent in their roles. JOEL ROSE 105

Reason #4: Stifling Procurement Barriers

As of 2013, 42 states, three US territories, and Washington, DC, had legislative provisions that require providing free textbooks for all students.9 State laws vary for grade-level textbook adoption; some are statewide, while others are more focused at the district or school levels. But the basic notion that grade-level textbooks, despite their profound limitations, shall serve as the primary “technology” that teachers use in classrooms is deeply embedded in most states’ educational codes. With statewide mandates in place, states and districts then set multi- year schedules and annual budgets around textbook adoption for different subjects and grade spans in different years. Savvy textbook publishers then map their product development around these cycles before aiming their hefty national sales forces at winning as many contracts as they can. Three major publishers (Pearson, McGraw-Hill Education, and Houghton Mifflin Harcourt) dominate the marketplace. This is how the vast majority of educational materials are purchased. Procurements for instructional materials that might fall outside these cadences with their requests for proposals, specification gathering, bid conferences, board agenda items, and contracting procedures can be a pain for central office staff that might have other priorities. I once worked at the New York City Department of Education and saw firsthand how it can take at least a year or more to buy anything that’s different than what’s been purchased in the past. Sometimes district administrators don’t realize how convoluted their procurement infrastructure actually is. We worked with a major US district that held a mayoral press conference to promote our partnership at the outset with a promise to figure out the paperwork later. The procurement passed us along, filling out one form or another for months on end. By the time the school year ended, no contract had been signed, and we had no way of getting paid for the services we rendered. 106 RETHINKING K–12 EDUCATION PROCUREMENT

The Path Forward

An essential function of our K–12 public education system is to ensure that the 50 million students it serves each year are prepared for college or a career once they graduate. The current grammar of school may work for some students, but most have little choice but to endure and hope they can find a different path to a fulfilling life once they leave the system. Innovative learning models provide an alternative way of thinking about how best to address current educational inequities and inadequacies. They stem from a core belief that the basic grammar of school that originated more than 100 years ago is likely not the optimal grammar for all students in the 21st century and that it’s time to develop others that reflect our nation’s current know-how and capabilities. But innovative learning models will not gain much traction if we simply rely on the basic laws of supply and demand to work their economic magic. If the past century has taught us anything, it’s that the K–12 sector operates under a different set of conditions that makes it highly resistant to the transformational change we have seen in nearly every other sector of our society. Overcoming these barriers will require a combination of intentionality, creativity, and patience to play the long game. So how exactly would policymakers work to overcome the legislative barriers, design capacity limitations, mindsets, and procurement barriers that stand in the way of new grammars of school? Here’s a potential road map.

Create Space in Accountability Policy for New Models to Develop. Just as successful organizations create space for new products and ideas that can serve as the basis for longer-term success, assessment and accountability policies must also leave room to responsibly explore their future iterations. The importance of creating space for innovation was contemplated in the Every Student Succeeds Act (ESSA) through its provisions around innovative assessment pilots. But these provisions include several requirements that continue to reinforce the current grammar of school; most notably, they require states to make an annual, comparable determination around grade-level proficiency. While these assessments can also extend JOEL ROSE 107 to below- and above-grade-level skills, the practical limitations around test length and time can make that hard to pull off. Nonetheless, states and districts can look for ways to push ESSA’s boundaries to create the space for innovative learning models, including using adaptive assessments to measure learning growth, modifying state accountability systems to focus on learning growth over multiple years, and weighting key transition points more heavily.10 They could even conduct requests for proposals to explore other ways of more precisely measuring growth, including approaches that could track student mastery down to the skill level. All this could happen in parallel to what federal law requires.

Fund the “D” in R&D. Plenty of academic researchers conduct studies on various K–12 programs and practices. But without real investment in the “D” part of R&D—the work involved in designing innovative learning models—most of the “R” will continue to underwhelm. Who would actually do the “D”? In today’s educational delivery model, local education agencies (LEAs) already do too much. As school managers, they hire and develop teachers and leaders, manage buildings, organize buses, coordinate meals, engage with communities, and oversee the myriad of details that go into running a school. Their complex work is made only harder when done within the constraints of public-sector management and under the auspices of a board of elected local officials with their own political interests. Given these constraints, LEAs cannot be relied on to also serve as cen- ters of gravity for developing innovative learning models. It’s just not what they’re built to do. Hospitals don’t invest in new MRI machines, practicing physicians don’t push the boundaries of genomics, and soldiers aren’t developing artificial intelligence–powered drones. The best innovations are informed by practitioners, but burden of design and iter- ation rests elsewhere. That’s why a new organization, which we’d call a “model provider,” is so essential to driving the agenda around innovative learning models. Model providers would be responsible for (1) designing innovative learning models through intensive R&D efforts, in partnership with schools and districts, and (2) supporting the adoption of new models 108 RETHINKING K–12 EDUCATION PROCUREMENT in existing schools while ultimately sharing in the accountability for student outcomes. NewSchools Venture Fund has prioritized this concept as part of its organizational strategy and has been steadily building a portfolio of early-stage model providers over the past 18 months.11 In a future-state educational delivery model, while LEAs would continue to operate the day-to-day management of schools, they would also select and partner with model providers in lieu of selecting textbooks. Some model providers might focus on particular subjects and grade spans, while others might cover more. LEAs would decide based on the model’s robustness, the track record of results, and their reputation for high-quality supports. And model providers would work only in schools where they felt they could demonstrate impact. In one sense, a bifurcated delivery model that pairs LEAs and model providers working toward a common outcome would be analogous to how the aviation sector operates—with plane makers focused on the sourcing and assembly of aircrafts and airlines focused on reservations, personnel, and maintenance. Each plays a vital role in ensuring reliably safe service, but each plays a distinct and complementary role in the process. Where might model providers emerge from? With the right policies, market signals, and incentives, existing product companies and educational publishers would shift their capacities to develop innovative learning models. So might charter management organizations, big technology companies, higher education institutions, operating foundations, and professional development organizations, which have some, but not all, of the key competencies required. Entirely new entities likely would be built and organized to design and deliver new models, much in the same way charter management organizations emerged over the past two decades.

Identify and Support Early Adopters. Sir Michael Barber, the one-time architect of Tony Blair’s education efforts in the UK, once shared an interesting perspective about educational mindsets. Most policymakers, he argued, believe that changing hearts and minds is a necessary precondition to changing behavior. But he’s found the opposite has proved true: Chang- ing behavior first is what ultimately changes hearts and minds. Thus, the best path to challenging the inertia that unwittingly protects the current JOEL ROSE 109 grammar of school is to identify and support early adopters that want to be pioneers for new learning models while addressing the barriers that resis- tors employ to rationalize keeping things as they are. The kernels for how a state might incentivize the adoption of innovative learning models are playing out in Texas, where the state recently launched Math Innovation Zones. Under the program, the state selected a set of providers (though mostly products, not models), matched them with a set of volunteer schools, allocated state dollars to support implementation, and set up a parallel accountability system to more closely and accurately measure success. Over time, as the state gathers more performance data, it will be able to expand the impact of providers that are having an effect while eliminating those that are not. Funding for programs such as Math Innovation Zones can come from provisions under ESSA that enable 7 percent of Title I funding for school interventions and another 3 percent for direct student services. That’s roughly $1.6 billion, though few states are fully taking advantage of these provisions. Another $2 billion is available in professional development dollars through Title II (teacher training and professional development) that could also be designated for supporting early adopters of innovative learning models. And those are just federal funding streams; states may have their own programs that could be reconfigured to better support innovation. Forward-leaning district leaders will also need to create space in their organizations to pilot and support innovative learning models—even at the risk of creating organizational agita. In our experience, organizational turf wars between chief innovation officers and chief academic or school officers are more often won by the latter—particularly since that’s whom school leaders report to. Superintendents who are serious about driving innovation and new learning models need to be unambiguous and resolute in driving this agenda while ensuring their teams are properly structured and appropriately aligned to get the job done.

Address Procurement Barriers. Ensuring procurement policies do not stand in the way of innovation will require state offices of education to comb through their education codes to stop well-intentioned regulations oriented around grade-level textbook adoptions from obstructing the development of innovative learning models. That’s probably the easy part. 110 RETHINKING K–12 EDUCATION PROCUREMENT

Addressing in more detail the nuances of state and district procurement procedures and how they could be improved to better support innovation is well beyond the scope of this chapter. Many of the rules, systems, and procedures that govern public procurements have evolved over decades, with new layers added each time some level of waste, fraud, or abuse is uncovered. While not the sexiest work, it is incumbent on each state and district to take a hard look at how they do business to ensure that systems in place to protect the public interest are not also the invisible forces rein- forcing the same way of doing school.

Conclusion

The grammar of school our country stumbled on in the middle of the 19th century has become the foundation for one of our greatest national accomplishments. It’s easy to take for granted today, but the fact that we’ve funded, built, staffed, and supplied more than 130,000 schools in every corner of the country so that any child can access a K–12 education is no small feat. Yet to better drive quality, it’s long past time to create the space for fundamental redesign. The past 100-plus years of human progress and educational know-how provide a fresh opportunity to imagine new grammars of school. They can now be designed in ways that are more attuned to the unique strengths and needs of each student, more mindful of students’ social and emotional development, more reflective of deeper and more authentic learning experiences, more creative in their use of modern tools and capabilities, and more inclusive of sustainable and fulfilling roles for educators. One-off efforts focused on any of these objectives that do not fundamentally redesign the classroom experience are unlikely to yield notewor- thy and scalable outcomes that students deserve. And yet as we’ve learned, comprehensive efforts to redesign the classroom itself to account for all these objectives will run up against defenses that unwittingly protect against threats to the current grammar of school. The work required to design, implement, and scale innovative learning models will not happen by osmosis. The economic forces that invis- ibly drive disruption and innovation in most sectors do not apply in the JOEL ROSE 111 same way in K–12 education. That’s why developing novel ways to enable and support innovation around new learning models is essential for long- term change. Creating the space in policy, catalyzing real investment in R&D, challenging bureaucratic inertia, and fixing procurement are central pillars to facilitating the shift to innovative learning models. While it’s no quick fix, it represents the most promising path to reshaping our educational system in ways that can work far better for the students it serves.

Notes

1. David D. Tyack and Larry Cuban, Tinkering Toward Utopia: A Century of Public School Reform (Cambridge, MA: Harvard University Press, 1995). 2. The Every Student Succeeds Act also requires states to administer a summative assessment in high school. 3. Jesse Margolis, “Three-Year MAP Growth at Schools Using Teach to One: Math,” MarGrady Research, February 2019, www.margrady.com/tto/. Overall, students served through Teach to One: Math over their three years in middle school grew 20 percentile points relative to national norms on the NWEA MAP assessment. However, schools that operated within accountability systems that valued learning growth (as reflected on the MAP) grew 38 percentile points, while those focused largely on state proficiency grew 7 percentile points. 4. More on this topic can found in New Classrooms Innovation Providers, “The Iceberg Problem: How Assessment and Accountability Policies Cause Learning Gaps to Accumulate . . . and What to Do About It,” Fall 2019. 5. The challenges of implementing Teach to One in grade-level accountability systems were explored in Douglas D. Ready et al., Final Impact Results from the i3 Imple- mentation of Teach to One: Math, Columbia University, Teachers College, Consortium for Policy Research in Education, January 2019, https://www.newclassrooms.org/wp- content/uploads/2019/02/Final-Impact-Results-i3-TtO.pdf. As part of the study, researchers enumerated the many ways the program was locally modified to prioritize grade-level exposure over individual student needs. This ultimately prevented researchers from drawing any generalizable conclusions about the program itself. 6. National Science Board, “Research and Development: U.S. Trends and Inter- national Comparisons,” 2018, www.nsf.gov/statistics/2018/nsb20181/report/sections/ research-and-development-u-s-trends-and-international-comparisons/introduction. 7. National Science Board, “Research and Development.” 8. US Energy Information Administration, “Today in Energy: U.S. Fuel Eth- anol Production Continues to Grow in 2017,” July 21, 2017, https://www.eia.gov/ todayinenergy/detail.php?id=32152. 112 RETHINKING K–12 EDUCATION PROCUREMENT

9. Vincent Scudella, “K–12 Textbook Adoption: State Textbook Adoption,” Education Commission of the States, September 2013, www.ecs.org/clearinghouse/01/09/23/10923. pdf. 10. For more on this topic, see New Classrooms Innovation Partners, “The Iceberg Problem.” 11. For more on NewSchools’ work with model providers, see NewSchools, “Model Providers: New Pathways to Innovation,” June 2019, www.newschools.org/wp-content/ uploads/2019/06/Model-Providers-Pathways-to-Innovation.pdf. Procurement Practices That Inhibit Innovation: A Case Study from the Edtech Startup Perspective

LAUREN DACHILLE

e can all agree that competition is a good thing in most indus- Wtries. Where there’s competition, there’s innovation, the standard of service is higher, and consumers have more power. In the K–12 world, barriers to entry have created an environment in which a few incumbents dominate the market and there is little innovation. This can often mean districts are left with clunky, outdated products and poor customer service. In fact, districts spend about $8.4 billion annually on edtech software; yet, the vast majority of those technology tools do not fully meet users’ needs.1 As a former member of a school district human resources team and now an entrepreneur in the K–12 HR space, I’ve experienced this phenomenon firsthand. In this chapter, through the lens of my own experience at DC Public Schools and now as the founder of Nimble, I examine what districts themselves might do to drive more innovation in the edtech market and why this will ultimately serve them—and their students—well. To more successfully identify and select innovative products and services, districts must first fundamentally shift the way they think about outside vendors, products, and services. They must then restructure their procurement processes to incentivize the products and vendor standards they want. There are two key perspectives that inform districts’ attitudes toward products and vendors that, if shifted, would clear a path for districts to take advantage of more innovative tools.

113 114 RETHINKING K–12 EDUCATION PROCUREMENT

Myth #1: Vendors Are to Be Avoided

Most HR leaders have a negative view of vendors; I was part of a district HR team and am now a vendor, so I’ve seen it from both sides. Vendors are known for overpromising before the contract is signed and under-delivering afterward. In turn, districts regard them with distrust. For this chapter, I interviewed several HR leaders about procurement, and I asked them how the word “vendor” makes them feel. Some of the most common responses were “broken promises” and “pushy.” Rather than being seen as creating value for districts through their products and services, vendors are seen as taking key resources away from school systems and students. There is a general assumption in K–12 that private companies are about profiting from students and school systems, rather than contributing to positive outcomes. However, the structure of the entity as for-profit or nonprofit can’t tell you much about its mission and values; many mission-oriented companies incorporate as private companies (rather than nonprofits) for good reason. For example, in the case of my own company, Nimble, the product development over the first three to five years was relatively capital-intensive and required outside investment. Given that limited philanthropic entities invest in software solutions for back-office tools in finance and HR, there was little option other than to incorporate as a private company. Further, if an organization provides a software tool to districts, a core competency is its ability to recruit and retain top-tier technical talent. Equity in a private company is a tool companies have that nonprofits don’t, which can attract the strongest technical talent in the early stages, when cash is less readily available. Ultimately, an entity might choose to structure as a for-profit versus nonprofit for many reasons, unrelated to mission and values. In fact, depending on the product or service, there is a strong argument to be made that if software founders want to maximize their impact, they should incorporate as a for-profit company. So why does this perspective matter? Well, first, it comes hand in hand with avoiding sales outreach related to products or services. And perhaps this is for good reason. Throughout my interviews, I heard from several HR LAUREN DACHILLE 115 leaders who regularly received emails and calls from vendors. In fact, one received an email from a vendor during our short conversation. Whether the vendors’ excessive outreach is a cause or effect of the districts’ avoidance is hard to say, but the ultimate outcome is that many talent leaders are not keeping a pulse on the new products and tools available to them. The attitude toward salespeople is very much: “If we’re ever shopping for a new [insert product], we’ll let you know.” But, let’s consider why this is insufficient. Sometimes the very nature of innovation is that it solves a problem we don’t know we have with a solution we couldn’t have fathomed. For example, back when everyone was using BlackBerry phones, most of us might not have said we were on the market for a new phone, because, well, our phone already had an internet connection and worked just fine. It was only once the iPhone was launched that consumers realized exactly what they had been missing. Imagine if consumers had been so resistant to new products that they never took the time to see what the iPhone could do for them. We’d all still be plugging away at those tiny BlackBerry keys. By the same token, I constantly interview HR teams for Nimble’s product development, and I often hear them describe the over 20 hours they spend weekly on manual workarounds to accomplish tasks that new and innovative tools like Nimble can actually solve for them. Yet, none of them are in the market for new software solutions. Like the BlackBerry users, they don’t imagine more efficient and user-friendly systems because they’ve been exposed only to the antiquated tools they have. They are comfortable with their workarounds—and, in fact, consider them a core part of their daily work. If you engage with vendors and products only when you’re seeking to solve a specific problem, you are likely to entirely miss new innovations that solve problems you never thought could be solved in ways you hadn’t considered. This orientation toward vendors is also problematic because it ensures districts experience the exact “pushy” vendor behavior they hate. When K–12 decision makers are vendor averse and spend little time proactively exploring the true differences among products, the only differentiator K–12 companies can pursue to get their products into schools is a giant, ubiq- uitous sales force. And the market for K–12 recruitment software reflects 116 RETHINKING K–12 EDUCATION PROCUREMENT this. The two incumbents in the space invest substantially more in their sales forces than in their product development. Thus, we live in a world where strong K–12 products lose out to strong sales teams. So, what is the alternative? What if each quarter district decision makers spent a day proactively examining new products on the market or the new features of existing tools, with the help of their most tech-savvy teammates? This might mean reviewing vendors’ materials or setting up demos—even if their district is not on the hunt for a new system at that moment. When vendors reach out at other times, it would then become easy for districts to say, “I’ll be doing my semiannual product review next month. Please send me information or an updated demo recording before then.” It would mean districts would stay up to speed on new innovations and tools that could make their teams more efficient in ways they hadn’t previously considered. Over time, if districts became more sophisticated curators of strong products, it would also incentivize companies to invest more in product innovation and less in that pushy sales outreach.

Myth #2: Early-Stage Companies Are Riskier Than More Established Companies

I can’t count the number of conversations I’ve had or “request for proposal” (RFP)2 responses I’ve completed in which it’s become obvious that districts consider early companies too risky to work with. Some RFPs actually explicitly state that companies with fewer than five years of experience need not apply. If every district did that, then no new companies or innovations would ever sprout up in K–12 because no startups could attract the early customers and revenue to survive. Thus, by rejecting early-stage companies across the board, districts are setting themselves up to work with only the antiquated monopolies and duopolies that dominate the software space. Let’s explore this logic with an analogy. What if all districts hired only teachers with over five years of experience because they felt that new teachers were too risky? This would mean there would be no jobs for new teachers, and, ultimately, few would come into the profession. That might be fine for districts over the next few years, but a decade or two later, as LAUREN DACHILLE 117 their population of teachers aged out, this would create a massive pipeline problem. Thus, most districts instead find ways to screen new teachers to strategically identify the best of the best. The same should be true of districts’ relationships with early-stage companies. First, rather than making the blanket assumption that early stage means risky, districts must identify the discrete and specific risks they see in working with young companies. (Is the company financially sustainable? Does it have the core competencies necessary to execute? Does it have a history of following through on promises?) Then they can work with the company to gather the data necessary to address these risks. If the company cannot adequately address the specific risks, then perhaps it isn’t the right fit. The other mistake districts make in evaluating the risks of early-stage companies is to disregard the known risks of working with more established companies. Just because a risk is known and common doesn’t mean it isn’t a risk. Late-stage education companies are constantly acquired by private equity firms, which then raise prices and make cuts to customer service. Districts in relationships with these companies (even large districts) are typically small fish in a big pond, which means they have little leverage to hold the companies accountable. If one district leaves, these companies have hundreds or thousands more, so it often matters little (especially relative to the value early-stage companies place on accommodating every customer). If districts could systematically evaluate the risk of working with each individual early-stage company and the risks inherent in working with many larger companies, they would likely choose to use innovative products more often. They would also be laying the foundation for an industry with much healthier product competition and ultimately more leverage for district consumers.

Changing the Procurement Process

Incentivizing innovation is not just about changing districts’ perspectives on products and companies though. It’s also about changing procurement processes in a way that allows district leadership to be more open to innovation while still adequately de-risking the process. 118 RETHINKING K–12 EDUCATION PROCUREMENT

Job Postings and Applications

• The system allows full customization of job postings and supports several types of applications based on job function and internal or external to the district. • The system is connected to or is able to integrate with job boards. • The system can create job postings from scratch or from previous postings. • The system allows applicants to upload different documents. • Applicants can start an application, save, and return to edit.

Screening and Communication

• The system has workflow functionality to progress candidates through various levels of screening. • The system contains email capability to communicate with applicants. • The system contains a robust applicant search feature to include a keyword search. Software Features and Functionality 30 Price 25 Scalability 20 Training and Support Available 15 Compatibility with Existing HRIS Infrastructure 10 Total Points Possible: 100

First, district leadership should consider the placement of the procurement team within the larger hierarchy, the established goals for this team, the expertise and background of the team’s leadership, and what all that means for its incentives. I asked every district leader I spoke with for this chapter whether they felt that their district’s procurement process focused LAUREN DACHILLE 119 more on risk avoidance or excellence. This may not be surprising, but none said excellence. Consider district decision makers’ incentives. If they choose to implement a product that has already been used by hundreds of other districts, and they find themselves with a mediocre service that makes some but not much positive change, that is typically regarded as a relatively successful procurement process. If they choose to implement a new solution that could bring major efficiencies, perhaps they would receive some recogni- tion for that a year or two down the line (or perhaps not), but with that small potential upside comes the huge downside risk of having to explain why they chose a less-proven solution if it doesn’t work out. In most cases, the trade-off is just not worth it for decision makers because supporting innovation is not an explicit part of their job description. Part of this relates to which department procurement is in. In some districts, this is operations; in others, it may be finance or legal. While placement in finance might lend itself to a process more focused on return on investment than placement in legal does, it’s equally possible that this leads to a focus exclusively on cost, rather than product differentiation and value. Without the right goals, none of these teams are naturally incentivized toward procuring innovative tools. In today’s day and age, where so much district spending is on technology, procurement teams must employ (or collaborate heavily with) experts in modern technology, who can evaluate and understand the relative potential of different products. Ideally, these individuals would have experience with software designed not only for the K–12 market but also to serve the private sector, including newer solutions from Silicon Valley. Experience with technology across industries can bring a more sophisticated eye to product evaluation, since other industries have had the benefit of more competition and rigor across technology tools for many years. Procurement teams must also be given explicit goals that provide an upside incentive for sourcing systems that create efficiencies and help move the district further toward achieving its desired outcomes. Procure- ment teams with tech expertise that are properly incentivized can move away from being the compliance police of the procurement process toward providing a valuable curation service for district leaders that helps shed light on innovative solutions that can add value. 120 RETHINKING K–12 EDUCATION PROCUREMENT

Second, districts must craft RFPs and procurement processes that elicit the outcomes they want from their vendors and products. The typical RFP that Nimble has responded to in the past three years has had between 50 and 100 requirements. Almost all these requirements have been problematic in three ways: First, they tend to reflect only existing basic product capabilities that the district has had in current or past tools; second, responses to requirements are binary, which limits important information districts might consider in scoring; and lastly, requirements often reflect specific ways of addressing issues, rather than a broad invitation for the vendor to describe how its solution will help the district meet its goals. Let’s make this more concrete with an example. The side bar contains an excerpt from real-life requirements from a district applicant tracking system RFP, along with a scoring rubric from the same RFP.

Problem #1: Requirements and Scoring Reflect Only Existing or Basic Capabilities

The ability to email candidates, integrate with job boards, and upload documents to an application are some of the most basic possible functions of an applicant tracking system. It’s unlikely that vendor responses to these questions will provide much, if any, differentiation across their tools, which puts substantial weight on price and scalability as deciding factors. (For what it’s worth, scalability criteria are not explicitly defined in this RFP, so most of the weight probably ends up on price.) So, how do districts develop these requirements lists? One human capital leader I spoke with likened the process to creating a job description. Basically, you pull an old one as a template and make a few updates based on the issues that are top of mind. You can see how, especially for procuring technology, in which the functionality available to you is constantly changing and advancing, using a 5- or 10-year-old requirements list would be quite problematic. If, however, districts shifted to a culture in which procurement teams and division leaders actively made time to review new technology each quarter, requirements lists could be living documents, constantly updated with more advanced features, allowing them to differentiate based on product quality, rather than just price. LAUREN DACHILLE 121

Problem #2: Responses to Requirements Are Binary

The other issue with requirements lists like these is that each vendor is typically asked to mark them as “Y or N,” to indicate whether they can meet this requirement. This tends to advantage established vendors, whose product offerings can be stagnant and which are less likely to develop innovative solutions. As a simplified example, let’s imagine a startup that met 40 of 100 requirements at the time of the RFP but built features at a rate of 20 per year and an established vendor that met 60 of 100 requirements but built features at a rate of five per year. The startup would far surpass the established vendor in quality of service starting in year three, and that gap would continue to grow with time. However, that would not be captured with a binary “Y or N” process. One shift districts can make to procurement requirements is to consider the lifetime value of a vendor partnership, rather than a snapshot in time. One charter school network RFP that did this well asked vendors to mark each requirement with one of the following codes rather than a binary “Y or N.”

• SR. Satisfies requirements as delivered without modification.

• AM. Additional module is needed. Pricing must include the cost of this module and should be listed as a separate line item.

• SRM. The software is capable of meeting the requirement, but some modification or customization is required. Pricing should list sepa- rately the cost of any such modifications.

• RM. Software cannot currently satisfy requirement. However, this functionality is included in the current product road map. For items marked RM, please note when they will be completed.

• NA. The functionality is not available or customization would be so complex as to make it impractical or too expensive.

This particular RFP also featured a much more sophisticated list of requirements that likely led to substantial differentials in scoring across 122 RETHINKING K–12 EDUCATION PROCUREMENT vendors. This response framework allowed the charter network to evaluate vendors’ current capabilities while keeping in mind the likelihood of future advancements.

Problem #3: Requirements Ask for Specific Solutions, Rather Than Expressing the End Goal and Leaving the Solution Open-Ended

Districts often name system requirements that support the current processes they have in place. This becomes a problem, however, when the processes they have in place are workarounds to address insufficient systems. For example, the aforementioned district RFP contained the following requirement: “Ability to export current open requisitions to an excel file to monitor vacancies.” The fact that districts spend their time monitoring vacancies in spreadsheets is a symptom of a larger shortcoming of existing applicant tracking systems. Nimble has innovated in this area by developing an in-system vacancy management dashboard that elimi- nates the need to export vacancies into spreadsheets for tracking. How- ever, given the structure of this RFP, Nimble might fail to meet the stated requirement. What if, instead of articulating a solution based on current systems and processes, the district had instead articulated a challenge and left it to the vendor to propose the solution in the context of the vendor’s tool? Rather than advantaging existing systems, this approach would leave more room for the bid evaluation process to consider innovative solutions to district problems. Rather than focusing the scoring for product features and functionality on dozens of basic requirements, districts should instead list their departments’ top-line goals and ask each vendor to speak to how the vendor’s product would help the department meet those goals. In addition, districts could also ask vendors to specifically address any of the 100 basic requirements that they do not currently meet. Contrary to the current district RFP practices that reward status quo features, a process like this would advantage innovative and customer-centric solutions. While not all procurement goes through an RFP process, these same principles can be applied to non-RFP procurement as well. LAUREN DACHILLE 123

Finally, once districts have restructured their selection process to incentivize more innovative solutions, they must also adjust contract structures to align vendors’ interests with districts’ interests. This means thinking about the measurable ways in which a given product helps advance depart- mental goals (which ideally should have been done in the requirements development process) and then structuring tiered contract payment terms so there is an upside to high-performing products and a downside to those that do not deliver on the promises they’ve made. This deters vendors from making promises they can’t keep during the sales process and advantages the innovative products and companies whose tools actually serve their customers effectively. In all my interviews for this chapter, few department leaders could think of why, aside from inertia, performance-based contracts should not be more common. Ultimately, technology is a powerful tool to help us build capacity on resource-constrained district teams. By reevaluating the way they view vendors and products and realigning the incentives in their procurement processes, districts can promote innovation and get more from their technology products.

Notes

1. Michelle Davis, “K–12 Districts Wasting Millions by Not Using Purchased Soft- ware, New Analysis Finds,” Education Week, May 14, 2019, https://marketbrief.edweek. org/marketplace-k-12/unused-educational-software-major-source-wasted-k-12-spending -new-analysis-finds/. 2. A “request for proposal” is the process by which school districts solicit bids from vendors for new software or services. Conclusion

Too often, conversations about school improvement have treated education procurement as an afterthought. While talk of “social and emotional learning” or “equity” garner plenty of head nodding from policymakers, foundations, and school leaders, talk of “education procurement” tends to elicit mostly yawns. Yet, as the contributors to this volume make clear, procurement makes a huge difference in theday-to -day experiences of students and educators. From the textbooks and curricula available to students, to the professional development programs offered to teachers, to the human resources software used to hire teachers, there is little in education that smart procurement practices can’t improve. When districts purchase materials that are ineffective, unwanted, or faddish, students suffer. When district administrators make procurement decisions based on what they imagine to be the permissible use of funds rather than what they deem to be the most effective use of funds, students get shortchanged. Thomas Arnett of the Clayton Christensen Institute puts it well in his chapter, observing that “teaching without good resources is like trying to build a house without power tools or scaling a mountain without good hiking boots. It isn’t impossible. But having the right resources makes a huge difference.” While it may not be the flashiest topic, it’s clear that education procurement sorely needs innovation and improvement. Especially now, as districts face substantial budget cuts in the wake of the coronavirus pandemic, it’s crucial that policymakers and education leaders consider whether each of the countless billions of dollars spent on textbooks, curricula, technology licenses, and all the rest is being put to the best possible use. The ideas in this collection are, of course, just a start. The challenges of improving education procurement are complex and not susceptible to simple, silver-bullet fixes.

124 CONCLUSION 125

My hope is that the problems and solutions presented in this volume will provide a jumping-off point for policymakers, advocates, educators, and vendors looking to ensure that schools find ways to do better. —Frederick M. Hess About the Authors

Thomas Arnett is a senior research fellow in education for the Clayton Christensen Institute. His work focuses on studying innovations that amplify educator capacity, documenting barriers to K–12 innovation, and identifying disruptive innovations in education.

Lauren Dachille is the founder and CEO of Nimble and a former member of the Human Capital Team at DC Public Schools.

David DeSchryver is a senior vice president at Whiteboard Advisors. He previously worked as an attorney at the national education law firm of Brustein & Manasevit, where he helped state and local educational agencies with federal fiscal and policy challenges.

Mike Goldstein is an education consultant in Boston. He is the founder of Match Education and was chief academic officer of Bridge International Academies.

Frederick M. Hess is a resident scholar and the director of education policy studies at the American Enterprise Institute.

Melissa Junge is an attorney and cofounder of Federal Education Group, a law and consulting firm that helps states, school districts, and other educational organizations understand federal law so they can use federal money to achieve their goals.

Eric Kalenze is an educator, author, and the US director of researchED, a grassroots organization committed to improving educators’ research literacy.

126 ABOUT THE AUTHORS 127

Julia Kaufman is a senior policy researcher at RAND Corporation. Kaufman’s research examines how education policies and programs— from curriculum adoption to educator professional learning—are implemented in schools and how those policies and programs affect classroom teaching and learning.

Rebecca Kockler is the former assistant superintendent of academic content at the Louisiana Department of Education, where she was the architect of Louisiana’s nationally recognized academic model. She currently supports school systems and national nonprofits focused on curriculum implementation and teacher training.

Sheara Krvaric is an attorney and cofounder of Federal Education Group, a law and consulting firm that helps states, school districts, and other educational organizations understand federal law so they can use federal money to achieve their goals.

Joel Rose is the cofounder and CEO of New Classrooms Innovation Part- ners. He is the former chief executive for Human Capital and the New York City Department of Education, where he also spearheaded the creation of School of One. Edited by Frederick M. Hess

Praise for RETHINKING K–12

EDUCATION PROCUREMENT RETHINKING K–12 EDUCATION PROCUREMENT K–12 EDUCATION RETHINKING “The vexing question of why practices, programs, and products don’t take hold in classrooms, even when available and effective, is as salient as ever in the wake RETHINKING of the pandemic. This volume is full of new ideas and fresh takes on old ones. Taken together, they suggest a compelling multipronged approach to tackling the procurement challenge.” K–12 EDUCATION —Stacey Childress, CEO, NewSchools Venture Fund “While education procurement, as Frederick Hess and these leaders note, rarely PROCUREMENT gets attention, it is fundamental if we are serious about improving teaching and learning and creating more equitable classrooms. This volume addresses questions I have wrestled with from many angles, and it suggests practical solutions and ideas for how we can ensure what works is what gets used.” —Candice McQueen, CEO, National Institute for Excellence Why Promising Programs, Practices, in Teaching; Former Commissioner of Education, Tennessee and Products Seem to Rarely Get “Great ideas only make a difference for kids if educational leaders and teachers can bring them to a reality in the classroom. Too often, the procurement process Adopted, Implemented, or Used gets in the way of making great ideas a reality. Rethinking K–12 Education Pro- curement gives educators the tools necessary to make it happen for kids.” —Duncan Klussmann, Former Superintendent, Spring Branch Independent School District

“This is a compact, bracing, and intelligent book that crosses swords with our often-sclerotic K–12 system: Read it to be necessarily provoked.” —David Steiner, Executive Director, Johns Hopkins Institute Edited by Frederick M. Hess for Education Policy; Former Commissioner of Education, New York State

AMERICAN ENTERPRISE INSTITUTE