Incentives for Mayors to Improve Learning

Policy Research Working Paper 9509 Public Disclosure Authorized

Evidence from State Reforms in Ceará, Brazil Public Disclosure Authorized Ildo Lautharte Victor Hugo de Oliveira André Loureiro Public Disclosure Authorized Public Disclosure Authorized

Education Global Practice January 2021 Policy Research Working Paper 9509

Abstract Financial incentives for students, teachers, and schools are higher on mathematics and language tests. These impacts often used to promote learning. Yet, little is known about increase twofold when Ceará offers technical assistance to whether similar incentives for mayors produce analogous municipalities (pedagogical and managerial) and become findings. This paper investigates this question by exploring significant for fifth graders. These gains are seen among stu- a results-based financing reform in Ceará, Brazil, which dents in the top performance quantiles, but reformulating redistributes state resources to municipalities based on edu- the results-based financing rule to penalize municipalities cation performance. Comparing schools on both sides of with more low performers significantly reduces learning Ceará’s border over key implementation periods, the paper gaps. The paper discuss several mechanisms: the selection of shows that ninth grade students who were exposed to the school principals, teacher training, the provision and quality results-based financing performed 0.15 standard deviation of textbooks, curriculum coverage, and school homework.

This paper is a product of the Education Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at [email protected].

The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent.

Produced by the Research Support Team Incentives for Mayors to Improve Learning: Evidence from state reforms in Ceará, Brazil ∗

Ildo Lautharte† Victor Hugo de Oliveira‡ André Loureiro†

Keywords: Education Incentives for Mayors, Results-Based Financing, Technical Assistance.

JEL Classiﬁcation: H52, I21, I22.

∗We would like to thank Halsey Rogers, Pedro Cerdan−Infantes, Renata Lemos, Emanuela DiGroppelo, Caio Piza, Leandro Costa, Bruno Ottoni, and David Evans for the detailed comments. The authors gratefully acknowledge ﬁnancial support from the REACH Trust Fund at the World B ank. We also thank seminar participants at the World Bank, Instituto IDados, Todos Pela Educação and Brazilian Society of Econometrics for valuable feedback on early drafts of this manuscript. The views expressed by the authors do not reflect the views of IPECE or the World Bank. All errors are, naturally, our own. The authors gratefully acknowledge financial support from the REACH Trust Fund at the World Bank. †Education Global Practice, The World Bank Group ‡Instituto de Pesquisa e Estratégia Econoômica do Ceará 1 Introduction

Much has been written on how financial incentives can play an important role in promoting student learning. Numerous studies have examined policies rewarding students based on pre- defined learning goals (Fryer 2011; Bettinger 2012). Others tested, and compared, particular configurations of student incentives across different ages, types of incentives and timeframes (Levitt et al. 2016). A not smaller field investigates the implications of conditioning scholarships on school attendance (Kremer et al. 2009); how to design effective pay-for-performance programs for teachers

(Glewwe et al. 2010; Barlevy and Neal 2012); and the eﬀects of distributing school grants based on education outcomes (Das et al. 2013; Gordon 2004). Such an abundance contrasts sharply with relatively no progress made on the design and impacts of similar incentives focused on city mayors.

A context-specific reason for this imbalance may be the result of constitutional arrangements prohibiting top-down fiscal incentives tied to education performance. Or, when legally possible, it may not be politically feasible to design such incentives, because learning is argued to depend on factors outside of the control of the mayor (Hansen et al. 2004). Another explanation for limited empirical evidence is the lack of credible experiments linking financial incentives to education performance at the city level. This paper aims to investigate these gaps. It explores a state reform in Ceará, Brazil, in which state tax revenues are redistributed based on municipal competition on education outcomes. Different from many countries, Brazil provides an ideal opportunity to investigate education incentives to mayors because (i) there is administrative autonomy among federal, state and municipal governments and (ii) primary and lower secondary education are, in most cases, the responsibility of municipalities.

In 2007, the state government also started to implement non-mandatory technical assistance (TA) for municipalities to address the heterogeneous capacity of municipal governments to implement education programs. The TA was initially targeted at literacy, and in 2011 was expanded to include support for grades 3 to 5. The state government provided pedagogical and management support to the municipal secretariats of education, including elements like scripted materials for teachers, teacher training focused on classroom practice, eﬀective use of learning assessments, and knowledge exchanges between schools. Another important milestone of the education reforms in Ceará was the reformulation of the RBF rule in 2013 to penalize municipalities with higher percentages of students below minimum thresholds of performance in mathematics and language

(Portuguese).

Our empirical strategy explores this timeline and the location of schools across Ceará’s border in other states to identify RBF eﬀects on learning at the end of primary (5th grade) and lower secondary (9th grade) education. Using data from students, teachers, and principals from the

Sistema de Avaliação do Ensino Básico (SAEB), we compare the average differences in education outcomes on both sides of the border before and during each implementation stage.1 In total, this approach generates four parameters. To benchmark our findings and control for pre-existing disparities, we compute the baseline difference in pre-RBF periods (2005 and 2007). The second parameter refers to periods when the RBF was introduced without any complementary program in

2009. The third parameter measures the average difference in periods when the RBF was combined with TA (in 2011 for 5th grade, and in 2015 for 9th grade). The last difference considers the reformulation of the RBF rule in 2013 until 2017. We argue that, by comparing these parameters and controlling for student and municipal characteristics, time trends, and border fixed effects, it is possible to estimate whether financial incentives for mayors improved learning and the impacts of providing TA and reformulating the RBF rule.

1It is crucial to highlight two points in this strategy. First, SAEB is not part of the RBF rule in Ceará. Instead, Ceará bases the redistribution criteria on its own state assessment called SPAECE. SAEB is a national assessment with no influence on fiscal redistribution. In this context, using SAEB is empirically important, because it is less susceptible to gaming by municipalities. The second point concerns our definition of schools at the border. Throughout the analysis, we restrict our sample to schools located in the immediate three municipalities on each side of the border between Ceará and adjacent states (See Figure 1). We provide a set of robustness checks testing alternative border definitions.

3 This framework indicates positive and significant impacts of the RBF reform on student test scores. According to our estimates, students in 9th grade exposed to the RBF mechanism in Ceará presented 0.15SD higher scores in mathematics and Portuguese compared to similar students across the state border who were not exposed to the reform. This increase is equivalent, on average, to an additional 3 months of learning. In contrast, at this stage no clear improvement was observed for 5th graders. However, in periods when the RBF was combined with TA for municipalities, the effects of RBF alone almost doubled, and significant impacts on test scores for 5th grades emerge. The gains of combining RBF and TA for both grades is equivalent to 5 months of learning in Portuguese and 3 months in mathematics on the top of the effects of RBF alone. Further estimates, disentangling the TA component, indicate that most of these ‘’RBF + TA” effects come from providing training and textbooks to schools. Another particularity is the distribution of RBF effects. Quantile regressions show that RBF periods disproportionately benefited students at the top of the distribution, − increasing learning gaps. Providing TA slightly reduces these disparities, but only for periods when the reform reformulated the redistribution rule to penalize municipalities with higher incidences of students below basic levels of performance.

We investigate potential mechanisms for these ﬁndings in three groups. As local authorities in

Brazil are responsible for selecting and training local staff, we first check for changes in the selection and training of principals and teachers for schools at the border. The estimates show that mayors in Ceará are significantly less likely to directly assign principals to schools and are more likely to appoint them through formal selection processes (10pp−20pp). This result suggests that replacing political appointments in Ceará with a formal process may represent one channel by which the

RBF impacts operate (Mitra Akhtari 2020). Another observed channel is that enrollment rates and the quality of training significantly increased in Ceará relative to its counterparts. After offering TA, and relative to the baseline difference at the border, school principals and teachers were significantly more likely to enroll in TA (11.5pp for 5th grades and 8.8pp for 9th grades) and

4 report that it was useful for their daily work (7.8pp for 5th grades and 12pp for 9th grades).

Next, we analyse textbook quality and distribution. Using the SAEB questionnaire of principals, the estimates indicate that schools in Ceará are 10pp to 25pp less likely to report a lack of textbooks than similar principals on the other side of the border. This reduction only occurs after TA is introduced. At the same time, 5th grade teachers in Ceará report that textbooks are at least

“good” or “great” around 9.5pp more frequently than their counterparts. Considering concerns raised in previous literature regarding the impacts of textbooks on learning (Glewwe et al. 2009), note that the state of Ceará structured these textbooks to support over-age students. Thus, the distribution of good quality textbooks to municipal schools is argued to be a mechanism explaining our ﬁndings.

The last group of mechanisms analyzed is related to pedagogical practices. During “RBF+TA” periods, 5th grade students in Ceará became 2pp more likely to report that their teachers checked their homework (in both mathematics and Portuguese) than students studying in schools located in the circumventing municipalities outside of Ceará. For 9th grades no signiﬁcant result was found for this particular outcomes. Importantly, teachers in Ceará were around 8pp more likely to cover at least 80% of the school curriculum for both grades, and 11.6pp for 9th grades teachers, after the

RBF rule was reformulated in 2013.

We also show evidence that the introduction of the RBF increased public spending on education per pupil by 1.2% relative to the baseline mean in the municipalities of Ceará when compared to municipalities across the border. The introduction of RBF led mayors to invest in school facilities

(e.g., internet connections, computer labs, libraries, and sports courts). Moreover, the increase in public spending per capita on complementary public services is even larger, varying from 10.5% to

21% depending on the type of service. In contrast, the introduction of TA jointly with the RBF and the reformulation of the RBF mechanism did not translate into more education spending in

5 Ceará, and the results suggest that the large gains in student learning after introducing TA and reformulating the RBF rule were attained without increasing municipal public spending.

The paper is organized as follows. After this introduction, section2 provides background information on Ceará and explains the RBF reform that introduced education incentives for mayors.

Section3 describes the data and presents descriptive statistics. The identiﬁcation strategy is dis- cussed in section4. Section5 illustrates the RBF eﬀects, presents estimation results on test scores, and disentangles the components of TA. Section6 investigates several mechanisms explaining our results. Section7 concludes.

2 Background

The state of Ceará is located in northeast Brazil and has approximately 9.1 million inhabitants

(equivalent to Austria or Israel) living in an area of 148, 895km2 (equivalent to Bangladesh or

Greece). According to 2017 data, its GDP per capita reached R$16, 395 (or US$7, 378 PPP), similar to Vietnam (US$7, 448) and Cabo Verde (US$7, 454), which translates into the ﬁfth smallest

GDP per capita in Brazil. Three-fourths of its population is concentrated in urban areas, infant mortality is around 13.2 per 1, 000 live births, and life expectancy is 74.3 years − 2 years less than the national average.2

2.1 The Results−Based Financing Reform in Ceará

In this socioeconomic context, in December 2007 Ceará initiated a reform modifying how state consumption tax revenues were redistributed to municipalities. The tax in question is the Im- posto sobre Circulação de Mercadorias e Serviços (henceforth, ICMS), an important source of

2The information about economic activity, demography, and socioeconomic characteristics of Ceará is from Cavalcante et al.(2019) and de Oliveira et al.(2020).

6 tax revenue for sub-national governments that taxes services and family consumption.3 A crucial characteristic of the ICMS structure is that the Brazilian constitution establishes that 1/4 of

ICMS revenues must be redistributed to municipalities (or 6.25% of total ICMS revenues), referred to here as Quota−Parte. This redistribution should be based on discretionary state legislation.

Thus, the ﬁscal reform per se focused on changing the Quota-Parte redistribution rule and did not involve the reallocation of additional tax revenues.

Historically, the Quota-Parte rules have been largely based on a common set of variables: municipal population size, municipal area in kilometers, equal distribution, and percentage of treated trash residuals.4 The state government of Ceará enacted Legislation 14,023/2007 in December 2007 changing that pattern and introducing an results−based ﬁnancing (RBF ) model to redistribute

ICMS Quota-Parte based on education, health, and environmental performance (Holanda 2006).

The law deﬁned indicators for these RBF components and aimed to foster competition among city mayors for better outcomes in critical areas.

The RBF model can be described by the following equation:

QuotaICMS,c = 0.18 · Educc + 0.05 · Healthc + 0.02 · Environmentc

where QuotaICMS,c is the Quota−Parte for municipality c; Educc represents the education quality index for municipality c; Healthc represents average levels and improvements in infant mortality

3In most states, the ICMS tax rate varies signiﬁcantly per product or service, from 7% to 35% according to the state and product; it is the main source of revenue for state governments. 4Among the adjacent states of Ceará, Rio Grande do Norte has a Quota−Parte rule largely based on equitable distribution among municipalities, and a smaller fraction based on demographic and/or territorial variables. Other states, such as Piauí and Paraíba, use a similar rule but add a percentage for trash treated and an environment indicator (less than 10%). In the state of Pernambuco, until 2011 tax value added and conservation units also had weight. Still for Pernambuco, from 2011 on-wards 2% of the Quota−Parte was shared with municipalities based on education quality, and only recently, in Law 16, 616/2019, the quota increased to 18% until 2025, as is the case in Ceará. Despite the fact that Pernambuco has adopted the same RBF mechanism as Ceará, it will be only implemented in 2025. Excluding Pernambuco from our estimates results does not change the interpretation in any meaningful way (see Table A.2).

7 rates for municipality c, and Environmentc is a dummy variable indicating whether the municipal-

5 ity c has an operational solid waste management system . As can be seen, Educc receives, by far, the highest weight in the QuotaICMS,c redistribution rule: 18% out of 25% transferred to municipalities depends on education performance. It is also important to mention that the QuotaICMS,c is calculated by an independent think−thank (Instituto de Pesquisa e Estratégia Econômica do

Ceará - IPECE) using information collected by the state government. Municipalities do not collect or report on their Educc, Healthc, and Environmentc data.

This paper focuses on Educc because of its preeminence in the QuotaICMS,c redistribution rule. It has two main components: literacy rates (Literacy) and learning scores (Portuguese and Mathe- matics) in primary education (Learning). Literacy rates and learning scores are calculated with a state-managed assessment called SPAECE that is applied to students yearly to monitor the performance in basic education in Ceará. In the original RBF model, proposed in 2007, literacy performance is weighted twice as much as Learning performance to induce greater eﬀorts towards improvements in basic literacy as shown in the following Educc equation:

2 1 Educ = · Literacy + · Learning c 3 c 3 c

where Literacyc averages the level and time variation of literacy rates per municipality c. On

w literacy level (Lc ) the formula weights the average literacy rate (Lc) by SPAECE enrollment rates

w Enroll%,c (Enroll%,c) times half the standard deviation of Portuguese scores: Lc = Lc × c . Adding 0.5σL these two weights aims to reduce the potential negative selection of students into SPAECE and to punish municipalities with highly unequal learning performance. Based on the weighted literacy

w w w Lc −Lmin rates (Lc ), the ﬁnal index for level is Iliteracy,c = w w divided by the sum of Iliteracy,c for all Lmax−Lmin P municipalities, c Iliteracy,c. For the time variation component, the RBF formula considers the 5The methodology for computing these indexes was established by Holanda et al.(2008a,b).

8 w w 6 time diﬀerence between Lct and Lc(t−1) in the same equations.

The Learningc component of the RBF formula has the same structure as the Literacyc component.

w w It also considers levels and time variation at the municipal level but, instead of Lc and Lct, the

th w w formula considers weighted test scores in 5 grade named Tc and Tct . All underlying calculations to find ILearning,ct follow the same procedures as for Iliteracy,ct. But different from Literacyc, this component includes pass rates (Enroll%,c). More explicitly, the final Learningc indicator becomes:

Enroll%,c Ilearning,c ∆Ilearning,ct Learningc = 0.2 · P + 0.8 · 0.4 · P + 0.6 · P . c Enroll%,c c Ilearning,c c ∆Ilearning,ct

In summary, the original RBF model shows that municipalities with greater improvements in literacy rates, pass rates, and learning scores will receive a higher fraction of Quota − P arte than municipalities with lower overall performance. However, if better test scores are achieved by restricting the participation of students in SP AECE or by widening the gap between high and low performers, municipalities are penalized with a lower Quota−P arte. Overall, the RBF model intends to generate incentives to local authorities to amplify the coverage of students participating in standard exams and, at the same time, reduce learning disparities.

2.2 The Reformulation of the RBF Model

The RBF formula described above was partially reformulated in 2012 by the state government of

Ceará. The new decree, 30,796/2011, took eﬀect in 2013 and focused on the Educc component, with Healthc and Environmentc unaltered. In this decree, four reformulations were implemented:

(i) Enroll%,c was removed from Learningc and added to the Educc component (weighting 0.05);

(ii) Literacyc replaced the weight for levels by 0.75 and for time variation by 0.25, instead of

6 Explicitly, the Literacyc equation is illustrated in the RBF mechanism:

Iliteracy,c ∆Iliteracy,ct Literacyc = 0.5 · P + 0.5 · P c Iliteracy,c c ∆Iliteracy,ct

w w NAc where ∆Iliteracy,ct = L − L . Enroll = NAm and NEm are the number of pupils assessed and ct c(t−1) %,c NEc enrolled in the municipal network, respectively.

9 0.5 each; (iii) Learningc averaged test scores in Mathematics and Portuguese. These ﬁrst three modiﬁcations focused on increasing the importance of learning levels in the RBF mechanism, inducing equity and preventing gaming (Loureiro et al. 2020).7

The last modiﬁcation (iv) substitutes the standard deviation in literacy rates (Lw) and test scores

(T w) with the percentage of students below critical levels of learning. Along with simplifying the

RBF formula, this particular change reﬂects the low response of σc to local policies compared to the percentage of students below thresholds of performance. In speciﬁc terms, Literacyc replaces

c 3 1 2 σL by (1 − Lcritical,c) · (1 − Lpartial,c) · (1 + Lsatisf,c) , where Lcritical,c is the proportion of students deﬁned as illiterate, Lpartial,c is the proportion of students deﬁned as partially literate, and Lsatisf,c refers to the proportion deemed satisfactorily literate. For Learningc, the weight becomes (1 −

2 2 P ortlow,c) ·(1−P orthigh,c) for Portuguese and Mathematics follows the same structure. In general terms, the reformulation concentrates the ﬁscal incentives of Quota−Parte in the lower tail part of the distribution by giving more weight to municipalities seeing improvements in students who are lagging behind.

2.3 Technical Assistance

In 2007, the State of Ceará established the goal that "all students in public schools should read by the end of 2nd grade (age 7)".8 Unlike most Brazilian states, in 2007 the large majority of primary

7After these modiﬁcations, the RBF model has the form:

Enroll%,c Educc = 0.5 · Literacyc + 0.45 · Learningc + 0.05 · P c Enroll%,c

The second change was in the weights of the Literacyc formula:

Ilit,c ∆Ilit,ct Literacyc = 0.75 · P + 0.25 · P c Ilit,c c ∆Ilit,ct

The third modiﬁcation adds mathematics to the Learningc formula:

Learningc = 0.5 · P ortuguesec + 0.5 · Mathematicsc

where P ortuguesec and Mathematicsc are respectively the indicators of learning in Portuguese and Mathematics. 8Formally, the program is called Programa de Alfabetização na Idade Certa (PAIC).

10 education students in Ceará were enrolled in public schools managed by municipalities.9 Based on this ambitious goal, the government of Ceará elaborated a non−mandatory package of technical assistance (TA) available to all municipalities. This package has three fundamental components:

1. Literacy Support to Municipalities: Provide structured literacy textbooks10 and teacher

training focused on classroom practice.

2. Knowledge Exchange Practices: (i) the provision of training for education staﬀ, with a focus

on the management of education systems and schools; (ii) solid monitoring & evaluation of

education policies; (iii) incentives for knowledge exchange of best practices among schools;

(iv) support to reform teacher careers; and (v) promotion of meritocratic selection for school

principals.

3. Pedagogical Use of Learning Assessments (SPAECE): state level diagnostics of literacy levels

at the beginning and end of each school year to guide pedagogical action.

To coordinate these activities, the state secretariat of education in Ceará created a collaborative governance arrangement structured in three levels. The ﬁrst is the state coordination team that hires specialists to train and guide regional teams. The regional teams, for instance, disseminate the training to municipal teams (second level) that are, in turn, responsible for training teachers

(third level).11 Because these training activities focus on the use of structured materials and classroom activities12, the empirical section investigates each component separately: the incidence and quality of teacher training, curriculum coverage, types of principal selection processes, pedagogical

9In the following years, many states accelerated the process of devolving public primary schools to municipal administrations. As of 2018, all states had the majority of primary education students enrolled in municipal schools; in many states this percentage is close to 100%. 10These materials proposes detailed guidance on time-use, activities, and pedagogical routines for each class. The materials are aligned with the state curriculum, prioritization of basic skills, and a particular focus on literacy at the right age. 11For more details on the technical assistance from the state secretariat of education to municipal governments in Ceará, see (Loureiro et al. 2020). 12These activities involved classroom observations by trainers based on text books used daily in the classroom.

11 practices, and quality of materials.

3 Data and Descriptive Statistics

Our main source of information is the Sistema de Avaliação da Educação Básica (SAEB) provided by the Brazilian Ministry of Education between 2005 and 2017. SAEB is a learning quality assessment implemented biannually in Brazil that provides standard test scores in Mathematics and Portuguese, and collects detailed information about students, teachers, and school principals.

The SAEB assessment is undertaken at the end of each educational cycle: 5th grade of primary education, 9th grade of lower secondary education, and 3rd grade of upper secondary education.

Because virtually only primary and lower secondary education are managed by municipalities in

Ceará, and are therefore part of the RBF mechanism, our regression analysis is restricted to 5th and 9th grades. Importantly, to reduce the influence of gaming in our findings, note that SAEB is not part of RBF indicators in Ceará. Instead, the RBF uses an independent state-level assessment called SPAECE. An additional advantage of SAEB is to provide unique city identifiers that allow us to explore geographic properties of schools in our identification strategy and control for city

ﬁxed eﬀects. In 2005, only a representative sample of schools participated in the assessment; from

2007 on-wards it included all public schools with at least 30 students enrolled in the 5th or 9th grade.

The rich information in SAEB allows investigating a comprehensive set of outcomes. At the student level, it provides general characteristics such as gender, age, and race and speciﬁc household characteristics as single parenting, household size, number of bathrooms, and parental education.

The teacher questionnaire asks about their level of qualiﬁcation and experience in education, recent participation in training programs, basic contract characteristics, and pedagogical practices in the classroom (e.g., homework correction, materials used, and curriculum coverage). For school prin-

12 cipals, SAEB similarly provides data on qualiﬁcations, experience, and participation in training, but adds how principals were selected and the provision of textbooks. Exploring this information allows us to investigate some mechanisms explaining the RBF eﬀects on learning outcomes.

As our empirical strategy compares schools that are geographically close during the critical periods of the RBF reform, we restrict the working sample to municipalities at the border of Ceará and adjacent states: Piauí, Rio Grande do Norte, Paraíba and Pernambuco. However, the narrower the deﬁnition of ‘’border", the smaller is the sample power. Or alternatively, a wide deﬁnition of

‘’border" may not effectively control for unobservable factors explaining the differences in learning performance between schools. Thus, we define border as the first three municipalities on each side of the Ceará border. By using this definition, approximately one-third of the municipalities in these five states become part of the working sample (see Figure 1 for an illustration of all municipalities in the sample). Table 1 presents descriptive statistics using this definition.

An immediate evidence from Table 1 is that, during the baseline (2005 − 2007), there is no systematic diﬀerence between Ceará and its adjacent states. This is valid for 5th grade or 9th grade, test scores, and student characteristics. Other evidence is that for RBF periods from 2009 to 2017, test scores in Ceará steadily increase relative to adjacent states, especially in “RBF (re) + TA” periods. Considering students in Panel A, around 50% are boys, the students are around 11 years old and, on average, there are 5 people per household. When the re-formulated RBF started

(2013), the diﬀerence in student performance in test scores between Ceará and adjacent states is

7.6 points in Portuguese and 9.2 points in Mathematics comparing 2013 to 2011.

We complement the empirical analysis by merging additional data-sets. In terms of including school−level controls, a useful source of information is the Censo Escolar, which provides comprehensive information on school infrastructure and unique ids that allow us to merge it with the

SAEB data. For municipal-level controls, we include data on GDP per capita and population size

13 Table 1 Test Scores and Student Characteristics at the Border of Ceará

Panel A. Students in 5th grade Period 1. Baseline 2. RBF 3. RBF + TA 4. RBF (re) + TA Years (2005/07) (2009) (2011) (2013/15/17) Ceará Adj. States Ceará Adj. States Ceará Adj. States Ceará Adj. States

Portuguese Scores 157.7 155.9 168.9 163.3 183.7 169.0 208.2 183.2 (37.7) (35.5) (42.1) (37.1) (45.4) (38.8) (52.8) (45.1) Mathematics Scores 172.5 172.5 184.8 180.8 201.6 184.6 220.7 194.4 (38.2) (37.0) (44.7) (39.1) (48.4) (38.8) (54.6) (42.8) % of Boys 0.50 0.49 0.52 0.51 0.52 0.51 0.51 0.51 (0.50) (0.50) (0.49) (0.50) (0.49) (0.50) (0.50) (0.50) % of White Students 0.27 0.29 0.27 0.28 0.24 0.25 0.22 0.26 (0.44) (0.45) (0.44) (0.45) (0.43) (0.43) (0.41) (0.44) Student Age (in years) 10.9 11.1 11.0 11.3 11.0 11.2 10.8 11.1 (1.31) (1.53) (1.37) (1.52) (1.25) (1.44) (1.05) (1.27) Household Size 5.02 4.98 4.99 4.97 5.07 5.10 5.40 5.37 (1.13) (1.09) (1.13) (1.10) (1.39) (1.36) (1.26) (1.31) % of Mothers with Diploma 0.12 0.14 0.11 0.13 0.12 0.14 0.17 0.18 (0.33) (0.34) (0.31) (0.33) (0.33) (0.35) (0.37) (0.38) # of Student Observations 37,437 17,352 47,768 22,011 50,346 21,401 129,350 82,395 Panel B. Students in 9th grade Period 1. Baseline 2. RBF 3. RBF (re) 4. RBF (re) + TA Years (2005/07) (2009/11) (2013) (2015/17)

Portuguese Scores 211.0 215.2 226.0 223.0 237.9 227.3 257.8 239.6 (40.3) (40.5) (44.0) (42.5) (46.3) (43.7) (47.4) (44.5) Mathematics Scores 222.8 228.4 230.8 229.8 242.2 231.9 259.6 240.3 (38.9) (40.4) (44.8) (42.1) (48.8) (42.8) (51.1) (43.0) % of boys 0.45 0.40 0.46 0.41 0.48 0.44 0.48 0.46 (0.49) (0.49) (0.49) (0.49) (0.50) (0.49) (0.50) (0.49) % of white students 0.24 0.27 0.22 0.25 0.20 0.23 0.20 0.22 (0.43) (0.44) (0.41) (0.43) (0.40) (0.42) (0.40) (0.41) Student Age (in years) 15.2 15.6 15.2 15.4 14.9 15.1 14.9 15.1 (1.56) (1.76) (1.31) (1.50) (1.18) (1.35) (1.01) (1.24) Household Size 5.23 5.06 4.99 4.90 5.40 5.46 5.31 5.38 (1.67) (1.63) (1.53) (1.49) (1.24) (1.21) (1.22) (1.20) % of Mothers with Diploma 0.12 0.16 0.13 0.17 0.17 0.19 0.19 0.22 (0.33) (0.36) (0.33) (0.38) (0.38) (0.39) (0.39) (0.41) # of Student Observations 23,285 10,793 81,125 27,958 40,409 14,619 84,395 32,747 Notes. This table presents descriptive statistics of students at the border between Ceará and adjacent states. All data is from the Sistema de Avaliação da Educação Básica (SAEB), a national assessment implemented biannually. In additional to Ceará, we consider the following adjacent states: Piauí, Rio Grande do Norte, Paraíba and Pernambuco. The definition of border considers the first three municipalities on each side of Ceará’s border. Panel A considers 5th graders. Panel B is for 9th graders. The baseline period of 2005 and 2007 is when neither Ceará nor adjacent states had implemented RBF incentives. The column "RBF " considers 2009 in Panel A, because the RBF law started on January 1, 2009. For 5th grades, technical assistance (TA) initiated in 2011, presented in column "RBF + TA" in Panel A. In Panel B, the column ‘’RBF (re)" presents the statistics for 9th grade when the RBF formula was re-calibrated in 2013, but TA was offered only to 5th grade classes. In Panels A and B, periods of re-calibration and TA are shown in columns "RBF (re) + TA". at the municipal level from the Instituto Brasileiro de Geografia e Estatística (IBGE) and live birth records from the Ministry of Health.13 To investigate potential RBF mechanisms, we use municipal public spending data provided by the National Treasury Bureau (Secretaria do Tesouro

13Municipal data on live births can be obtained at the url: http://www2.datasus.gov.br/DATASUS/index. php?area=0205.

14 Nacional - STN).14.

4 Identiﬁcation Strategy

The results−based financing (RBF) reform in Ceará configures an opportunity to estimate if education incentives for mayors triggers student learning. Given that such a reform occurred at the state level and at clear points in time allows us to explore empirically the geographic and time characteristics in our identification strategy. That being said, we start by restricting our sample to schools located in the three immediate municipalities at the border between Ceará and adjacent states (Piauí, Paraíba, Rio Grande do Norte and Pernanbuco). Figure 1 illustrates the municipalities included in the analysis. By limiting our sample to municipalities that are geographically close, but on different sides of the border, we intend to minimize the influence of time-invariant unobserved characteristics of schools and municipalities that affect student learning.

Put simply, schools right across the border serve as a good counterfactual for the case that the

RBF reform had not been implemented in Ceará.

Another useful characteristic of the Ceará reform is its clear timeline. We leveraged that the

RBF and the technical assistance (TA) component were introduced at defined dates to compare the evolution of learning outcomes at the border at different points in time. To make this point clearer, consider the RBF and TA reform for 5th grades. In the first moment, the state congress enacted the RBF reform in mid-December 2007; so, from January 2008 on-wards, a fraction of

Quota-Parte revenues started to be redistributed to municipalities based on learning outcomes.

Then, in 2011, schools with students from 3rd − 5th grade received a non−mandatory package of training and pedagogical materials (the TA component). In a third and ﬁnal stage, in 2012 the RBF formula was re-formulated to penalize municipalities with higher percentages of students

14Public spending data can be accessed at https://siconfi.tesouro.gov.br/siconfi/index.jsf.

15 performing below pre-deﬁned minimal thresholds on test scores.

By exploring the state borders and timeline, we identify the RBF effects by comparing the average difference between schools at the border at every stage of the reform in Ceará relative to the average difference at the border during baseline years (2005 − 2007). We estimate the following regression for 5th grade:

pre rbf yict = αborder + αyear + tc + β (PRE-RBFt × Cearac) + β (RBFt × Cearac)

rbf+ta rbf(re)+ta + β ([RBF -TAt] × Cearac) + β [RBF(re)-TAt] × Cearac + γXi + εict, (1)

where yict is the outcome of interest; Cearac is a dummy variable equal to 1 for municipalities in Ceará − i.e., the ﬁrst three municipalities on the Ceará’s side of the border − and zero for municipalities on the opposite side. The variable PRE-RBFt indexes years 2005 and 2007, the

Figure 1: Municipalities at the Border between Ceará and Adjacent States

16 baseline years; RBFt equals one for 2009, two years after the RBF was introduced in Ceará.

The dummy RBFt-TA equals one for 2011, when the TA to municipalities was initiated; and

RBF (re)-TA refers to the period after 2013, when the RBF formula was re-calibrated.

Eq. (1) also includes a set of fixed-effects and time trends. αborder is a border fixed-effect that captures unobservable characteristics shared by municipalities at the same border. Including this variable is important, because it makes the RBF impacts emerge from comparing schools that are geographically close instead of simply averaging both sides of the frontier. αyear is a time trend and tc is a city−specific time trend. Time-varying controls in Xi include a set of student (age, race, mother education) and school (if rural school and a dummy if the school offers primary and lower secondary education) characteristics. Finally, εict is the error term clustered by municipality to allow for contemporaneous spatially correlated shocks.

There are four coefficients on which the analysis is based. The baseline difference βpre captures pre-determined differences between Ceará and adjacent states prior to the reform. The parameter

βrbf estimates the average diﬀerence in the RBF periods before the TA was introduced (2009 for

5th grade, and 2009 to 2011 for 9th grade). Next, βrbf+ta represents the estimated difference when the RBF and TA were offered, and βrbf(re)+ta shows the average difference when the RBF formula was reformulated. Based on these coefficients, we report three parameters of interest. First, we estimate the “RBF effect” by βrbf − βpre. This estimate discounts from the average difference in

RBF periods the average difference during the baseline years. The second is βrbf+ta − βrbf which measures the incremental effect of TA and RBF on education outcomes. Throughout the analysis, we report this difference as “RBF and Technical Assistance”. Lastly, βrbf(re)+ta − βrbf+ta captures the effect of reformulating the RBF mechanism, and “Results−Based Financing, re” alludes to this parameter in our tables.

However, using these diﬀerences to estimate causal eﬀects is only valid under at least two assump-

17 tions. Primarily, it is assumed that the baseline diﬀerence βpre would have remained unaltered if the RBF reform had not been implemented. This assumption echoes the necessary condition that outcomes should have the same trend in the absence of treatment. Secondly, no parallel shock in education outcomes occurred in Ceará at the same time that RBF and TA were introduced. If that is the case, Eq. (1) will erroneously attribute to RBF /TA what is in fact the impact of a contemporaneous shock on education outcomes.

5 Estimation Results

This section starts by presenting the estimated impacts of results−based financing and technical assistance on Mathematics and Portuguese scores. It graphs the evolution of test scores for students at the border across time and presents the results for 5th and 9th graders. In the following section, we disentangle the effects of TA components on test scores to understand the mechanisms explaining our findings.

5.1 Student Test Scores

The eﬀects of RBF on test scores are presented in Figure 2. It plots average scores of Mathematics and Portuguese from 2005 to 2017 for municipalities at the border, in accordance with Figure 1.

As the federal government applies SAEB biannually and at the end of each school cycle, Figure 2 only includes students in the 5th and 9th grades. Dashed vertical lines indicate each stage of the education reform in Ceará. The ﬁrst line marks when the RBF model starts. The next two lines refer to the introduction of a non-mandatory TA component and the reformulation of RBF (note that these dates diﬀer depending on the grade).

A ﬁrst indication of how education incentives to local authorities aﬀect test scores is presented

18 A. Portuguese, 5th Grade B. Mathematics, 5th Grade

C. Portuguese, 9th Grade D. Mathematics, 9th Grade

Figure 2: Student Test Scores at the Border of Ceará

Notes. This figure shows average test scores in Mathematics and Portuguese for students in the 5th and 9th grades from 2005 to 2017. Data on test scores is from Sistema de Avaliação de Educação Básica (SAEB). The empirical analysis uses SAEB data because it is applied by the federal government independently every two years and is not part of the RBF mechanism in Ceará. All figures only include municipalities at the border between Ceará and adjacent states (See Figure 1). Blue lines indicate Ceará, while red lines represent municipalities on the other side of the border. Figures A and B are for students in the 5th grade. Figures C and D are for students in the 9th grade. Portuguese scores are presented in Figures A and C, and Figures B and D plot Mathematics scores. Dashed vertical lines indicate when the RBF reform, Technical Assistance (TA) and the RBF reformulation were initiated for each grade. in Figure 2. During the baseline years (2005 − 2007), Figures 2.A and 2.B show no significant difference in test scores between students in municipalities at the border. This conclusion does not change, even when the government of Ceará introduces the RBF mechanism in 2009: the

19 average difference in test scores at the border remains equal when only the RBF mechanism is in place. However, still in Figures 2.A and 2.B, significant differences start to emerge only after TA is offered in 2011 and reinforced after RBF is reformulated in 2013 (see Section 2 for details). As the regression analysis indicates, the provision of TA and the introduction of weights for students below minimum thresholds of performance in the RBF formula had an essential role in these improvements.

The scenario for 9th grades in Figures 2.C and 2.D is slightly diﬀerent than 5th grades. In baseline years, 9th grade students in Ceará performed signiﬁcantly lower than those at the border of adjacent states. Additionally, Figure D shows a clearer case for parallel trends during pre-RBF years than Figure C, despite the fact that formal testing cannot reject the equal trends hypothesis before

RBF was introduced. This diﬀerence in test scores vanishes entirely in RBF years (2009 − 2011) and reﬂects increases in test scores in Ceará that are not followed by circumventing municipalities.

In fact, and similar to 5th graders, reformulating the RBF model in 2013 and introducing the TA component from 2015 to 2017 consolidated these improvements in the following years in Ceará relative to its counterparts.

Table 2 complements these ﬁndings by presenting the RBF eﬀects on test scores using Eq. (1).

Columns (1) and (3) present the impacts on Mathematics scores. Columns (2) and (4) are for

Portuguese scores. Impact estimates for 5th graders are presented in columns (1) and (2) and for 9th graders in columns (3) and (4). The baseline difference between students in Ceará and across the border is shown as “Baseline Difference (βpre)", and a parallel trends test in pre-RBF years is presented as “Diff(2007) − Diff(2005)". Standard errors are clustered by municipality and p−values are presented in brackets.

At least three main conclusions are drawn from Table 2. Panel A shows that the RBF alone in

Ceará increased test scores only for students at the end of lower secondary (9th grade). In this

20 Table 2 The Effects of Results−Based Financing on Test Scores (Clustered Standard Errors)

Mathematics Portuguese Mathematics Portuguese 5th Grade 5th Grade 9th Grade 9th Grade (1) (2) (3) (4)

A. DD, Results−Based Financing (RBF) 0.036 0.061∗ 0.155∗∗∗ 0.153∗∗∗ βrbf − βpre (0.040) (0.037) (0.026) (0.023) B. DD, RBF and Teaching Assistance 0.220∗∗∗ 0.162∗∗∗ βrbf+ta − βrbf (0.029) (0.021) 0.148∗∗∗ 0.118∗∗∗ βrbf(re)+ta − βrbf(re) (0.032) (0.028) C. DD, Results−Based Financing, re 0.170∗∗∗ 0.166∗∗∗ βrbf(re)+ta − βrbf+ta (0.038) (0.032) 0.092∗∗ 0.096∗∗∗ βrbf(re) − βrbf (0.035) (0.028)

Baseline Difference (βpre) −0.022 −0.013 −0.199 −0.142 [0.595] [0.738] [0.000] [0.000] Diff(2007) − Diff(2005) 0.069 0.080 −0.022 0.018 [0.512] [0.343] [0.891] [0.853]

City/Student Controls Yes Yes Yes Yes City−Specific Time Trend Yes Yes Yes Yes Border Fixed−Effects Yes Yes Yes Yes # of Student Observations 233, 830 223, 820 194, 852 194, 862 Adj. R−squared 0.188 0.247 0.149 0.185 Notes. ∗ ∗ ∗ denotes significance at 1 percent, ∗∗ at 5 percent, and ∗ at 10 percent level. This table shows the estimated impacts of the results−based financing (RBF) reform on student test scores. Data on test scores is from the Sistema de Avaliação da Educação Básica (SAEB), a national standardized exam undertaken by the federal government every two years. Importantly, SAEB is not part of the RBF mechanism in Ceará (instead, they use the state assessment called SPAECE) and for this reason is more appropriate to minimize potential gaming. All regressions restrict the sample to the first three immediate municipalities on each side of the border between Ceará and adjacent states as shown in Figure 1. The municipality controls are: number of births in the municipality−year (Sistema de Nascidos Vivos (SINASC)); a dummy for rural schools (from SAEB); and a dummy for schools offering grades 1 − 9 (School Census). Student controls come from SAEB and include: a dummy if the student is male, if the student is white, age in years, number of bedrooms, if the mother has higher education and their household size. The estimates also include time trends and city-specific time trends. Border fixed-effects introduces dummies for each border between Ceará and its adjacent states (see the different colors in Figure 1). By including these dummies, the regressions compare students at the same border instead of simply averaging test scores on both sides of the border. Panel A presents DD estimates of RBF impacts. Panel B refers to the effects after introducing the TA component. Panel C shows DD estimates after the RBF formula included weights for the percentage of students below minimum thresholds of performance in Mathematics and Portuguese. The baseline difference at the border, βpre, and a parallel trend test estimating changes in the trends during pre-RBF periods is shown at the bottom of each column. Standard errors are clustered by municipality and shown in parenthesis. P−values are shown in brackets.

21 case, the RBF generates an impact of approximately 0.15 standard deviation on Mathematics and

Portuguese scores. No impacts are found for students in 5th grade. As shown in Figures 1C and

1D, these increases close the score gaps at the border observed during pre-RBF periods. To put these results into another perspective, the RBF eﬀect on 9th grade translates into an increase equivalent to 3.7 months of schooling in Mathematics and 3 months in Portuguese.15

The second ﬁnding relates to the gains of providing TA to municipalities. Results from Panel

B show that 5th graders have a significant improvement of 0.22SD (5 months of schooling) in mathematics and 0.162SD (3 months of schooling) in Portuguese relative to the difference at the border when only the RBF was in place. For 9th grade, columns (3) and (4) illustrate a similar conclusion: the TA component almost doubled the RBF impacts in Panel A. One explanation for such gains is that the TA component provides pedagogical materials, teacher training, and knowledge exchange between schools, which benefit municipalities with particularly low capacity to implement similar policies.

Finally, Panel C suggests that reformulating the RBF mechanism by penalizing higher percentages of students performing below minimum levels in SPAECE generates further increases on test scores. Intuitively, this reformulation may reflect that mayors and schools focus on the students who are significantly lagging behind. Observing the estimates in Panel C reinforces this interpretation. When the reformulation occurred without the support TA (βrbf(re) − βrbf ), the impacts are approximately 0.092SD (2.2 months of schooling) for 9th grade. Yet, when minimum thresholds are introduced in the RBF formula and the TA component is available to municipalities, the impacts on test scores are twice as large (columns (1) and (2), Panel C). Thus, it seems that if mayors understand which students have a stronger impact on the RBF redistribution and TA is available, there are significantly larger impacts on test scores.

15We calculated schooling-equivalent months by the diﬀerence in SAEB test scores for the same cohort of students in municipalities outside Ceará between 2007 (when students were in 5th grade) and 2011 (when these students were in 9th grade) divided by two. According to the exercise, every two years represents an average gain of 0.47 − 0.495 standard deviation in mathematics and 0.56 − 0.62 in Portuguese.

22 Two points about these findings are important to highlight. If mayors orient teachers to teach to the SAEB test, our impact coefficients will capture the ability of mayors to artificially influence test scores rather than improvements in learning (for a general discussion, see Barlevy and Neal

(2012); Jacob 2005; Glewwe et al. 2010; Carrell and West 2010). However, we argue that this form of gaming is unlikely, because the RBF mechanism is based on SPAECE (the state−level assessment in Ceará) while our estimates use SAEB (the Brazilian national assessment). As SAEB scores have no influence on the RBF formula and Quota − P arte redistribution in Ceará, mayors have no incentive to game using the test to receive additional funding. Another relevant point is that these conclusions remain consistent after considering different sample definitions, alternative controls, time trends, and combinations of municipalities at the border (see Tables A.1 and A.2 in the appendix for the results).16

Another legitimate concern, inherent in outcomes−based policies, is whether the RBF mechanism increases learning gaps between low and high performing students. That might be the case, for example, if mayors were to pool their resources to reach high−achieving students or schools to increase average test scores. An opportunity to check whether the RBF reform widened learning gaps is by re-estimating Eq. (1) per quantile. Figure 3 presents the parameters of interest and their respective 95% conﬁdence interval for 5th and 9th grades in Portuguese and Mathematics per quantile.

Figure 3 outlines three aspects of RBF effects on low-performing students. First, the RBF alone tends to increase learning gaps, especially for students in the 5th grades. In Figure 3.A, the effects of RBF measured by βrbf − βpre become significant only for quantiles above 0.6, while it is not different from zero for students below this point. Secondly, introducing the TA component

16An additional concern is whether our estimates are capturing changes in student cohorts in Ceará rather than in performance. To verify this hypothesis, we test whether enrollment rates in public and private schools (in log terms) change after the introduction of RBF and the TA. Table A.3 in the appendix shows that this is not the case. There is no signiﬁcant change in enrollment rates in public schools in Ceará or negative changes in enrollment rates in private schools.

23 βrbf − βpre βrbf+ta − βrbf βrbf(re)+ta − βrbf+ta

(A) − Mathematics, 5th Grade

βrbf − βpre βrbf+ta − βrbf βrbf(re)+ta − βrbf+ta

(B) − Portuguese 5th Grade

rbf(re)+ta rbf(re) βrbf − βpre βrbf(re) − βrbf β − β

βrbf − βpre βrbf(re) − βrbf βrbf(re)+ta − βrbf(re)

(D) − Portuguese 9th Grade

Figure 3: The effects of RBF on Test Scores per Quantile

24 in the RBF model (Figures 3.A and 3.B) aﬀects signiﬁcantly low-achieving students but is still not enough to narrow the learning gaps − i.e., βrbf+ta − βrbf is around three times as large for

90th quantile students compared to 10th quantile students. Lastly, including in the RBF formula an explicit measure of students below minimum performance thresholds tends to decrease the learning gaps, particularly for students in 9th grade. In this case, the eﬀects for students in the top quantiles are only slightly larger than students in low quantiles, or even smaller when considering Portuguese scores in 9th grade. Altogether, Figure 3 indicates that when the RBF formula conditions the redistribution of resources to the performance of students at the lowest bottom (minimum thresholds), it tends to reduce, but not eliminate, the learning enlarged by the

RBF mechanism.

5.2 The Eﬀects of Technical Assistance

One explanation for our results is that reformulating the RBF formula and providing technical assistance to municipalities generated larger and more equitable impacts on test scores. As dis- cussed in Section 2, it might be a consequence of the design of the TA program that supports students who are lagging behind in each grade by providing structured materials to schools, oﬀering teacher training and promoting knowledge exchange between schools (Escola Nota 10 program).

This section investigates which of these components plays a more important role in RBF impacts on learning.

For that, the ﬁrst step replaces the RBF − TA dummy in Eq. (1) by three variables indicating each TA activity: one indexing the percentage of teachers that received textbooks and another calculating the percentage of teachers participating in training in the last two years − both as percentages at that school level, and a dummy for schools that participated in the knowledge exchange program Escola Nota 10. Note that in excluding the knowledge exchange dummy, we

25 are unable to measure directly if textbooks or teacher training are oﬀered by the state TA program

(PAIC) or a local program in the municipality. So, we only measure TA components indirectly, as textbooks and teacher training can be similarly oﬀered in municipalities on both sides of the border.

As a next step, we interacted these variables with Cearac to account for diﬀerences at the border between municipalities in Ceará and adjacent states (see Figure 1), exactly as performed in Eq. (1).

We therefore estimate the impact of each TA component by using the difference−in−differences in learning scores of: (i) the difference in performance between schools/teachers reporting one TA component at the border; and (ii) the difference in performance between schools at the border during the RBF periods. Table 3 presents the estimated impacts using this strategy for 5th and

9th grades.

Results from columns (1)−(4) in Table 3 add important details to our previous ﬁndings. Combining the RBF incentives for local mayors with the provision of textbooks to teachers seems to account for a great part of TA eﬀects on learning scores in Panel B. As indicated in columns (1) and (2) in Panel

B, if all teachers received textbooks, test scores would increase by 0.162SD for Portuguese and

0.253SD for Mathematics, equivalent to around 4 months of learning. In contrast, participation in the knowledge exchange program or providing teacher training does not show signiﬁcant impacts.

A possible explanation for absent impacts of teacher training may emerge from the results in

Panel C. After the RBF takes into account clear thresholds of minimum student performance at schools, teacher training shows a strong and signiﬁcant eﬀect on test scores. For example, for 5th grade, its impacts are twice as large as providing textbooks to teachers: an increase of 0.244SD for Portuguese and 0.290SD for Mathematics, or around 5 months of learning equivalent. The impacts are similarly high for students in 9th grade, as seen in columns (3) and (4).

26 Table 3 The Effects of Technical Assistance Components on Test Scores (Clustered Standard Errors)

Portuguese Mathematics Portuguese Mathematics 5th Grade 5th Grade 9th Grade 9th Grade (1) (2) (3) (4)

A. DD, Results−Based Financing (RBF) 0.033 0.012 0.156∗∗∗ 0.149∗∗∗ βrbf − βpre (0.043) (0.042) (0.025) (0.028) B. DD, RBF + Technical Assistance

−0.019 0.022 Knowledge Exchange, βrbf+Knowledge − βrbf (0.054) (0.067) 0.007 −0.002 Teacher Training (%), βrbf+T raining − βrbf (0.050) (0.061) 0.162∗∗∗ 0.253∗∗∗ Textbooks (%), βrbf+T extBooks − βrbf (0.055) (0.064)

C. DD, Results−Based Financing, re

0.127∗∗∗ 0.167∗∗∗ βrbf(re) − βrbf (0.029) (0.032) 0.016 0.103 −0.023 0.160 Knowledge Exchange, βrbf(re)+Knowledge − βrbf (0.071) (0.088) (0.182) (0.294) 0.244∗∗∗ 0.290∗∗∗ 0.182∗∗∗ 0.177∗∗ Teacher Training (%), βrbf(re)+T raining − βrbf (0.045) (0.054) (0.061) (0.071) 0.109∗ 0.128∗ 0.045 0.144∗∗ Textbooks (%), βrbf(re)+T extBooks − βrbf (0.055) (0.066) (0.067) (0.073)

City/Student Controls Yes Yes Yes Yes City−Specific Time Trend Yes Yes Yes Yes Border Fixed−Effects Yes Yes Yes Yes # of Student Observations 228, 418 228, 428 191, 285 191, 275 Adj. R−squared 0.244 0.184 0.187 0.150 Notes. ∗ ∗ ∗ denotes significance at 1 percent, ∗∗ at 5 percent, and ∗ at 10 percent level. This table shows the impacts of technical assistance on test scores for 5th and 9th grades. Test scores are from Sistema de Avaliação da Educação Básica (SAEB), the national standardized exam undertaken every two years. All regressions restrict the sample to students in municipalities at the border as indicated in Figure 1. The municipality controls are: the number of births per year (Sistema de Nascidos Vivos, SINASC); a dummy for rural schools (SAEB); and a dummy for schools offering grades 1 to 9. Student controls include: if male, if white, age in years, the number of bedrooms, if the mother has higher education and household size. Municipality-specific time trends and border fixed-effects are included. Panel A presents the impacts of the RBF mechanism. Panel B shows the impacts of providing RBF and each technical assistance component. The variable textbooks is the percentage of teachers in the school who received textbooks. Training is the percentage of teachers participating in training in the last two years, and Knowledge is a dummy for schools that participated in the knowledge exchange program Escola Nota 10. Panel C estimates the effects by introducing in the RBF formula the percentages of students below minimum performance thresholds in the city.

27 6 Mechanisms

The next sections explore four potential channels for our ﬁndings: (i) the selection and training of school principals, (ii) the provision of training to teachers, and (iii) the distribution of pedagogical materials. For these, we use detailed questionnaires from SAEB applied to school principals, teachers, and students. The last mechanism, (iv), is the changes in diﬀerent domains of public spending and school facilities. The following sections discuss these points individually.

6.1 Selection and Training of School Principals

The results so far suggest that combining education incentives for mayors with technical assistance produces sharp improvements in test scores. Interestingly, these eﬀects reduce by half when RBF was implemented without any technical support. Because the selection and training of school principals in Brazil is generally provided by municipal authorities, even when the selection occurs by community voting, we now turn our attention to how RBF reform has altered the selection and training of school principals in Ceará.

Two additional challenges emerge from using the questionnaire of principals from SAEB. Primarily, a large proportion of principals manage schools with either primary or lower secondary students.

Therefore, it is not possible to perform separate regressions for each school type and identify when the technical assistance was introduced; if it was in 2011 or 2015. Secondly, the questionnaire for principals and teachers only became comprehensive after 2005, giving us only 2007 as a baseline diﬀerence. Because of these limitations, we had to adapt our empirical strategy in Eq. (1) to a dynamics model as in the following regression:

2017 X yict = αborder + tct + αt + αtime + β0Ceara + βtDiDt + γXi + it, (2) t=2009

28 where yit refers to the outcome of interest for principal i in time t; αborder is the border fixed-effects as defined for Eq. (1); αt is a time trend; and αct is a municipality-specific time trend. Ceara is a dummy for schools at Ceará’s borders and zero for schools on the other side of the border (Figure

1); therefore, β0 measures the DD in the baseline period. αyear is a set of exclusive dummies per year, i.e., from 2007 to 2017. The variable DiDt interacts αyear and Ceara; γXi is a set of controls for principals’ characteristics (five age categories, race, and gender); and it is the error term. Because principals manage schools of different sizes, Eq. (2) weights the coefficients by school size, according to the school census. As before, the parameters of interest are the yearly differences in outcomes (βt) at the border relative to the difference during the baseline year of

2007.

Figure 4 plots the DD coeﬃcients of interest for principal selection. Selection types are merged in four categories: "Election", "Designated", "Formal Selection", and "Hybrid".17 Thus, Figures

4.A and 4.B present the βt, and their respective 95% conﬁdence intervals, relative to the diﬀerence in the baseline, in this case 2007.

The ﬁrst result from Figure 4 is the opposite behavior shown by designations and formal selection of principals. Figure 4 demonstrates that municipalities in Ceará had around a 10pp decrease in the probability of school principals being designated during the RBF period (2009 − 2011) and almost a 20pp decrease after the RBF was reformulated (2013 − 2017). In contrast, there is an increase in formal selection: when the RBF was introduced, principals in Ceará were around

10pp − 15pp more likely to go through a formal selection process than principals on the other side of the border. The second finding is that either "Hybrid" or “Election” selection processes present no significant change. Overall, these findings are in accordance with the interpretation

17The referred question in SAEB is "Did you assume this position by:". The outcome "Election" equals one when principals respond "Election only" and zero otherwise. The outcome "Designated" equals one when principals answer "Designation only" and zero otherwise. From SAEB 2011 backwards, this variable also includes principals who respond "Political Indication" and "Technical Indication". "Formal Selection" considers principals admitted by "Selection Only". "Hybrid" encompasses the combinations of "Selection and Election" or "Selection and Indication". This combination was chosen because these options remain in SAEB from 2007 to 2017.

29 (A) − Election and Designation

(B) − Formal and Hybrid Selection

Figure 4: The RBF Effects on the Selection of Principals

Notes. This Figure shows DD coefficients from Eq. (2) for four types of principal selection. The regressions only include schools located in municipalities at the border between Ceará and adjacent states (as shown in Figure 1). Figure (A) presents yearly coefficients for principals selected by election (black line) or Designation (orange line). Figure (B) shows year coefficients for Formal (black line) and Hybrid Selection (orange line). The vertical dotted line indicates when the RBF reform starts (2009). Data on school principals is drawn from SAEB, and all regressions control for municipality and principal characteristics (age, race, gender), time trends, and border fixed-effects. The x−axis presents each stage of the RBF reform: it starts by the baseline (2007), "RBF" indicates periods with "Results−Based Financing", "TA G5" indicates when technical assistance was introduced in 5th grades, "RBF(re)" refers to periods when the RBF formula was reformulated and "TA G5&9" indicates when the TA component was extended to 9th grade. 95% confidence intervals are shown for each coefficient.

30 that mayors become more likely to undertake formal selection processes after tax resources are conditioned upon education performance.

By formalizing the selection process, mayors may attract high−quality principals and avoid hiring low−quality professionals. A fast−growing body of research has shown that school principals play an important role in student performance (Lavy 2008; Grissom and Bartanen 2019). It has been found that better−qualiﬁed principals aﬀect graduation rates and English test scores (Coelli and

Green 2012), and performance in primary education (Dhuey 2018). In addition to formal selection, local authorities can aim to improve the general quality of principals by providing, or by adopting, the training oﬀered by the state government. To examine the provision and quality of training, we use two outcomes of principals derived from SAEB: a dummy "if principals enrolled in training in the last 2 years" and a dummy "if they considered the training useful for their daily work".

According to Figure 5, school principals in Ceará were significantly more likely to participate in training after 2013. The DD coefficients indicate that training participation increased by 20pp after RBF reformulation, maintaining a positive and significant impact between 20pp − 30pp in the following years. Along with training participation, principals in Ceará are around 10pp more likely to report that the training was useful than principals on the other side the border. For instance, as the question used to identify participation in training refers to the last two years, such increases might reflect the availability of technical assistance since 2011 in the state of Ceará.

Another argument is that it may have taken some time for principals to participate in training and, consequently, evaluate its usefulness.

6.2 Selection and Training of Teachers

The second part of our analysis relates to the selection and training of teachers. Table 4 presents the estimations for several teacher outcomes using Eq. (1). Panel A column (1) shows the results

31 Figure 5: Principal Training and its Perceived Usefulness

Notes. This figure shows the DD coefficients from Eq. (2) on the participation of principals in training in the last two years and the perceived usefulness of training for daily work. The black line refers to the RBF impacts on the participation of school principals in training (= 1). The orange line plots the DD coefficients for principals indicating that the training was useful (= 1). As in previous analysis, these regressions compare schools located in municipalities at the border between Ceará and adjacent states (see Figure 2). The vertical dotted line indicates when the RBF reform starts. The data on school principals is from SAEB. The regressions controls for municipality, principal characteristics, time trends, and border fixed-effects. The x−axis presents the estimates for the baseline year (2007). The "RBF" indicates "Results−Based Financing" periods, "TA" indicates Technical Assistance periods, "TA G5" indicates when the TA was offered to 5th grade, and "RBF(re)" refers to periods when the RBF was re-calibrated. Finally, "TA G5&9" indicates when the TA component was extended to 9th grade. 95% confidence intervals are shown for each coefficient. using as outcome a dummy equal to one "if teachers had training in the past 2 years" and zero otherwise; column (2) considers a dummy equal to one "if teachers perceived the training as useful" and zero otherwise.18 Panel B refers to selection type and classroom hours per week. Panel B column (3) shows the DD coefficients if teachers had an open-ended contract, and column (4) is if the teacher has a temporary contract. Columns (5) and (6) refer to teachers with 20 or 40 hours per week in the classroom respectively. Table 4 shows the results for 5th and 9th grades. Standard

18Similar to school principals, the teacher questionnaire in SAEB asks: (i) "Did you participate in any in-service training activity in the past two years?". Teachers can answer "yes", to which we assign one, and "no", to which we assign zero. For column (2), the question is: "Did the in-service training contributed to improving your classroom practice?". The indicator consider when teachers answer "Almost always" and "Always".

32 errors are clustered at the municipal level and shown in parenthesis. st 1 (or year, ∗∗ ∗∗∗ − rbf is for a 053 . 109 042) 012 038) 003 042) 040) 054) 049) 133 029 ...... 0 0 (1) 0 0 0 (0 − − ) and th 9 . The definition of for is if teachers had an 1 ta (3) ) ∗∗∗ ∗∗∗ ∗∗∗ ∗∗∗ Based Financing" reports ) + − 035) (0 018) (0 033041) 0 (0 012) (0 046 015 029) (0 129 095 162 081 ...... re . . . . (SAEB). Column ( 0 0 0 0 hours per week. All regressions − − − − rbf 40 (or or ta 20 categories. It also includes: time trends, ∗∗ + 7 as illustrated in Figure 010 053 0 . . 076 037) (0 036) (0 032 027029) 0 (0 035) (0 023) (0 033) (0 057 . rbf ...... 0 0 Teachers Teachers 0 0 ; and a dummy for schools offering from the − − − − RBF periods. The line "DD, Teaching Assistance" − Paraíba SAEB Grade Grade and th th ∗ ∗∗∗ 5 9 014 007 021 0 068 . . . 036) (0 039) (0 058 0 041) (0 041) (0 033) (0 044) (0 . 133 ...... 0 0 0 . 0 0 − − − − − Pernambuco , is for teachers with contracts of Sistema de Avaliação da Educação Básica (6) ∗∗∗ ∗∗∗ 023) (0 037) (0 011 0 006) (0 034) (0 008) (0 011) (0 001 017 078 120 021 ...... and 0 0 0 0 0 0 − (5) considers if teachers perceived the training as useful. column Table 4 percent level. This table presents the DD coefficients of RBF effects on teacher training, (2) ∗∗ ∗∗ ∗∗∗ ∗∗∗ 10 ∗ A. Training B. Selection and Classroom Hours 075 087 027) (0 029) (0 007 026) (0 027) (0 035) (0 033) (0 115 088 Rio Grande do Norte . 057 . (1) (2) (3) (4) (5) (6) ...... 0 0 0 , 0 0 0 If had if training if permanent if temporary 20 Classroom 40 Classroom (0 (0 (0 (0 (0 (0 − training helped Teacher Teacher Hours per wk Hours per wk Piauí ) ) ) ) ) ) ) (Clustered Standard Errors) ta re Based Financing, re" reports the estimated impacts of reformulating the RBF model. Standard errors pre pre rbf rbf + ( years. Column β β β β and − 2 rbf rbf − − − − percent, and * at β β ) ta 5 effects. Regression coefficients are weighted by school Size. The line "DD, Results − re − rbf rbf + ( β β − Ceará ta ta rbf rbf )+ )+ β β re re (SINASC); a dummy for rural schools provided provided by ( ( Teacher Training, Selection and Classroom Hours rbf rbf β β is for temporary contracts. Columns alone by comparing the differences at the border between RBF and pre percent, ** at 1 (4) RBF Based Financing ( Based Financing ( − − Based Financing, re ( − Sistema de Nascidos Vivos Based Financing, re ( DD, Teaching Assistance ( ) periods. Finally, the line "DD, Results − th DD, Results DD, Results 9 specific time trends, and border fixed grades. The students controls include: if male, if white, dummies per age group (as they are provided in DD, Teaching Assistance ( − *** denotes significance at for th 9 ended contract and column ) DD, Results − re ( Questionnaire Respondent Teacherrbf Teacherare clustered by municipality and shown in parenthesis. Teacher Teacher Teacher Teacher DD, Results estimates the impacts of combining the RBF with technical assistance by testing the difference in the coefficients Notes. selection and classroom hours. All outcomes come from teacher questionnaires in the dummy equal to oneopen teachers had training in the past the impacts of introducing the to the municipal restrict to municipalities atborder the considers border the between according three to immediate the municipalities at each side. Regressions include municipality controls: the number of births in the municipality

33 Table 4 indicates significant increases in the participation of teachers in training, particularly after the technical assistance periods. From Column (1), 5th grade teachers are 11.5pp more likely to participate in training, while 9th grade teachers are 8.8pp more likely than their counterparts across the border. Still in column (1), DD coefficients during RBF periods show negative impacts for 5th grade (8.7pp) and positive impacts for 9th grade (7.5pp). Similar to both grades, after TA teachers report significantly more that training was useful, reaching 12pp for 9th grade. Relative to the baseline difference, for either 5th or 9th grade, there is a significant increase in the percentage of teachers Ceará enrolling in training and reporting that it was useful.

The second piece of evidence comes from Panel B, Table 4. Despite signiﬁcant changes in the selection of principals, no clear pattern is seen for teachers in Figure 4, Table 4 columns (3) and

(4). On the other hand, columns (5) and (6) suggest a strong substitution of teachers from 20- hours per week contracts to 40-hours per week contracts. In column (5), it is possible to observe a decrease of 16.2pp in the percentage of 5th grade teachers with 20-hours per week contracts; for

9th grade, the decrease is 8.1pp. At the same time, the percentage of teachers reporting a 40-hour per week contracts in column (6) increases by 10.9pp for 5th grade and 13.3pp for 9th grade. This reduction repeats when the RBF is reformulated for both grades. Therefore, while the TA impacts on the incidence and quality of training, introducing the RBF model results in signiﬁcantly more teachers with 40-hours per week contracts than on the other side of the border.

If teacher training and selection improves pedagogical skills and learning, the results from Table

4 may partially explain how the RBF mechanism and TA influence student outcomes. Previous literature reinforces this argument. Angrist and Lavy(2001) evaluate the impacts of in −service training for teachers in Jerusalem and find significant positive effects on test scores. Bressoux et al.(2009) show that teacher training substantially improves student test scores in mathematics in France. Therefore, better training and longer classroom hours are potential channels explaining

34 our ﬁndings.19

6.3 Pedagogical Support

Table 5 presents the impacts of RBF and TA policies on pedagogical support. It divides the analysis into three parts: Panel A shows if teachers are more likely to check school homework in mathematics (Column (1)) and Portuguese (Column (2)); Panel B concentrates on book quality, according to teachers’ perspectives20; Panel C shows if teachers cover more than 80% of the curriculum in column (4) and if they cover less than 40% in column (5). Table 5 reports the results for 5th and 9th grades.

Table 5 suggests that teachers in Ceará are signiﬁcantly more likely to check school homework and see their pedagogical materials as "good" or "great" than their cross-border counterparts. Table

5 column (1) shows that 1.8pp more teachers check school homework in mathematics, and 2.3pp in Portuguese, when technical assistance was introduced. For 9th grade, the results are generally insigniﬁcant. Similarly, improvements in textbook quality in Panel B are only observed for 5th grade. That can be seen in column (3) by signiﬁcant increases of 9.4pp with TA and more than a

10.3pp increase after RBF was reformulated. If the use of better pedagogical materials inﬂuences performance as previous literature suggests (Leme et al. 2012), textbooks of higher quality may explain the improvements of student performance in 5th grade in Ceará.

Another potential channel explaining our ﬁndings comes from Panel C columns (4) and (5). Col- umn (4) Panel C shows 5th grade teachers in Ceará were 8.1pp more likely to cover 80% of the curriculum after TA was implemented than teachers on the other side of the border. The eﬀect

19Further analysis using the probability of teachers having a master’s shows insignificant results for both grades. So, we cannot consider that better-certified teacher is a channel explaining our findings. (See Ehrenberg and Brewer (1994) and Clotfelter et al.(2006) for similar results in the literature.) 20The question used in SAEB is: "What is your opinion about the book(s) used in the subject you lecture in the class?". The "book quality" variable in Panel B equals one when teachers say "Good" or "Great" and zero otherwise.

35 Table 5 Pedagogical Practices, Textbook Quality and Curriculum Coverage (Clustered Standard Errors)

A. Pedagogical Practices B. Book Quality C. Curriculum Coverage Teacher Checks Teacher Checks Textbook Covers Covers Maths Homework Port Homework is good/great ≥ 80% ≤ 40% (1) (2) (3) (4) (5) 5th Grade − Teachers/Students

DD, Results−Based Financing (βrbf − βpre) 0.008 0.006 −0.051 0.002 0.018 (0.010) (0.010) (0.056) (0.032) (0.019) DD, Teaching Assistance (βrbf+ta − βrbf ) 0.018∗ 0.023∗∗ 0.094∗∗ 0.081∗ −0.036∗∗ (0.010) (0.010) (0.046) (0.043) (0.017) DD, Results−Based Financing, re (βrbf(re)+ta − βrbf+ta) 0.010 0.004 0.103∗∗ 0.010 0.004 (0.009) (0.009) (0.043) (0.036) (0.011) # of Observations 225, 869 224, 593 2, 310 2, 310 2, 310 Adj. R−squared 0.006 0.634 0.084 0.549 0.484

9th Grade − Teachers/Students

DD, Results−Based Financing (βrbf − βpre) 0.016∗ 0.010 0.000 0.051 −0.005 (0.008) (0.008) (0.038) (0.042) (0.014) DD, Teaching Assistance (βrbf(re)+ta − βrbf(re)) 0.009 0.010 −0.014 0.081∗∗ 0.018∗∗ (0.009) (0.008) (0.038) (0.039) (0.008) DD, Results−Based Financing, re (βrbf(re) − βrbf ) 0.007 0.006 0.034 0.116∗∗∗ −0.027∗∗∗ (0.008) (0.008) (0.044) (0.042) (0.009) # of Observations 191, 503 191, 764 2, 310 2, 310 2, 310 Adj. R−squared 0.005 0.005 0.084 0.549 0.484 Questionnaire Respondent Student Student Teacher Teacher Teacher Notes. *** denotes significance at 1 percent, ** at 5 percent, and * at 10 percent level. This table presents the effects of RBF and technical assistance on pedagogical practices, textbook quality, and curriculum coverage. Data comes from teacher questionnaires in the Sistema de Avaliação da Educação Básica (SAEB) and all regressions are restricted to municipalities at the border between Ceará and its adjacent states as shown in Figure 1. The regressions include municipality controls for the number of births per year in Sistema de Nascidos Vivos (SINASC), a dummy for rural schools (SAEB); if schools offer from the 1st to the 9th grade. A time trend, municipal-specific trends, and a border fixed-effect are also included. The teacher/student controls are: if male, if white, and dummies per age group. Weights for school size, in terms of number of students, are considered in columns (3) to (5). The “DD, Results−Based Financing" shows the RBF impacts. The “DD, Teaching Assistance” refers to the effects of introducing the technical assistance component and “DD, Results−Based Financing, re” shows DD estimates reformulating the RBF mechanism. The baseline difference at the border is βpre. Standard errors are clustered by municipality and shown in parenthesis. on curriculum coverage is even stronger for teachers in 9th grade. In addition to the 8.1pp increase during TA periods, teachers were 11.6pp more likely to cover 80% of the curriculum after RBF reformulation. However, reinforcing this argument, an opposite movement is observed in column

(5). It shows a 3.6pp decrease in the percentage of 5th grade teachers reporting covering less than

40% of the curriculum during the TA periods, and 2.7pp less likely among 9th grade teachers after

RBF was reformulated. In the same way that textbook quality plays a prominent role in explaining the gains in performance for students in 5th grade, greater curriculum coverage by teachers seems to be an important driver for learning improvement in 9th grade.

Figure 6 complements Table 5 by showing that Ceará is distributing textbooks where they are scarcer. Using school principal questionnaires from SAEB and plotting the DD coeﬃcients from

Eq. (2), Figure 6 shows no change in the provision of pedagogical materials by the municipal council (blue bars), but signiﬁcant decreases in the incidence of schools reporting a lack of text-

36 books (red bars). This eﬀect becomes particularly high after TA reforms in 2011. Note that the textbooks from the RBF+TA component are provided by the state of Ceará, not by the munic-

ipalities. Thus, the better-quality textbooks reported by teachers in Table 6 might have been

distributed by the state government, according to school principals in Figure 6.

Figure 6: Provision of Textbooks

Notes. This figure shows differences in the pedagogic elaboration and provision at the border of Ceará and adjacent states. It plots the diff−in−diff coefficients from Eq. (2) between schools in Ceará and on the other side of the border. Blue bars illustrate the difference in the incidence of schools having pedagogical materials from the municipality’s council; red bars refer to when principals report a lack of pedagogical materials. This data uses principal responses to the SAEB questionnaire. The vertical dotted line indicates when the RBF starts. The x−axis indicates the estimates in the baseline period (2007); RBF indicates "Results-Based Financing" periods, "TA" indicates Technical Assistance periods; "TA G5" indicates when technical assistance was provided for 5th graders, "RBF(re)" refers to when the RBF was re−calibrated; and finally "TA G59" refers to the technical assistance is offered to 5th and 9th grades. Each bar includes 95% confidence intervals.

However, additional considerations are needed when considering the provision and quality of textbooks as mechanisms for RBF eﬀects. Glewwe et al.(2009), for example, suggest that textbooks had little eﬀect on primary schools in rural Kenya. In this case, the authors argue that the program elaborated textbooks for high achieving students and in their second or third language. So,

37 content and language barriers hampered the capacity of textbooks to improve learning in Kenya.

In contrast, textbooks in Ceará overcome both limitations: they are designed for students who are lagging behind in their grade and are produced in students’ native language (Portuguese).

6.4 Municipal Public Spending

An important aspect of Ceará’s RBF model is that ﬁscal resources conditioned on education performance do not necessarily need to be spent on education. If mayors foresee the need to build a new hospital or a bridge or even invest in security, there is nothing that prevents them from using the resources disbursed based on education performance. The idea underlying the fact that the RBF resources are not earmarked to education expenditures is to generate stronger incentives for mayors to prioritize improving learning. This section estimates whether RBF, and TA, aﬀects municipal public spending. Table 6 presents the estimation results on several public spending categories.

The effect related to RBF on municipal public spending is displayed in Panel A of Table 6. In column (1), the RBF increased public spending on education per pupil by 1.2% relative to the baseline mean. For the other sectors, the variations are even larger. Public spending per capita on sanitation, housing, and urbanization increased by 13.9% with the introduction of the RBF. On social spending (i.e., social security and social assistance), per capita spending increased by 10.5%, and by 21.6% for spending on culture, sports and leisure. Municipal public spending on health also increased with the introduction of the RBF, but the coefficient is statistically significant only at the 10% level. As a whole, the introduction of the RBF contributed to an increase of 2% in municipal public spending in Ceará relative to the baseline mean when compared with their counterparts in the adjacent states.

Panel B displays the incremental eﬀect of combining TA and RBF. The introduction of teaching

38 Table 6 The RBF effects on Public Spending of Municipalities (Clustered Standard Errors)

Sanitation, Sport, Education Health Housing and Social Culture Total Urbanization and Leisure spending (1) (2) (3) (4) (5) (6)

A. DD, Results−Based Financing 0.095*** 0.223* 0.671*** 0.462*** 0.691*** 0.153*** βrbf − βpre (0.021) (0.125) (0.139) (0.090) (0.112) (0.024) B. DD, Teaching Assistance 0.038 -0.259* 0.081 0.054 -0.015 -0.05 βrbf+ta − βrbf (0.031) (0.141) (0.175) (0.125) (0.173) (0.033) C. DD, Results−Based Financing, reformulated -0.006 0.039 -0.213 -0.056 -0.134 -0.024 βrbf(re)+ta − βrbf+ta (0.024) (0.070) (0.174) (0.184) (0.182) (0.021) Variation relative to the baseline mean βrbf − βpre 1.16% 3.70% 13.88% 10.46% 21.57% 2.05% βrbf(re)+ta − βrbf+ta 0.46% -4.30% 1.68% 1.22% -0.47% -0.67% βrbf(re)+ta − βrbf+ta -0.07% 0.65% -4.41% -1.27% -4.18% -0.32% Spending Deﬁnition per student per capita per capita per capita per capita per capita Dep. var. mean 8.193 6.021 4.834 4.417 3.204 7.466 Dep. var. SD 0.375 0.504 0.948 0.843 0.988 0.276 # of municipality Observations 4,163 4,160 4,256 4,256 4,256 4,163 Adj. R−Squared 0.702 0.223 0.065 0.13 0.078 0.381

Notes. ∗∗∗ denotes significance at 1 percent, ∗∗ at 5 percent, and ∗ at 10 percent level. The dependent variables come from National Treasury Bureau (Secretaria do Tesouro Nacional (STN). All regressions restrict the sample to the 330 municipalities at the border between Ceará and adjacent states. The definition of border considers the three imediate municipalities at each side of Ceará and Piaúi, Rio Grande do Norte, Pernambuco and Paraíba. municipality controls include the number of births in the municipality−year according to the Sistema de Nascidos Vivos (SINASC), and GDP per capita as provided by IBGE. The vector of dependent variables includes: (1) Natural log of education spending per student; (2) Natural log of health spending per capita; (3) Natural log of spending on sanitation, housing and urbanism per capita; (4) Natural log of spending on social assistance and social security per capita; (5) Natural log of spending on sport, leisure and culture per capita; (6) Natural log of total spending per capita. Public spending variables are deflated by the General Price Index (2017 = 100) of the Fundação Getúlio Vargas. Time trends and Municipal specific time trends are included. To compare municipalities close geographically, sharing the same state border, all regressions include Border Fixed Effects. The bottom of each column presents in Panel A the DD estimates of RBF impacts, Panel B presents the effects after introducing teaching assistance, the TA component in our model; and Panel C shows the estimates of reformulating the RBF to weights for minimum standards. Standard errors are clustered by municipality and shown in parenthesis. assistance did not affect the public spending of municipalities, especially on education. The exception is the reduction in health spending per capita (see column (2)), about −4.3% relative to the baseline mean but significant only at the level of 10%. The health sector shares some complementary services with education, such as health prevention (e.g., obesity prevention and immunization), guidance on sexual and reproductive health, etc. The efficient provision of such services with the introduction of teaching assistance might have contributed to reducing health spending, especially on primary health care. Panel C, in turn, shows that the reformulation of the RBF did not have implications on municipal public spending. The large gains in leaning with the introduction of TA and the reformulation of the RBF (see Panels B and C of Table 2) were

39 attained without increasing spending on education per pupil.

An emerging literature investigates the importance of how governments are financed for public investment. Gadenne(2017) argues that increases in municipal governments spending based on tax revenues tends to benefit citizens more than increases based on grants. In the specific cases of constitutional transfers, Litschig and Morrison(2013) show that extra transfers from the federal government to municipalities based on the Fundo de Participação dos Municípios (FPM) increased local public spending per capita by approximately 20%, leading to an increase of 7% in schooling and 4pp in literacy rates. Carvalho Filho and Litschig(2020) show that extra resources from the

FPM transfers not only increased spending on education but also on housing, urban infrastructure, and transportation by about 20% in early 1980s. More recently, Cruz and Silva(2020) have analyzed the importance of budget structure for government spending in education by estimating the eﬀects of a minimum spending rule and a set of intergovernmental transfers. The authors show that the minimal spending rule decreases the eﬃciency of education spending for high-spending municipalities, despite increasing the education investment of low-spending municipalities.

Our results are in accordance with this literature and indicate that the introduction of RBF led municipalities to increase their spending per capita not only in the provision of education, but also of other public services. The reforms have led to an increase in the total public spending of municipalities in Ceará compared with municipalities across the border. In the case of education services, the introduction of the RBF led mayors to invest in the quality of school facilities, such as internet connections, computer labs, libraries, and sports courts (see Table A4 in the appendix).

School facilities improved with the introduction of TA as well, suggesting that mayors see them as complementary inputs to improve learning (Glewwe and Kremer 2006; Glewwe and Muralidharan

2016). For Latin American countries, evidence has shown that the provision of school facilities improves student learning, despite the lack of high-quality research (Cuesta et al. 2016).

40 7 Conclusion

This paper presents new evidence that fiscal incentives to local governments can be an effective tool to improve learning. Exploring a results−based financing (RBF) reform in the state of Ceará

(Brazil) that redistributes resources to municipalities based mainly on education outcomes, we investigate whether education incentives for mayors affect learning. While vast literature has already shown the importance of financial incentives for students, teachers, and schools for academic performance, little is known about whether similar mechanisms targeting head of local governments would produce similar impacts. Because city mayors in Brazil are responsible for delivering most educational services and to allocate budget to programs, distributing fiscal resources in exchange for better education outcomes presents itself as a valuable option for public policy.

Comparing test scores in schools located in municipalities at the border between Ceará and adjacent states, we show that the RBF increased student performance in Portuguese and mathematics by approximately 3 months’ learning equivalent for 9th grade. For 5th grade, no impacts were found. For instance, when the reform combines RBF and technical assistance (TA) to municipalities (i.e., provision of textbooks, training, and knowledge exchange between schools) the RBF +TA increased learning outcomes around 3 months’ learning equivalent for both grades, but now significant and slightly higher for students in 5th grade. Investigating the role of each TA component, it seems that providing textbooks and training teachers were the main drivers of TA impacts.

Our findings also confirm the concern that RBF programs increase inequality. Quantile estimates suggest that implementing the RBF produces effects that are three, sometimes four, times higher for students from 0.20 higher quantiles than for students from the top 0.20 quantiles, increasing learning gaps. This scenario only changes after the reformulation of the RBF criteria penalized municipalities based on the percentage of students performing below minimum threshold scores.

During the period of the reformulated RBF, the impacts are smaller than in previous periods

41 0.92SD (or 2 months of schooling) but with strong implications for distribution, with greater impacts on 9th grade students from low quantiles and signiﬁcantly reduced impacts for 5th grade students.

We use detailed data on teachers, principals and public spending to analyze several mechanisms for these findings. First, there are significant reductions in principals “Designated" to schools and replaced by those going through formal selection processes. Second, there is a strong indication that both principals and teachers in Ceará are more likely to be involved in training and report it as useful for their daily work compared to similar principals and teachers in the immediate municipalities outside Ceará’s border. Fourth, after the RBF reform, teachers in Ceará are more likely to be hired on 40-hour contracts (both grades), check school homework more often (5th grade), and cover at least 80% of the curriculum (both grades). Finally, schools in Ceará report significantly less frequent lack of pedagogical materials at the same time that significantly more teachers consider these books to be at least “good”.

Exploring diﬀerent formats of the RBF model in Ceará in time provides a clear contribution to public policy design. Particularly, introducing the RBF in combination with technical assistance boosts learning impacts in at least two−fold relative to RBF alone. Elaborating textbooks and providing training to teachers, based on student test scores, is the main source of these impacts.

Another important contribution to policy design is that redistributing ﬁscal resources exclusively based on education performance tends to widen learning gaps. Such inequalities are signiﬁcantly reduced, when penalizing municipalities where there is a high percentage of students below certain thresholds of performance.

All these gains in student learning in Ceará were obtained without increasing public spending on education. This means that the large gains in student learning and education quality followed the eﬃcient allocation of public spending to municipalities in Ceará.

42 References

J. D. Angrist and V. Lavy. Does teacher training aﬀect pupil learning? evidence from matched

comparisons in jerusalem public schools. Journal of Labor Economics, 19(2):343–369, 2001. doi:

10.1086/319564. ↑34

G. Barlevy and D. Neal. Pay for percentile. American Economic Review, 102(5):1805–31, May

2012. doi: 10.1257/aer.102.5.1805. URL https://www.aeaweb.org/articles?id=10.1257/

aer.102.5.1805. ↑2, ↑23

E. P. Bettinger. Paying to learn: The eﬀect of ﬁnancial incentives on elementary school test scores.

The Review of Economics and Statistics, 94(3):686–698, 2012. doi: 10.1162/REST\_a\_00217.

↑2

P. Bressoux, F. Kramarz, and C. Prost. Teachers’ Training, Class Size and Students’ Outcomes:

Learning from Administrative Forecasting Mistakes. The Economic Journal, 119(536):540–561,

02 2009. ISSN 0013-0133. doi: 10.1111/j.1468-0297.2008.02247.x. URL https://doi.org/10.

1111/j.1468-0297.2008.02247.x. ↑34

S. E. Carrell and J. E. West. Does professor quality matter? evidence from random assignment of

students to professors. Journal of Political Economy, 118(3):409–432, 2010. doi: 10.1086/653808.

↑23

I. Carvalho Filho and S. Litschig. Long-run impacts of intergovernmental transfers. Journal of

Human Resources, 2020. doi: 10.3368/jhr.57.3.0917-9064R2. ↑40

A. L. Cavalcante, D. C. Suliano, W. L. Paiva, N. Trompieri Neto, P. A. Pontes, C. Lima,

J. Freire Júnior, and R. Soares. Indicadores Econômicos do Ceará 2019. Instituto de Pesquisa

e Estratégia Econômica do Ceará, 2019. ↑6

C. T. Clotfelter, H. F. Ladd, and J. L. Vigdor. Teacher-student matching and the assessment of

43 teacher eﬀectiveness. Journal of Human Resources, XLI(4):778–820, 2006. doi: 10.3368/jhr.

XLI.4.778. ↑35

M. Coelli and D. A. Green. Leadership eﬀects: school principals and student outcomes. Eco-

nomics of Education Review, 31(1):92 – 109, 2012. ISSN 0272-7757. doi: https://doi.org/

10.1016/j.econedurev.2011.09.001. URL http://www.sciencedirect.com/science/article/

pii/S0272775711001488. ↑31

T. Cruz and T. Silva. Minimum spending in education and the ﬂypaper eﬀect. Economics of

Education Review, 77, 2020. doi: https://doi.org/10.1016/j.econedurev.2020.102012. ↑40

A. Cuesta, P. Glewwe, and B. Krause. School infrastructure and educational outcomes: A litera-

ture review, with special reference to latin america. Economía, 17(1):95–130, 2016. ↑40

J. Das, S. Dercon, J. Habyarimana, P. Krishnan, K. Muralidharan, and V. Sundararaman. School

inputs, household substitution, and test scores. American Economic Journal: Applied Eco-

nomics, 5(2):29–57, April 2013. doi: 10.1257/app.5.2.29. URL https://www.aeaweb.org/

articles?id=10.1257/app.5.2.29. ↑2

J. L. de Oliveira, L. O. Rodrigues, R. S. Sales, V. H. de Oliveira, and R. Pereira. Síntese de

indicadores sociais 2018. Technical report, Instituto de Pesquisa e Estratégia Econômica do

Ceará, 2020. ↑6

S. J. Dhuey, E. How school principals inﬂuence student learning. Empirical Economics, 54:

851–882, 2018. doi: 10.1111/j.1467-9442.2008.00555.x. URL https://doi-org.libproxy-

wb.imf.org/10.1007/s00181-017-1259-9. ↑31

R. G. Ehrenberg and D. J. Brewer. Do school and teacher characteristics matter? evidence from

high school and beyond. Economics of Education Review, 13(1):1 – 17, 1994. ISSN 0272-7757.

doi: https://doi.org/10.1016/0272-7757(94)90019-1. ↑35

44 J. Fryer, Roland G. Financial incentives and student achievement: Evidence from randomized

trials. The Quarterly Journal of Economics, 126(4):1755–1798, 11 2011. ISSN 0033-5533. doi:

10.1093/qje/qjr045. ↑2

L. Gadenne. Tax me, but spend wisely? sources of public ﬁnance and government ac-

countability. American Economic Journal: Applied Economics, 9(1):274–314, 2017. doi:

10.1257/app.20150509. ↑40

P. Glewwe and M. Kremer. Chapter 16 schools, teachers, and education outcomes in developing

countries. volume 2 of Handbook of the Economics of Education, pages 945 – 1017. Elsevier,

2006. doi: https://doi.org/10.1016/S1574-0692(06)02016-2. ↑40

P. Glewwe and K. Muralidharan. Chapter 10 - improving education outcomes in developing

countries: Evidence, knowledge gaps, and policy implications. volume 5 of Handbook of the

Economics of Education, pages 653 – 743. Elsevier, 2016. doi: https://doi.org/10.1016/B978-0-

444-63459-7.00010-5. ↑40

P. Glewwe, M. Kremer, and S. Moulin. Many children left behind? textbooks and test scores in

kenya. American Economic Journal: Applied Economics, 1(1):112–35, January 2009. doi: 10.

1257/app.1.1.112. URL https://www.aeaweb.org/articles?id=10.1257/app.1.1.112. ↑5,

↑37

P. Glewwe, N. Ilias, and M. Kremer. Teacher incentives. American Economic Journal: Applied

Economics, 2(3):205–27, July 2010. doi: 10.1257/app.2.3.205. URL https://www.aeaweb.org/

articles?id=10.1257/app.2.3.205. ↑2, ↑23

N. Gordon. Do federal grants boost school spending? evidence from title i. Journal of Public

Economics, 88(9):1771 – 1792, 2004. ISSN 0047-2727. doi: https://doi.org/10.1016/j.jpubeco.

2003.09.002. ↑2

45 J. A. Grissom and B. Bartanen. Principal eﬀectiveness and principal turnover. Education Finance

and Policy, 14(3):355–382, 2019. doi: 10.1162/edfp\_a\_00256. ↑31

K. T. Hansen, J. J. Heckman, and K. J. Mullen. The eﬀect of schooling and ability on achievement

test scores. Journal of Econometrics, 121(1):39 – 98, 2004. ISSN 0304-4076. doi: https:

//doi.org/10.1016/j.jeconom.2003.10.011. Higher education (Annals issue). ↑2

M. C. Holanda. Ceará: A prática de uma gestão pública por resultados. IPECE, 2006. ISBN

85-98664-06-5. ↑7

M. C. Holanda, M. P. Barbosa, and L. O. Costa. Memória de cálculo dos coeﬁcientes de distribuição

do icms municipal 2009. Technical Report Nota Técnica N. 32, IPECE, Fortaleza-CE, 2008a.

↑8

M. C. Holanda, M. P. Barbosa, and L. O. Costa. Metodologia de cálculo da nova lei do icms

municipal. Technical Report Nota Técnica N. 33, IPECE, Fortaleza-CE, 2008b. ↑8

B. A. Jacob. Accountability, incentives and behavior: the impact of high-stakes testing in the

chicago public schools. Journal of Public Economics, 89(5):761 – 796, 2005. ISSN 0047-2727.

doi: https://doi.org/10.1016/j.jpubeco.2004.08.004. URL http://www.sciencedirect.com/

science/article/pii/S0047272704001549. ↑23

M. Kremer, E. Miguel, and R. Thornton. Incentives to learn. Review of Economics and Statistics,

91(1):437– 456, 2009. ↑2

V. Lavy. Does raising the principal’s wage improve the school’s outcomes? quasi-experimental

evidence from an unusual policy experiment in israel. The Scandinavian Journal of Economics,

110(4):639–662, 2008. doi: 10.1111/j.1467-9442.2008.00555.x. URL https://onlinelibrary.

wiley.com/doi/abs/10.1111/j.1467-9442.2008.00555.x. ↑31

46 M. C. Leme, P. Louzano, V. Ponczek, and A. P. Souza. The impact of structured teaching

methods on the quality of education in brazil. Economics of Education Review, 31(5):850 –

860, 2012. ISSN 0272-7757. URL http://www.sciencedirect.com/science/article/pii/

S0272775712000660. ↑35

S. D. Levitt, J. A. List, S. Neckermann, and S. Sadoﬀ. The behavioralist goes to school:

Leveraging behavioral economics to improve educational performance. American Economic

Journal: Economic Policy, 8(4):183–219, November 2016. doi: 10.1257/pol.20130358. URL

https://www.aeaweb.org/articles?id=10.1257/pol.20130358. ↑2

S. Litschig and K. M. Morrison. The impact of intergovernmental transfers on education outcomes

and poverty reduction. American Economic Journal: Applied Economics, 5(4):206–240, 2013.

doi: 10.1257/app.5.4.206. ↑40

A. Loureiro, L. Cruz, and D. K. Lautharte Junior, Ildo Jose; Evans. The state of ceara in brazil is

a role model for reducing learning poverty. Technical report, World Bank Group, Washington,

D.C., June 2020. URL https://openknowledge.worldbank.org/handle/10986/34156. ↑10,

↑11

L. T. Mitra Akhtari, Diana MOreira. Political turnover, bureaucratic turnover, and the quality of

public services. Working Paper, 2020. ↑4

47 A Appendix

Table A.1 − ICMS Quota−Parte Redistribution Criteria

48 Table A1 Robustness Checks, Mathematics Test Scores (Clustered Standard Errors)

Border Excluded Excluded Excluded All municipalities Ceará Ceará Ceará Ceará Year 2005 Year 2017 municipalities in 2005 Pernambuco Piauí Rio G. do Norte Paraíba (1) (2) (3) (4) (5) (6) (7) (8) 5th Grade Students DD, Results−Based Financing (βrbf − βpre) 0.036 0.039 0.091∗∗∗ −0.044 0.047 0.009 0.070 0.022 (0.039) (0.039) (0.033) (0.081) (0.036) (0.056) (0.037) (0.046) DD, Technical Assistance (βrbf+ta − βrbf ) 0.220∗∗∗ 0.219∗∗∗ 0.169∗∗∗ 0.194∗∗∗ 0.235∗∗∗ 0.146∗∗∗ 0.253∗∗∗ 0.222∗∗∗ (0.029) (0.029) (0.024) (0.057) (0.032) (0.031) (0.035) (0.032) DD, Results−Based Financing, re (βrbf(re)+ta − βrbf+ta) 0.171∗∗∗ 0.149∗∗∗ 0.077∗∗∗ 0.143∗∗ 0.196∗∗∗ 0.096∗∗∗ 0.174∗∗∗ 0.199∗∗∗ (0.038) (0.036) (0.021) (0.056) (0.043) (0.038) (0.048) (0.044) # of Student Observations 233, 541 188, 179 1, 058, 315 75, 836 176, 911 143, 490 180, 509 200, 580 Adj. R−squared 0.187 0.158 0.155 0.187 0.204 0.161 0.191 0.195 9th Grade Students DD, Results−Based Financing (βrbf − βpre) 0.155∗∗∗ 0.154∗∗∗ 0.099∗∗∗ 0.094∗ 0.156∗∗∗ 0.120∗∗∗ 0.171∗∗∗ 0.167∗∗∗ 49 (0.026) (0.026) (0.016) (0.054) (0.030) (0.032) (0.030) (0.030) DD, Teaching Assistance (βrbf(re)+ta − βrbf(re)) 0.092∗∗∗ 0.055 0.053∗ 0.055 0.110∗∗∗ 0.043∗∗∗ 0.103∗∗ 0.099∗∗ (0.032) (0.032) (0.019) (0.078) (0.036) (0.044) (0.034) (0.035) DD, Results−Based Financing, re (βrbf(re) − βrbf ) 0.148∗∗∗ 0.148∗∗∗ 0.145∗∗∗ 0.138∗∗∗ 0.151∗∗∗ 0.117∗∗∗ 0.164∗∗∗ 0.150∗∗∗ (0.035) (0.034) (0.032) (0.062) (0.040) (0.037) (0.042) (0.040) # of Student Observations 194, 618 155, 897 735, 318 55, 773 160, 695 106, 291 151, 591 165, 979 Adj. R−squared 0.149 0.132 0.138 0.147 0.149 0.152 0.148 0.150 Student Controls Yes Yes Yes Yes Yes Yes Yes Yes Border Fixed−Effects Yes Yes No Yes Yes Yes Yes Yes municipalities at the Border Yes Yes No Yes Yes Yes Yes Yes City−Specific Time Trend Yes Yes Yes Yes Yes Yes Yes Yes Notes: *** denotes significance at 1 percent, ** at 5 percent, and * at 10 percent level. This table presents robustness checks using mathematics scores of 5th grade students. The sample restricts to municipalities at the border between Ceará, the treatment state, and the states of Pernambuco, Paraíba, Rio Grande do Norte, and Piauí. All states share their border with Ceará. The regressions assign into the sample the first three municipalities in the immediate border between Ceará and one of the adjacent states. The data on student performance is from the Sistema de Avaliação da Educação Básica (SAEB). All regressions include municipality controls as the number of births in the municipality-year, according to the Sistema de Nascidos Vivos (SINASC); a dummy for rural schools (SAEB); and a dummy for schools offerering from the 1st to the 9th grade. Student controls consider: a dummy if the student is male, if white, age in years, number of bedrooms at home, if the mother has higher education and the household size. Column 1 excludes observations from the year 2005. Column 2 excludes the year 2017. Column 3 applies the same approach to all municipalities, unconstrained to their location. Column 5 uses the sample to schools offering exclusively from the 1st to the 5th grades. Column 6 refers to schools offering primary and lower secondary years, i.e. from the 1st to the 9th grades. From columns 7 to 10, the estimates exclude schools located at each border separately. The impact coefficients are presented at the bottom of each column and estimates the effects of the RBF model and RBF + Technical assistance programs separately in comparison to the difference at the border in the pre RBF period. Standard errors are clustered by municipality level and 95% confidence intervals are in brackets. Table A2 Robustness Checks, Portuguese Test Scores (Clustered Standard Errors)

Border Excluded Excluded Excluded All municipalities Ceará- Ceará Ceará Ceará Year 2005 Year 2017 municipalities in 2005 Pernambuco Piauí Rio G. do Norte Paraíba (1) (2) (3) (4) (5) (6) (7) (8) 5th Grade Students DD, Results−Based Financing (βrbf − βpre) 0.060 0.064∗ 0.114∗∗∗ −0.001 0.078∗∗ 0.008 0.096∗∗∗ 0.058 (0.036)(0.036)(0.024)(0.080)(0.035)(0.048)(0.033)(0.043) DD, Technical Assistance (βrbf+ta − βrbf ) 0.162∗∗∗ 0.161∗∗∗ 0.127∗∗∗ 0.163∗∗∗ 0.165∗∗∗ 0.116∗∗∗ 0.195∗∗∗ 0.159∗∗∗ (0.021)(0.021)(0.019)(0.040)(0.024)(0.023)(0.025)(0.023) DD, Results−Based Financing, re (βrbf(re)+ta − βrbf+ta) 0.166∗∗∗ 0.155∗∗∗ 0.111∗∗∗ 0.123∗∗∗ 0.189∗∗∗ 0.100∗∗∗ 0.169∗∗∗ 0.191∗∗∗ (0.032)(0.031)(0.017)(0.044)(0.041) (1.69) (1.61) (0.037) # of Student Observations 233, 541 188, 169 1, 058, 331 75, 831 176, 900 143, 479 180, 506 200, 575 Adj. R−squared 0.244 0.206 0.216 0.257 0.223 0.237 0.244 0.251 9th Grade Students DD, Results−Based Financing (βrbf − βpre) 0.151∗∗∗ 0.163∗∗∗ 0.115∗∗∗ 0.103∗∗ 0.161∗∗∗ 0.122∗∗∗ 0.162∗∗∗ 0.160∗∗∗ (0.023) (0.023) (0.016) (0.047) (0.027) (0.030) (0.023) (0.026) 50 DD, Teaching Assistance (βrbf(re)+ta − βrbf(re)) 0.096∗∗∗ 0.070∗∗∗ 0.053∗∗ 0.086∗ 0.107∗∗∗ 0.065∗ 0.102∗∗∗ 0.104∗∗∗ (0.028) (0.029) (0.023) (0.050) (0.031) (0.033) (0.032) (0.030) DD, Results−Based Financing, re (βrbf(re) − βrbf ) 0.118∗∗∗ 0.113∗∗∗ 0.149∗∗∗ 0.077 0.121∗∗∗ 0.103∗∗∗ 0.126∗∗∗ 0.113∗∗∗ (0.028) (0.028) (0.017) (0.066) (0.030) (0.038) (0.030) (0.031) # of Student Observations 194, 618 155, 907 735, 343 74, 842 160, 705 206, 536 151, 597 165, 994 Adj. R−squared 0.185 0.162 0.165 0.185 0.185 0.182 0.186 0.189 Municipality Controls Yes Yes Yes Yes Yes Yes Yes Yes Student Controls Yes Yes Yes Yes Yes Yes Yes Yes Border Fixed−Effects Yes Yes No Yes Yes Yes Yes Yes municipalities at the Border Yes Yes No Yes Yes Yes Yes Yes City−Specific Time Trend Yes Yes Yes Yes Yes Yes Yes Yes Notes: ∗∗∗ denotes significance at 1 percent, ∗∗ at 5 percent, and ∗ at 10 percent level. This table presents robustness checks using portuguese scores of 5th grade students. The sample restricts to municipalities at the border between Ceará, the treatment state, and the states of Pernambuco, Paraíba, Rio Grande do Norte, and Piauí. All states share their border with Ceará. The regressions assign into the sample the first three municipalities in the imediate border between Ceará and one of the adjacent states. The data on student performance is from the Sistema de Avaliação da Educação Básica (SAEB). All regressions include municipality controls as the number of births in the municipality-year, according to the Sistema de Nascidos Vivos (SINASC); a dummy for rural schools (SAEB); and a dummy for schools offering from the 1st to the 9th grade. Student controls consider: a dummy if the student is male, if white, age in years, number of bedrooms at home, if the mother has higher education and the household size. Column 1 excludes observations from the year 2005. Column 2 excludes the year 2017. Column 3 applies the same approach to all municipalities, unconstrained to their location. Column 5 uses the sample to schools offering exclusively from the 1st to the 5th grades. Column 6 refers to schools offering primary and lower secondary years, i.e. from the 1st to the 9th grades. From columns 7 to 10, the estimates exclude schools located at each border separately. The impact coefficients are presented at the bottom of each column and estimates the effects of the RBF model and RBF + Technical assistance programs separately in comparison to the difference at the border in the pre−RBF period. Standard errors are clustered by municipality level and 95% confidence intervals are in brackets. Table A3 School Enrollments (Clustered Standard Errors)

Municipal Schools Private Schools Primary Low Secondary Primary Low Secondary All All Education Education Education Education (1) (2) (3) (4) (5) (6) A. DD, Results−Based Financing 0.045* -0.002 0.069* -0.042 -0.061 0.123 βrbf − βpre (0.025) (0.028) (0.041) (0.078) (0.074) (0.079) B. DD, Teaching Assistance 0.001 -0.015 -0.135 -0.039 -0.062 0.023 βrbf+ta − βrbf (0.015) (0.021) (0.157) (0.058) (0.060) (0.062) C. DD, Results-Based Financing, reformulated -0.090*** -0.116*** -0.094** -0.077 -0.073 -0.014 βrbf(re)+ta − βrbf+ta

51 (0.018) (0.021) (0.042) (0.061) (0.058) (0.065) # of municipality Observations 2,310 2,310 2,310 2,310 2,310 2,310 Adjusted R-squared 0.706 0.688 0.572 0.477 0.491 0.511 Notes: *** denotes significance at 1 percent, ** at 5 percent, and * at 10 percent level. The dependent variable comes from School Census of the Ministry of Education. All regressions restrict the sample to the 330 municipalities at the border between Ceará and adjacent states. The definition of border considers the three immediate municipalities at each side of Ceará and Piaúi, Rio Grande do Norde, Pernambuco and Paraíba. In Column 3 onwards, municipality controls include the number of births in the municipality-year according to the Sistema de Nascidos Vivos (SINASC) and GDP per capita as provided by IBGE. All columns account for "Border Fixed effects" which is a set of dummies for municipalities sharing the same border; i.e; Ceará with Piauí, Ceará and Rio Grande do Norte, Ceará and Pernambuco and Ceará and Paraíba. The vector of dependent variables includes: (1) Natural log of total enrollment in primary and secondary low education of municipal schools; (2) Natural log of enrollment in primary education of municipal schools; (3) Natural log of enrollment in low secondary education of municipal schools; (4) Natural log of total enrollment in primary and secondary low education in private schools; (5) Natural log of enrollment in primary schools of private schools; (6) Natural log of enrollment in low secondary schools of private sector. Time trends and Municipal specific time trends are included. To compare municipalities close geographically, sharing the same state border, all regressions include Border Fixed Effects. The bottom of each column presents in Panel A the DD estimates of RBF impacts, Panel B presents the effects after introducing teaching assistance, the TA component in our model; and Panel C shows the difference-in-difference estimates of reformulating the results-based financing model to include pass rates and higher weights for lower secondary grades. Standard errors are clustered by municipality and shown in parenthesis. Table A4 The RBF effects on School Facility (Clustered Standard Errors)

Internet Computer Science Sports Library Classrooms Connection Lab Lab Court (1) (2) (3) (4) (5) (6)

A. DD, Results−Based Financing 0.070*** 0.075*** 0.000 0.071*** 0.051*** 0.248* βrbf − βpre (0.024) (0.022) (0.002) (0.024) (0.014) (0.143) B. DD, Teaching Assistance 0.082*** 0.181*** 0.007* 0.028** 0.008 0.286** βrbf+ta − βrbf (0.021) (0.038) (0.004) (0.012) (0.010) (0.112) C. DD, Results-Based Financing, reformulated 0.049* −0.031 0.005 0.005 0.053*** 0.029 βrbf(re)+ta − βrbf+ta (0.029) (0.033) (0.003) (0.021) (0.011) (0.178) Variation relative to the baseline mean βrbf − βpre 21.9% 22.7% 0.0% 20.6% 30.2% 4.9% βrbf(re)+ta − βrbf+ta 25.7% 6.6% 20.0% 7.0% 8.3% 2.8% rbf(re)+ta rbf+ta

52 β − β 15.4% -9.4% 50.0% 1.4% 31.4% 0.6% Dep. var. mean 0.319 0.331 0.010 0.345 0.169 5.077 Dep. var. SD 0.316 0.330 0.027 0.267 0.173 2.594 # of municipality Observations 2,310 2,310 2,310 2,310 2,310 2,310 Adj. R−Squared 0.722 0.634 0.084 0.549 0.484 0.659

Notes: *** denotes significance at 1 percent, ** at 5 percent, and * at 10 percent level. The dependent variable comes from School Census of the Ministry of Education. All regressions restrict the sample to the 330 municipalities at the border between Ceará and adjacent states. The definition of border considers the three imediate municipalities at each side of Ceará and Piaúi, Rio Grande do Norte, Pernambuco and Paraíba. In Column 3 onwards, municipality controls include the number of births in the municipality-year according to the Sistema de Nascidos Vivos (SINASC) and GDP per capita as provided by IBGE. All columns account for "Border Fixed effects" which is a set of dummies for municipalities sharing the same border; i.e; Ceará with Piauí, Ceará and Rio Grande do Norte, Ceará and Pernambuco and Ceará and Paraíba. The vector of dependent variables includes: (1) Share of fundamental schools with access to internet; (2) Share of fundamental schools with computer lab; (3) Share of fundamental schools with science lab; (4) Share of fundamental schools with library; (5) Share of fundamental schools with sports court; (6) Average number of classrooms. Time trends and Municipal specific time trends are included. To compare municipalities close geographically, sharing the same state border, all regressions include Border Fixed Effects. The bottom of each column presents in Panel A the DD estimates of RBF impacts, Panel B presents the effects after introducing teaching assistance, the TA component in our model; and Panel C shows the difference-in-difference estimates of reformulating the results- based financing model to include pass rates and higher weights for lower secondary grades. Standard errors are clustered by municipality and shown in parenthesis.