David Deming DRAFT – DO NOT CIRCULATE September 2009 1

School Choice and College Attendance Evidence from Randomized Lotteries

David Deming Justine Hastings Thomas Kane Douglas Staiger

This is a preliminary draft. Please do not circulate or quote it without prior permission. Comments are welcome and appreciated.

ABSTRACT

In 2002, Charlotte Mecklenburg school district implemented an open enrollment policy that allocated slots at oversubscribed schools via random lottery. To assess the impact of gaining admission to a highly demanded high school, we match administrative data from the district to the National Student Clearinghouse, a national administrative database of postsecondary enrollment. We find strong evidence that high school lottery winners from neighborhoods assigned to the lowest-performing schools benefited greatly from choice. Girls are 12 percentage points more likely to attend a four-year college. Boys are 13 percentage points more likely to graduate from high school but are less likely to attend a four-year college. We present suggestive evidence that changes in relative rank within schools may explain these puzzling gender differences. In contrast with the results for students from low-performing home school zones, we find little evidence of gains for students whose home schools are of even average quality. David Deming DRAFT – DO NOT CIRCULATE September 2009 2 School choice is an increasingly important feature of the U.S. education policy landscape. Scarce public resources and the rising return to education have led to a focus on policies that can enhance schools’ productivity. Proponents of school choice espouse policies which decouple neighborhood residence and school attendance, breaking the monopoly power of local school districts and causing schools to compete for students (Hoxby, 2003.) Aside from competitive pressure, school choice could also enhance welfare by improving match quality between students and schools (Hoxby, 2003). Improvement in outcomes for student applicants is a necessary condition for choice to be efficiency-enhancing. Yet evidence on the benefits of public school choice, at least in the U.S. setting, is weak. The most broad-based form of school choice is known as open enrollment. Open enrollment allows public school students to attend magnet or other public schools that are outside of their neighborhood zone, subject to the availability of slots. A number of school districts have implemented some form of open enrollment in recent years. Cullen, Jacob and Levitt (2006) study one such program in Chicago public high schools. They use lottery-based random assignment to oversubscribed schools to investigate the causal impact of being offered admission to a non-home school, and find no evidence of benefits across a number of outcomes despite sizeable changes in measured peer quality (Cullen et al 2006). Voucher programs in Milwaukee (Witte 1997, Rouse 1998), New York City (Howell and Peterson, 2002) and Washington DC (Wolf et al, 2008) have found mixed impacts on student achievement. Some of the best evidence for the benefits of school choice comes from recent studies of lottery-based admission to charter schools. Hoxby and Rockoff (2004) and Hoxby and Murarka (2007) find modest effects of charter schools on student in Chicago and New York City respectively. Very recent results from Boston (Angrist et al, 2009) and the charter schools in the Harlem Children’s Zone (Dobbie and Fryer, 2009) have found very large yearly gains on standardized test scores, particularly in middle school math. Among studies of school choice that do find positive impacts, they are largely limited to test scores as an outcome. Usually this is by necessity, since few other indicators of progress for school-age children are available. Although past research has demonstrated the connection between test scores and wages (Murnane et al 1995; Krueger 2003), postsecondary outcomes such as educational attainment, earnings, and health are of direct interest. Furthermore, some of the best evidence concerning long-term benefits of social programs comes from interventions David Deming DRAFT – DO NOT CIRCULATE September 2009 3 where test score gains “faded out” over time (Anderson 2008; Belfield et al 2006; Deming 2009). Recent studies of school responses to high-stakes accountability policies have found evidence of “teaching to the test” (Jacob 2005), or purely non-productive test score gains through strategic assignment to test-taking (Figlio 2006, Jacob 2005) and teacher cheating (Jacob and Levitt 2003). This raises concerns that schools may be inefficiently multi-tasking by acting to maximize measured achievement at the expense of postsecondary outcomes of direct interest such as college attendance (Holmstrom and Milgrom, 1991). In this paper we study the impact of open enrollment in high schools in Charlotte Mecklenburg school district (CMS) on subsequent college attendance. In 2002, CMS implemented a district-wide school choice plan. Students were guaranteed admission to their neighborhood school but were allowed to choose and rank up to three other schools in the district, including magnet schools. Nearly half of all rising ninth graders chose a non-guaranteed school. Where demand for school slots exceeded supply, allocation was determined by random lottery. Furthermore, the great majority of students who were offered admission chose to enroll, leading to a large one-year change in student assignments in the district. Since the youngest students in this sample were rising 9th graders in 2002, we are able to observe college-going for a minimum of two years after lottery participants were scheduled to graduate from high school. These are critical times in students’ lives. Over 90 percent of eventual bachelor’s degree recipients enter college within the first two years of graduating from high school (College Board, 2007). To analyze the effect of being offered admission to a non-guaranteed school, we match a long (1995-2008) and detailed panel of administrative data from CMS, including the lottery numbers and admission status of all applicants, to a national database of postsecondary enrollment called the National Student Clearinghouse (NSC). The match is done directly using personal identifying information. As a result, attrition from the college attendance data is limited only to colleges that are not covered by the NSC. As we discuss later, this is a very small fraction of four-year and public two-year colleges nationwide, and an even smaller fraction of colleges in North Carolina and the surrounding states. The central finding of this paper is that students with the lowest performing guaranteed (or “home”) schools benefit greatly from choice. We define “low performing” schools by their average test scores and rates of college attendance, but also by value added measures and David Deming DRAFT – DO NOT CIRCULATE September 2009 4 estimates of parental demand from the choice lottery. Students from neighborhoods that are assigned to one of the four lowest-performing schools stay in school longer, graduate at higher rates, and are more likely to attend college. In contrast, we find no consistent evidence of gains for students whose home school is of even average quality. In a pattern that is consistent with many previous studies, the effects of school choice are markedly different by gender (Kling et al 2007; Hastings et al 2006; Schanzenbach 2005; Angrist et al 2009). Females from low-performing schools who win the lottery to attend their first-choice school are about 12 percentage points more likely to attend a 4 year college. Although these students are still completing their schooling, the effects on persistence in college for females are proportionally even larger than for the enrollment margin. Male lottery winners are about 13 percentage points more likely to graduate from high school. In contrast we find no increase in college enrollment for males, and some evidence that lottery winners are actually less likely to persist in 4 year colleges. We present some evidence on changes in class rank for male lottery winners that may provide a possible explanation for this puzzling result. In general, our findings are corroborated by impacts on test scores and grades, course- taking patterns and disciplinary incidents. Lottery winners from low-performing neighborhood school zones had higher grade point averages and took more math and science classes. Evidence on high school test scores is compromised by selection into test-taking across years and subjects, but we find some suggestive evidence of gains in Algebra I and Chemistry for males. Lottery winners of both genders were less likely to be absent and suspended from school in the two school years following the lottery. In all cases, effects were large for children from low- performing school zones and small or near zero for all others. Changes in average peer characteristics also match these patterns – lottery winners from the lowest-performing schools had peers with significantly higher test scores and graduation and college enrollment rates, but this was much less true for lottery winners from other schools. One unavoidable limitation of lottery-based studies is that they rely on oversubscription. Any school with excess demand for enrollment is likely to be of above-average quality. Parents of students that apply to these schools are likely to place a high value on education and be more motivated than the average parent. Similarly, parents of lottery losers might be more motivated to find a good alternative for their child. In that case, only relative scarcity of school quality will produce a meaningful difference between lottery winners and losers. David Deming DRAFT – DO NOT CIRCULATE September 2009 5 There are several features of the CMS choice plan that make it ideal for studying the impact of school choice for students with lower-quality neighborhood schools. From 1968 until 2001, CMS operated under a desegregation court order. As a result, school boundaries were redrawn dramatically in the year of the choice plan, and nearly half of rising 9th graders were assigned to a different school than what they would have attended had they been one year older. This sudden and dramatic change generated rich variation in student assignments. Unlike many other studies, school choice in CMS was broad-based. Over 95 percent of rising 9th grade students submitted a choice, and over 50 percent chose their allotted maximum of three non- guaranteed schools. Furthermore, students who used all three choices on non-guaranteed schools were far more likely to be from “inner city” schools and were well below the district average on eighth grade math and reading scores. Overall, CMS schools vary tremendously by this measure – the difference between the highest and lowest high school in average 8th grade test scores of incoming 9th graders is about 1.5 standard deviations. However, it is important to note that these high-scoring schools were not the most highly demanded schools. Rather, highly demanded schools were demographically similar to the lowest-performing schools and were located in the same neighborhoods. Three of the four were magnet schools, and the test scores of incoming freshman were around the district average. In contrast, the highest-performing schools in CMS were in the outer suburbs and consisted mostly of higher-income, white students. Not all of these schools were oversubscribed, yet inner city parents elected not to attend them. One possible reason is that they value proximity very highly (Hastings et al 2006), but even conditional on distance the correlation between average test scores and parental demand was modest. This paper makes several contributions. The first is to provide evidence on the benefits of a specific form of school choice – open enrollment in public high schools. The only other study to look at school choice in a similar setting found no impact of winning the lottery to attend a first-choice high school (Cullen et al, 2006). There are two possible reasons for this discrepancy. The first is that school choice has a disproportionate impact on students with a low quality default option. In the study of school choice in Chicago, students could apply to many schools, the chances of acceptance were much lower, and a relatively low fraction of accepted students actually enrolled. While standard instrumental variables methods can account for this relatively weaker first stage, a more substantive implication is that lottery applicants in Chicago were often David Deming DRAFT – DO NOT CIRCULATE September 2009 6 weighing several other acceptable alternatives. This may be due in part to the twenty year history of school choice in Chicago, where parents have had many years to exercise Tiebout choice through residential location. Although this is true in CMS as well, the sudden opportunity to attend a non-home school may offer immediate benefits that are less available in equilibrium, once parents have re-sorted and admission probabilities have adjusted. The second possibility is that school choice has the strongest impact on postsecondary outcomes that have not been available in previous studies. This paper is, to our knowledge, one of the first academic studies to make use of administrative data on college attendance from the National Student Clearinghouse.1 Tthe NSC data are available for all students in CMS in multiple years, enabling a comprehensive examination of college-going in a large urban school district. In contrast to a model of strict peer achievement maximization, parents seem to choose schools that are demographically similar but do a much better job of sending children to college. This may not be surprising if students’ test scores are relatively fixed by 9th grade yet institutional features of a school (i.e. a good guidance counselor) could still have a large impact on college enrollment. Finally, this paper provides an in-depth look at the relationship between school success, course-taking and the growing “reverse” gender gap in educational attainment. While boys and girls choose similar schools and enter high school with similar test scores, female lottery winners accumulate more credits and have higher GPAs, and this leads to increases in four year college enrollment. In contrast, boys are no more likely to accumulate math and science credits after 9th grade and have only marginally higher GPAs. One important possible explanation is that highly demanded schools are also more challenging, and that to succeed students must adjust their level of effort. I find that lottery winners of both genders enter school at a lower relative rank than lottery losers on an initial test score. However, female lottery winners adjust by improving their grades so that they are similar to lottery losers in the GPA distribution, whereas male lottery winners lose GPA rank. This may be a partial explanation for the pattern of findings on college attendance by gender.

1 Kane (2003), matches administrative data from the CalGrant program to the NSC in an NBER working paper. The Chicago Consortium for School Research (CCSR) has also made use of NSC data in several published policy briefs. David Deming DRAFT – DO NOT CIRCULATE September 2009 7 2. Institutional Detail and Data Description 2.1 – Background and Details of the Choice Plan Charlotte-Mecklenburg is the 20th largest school district in the nation. The school district encompasses all of Mecklenburg County, which includes both the inner city areas of Charlotte and the more affluent suburbs surrounding it. Thus neighborhoods in CMS and the schools that are assigned to them vary widely by race and income. In 1971, the Supreme Court (in Swann v. Charlotte-Mecklenburg Board of Education) ruled that this variation resulted in neighborhood schools that were de facto segregated, and for over 30 years CMS schools were forcibly desegregated under a court order. Students were bused all around the district to preserve racial balance in the schools.gb Particularly at the high school level, this meant in practice that inner- city and largely African-American neighborhoods were divided up and bused out to more affluent and white suburbs in different parts of the county. After several years of legal challenges, the court order was overturned and CMS was declared unitary and ordered to dissolve its busing plan. In 2001 the State Supreme Court declined to hear an appeal, signaling the end of desegregation. In December of 2001 the CMS School Board voted to move forward with district-wide open enrollment for the 2002-2003 school year. Because CMS was no longer allowed to use race explicitly in student assignments, they redrew school boundaries as traditional contiguous neighborhood school zones. Children who lived within each zone received guaranteed access to their neighborhood school. This resulted in a change in assigned neighborhood high school for about 35 percent of households. Figures 1a and 1b provide a visual illustration of this dramatic change. Figure 1a is a map of the high school boundaries in the year before the choice plan (2001-2002), and Figure 1b shows the boundaries in the following year. The hatched areas on both maps are areas that experienced a change in high school assignment. As the figures show, the majority of central city areas and substantial shares of the suburbs were reassigned. The school choice lottery took place in the spring of 2002. In order to maximize the number of parents that exercised choice, CMS conducted an extensive information campaign. They held a well-advertised fair at the Charlotte convention center, set up “choice booths” in local shopping malls, and sent volunteers door-to-door in low-income and non-English speaking neighborhoods to talk to families about the plan. CMS also developed a comprehensive booklet David Deming DRAFT – DO NOT CIRCULATE September 2009 8 with information about each school, as well as smaller brochures for individual schools. As a result, over 95 percent of parents submitted a choice application in the spring of 2002. Parents were allowed to submit up to three choices, which included schools as well as special programs within schools.2 Students were guaranteed admission to their neighborhood school, and admission for all other students was subject to grade-specific capacity limits that were set by the district beforehand but unknown to families at the time of the lottery (Hastings et al, 2006). Children with siblings already in enrolled in a school also received guaranteed access. CMS was also divided into four “choice zones” and transportation was provided by the district only within each zone, although families were free to provide their own transportation to any school.3 The district expanded capacity at schools where they anticipated high demand in an attempt to give every parent one of their top three choices. Still, most high schools were oversubscribed. When demand for slots among non-guaranteed applicants exceeded supply, admission was allocated by random lotteries that occurred within the following lexicographic priority groups. The first three groups consisted of students who had attended the school in the previous year, with the highest rising grade (12th) given the first grouping. In practice, this meant that most lotteried applicants (about 60 percent) were rising 9th graders. The next priority group consisted of free-lunch eligible (i.e. low income) students applying to schools where less than half of the school population was free-lunch eligible. The final priority group was students applying to a school within their choice zone. Applicants were sorted by priority group according to these rules, and then assigned a random lottery number. Slots at each school were first filled by students with guaranteed access, and then remaining slots were allocated within each priority group according to students’ lottery numbers. If all members of a priority group could be offered admission, slots were allocated to the next priority group in the order of lottery numbers. CMS administered the lottery centrally (i.e. schools did not conduct their own lotteries) and applied an algorithm known as a “first choice maximizer” (Abdulkadiroglu and Somnez, 2003). This meant that CMS first allocated slots to all those who listed a school as their first choice and only then moved to second choices. As the name indicates, this maximized the percentage of students who received their first choice, but it also meant that students who lost the

2 Parents who listed 3 non-guaranteed choices were automatically assigned their “home school” as a 4th choice. 3 The choice zones were constructed so that there was at least one predominately white suburban and at least one predominantly black inner-city school in each zone. David Deming DRAFT – DO NOT CIRCULATE September 2009 9 lottery to attend their first choice school often found that their second choice had already been filled up in the previous round. While there is the potential for strategic choice with this type of lottery mechanism, Hastings et al (2006) show evidence that this is not likely to be a large problem in CMS, at least in the first year of the choice plan. In any case, even if parents are of two types (“sincere” and “sophisticated”), each has the same ex ante probability of winning the lottery so it will not bias the results, although it may complicate their interpretation (Pathak and Somnez, 2008).

2.2 Data Description and Summary Statistics We match the lottery applicant file, with individual lottery numbers and priority groupings, to a panel of administrative data from CMS. The data span approximately six years before and after the choice lottery (from 1995 to 2008) and contain detailed information on students’ enrollment histories, test scores, course-taking and other outcomes of interest. The North Carolina Department of Public Instruction requires all school districts to assemble and send them a standardized set of files under the state’s accountability regime. In addition to enrollment records, this includes students’ scores on standardized End-of-Grade (EOG) exams in math and reading for grades 3-8 and End-of-Course (EOC) exam scores in high school subjects such as Algebra I and II, Geometry, English I, Biology and Chemistry. These tests are administered to all public school enrollees and schools are required to use the scores as some component of students’ grades. Importantly, these reporting requirements help to ensure that CMS’s records are of high quality, with longitudinally linked and consistent student records. See the Data Appendix for a more thorough description of the CMS administrative data. We match these files to information on college attendance from the National Student Clearinghouse (NSC), a non-profit organization that maintains enrollment information for over 90 percent of colleges nationwide. The NSC was originally started as a service to help banks and lenders track student borrowers. The data contain information on enrollment spells, full or part- time status and (in some cases) degree receipt for all covered colleges that a student has ever attended. Although not all colleges provide information to the NSC, the coverage is very good in North Carolina and the surrounding states. The Data Appendix contains a list of colleges by David Deming DRAFT – DO NOT CIRCULATE September 2009 10 coverage and a detailed analysis of the match process using data from the Department of Education’s Integrated Postsecondary Data Source (IPEDS) as a reference.4 The administrative data from CMS, while extremely detailed, have the downside that students are not followed if they leave the district. In this case the effect of being randomly offered admission to a first-choice school will only be observed for students who stay in the district. Since losing the lottery may be correlated with withdrawal from CMS, this presents a potential threat to the validity of lottery-based randomization. We match both external data sources to CMS records using personal identifying information such as name, date of birth and social security number. This overcomes the limitation of nonrandom attrition, because students who leave CMS are also followed in these data. Thus attrition is subject only to the NSC’s coverage and the quality of the match. However, unless coverage is differential for lottery winners and losers, the results may be attenuated but will not be biased. CMS received high school lottery applications from 29,584 students. We first limit the sample to students who were enrolled in any CMS school in the previous year. About five percent of applicants come from outside the district, and these students are much less likely to be enrolled in CMS the following fall. Since previous enrollment status is fixed at the time of the lottery, this sample restriction does not affect the validity of the randomization. We also drop the less than one percent of students who choose alternative schools. Finally, about five percent of this sample does not show up in any CMS school in the fall of 2002. Although these students can still be matched to the NSC and arrest record data and are included in those analyses, we have no other outcome information for them. Thus selective attrition is a concern for analyses that use CMS administrative data. This final restriction results in a sample of 26,242 rising 9th-12th graders. Even with mandatory busing, the demographic composition of CMS high schools varied widely. Not surprisingly, the redrawing of school boundaries as contiguous neighborhood school zones caused these already sizeable gaps to widen further. The first and fourth columns of Table 1 report the average 8th grade math score (in standard deviation units) of rising 9th graders and

4 The major two-year college in Charlotte, Central Piedmont Community College (CPCC), did not provide information to the NSC until 2006. To fill in this gap, I obtained enrollment data directly from CPCC for all years which was more detailed than what is typically provided by the NSC. The data from CPCC contain information on type of enrollment (i.e. degree-seeking or correspondence course) and credit accumulation and GPA. I also used this data to verify the NSC’s match process. See the Data Appendix for details. David Deming DRAFT – DO NOT CIRCULATE September 2009 11 the percent of black students in each high school for the year before the choice plan (2001-2002). The second and fifth columns show the change in composition for each school in the year of the choice plan (2002-2003). The third and sixth columns compute the average characteristics of students in each high school’s neighborhood zone. This shows what CMS schools would look like if the boundaries were redrawn but there was no school choice. The schools are ranked by the average test score of their 2002 neighborhood zone. The difference between the top and bottom school was already large before choice (about 1 standard deviation), but it grows further to about one and a half standard deviations in the year of the choice plan. Similarly, racial sorting increased sharply. This is not surprising given the composition of each school’s neighborhood zone. The seventh through tenth columns of Table 1 show enrollment patterns. The utilization rate is simply the number of children who attend a school divided by the total number zoned to that school. Lower test-score schools are under-enrolled, with capacity at or just above 50 percent in a few cases. This is partly due to the 3 magnet high schools in the district, which are located in inner city areas. One school, Berry Academy of Technology, was opened for the first time in 2002. Still, several lower-test score schools saw large drops in enrollment from 2001 to 2002, which can only be partially accounted for by increases in the supply of magnet slots. In contrast, most of the higher-scoring schools were close to capacity or oversubscribed.

2.3 Patterns of Choice in CMS There was a great deal of churning in CMS schools in the year of the choice plan. Nearly 50 percent of rising 9th graders chose a school other than the one to which they were assigned. This was due partly to the large one-year change in school boundaries, but choices were very heterogeneous even among parents whose home school remained the same. Table 2 shows these patterns of choice by neighborhood school zone. The first two columns show the number of rising 9th graders assigned to each neighborhood school, and the fraction of them that chose their neighborhood school first. This share, while never close to 100 percent for any school, varies greatly across schools. Like Table 1, schools are sorted by the average 8th grade test scores of students in the neighborhood zone. Not surprisingly, higher test-score schools are chosen more frequently. What is notable, however, is how few of the students in the low test-score neighborhoods choose their home school. Among the four lowest-scoring neighborhoods, only David Deming DRAFT – DO NOT CIRCULATE September 2009 12 about 45 percent of residents choose to attend their home school. For rising 9th graders, that number is less than 30 percent. As a consequence, a disproportionate share of the lottery sample consists of students from low-scoring neighborhoods. Two conditions must hold for students to be in the lottery sample. First, a student must apply to a non-guaranteed school. Second, the probability of admission for that student’s priority group must be greater than zero and less than one (i.e. a non-degenerate lottery). In the lottery analysis specification in Section 3, the weight given to each school in the overall analysis is the number of students in the lottery times p*(1-p) where p is the probability of winning the lottery. Intuitively, this is because lotteries with a more balanced (i.e. closest to 50 percent) proportion of winners and losers will have greater power to detect true differences in outcomes. The result, in column 7 of Table 2, is that students from the lowest-scoring neighborhoods constitute a disproportionate share of the lottery sample. Table 3 presents choice patterns organized by first choice rather than neighborhood school. The first thing to note is that the 4 highest-scoring schools admitted over two-thirds of non-guaranteed applicants. Three of these 4 schools admitted 100 percent of rising 9th grade applicants. This suggests that parents are not all strictly maximizing on average test scores since those schools were available but not chosen. Instead, the most frequently chosen schools rank in the middle in terms of test scores but are demographically similar to the lowest scoring neighborhoods. The set of lottery weights calculated in column 7 reveals that the 3 magnet schools account for about 60 percent of the lottery sample. It is also important to note that enrollment conditional on admittance is very high for all schools. This suggests that students were not relying on unobserved outside options, and is important for interpretation of the results presented in Section 4. The lottery sample consists of 1,230 rising 9th graders and 786 rising 10th-12th graders. Previous studies of school choice have been criticized as having limited external validity due to “cream skimming”. If students who opt out of their neighborhood schools are a small and very different population than those who stay, the effect of choice may be hard to generalize to a larger population. Table 4 shows that this is not a concern here. Columns 1 and 2 compare the characteristics of all students to those who chose their home school. Columns 3 through 5 organize students who chose a non-guaranteed school into 3 categories. The first and second David Deming DRAFT – DO NOT CIRCULATE September 2009 13 categories consist of priority groups where everyone was admitted and everyone was denied, respectively. The last group is the lottery sample, which was subject to randomization. In general, students who chose to remain at home are better off than all others. Their test scores are about 0.4 standard deviations higher than non-guaranteed choosers, and they are more likely to be white and less likely to be free lunch eligible. Among those who chose a non- guaranteed school, the all admitted group is more likely to be free-lunch eligible. This is because free lunch students who applied to a school with a low fraction of free-lunch eligible students were given an explicit priority boost in the lottery.5 Aside from that, the three groups are fairly equal. To show this simply, we first generate a predicted probability of 4 year college attendance by regressing four-year college attendance on a polynomial in prior test scores, student demographics and prior school and census tract fixed effects. We then plot this predicted probability for students who choose their home school and then for the three groups of non- guaranteed applicants in Figure 3A. We can see that applicants who choose their home school are much more likely to attend college than others, but that the three groups who choose non- guaranteed schools (all admitted, all denied, and the lottery sample) are very similar. Figure 3B plots this same predicted probability for students from low-performing school zones. Here we can see that there is very little difference between any of the groups. Although lottery applicants are clearly different in some way since they chose not to attend their home schools, it is important to note that they are similar on an index of observable predictors of college attendance. Finally, Column 7 shows the characteristics of the less than five percent of the sample who did not submit a choice. Nearly 50 percent are free lunch eligible, and they have average test scores about 0.75 standard deviations below the average. We cannot say anything about the effect of winning the lottery for this population.

2.4 Measures of School Quality Schools with low average test scores are not necessarily low-performing. If parents know this, they may choose schools that provide better “value added” over schools with better peers. Still, there is plenty of evidence that parents value test scores in their choice of school, either through higher housing prices in the residential location decision or through school choice

5 Note that this doesn’t affect the identification strategy since I estimate the effect of the winning the lottery for applicants within the same priority groups. David Deming DRAFT – DO NOT CIRCULATE September 2009 14 programs (Black 1999, Bayer et al 2006, Hastings et al 2006.) It is less clear, however, that improvements in measured peer quality are productive for students, despite parental demand. Perhaps incremental improvements are effective but large changes lead to mismatch (Cullen et al 2006). Alternatively, mean peer achievement in levels may not be as informative as in gains. School choice scholars define school productivity nearly exclusively as average academic achievement per dollar (Hoxby 2003, Hanushek 2003). Yet parents also show weak responsiveness to “value added” measures in existing studies of school choice (Cullen et al 2006; Hastings et al 2008). This could be due either to a lack of information about school effectiveness (Hastings and Weinstein, 2008), because they do not value effectiveness as much as other neighborhood features (Rothstein 2006), or because “value added” is not actually a good measure of school quality. Table 5 measures school value-added and parental demand. Value-added is calculated as the school-level mean residual from a student-level regression of each outcome on a 3rd order polynomial in 7th and 8th grade reading and math test scores, demographic characteristics, and census tract fixed effects. This effectively compares students who live in similar neighborhoods, with similar observable measures of academic preparation, but who go to different high schools. These estimates are likely to suffer from bias due to nonrandom sorting into high school zones that is not picked up in test scores (Rothstein, 2009). Still, they may be informative and are likely to be less biased than simple comparisons of average test scores. We calculate value-added measures for two outcomes. The first is students’ standardized score on the English I EOC exam, which is taken by nearly every student in the 9th grade. The second is the predicted probability that students will have enrolled in four total semesters of any college by the spring of 2008. Alternative measures of college enrollment, such as any enrollment or at least one semester at a four-year college, yield similar results.6 Although we do not report scores based on the 2001 9th grade cohort, the across-year correlation in value-added for both English I and college attendance is reasonably high, about 0.75 and 0.6 respectively. This is notable because both rezoning and choice greatly affected the selection process of students to schools.

6 Two year enrollment often includes things like non-degree correspondence courses and GED retraining, which are arguably not measures of the academic contribution of a school. Few of these enrollees stay for more than a semester or two, however. One option is to only count 4-year college enrollment, yet some schools might specifically target 2-year colleges. In any case the correlation between different value-added measures of college enrollment is always over 0.9. David Deming DRAFT – DO NOT CIRCULATE September 2009 15 We also calculate parental demand, first as the percentage of families that select each school as their first choice, weighted by the size of each schools neighborhood zone and then standardized. This is reported in column 4. Column 5 reports the mean school residual from a conditional logistic regression which predicts the probability that families will choose each high school, controlling for a 4th order polynomial in travel time, plus home school and choice zone fixed effects. This is intended to account for the fact that some schools are not as highly demanded because they are located in less dense parts of the district. See Long (2004) for an explanation of the conditional logit setup and Hastings et al (2006) for a richer specification of parental choice that uses all 3 choices and allows for substitution patterns in a mixed logit framework. There are 3 important takeaways from Table 5. The first is that value-added measures are positively correlated with average test scores. This is a combination of true productivity differences between the schools and the bias from unobserved sorting across neighborhoods that is not captured by test scores and other covariates. Interestingly, the correlation is modestly higher for college-going than for the 9th grade English score value-added (0.67 vs. 0.53). The second thing to note is that value-added measures also predict parental demand. Again, college value-added is a stronger predictor than English (0.46 versus 0.24), and it is actually more correlated with parental demand than average test scores (0.25). The last important point is that there are some schools which rate poorly on all measures. Strikingly, the same four schools rate at the bottom on average test scores, college value added, and parental demand. Students from these neighborhood zones also make up nearly two-thirds of the lottery sample. Clearly, parents are choosing to exit these low test-score and low value-added schools. It is thus tempting to conclude that these schools are low-quality and that winning the lottery to attend another school will greatly benefit students from these neighborhoods. Still, none of the measures in Table 5 can fully account for the non-random sorting of students into neighborhoods.

3. Empirical Strategy If winning the lottery is randomly assigned, the winners and losers of each lottery will on average have identical observed and unobserved characteristics. Thus in expectation, the effect of winning the lottery can be estimated as a simple difference in means. With sufficient sample David Deming DRAFT – DO NOT CIRCULATE September 2009 16 size, one could estimate this difference for every lottery, but here the sample is not large enough for such a calculation. As an alternative, we estimate ordinary least squares regressions of the form:

Yij = ΒXij + δ(Win Lottery) ij + Γj + eij (1)

where Y is the outcome variable of interest, Xij is a vector of covariates which includes gender, race, free or reduced price lunch status, special education and limited English proficiency status, prior math and reading test scores, suspensions, absences and 2001-2002 school fixed effects. Win Lottery is an indicator variable that equals 1 if the student won the lottery, Γj is a set of

lottery fixed effects and eij is a stochastic error term. This specification conforms closely to Cullen et al (2006). The number of observations is equal to the number of students in the lottery sample since there is only one first choice application per student. Standard errors are clustered at the lottery (i.e. grade by choice by priority group) level. Lottery fixed effects are necessary to ensure that the ex ante probability of admission to a first-choice school does not differ between losers and winners. If, for example, savvy families had some prior knowledge about the chance of admission, they might (all else equal) apply to schools with a higher probability of acceptance. Thus comparing winners and losers across lotteries may induce bias. In this specification, the δ coefficient gives the weighted average (with weights approximately equal to those in column 8 of Table 3) of treatment minus control differences summed over each individual lottery. Here δ corresponds to an intent-to-treat (ITT) effect of attendance at a first choice for students who choose to apply to schools where the probability of admission is greater than zero and less than one. We cannot estimate the effect of attendance for students who apply to a guaranteed school or to a school with a degenerate lottery. The ITT does not capture the effect of attending a first-choice school, but rather the effect of being offered admission. Not all students who win the lottery will subsequently enroll. In Table 3 we saw that enrollment conditional on admission was very high – however, some lottery losers may also attend the school, largely by moving into the neighborhood zone after the lottery or in subsequent years. David Deming DRAFT – DO NOT CIRCULATE September 2009 17 An alternative to estimating the ITT is to use lottery status as an instrument for attendance at the first-choice school. Under certain assumptions the ITT can be scaled up by the difference in attendance to become a local average treatment effect (LATE). However, some lottery winners stay in school all four years and some leave much sooner. If better schools are also more academically challenging, the effect of winning the lottery may depend on a student’s prior preparation. In this case, withdrawal is also part of the treatment. Perhaps the best reason to report ITT estimates is that the effect of being offered admission is a direct policy parameter. In any setting where a school choice plan is implemented, not all students will apply nor will all applicants take up an offer of admission. School districts cannot force students to attend a particular school, and since length of enrollment among lottery winners is not randomly assigned, the LATE may be of limited external validity.

3.1 Verification of Randomization and the Effect of Winning on Attendance If the lotteries were conducted appropriately, there should be no difference on average between winners and losers on any characteristic that is fixed at the time of application. Table 6 tests this proposition by estimating equation (1) with a set of fixed demographic covariates as outcomes. Column 1 shows the mean value of each variable for lottery losers, and Column 2 shows the estimated difference and standard error for lottery winners. Reassuringly, mean differences are small and not statistically different from zero, suggesting that the randomization worked. The estimated effect of winning the lottery could still be biased by selective attrition if leaving the CMS school district is correlated with lottery status. Columns 3 and 4 of Table 6 reestimate covariate differences for the approximately 95 percent of the sample that was still in enrolled in CMS in the fall of 2002, after the lottery was conducted. The bottom two rows test the hypothesis that lottery winners are more likely to remain in CMS, About 94 percent of lottery losers are still enrolled, and winning the lottery has a precisely estimated zero impact on remaining in the district. Furthermore, as stated in Section 2, we have data on college attendance even for this small fraction of students who leave CMS after losing the lottery.7 Thus, selective attrition is unlikely to be a concern here.

7 Subject to coverage – see Appendices on NSC and Mecklenburg County Arrest data for details. David Deming DRAFT – DO NOT CIRCULATE September 2009 18 In Columns 5 and 6 we conduct a similar randomization check for the subsample of students who are assigned to the four lowest-performing high schools, as defined in Table 5. Here there is some evidence of imbalance. Although lottery winners score no higher on 8th grade math or reading test scores and are no more likely to be absent or suspended from school in the previous year, they are somewhat more likely to be both male and African-American. While this may just be a statistical fluke, the difference is troubling. We went back through the lottery files to see if the imbalance was concentrated in one lottery, and it was not. We estimated the regressions in Table 6 for 8th grade scores and predicted 4-yr enrollment within male and African-American applicant groups. No estimates were statistically different from zero. As a final robustness check, we estimated regressions for all the main outcomes using race and gender as additional priority groups in the lottery, which effectively compares winners and losers only within these groups. The main results were unaffected. Table 7 presents estimates of the effect of winning the lottery on initial fall 2002 enrollment and characteristics of the school attended. We split the sample into students from one of the four lowest-performing school zones (Columns 3 and 4) and all other students (Columns 1 and 2). Overall, lottery winners were 55 percentage points more likely to subsequently enroll in their first-choice school. This shows that winning the lottery had a large and highly significant impact on initial enrollment. About 35 percent of lottery losers still manage to enroll in their first choice school. This is due to a combination of factors. In a few cases, students lost the lottery for a special program within a particular school (such as the International Baccalaureate program) when regular admission to that school was not oversubscribed. The most common ways that lottery losers managed to enroll in their first choice is through subsequent admission off the waiting list and relocation to the school’s home zone after the lottery. Separate estimates for 9th grade applicants show that only about 20 percent of losers managed to enroll, compared to nearly 50 percent for 10th-12th graders. Lottery winners are about 36 percentage points less likely to be enrolled in their neighborhood school and about 30 percentage points more likely to be enrolled in a magnet school. They are also enrolled in schools that are about 1.5 miles farther away from their residence. Interestingly, the difference in distance to assigned school is near zero for the group of applicants from low-performing neighborhood schools. This reflects the fact that the three highly demanded magnet schools are all located in inner city areas. Since previous research has shown David Deming DRAFT – DO NOT CIRCULATE September 2009 19 that low-income parents value proximity very highly, this may be an important explanation for the pattern of parental demand (Hastings et al 2006; Hastings and Weinstein 2007). Lottery winners experienced large changes in average peer characteristics.8 Importantly, this was concentrated mostly among winners with low-performing neighborhood school assignments. On measures such as average 8th grade math scores, grade point average, high school graduation and college attendance, average peer characteristics of lottery winners were about 0.5 school-level standard deviations better than for losers. Proportional gains were particularly large (about 0.63 school-level standard deviations) for four-year college matriculation. The differences for winners from all other areas, while often significantly different from zero, were much smaller. Finally, although lottery winners experienced large increases in peer quality across a wide variety of outcomes, it is important to note that the schools they attended were demographically similar to the ones attended by lottery losers. This is in contrast to Cullen et al (2006), who found that lottery winners attended schools with modestly higher high school graduation rates (about 2 percent) but also lower fractions of African-American, Hispanic and free lunch eligible children (about 3-4 percent).

4. Results 4.1 – Persistence, Graduation and Transfer In Section 3 we show that winning the lottery had a strong and statistically significant impact on enrollment and on measures of peer quality for students with low-performing neighborhood schools. This shows that lottery winners from these areas experienced a significant change in school environment. As previous research has shown, however, higher peer quality does not guarantee improvement on outcomes like test scores and educational attainment. In a study of open enrollment in public high schools in Chicago, for example, Cullen, Jacob and Levitt (2006) find no gains for lottery winners despite significant gains in average peer test scores and graduation rates. In Tables 8 and 9 we evaluate the effect of winning the lottery on educational attainment, overall and then separately for students whose neighborhood school assignment is one of the four lowest-performing schools in CMS. We also split that sample by gender, since many educational interventions have found that girls benefit much more than boys,

8 These average peer variables are calculating using enrollment in the fall of 2002, and they exclude students in the lottery sample. David Deming DRAFT – DO NOT CIRCULATE September 2009 20 particularly when the sample is very disadvantaged (Kling, Ludwig and Katz 2005; Hastings et al 2006; Schanzenbach 2003; Angrist et al 2009). Table 8 shows the effect of winning the lottery on persistence in school and graduation. The first three rows estimate the effect of winning the lottery on being in the first-choice school in each subsequent school year. The effect of winning the lottery could be attenuated over time, either because lottery winners transfer to another school or drop out of the CMS system altogether, or because lottery losers successfully enroll in subsequent years. In general, however, winning the lottery in the spring of 2002 has a strong and persistent impact on subsequent enrollment. Even by the fall of 2005, when rising 9th grade lottery applicants would be entering their senior year of high school, 9th grade lottery winners are about 34 percentage points more likely to be enrolled in their first choice school.9 Overall, winning the lottery led to substantial increases in grade attainment. Column 2 shows that lottery winners were four to five percentage points more likely to stay in school through grades 10, 11 and 12. In Columns 4 and 6, we see that these gains are concentrated among lottery winners from low-performing neighborhood school zones. The point estimates are of similar magnitudes by gender but more precise for males, who were 9.6 and 9.4 percentage points more likely to make it to 11th and 12th grade respectively. Estimates for grade repetition are consistently negative but only statistically significant at the ten percent level for 9th grade girls, who were about 9 percentage points less likely to be retained if they won the lottery. The bottom panel of Table 8 estimates the effect of winning the lottery on students’ withdrawal status from CMS. These data directly come from the district’s administrative records, which are maintained and reported out to the North Carolina Department of Public Instruction as part of the state’s accountability system. There are five mutually exclusive possibilities – graduate, transfer in state, transfer out of state, dropout and no code. High school graduation is defined conservatively, as students who have a confirmed diploma date and were present through the end of 12th grade in a CMS school. In-state transfers are also defined conservatively, because CMS is required to verify that a student has enrolled at another school in the state of North Carolina before they can designate a student as an in-state transfer. Out-of-state transfers are not

9 All estimates exclude rising grades that, in the absence of grade repetition, would have already graduated and left CMS. For example, the effect of winning the lottery on fall 2005 enrollment is estimated for 2002 rising 9th graders only. Likewise, the estimates for grade attainment in rows 4-6 exclude applicants who had already made it to 10th, 11th or 12th grade when the lottery was conducted. David Deming DRAFT – DO NOT CIRCULATE September 2009 21 verified as rigorously. A non-trivial share (about 12 percent) of CMS enrollees simply stop showing up in the data without receiving a withdrawal code. While it is difficult to know exactly how the decision is made to call a student a dropout versus not assigning a code, we can compare them on observables as well as match each category of student to the National Student Clearinghouse and see how many of them show up in a (North Carolina or other state) college. In the low-performing subsample, the average 8th grade math scores of students with no withdrawal code are very close to those of dropouts (about -0.70 versus -0.75). 2.1 percent of girls and 8.5 percent of boys with no withdrawal code show up in a 4 year college, compared to about 1 percent of those coded affirmatively as dropouts. Based on these data and on the pattern that “high risk” males are more likely to not receive a withdrawal code, it is likely that a substantial fraction of these students have dropped out of school. Overall, the pattern of increased grade attainment among lottery winners also holds for high school graduation. Lottery winners are about 4 percentage points more likely to graduate from a CMS high school. Part of this effect, however may be offset by transfers. Winners are a statistically insignificant 2 percentage points less likely to transfer in-state, and they are no more likely to transfer out of state or have no code. This estimate represents some combination of true increases in graduation and differences between winners and losers in attrition from CMS over time. If we assume that all in-state transfers graduate, for example, the mean difference in graduation would be about 2 percentage points. Again, however, the effects are concentrated entirely in the “low-performing” subsample. Female lottery winners from these schools are about 11 percentage points more likely to graduate from high school and about 8 percentage points less likely to transfer in-state. This large difference in transfers for females makes it difficult to know how to interpret the net effect of winning the lottery on high school graduation. A conservative approach might look only at the dropout variable, in which case the effect is about 3 percentage points and statistically insignificant. Thus the true effect is likely to be between 3 and 11 percentage points for females from low-performing schools. For males, however, the effects are much less ambiguous. Male lottery winners are about 13 percentage points more likely to graduate from a CMS school, and the effect is statistically significant at the p<0.01 level. They are only 1.7 percentage points less likely to transfer in state and over seven percentage points less likely to be affirmatively coded as David Deming DRAFT – DO NOT CIRCULATE September 2009 22 a dropout. Since slightly less than half of the control group graduates from a CMS high school, 13 percentage points represents a very large and economically significant effect.

4.2 – College Unlike the CMS withdrawal codes, data on college attendance from the National Student Clearinghouse are not limited by attrition from the district. Still, the NSC’s coverage is not perfect and the match process itself might also contain systematic errors. As long as these errors are unrelated to lottery status, the estimated effects will be attenuated but not biased. However we may worry if, for example, coverage in North Carolina or the Charlotte area is better than elsewhere and lottery losers are more likely to leave CMS. Given that the proportion of out-of- state transfers is relatively small (and as shown in the previous section, many of them are likely to be dropouts) and unrelated to winning the lottery, this is unlikely to be a concern. Still, exploring the coverage of the NSC is important for understanding any possible sources of bias as well as attenuation. Fortunately, the coverage of the NSC in North Carolina and surrounding states is excellent. Over 95 percent of total enrollment at four-year and public two-year colleges was in colleges covered by the NSC during the sample period. I also linked CMS administrative data directly with the largest two-year college in the area, Central Piedmont Community College. These data include information about degree-seeking enrollment and choice of program. They also allowed for an independent investigation of the match quality of the NSC data, which turned out to correspond very closely to the direct match to CPCC. See the Data Appendix for details. Table 9 contains results for postsecondary enrollment overall and in the low-performing subsamples by gender. The first three rows are outcomes on the extensive margin – enrollment overall and in 2 or 4 year colleges. The next six rows examine persistence in college over time, first by looking at enrollment beyond one year which is the most common “stop out” point for college students (Horn, 1998). I also calculate a measure of full-time equivalent semesters by multiplying enrollment status (full-time, half-time, less than half time) in each semester by the number of semesters.10 At the time of writing, many sample members were still enrolled in school. To incorporate later enrollees and to get a sense for the direction of effects over time, I also construct a variable that is equal to one if the student is enrolled in a 4 year college in the

10 About 15 percent of colleges in the sample do not report enrollment status. For these colleges I impute a mean status measure using a weighted average of enrollment in the IPEDS data. Results that exclude these colleges are unaffected. David Deming DRAFT – DO NOT CIRCULATE September 2009 23 last observed semester, and zero otherwise. I also look at the probability that students will transfer from a two to a four year college. Finally, I construct a measure of selectivity using the Barron’s College Guide, which is a dummy variable for enrollment in a college that is designated as “Competitive” or higher. In Column 2, we see that lottery winners are no more likely be enrolled in any college. However, they are about 6 percentage points more likely to be enrolled in a two-year degree- seeking program. This measure excludes CPCC enrollees who are taking basic skills, correspondence or other training courses that do not count towards a two-year degree (and are thus ineligible for financial aid). There is no overall impact on four-year enrollment or on measures of persistence or selectivity. Lottery winners are, however, about 5 percentage points (from a baseline of about 7) more likely to transfer from a 2 to a 4 year college. For the low-performing school subsample, the effects on college enrollment differ markedly by gender. Girls are more likely to enroll in a 4 year college, and the effect is large and statistically significant – about 12 percentage points off a baseline of 24 - for 4-year college enrollment (Table 9, Column 4). The size of the effect actually grows slightly with time. Female lottery winners are 9.6 percentage points more likely to enroll for more than one year in a 4-year college, an effect size that is a bit over half of the control mean. By the spring of 2008 they are 9.4 percentage points more likely to still be enrolled in a 4-year college, and they have already completed about 0.4 more full-time equivalent semesters. Importantly, there is little evidence that they attend more competitive colleges. Boys in the low-performing subsample, however, are not more likely to attend or persist in four-year colleges. The overall effect on enrollment for this sample is around zero, with offsetting but insignificant positive and negative effects for 2 and 4 year colleges respectively. Over time, however, male lottery winners actually attend less college, although the estimates are somewhat imprecise. They are about 7 percentage points less likely to be enrolled in a 4 year college in the spring of 2008, suggesting that the effect of winning the lottery may become more negative over time. What explains gender differences in college-going? Previous literature has consistently found that girls benefit more than boys from educational interventions (Kling et al 2005; Hastings et al 2006; Schanzenbach 2005; Angrist et al 2009). One possible explanation is that girls are more likely to choose schools that prepare all students for college, whereas boys choose David Deming DRAFT – DO NOT CIRCULATE September 2009 24 more career-oriented or technical programs. While boys are more likely than girls to choose the one career academy in the sample (Berry Academy of Technology), results are similar when this school is excluded (or when I limit the analysis to Berry only). More generally, the difference in college-going between lottery winners and losers is fairly constant across choice and neighborhood schools, so different choices are unlikely to explain much of the gender heterogeneity. Another possibility is that boys are less academically prepared than girls in the lottery sample, and that the benefits of choice are increasing in college readiness. To test this, I split the low-performing subsample into below and above-median on the predicted 4 year college attendance measure in Figure 3 and Table 6. The benefits of choice for males are much larger for the 55 percent who have below-median predicted college attendance. This group is about 17 percentage points more likely to graduate from high school and about 12 percentage points less likely to be arrested for a felony. The effect on college attendance is negative, but near zero and statistically insignificant. Strikingly, above-median males are actually 19 percentage points less likely to still be enrolled in a 4 year college at the end of the sample period. In contrast, the benefits of choice for girls are strictly increasing in academic preparation, although the differences are not as large. Thus it appears that the least-prepared boys and the most-prepared girls benefit from choice. One possible explanation for this puzzling pattern is that boys react differently than girls to changes in relative rank. Since lottery winners attend schools with academically stronger students, they are lower ranked initially among their peers than lottery losers.

4.3 – Other Outcomes In Table 10, we estimate the effect of winning the lottery to a first-choice school on possible indicators of school quality such as test scores, grades and course-taking, and behavioral outcomes. Panel A reports results on high school end-of-course (EOC) exams in various subjects. These tests are scored and standardized at the state level by the North Carolina Department of Public Instruction (NCDPI). Beginning in 2006, but not applicable to the lottery sample, passing all EOC exams became a high school graduation requirement. As part of North Carolina’s accountability system, school pass rates on EOC exams are reported on the NCDPI and CMS district websites. State law also requires that EOC exam scores form some percentage David Deming DRAFT – DO NOT CIRCULATE September 2009 25 of the final grade, so the exams are also high stakes for students. EOC exams are taken in multiple course types (for example, regular versus advanced Geometry) but 9th and 10th grade courses that do not offer EOC exams are usually remedial. For 11th and 12th graders, non-EOC courses are usually in more advanced subjects such as statistics or calculus. The difficulty in comparing EOC exam scores of lottery winners and losers is that not all students take the exams, and even those who do often take them in different years. This depends on the high school a student attends, but also on their middle school. Nearly half of students in the sample, for example, took the EOC Algebra I exam prior to 9th grade. For students from suburban school zones the percentage is even higher. Overall, the effect of winning the lottery on test scores is consistently negative but imprecisely estimated. The English I exam comes closest to universal coverage, with about 90 percent of rising 9th graders taking the exam at the end of the 2002-2003 school year. One possible reason for the negative test score impacts is that, on the margin, more students are taking the tests in first-choice schools. This is evident in Panel B, which shows overall increases in math and science EOC course-taking for lottery winners. If the marginal student induced to take an EOC test (rather than enrolling in a less demanding course) is less able than the average student, then the effect of winning the lottery on exam scores will appear negative because of selection bias into test-taking. Evidence on exam scores for the low-performing subsample is more mixed. Students from these schools are less likely to have taken EOC math exams prior to 9th grade. About 75 percent of rising 9th graders from these areas are enrolled in Algebra I, and most others are enrolled in remedial courses rather than EOC Geometry or Algebra II. The estimates for girls are imprecise and display no consistent pattern across the six EOC tests. Boys, however, score significantly higher on the Algebra I and Chemistry exams. The effects are large (about 0.25-0.3 standard deviations), but also imprecise. Because of selection into test-taking it is difficult to say anything definitive, but there is some evidence of test score improvements for boys from low- performing neighborhood school zones. The effects on grade point average, however, are large and statistically significant. Overall, lottery winners show a GPA improvement of about 0.08 on a 4 point scale. The effect is particularly large (about 0.2 grade points) for girls in the low-performing subsample. While the overall gains diminish by about 50 percent when grade point average is computed only for math David Deming DRAFT – DO NOT CIRCULATE September 2009 26 and science classes, they remain high for girls (0.175) and become statistically significant for boys (0.121) in the low-performing subsample. Panel C examines the impact of winning the lottery on absences and suspensions. Perhaps surprisingly, the effects are biggest for girls, who are absent around 3 fewer days and are about 9 percentage points less likely to have been suspended in the 2002-2003 and 2003-2004 school years. There is no statistically significant impact on absences for boys in the low-performing subsample, but they are over 10 percentage points less likely to be suspended in the 2002-2003 school year. Unfortunately I have very little information on the nature and degree of the disciplinary incident that warranted suspension. The final set of results in Panel D examines the effect of winning the lottery on students’ relative position in school. Each row ranks students in their 9th grade enrolled school cohort (for simplicity we limit to rising 9th graders) according to their 8th grade math score. As seen in the wide range of average test scores by high school shown in Table 1, students with a standardized math score of zero will be near the bottom at high-performing but near the top at low-performing high schools. The second and third rows rank students similarly by their 9th grade English score and overall GPA. For outcomes like college attendance, relative ranking could be just as important as objective measures such as state test scores if schools expend more resources (such as attention from guidance counselors) on higher-ranked students. Many colleges factor class rank specifically in their admissions decisions, although they may accept more students overall from higher-performing high schools. Overall, lottery winners enter their first choice school about 4 percentile ranks lower on the 8th grade math test. This difference is larger for the low-performing subsample, about 7 and 5 percentile ranks for girls and boys respectively. The last three rows of Table 10 compute students’ GPA rank, which is of course correlated with ranking by initial ability (i.e. 8th grade math) but also includes effort in high school. Interestingly, female lottery winners make up for their lower entering rank by improving their grades so that they rank about as highly as the control group (at around the 53rd percentile, above the overall average). Male lottery winners, however, rank about 6.5 percentile points lower than the control group, despite an absolute increase in GPA. This is even clearer when we look at the probability of being in the top or bottom quartile of class rank. Male lottery winners from low-performing school zones are about David Deming DRAFT – DO NOT CIRCULATE September 2009 27 10 percentage more likely and 6 percentage points less likely to be in the bottom and top quartile of class rank respectively. This does not hold for females. The drop in class rank for male lottery winners may help explain some of the decrease in 4-year college attendance that we observed in Table 9. Unconditionally, less than ten percent of boys in the bottom quartile of the class rank distribution attend a 4-year college. Among those in the top quartile, the four-year enrollment rate is over 60 percent, and these percentages hold roughly equally for students in one of the four lowest-performing and one of the three magnet schools in CMS. The estimates in Table 10 show that lottery winners are 10 percentage points more likely to be in the bottom quartile and about 6 points less likely to be in the top quartile. If we multiply these changes by the overall enrollment percentages in each quartile of the treatment and control groups, we find that rank changes could explain about 4 percentage points of the difference in 4-year college enrollment, which is a relative large share of the total effect.11 This strongly suggests that changes in relative rank are important in explaining patterns of 4 year enrollment by gender.

5. Discussion Overall, the pattern of results suggests that lottery applicants from low-performing home school zones benefit greatly by winning admission to their first choice school. We find large gains in grade attainment and high school graduation, particularly for males. Female lottery winners from these areas are more likely to attend and persist in four-year colleges. Males are actually less likely to attend a four-year college, although part of this may be explained by relative rank changes within a school. We find little or no evidence for students who live in neighborhoods that are assigned to schools of even average quality. This is perhaps unsurprising since in Charlotte many of the better-off students and families have already Tiebout-chosen their preferred school. A key issue for interpretation is how and why students’ first choice schools were different from the neighborhood schools they would have otherwise attended. We show that students from low-performing neighborhood school zones (but not from other neighborhoods) experience relatively large improvements in measured peer quality. Yet the schools they attend are not the

11 It can explain over 100 percent of the 4-year enrollment margin (about 3 percentage points) and about 60 percent of the difference in the “still enrolled, Spring 2008” variable (about 7 percentage points). David Deming DRAFT – DO NOT CIRCULATE September 2009 28 highest-performing in the district and are located in the same inner city areas as their neighborhood schools. Although the results are not reported here, we found very little difference across schools in measures of teacher quality such as certification, advanced degree receipt and years of experience. We also found no evidence of gains in traditional measures of school inputs such as pupil-teacher ratios or per pupil funding. One possible reason for the lack of difference in teachers and other inputs is that they were intentionally adjusted by CMS in response to anticipated student demand or actual changes in enrollment patterns in the year of the choice plan. Although previous research has found that more qualified teachers for grades 3-8 sorted toward better schools (Jackson 2008), we find no evidence of that at the high school level, although we were unable to calculate value-added measures cleanly due to the wide variety of course offerings across schools and programs. Although overall per-pupil funding increased only modestly in the year of the choice plan, CMS did open a new magnet high school located in the inner city (Berry Academy) in 2002. It also expanded capacity at oversubscribed schools. Though net spending did not increase, the district may have effectively reallocated resources toward disadvantaged students in an effort to make the transition from busing to choice more palatable. This suggests that we may not be able to generalize from the CMS case to other open enrollment plans that are implemented without any additional changes. Another difficult distinction to make is between the effect of choice itself and an increase in school quality (if any) that results from receiving admission to one’s first choice school. If the benefits of school choice came largely through idiosyncratic match quality between student and school, for example, there would be little reason to think that measured differences in quality between the home and choice school could predict academic gains. Suppose instead of randomizing over applicants, we randomized admission (and enrollment) to highly demanded schools over all the students in the lowest performing schools. If the effects were similar to those found here, we could interpret them as gains from school quality rather than from choice. Thus a key issue is the extent to which applicants from these schools are different than those who choose to remain at home. Unfortunately, we can say very little about this. Although the density of predicted 4 year enrollment in Figure 3 suggests that home and non-applicants are similar on observables, there is clearly something different about them since they chose to attend a different school. David Deming DRAFT – DO NOT CIRCULATE September 2009 29 Lottery-based randomization allows for an unbiased causal estimate of the effect on individual students of being offered admission to a first choice school. It does not, however, answer an equally important question – what is the total effect of implementing an open enrollment plan? Because all the estimates we present are differences between the treatment and control group, they could be masking overall level differences in outcomes that occurred as a result of school choice. One particular possibility is that what we call the benefits of winning are actually the negative impacts of losing the lottery, relative to what would have happened if CMS still bused or simply rezoned without implementing choice. If average peer quality matters, then the exodus of the “best” students from low performing schools may have made stayers worse off. One limited way to test this, however, is first to look at absolute rates of college-going by year. The rising 9th grade cohorts from 2000 to 2002 attended 4 year colleges at roughly equal rates. This suggests that (absent compositional changes) there was no large overall gain or loss from choice. To address relative changes within cohorts, we compare the predicted probability of 4- year college enrollment in Figure 3 to actual enrollment for lottery winners and losers. We find that girls from low performing schools who win the lottery outperform their prediction by about 2 percentage points, and lottery losers underperform by 5.3 percentage points. Thus about 70 percent of the unconditional difference between winners and losers comes from a reduction in college-going in the control group. For boys it is the treatment group that makes up 70 percent of the difference – winners underperform by nearly 4 percentage points and losers beat the prediction by 1.8 percentage points. A final issue concerns the nature of the treatment. Even assuming that highly demanded schools are “better” in some sense, the effect of being offered admission and even enrolling is not straightforward to predict. Better schools may also be more challenging. Unlike receiving cash or some other no-strings-attached benefit, the benefit of attending a better school are probably increasing in effort and may not be strictly positive. If courses are harder but students do not adjust their effort in response, we may see worse or better average outcomes, as well as increased heterogeneity depending on the ability of the student. If boys are less likely to increase effort, this may provide a partial explanation for the pattern of results. Still, the treatment of admission to a first choice school encompasses all these things, and so the combination of increased graduation and grade point average but reduced college attendance and relative rank is ultimately only suggestive. David Deming DRAFT – DO NOT CIRCULATE September 2009 30

6. Conclusion In this paper we study the impact of open enrollment in high schools in Charlotte Mecklenburg school district (CMS) on college attendance and criminal activity. The central finding of this paper is that students with the lowest performing guaranteed (or “home”) schools benefit greatly from choice. Students from neighborhoods that are assigned to one of the four lowest-performing schools stay in school longer, graduate at higher rates, and are more likely to attend college. In contrast, we find no consistent evidence of gains for students whose home school is of even average quality. This paper makes several contributions. It is, to our knowledge, the first academic study to link assignment to and performance in high schools to a national administrative database of college-going. This, combined with lottery-based random assignment of low-SES students to highly demanded schools, makes the CMS choice plan an ideal setting to observe the effects of broad-based school choice in a large and heterogeneous urban school district. It is also one of the only studies of changes in student assignment (and one of the few in education research more generally) to look at long-run outcomes other than test scores. Finally, given the recent policy emphasis on early childhood and elementary school interventions, it is important to emphasize that high school may not be too late to intervene. While IQ or cognitive ability may be fixed early in life, this paper shows that high schools can play an important role in getting children into college and out of prisons without improving their test scores substantially. David Deming DRAFT – DO NOT CIRCULATE September 2009 31 References Abdulkadiroğlu, Atila, and Tayfun Sönmez. (2003). “School Choice: A Mechanism Design Approach.” American Economic Review, 93(3):729-747.

Anderson, Michael. (2008). “Multiple inference and gender differences in the effects of early intervention: A reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects.” Journal of the American Statistical Association, 103, 1481-1495.

Angrist, Joshua, Daniel Lang, and Philip Oreopoulos. (2009). "Incentives and Services for College Achievement: Evidence from a Randomized Trial." American Economic Journal: Applied Economics, 1(1): 136–63.

Abdulkadiroğlu, Atila , et al (2009). “Informing the Debate: Comparing Boston’s Charter, Pilot and Traditional Schools”. The Boston Foundation.

Bayer, Patrick, Fernando Ferreira and Robert McMillan (2006). “A Unified Framework for Measuring Preferences for Schools and Neighborhoods”. Journal of Political Economy, 115(4): 588-638.

Black, Sandra E. 1999. “Do Better Schools Matter? Parental Valuation of Elementary Education.” Quarterly Journal of Economics, 114 (May): 577–99.

Cullen, Julie Berry, Brian A. Jacob, and Steven Levitt. (2006). “The Effect of School Choice on Student Outcomes: Evidence from Randomized Lotteries.” Econometrica, 74(5):1191- 1230.

Cullen, Julie Berry, Brian Jacob and Steven Levitt. (2005). “The Impact of School Choice on Student Outcomes: An Analysis of the Chicago Public Schools,” Journal of Public Economics, 89(5-6): 729-760.

Cullen, Julie Berry and Brian Jacob (2007). “Is Gaining Access to Selective Elementary Schools Gaining Ground? Evidence from Randomized Lotteries”, National Bureau of Economic Research Working Paper No. 13443.

Deming, David (2009). “Early Childhood Intervention and Life-Cycle Skill Development: Evidence from Head Start”. American Economic Journal – Applied Economics, forthcoming.

Dobbie, Will and Roland G. Fryer Jr.(2009). “Living in the Zone: An Analysis of a Bold Social Experiment in Harlem”. Working Paper.

Figlio, David (2006). “Testing, Crime and Punishment”. Journal of Public Economics, 90(4-5): 837-851.

Gamoran, Adam. (1996). “Student Achievement in Public Magnet, Public Comprehensive, and Private City High Schools.” Educational Evaluation and Policy Analysis, 18(1):1-18. David Deming DRAFT – DO NOT CIRCULATE September 2009 32 Hanushek, Eric (2003). “The Failure of Input-Based Schooling Policies.” Economic Journal, 113: F64-98.

Hastings, Justine S., and Jeffrey M. Weinstein. (2008). “Information, School Choice, and Academic Achievement: Evidence from Two Experiments.” Quarterly Journal of Economics.

Hastings, Justine S., Thomas J. Kane, and Douglas O. Staiger. (2006). “Gender and Performance: Evidence from School Assignment by Randomized Lottery,” American Economic Review Papers and Proceedings, 96(2):232-236.

Hastings, Justine S., Thomas J. Kane, and Douglas O. Staiger. (2008). “Heterogeneous Preferences and the Efficacy of Public School Choice”. Working Paper.

Holmstrom, Bengt and Paul Milgrom (1991). “Multitask Principal-Agent Analyses: Incentive Contracts, Asset Ownership and Job Design”. Journal of Law, Economics and Organization, 7(0): 24-52.

Howell, William G. and Paul Peterson (2002). The Education Gap: Vouchers and Urban Schools, Washington, DC: Brookings Institution Press.

Hoxby, Caroline M. (2003). “School choice and school productivity: could school choice be a tide that lifts all boats?” The Economics of School Choice, ed. CM Hoxby, 287-341. Chicago: The University of Chicago Press.

Hoxby Caroline M. and Sonali Murarka (2007). “Charter schools in New York City: who enrolls and how they affect their students’ achievement”. Working Paper, New York City Charter Schools Evaluation Project.

Hoxby Caroline M., and Jonah Rockoff (2004). “The impact of charter schools on student achievement.” Working Paper, Columbia University.

Jackson, C. Kirabo (2009). “Student Demographics, Teacher Sorting and Teacher Quality: Evidence from the End of School Desegregation”. Journal Labor Economics, forthcoming.

Jacob, Brian (2005). “Accountability, Incentives and Behavior: Evidence from School Reform in Chicago”. Journal of Public Economics, 89(5-6): 761-796.

Kane, Thomas J. (2003). “A Quasi-Experimental Estimate of the Impact of Financial Aid on College-Going.” National Bureau of Economic Research Working Paper 9703.

Krueger, Alan B. (2003). “Inequality: Too Much of a Good Thing” In Benjamin Friedman, ed., Inequality in America: What Role for Human Capital Policies? Cambridge, MA: MIT Press, 1- 77. David Deming DRAFT – DO NOT CIRCULATE September 2009 33 Long, Bridget Terry (2004). “How have college decisions changed over time? An application of the conditional logistic choice model.” Journal of Econometrics, 121(1-2): 271-296.

Murnane, Richard J., Frank Levy and John B. Willett (1995). “The Growing Importance of Cognitive Skills in Wage Determination”. The Review of Economics and Statistics, 77(2):251- 266.

Pathak, Parag A., and Tayfun Sönmez. (forthcoming). “Leveling the Playing Field: Sincere and Sophisticated Players in the Boston Mechanism.” American Economic Review.

Rothstein, Jesse (2006). “Good principals or good peers? Parental valuations of school characteristics, Tiebout equilibrium, and the incentive effects of competition among jurisdictions.” American Economic Review, 96(4):1333-50.

Rothstein, Jesse (2009). “Student sorting and bias in value added estimation: Selection on observables and unobservables”. National Bureau of Economic Research Working Paper No. 14666.

Rouse Cecilia E. (1998). “Private school vouchers and student achievement: an evaluation of the Milwaukee Parental Choice Program”. Quarterly Journal of Economics, 113(2):553-602.

Schanzenbach, Diane (2006). “What have researchers learned from Project STAR?” Brookings Papers on Education Policy, 205-228.

Witte, John F., Troy D. Sterr, and Christopher A. Thorn. (1995). Fifth-Year Report: Milwaukee Parental Choice Program. Madison, Wisconsin: University of Wisconsin.

Witte, John F. (1997). “Achievement affects of the Milwaukee voucher program.” Working Paper, University of Wisconsin at Madison.

Wolf, Patrick et al (2008). “Evaluation of the DC Opportunity Scholarship Program: impacts after two years.” U.S. Department of Education, Institute of Education Science (NCEE 2000- 4023). David Deming DRAFT – DO NOT CIRCULATE September 2009 34

Figure 1: Map of CMS High School Zones in the 2001-2002 School Year

Notes: 2001 is the last year of busing. The diagonal lines represent areas that were rezoned the following year. “New Southwest” becomes Waddell and “New North” becomes Hopewell. David Deming DRAFT – DO NOT CIRCULATE September 2009 35

Figure 2: Map of CMS High School Zones in the 2002-2003 School Year

Notes: 2002 marked the end of busing and the beginning of school choice in CMS. The diagonal lines represent areas that were rezoned in the year of the choice plan. David Deming DRAFT – DO NOT CIRCULATE September 2009 36 Figure 3A

Predicted Probability of 4 Year College Attendance by Choice Status 2 5 . 1 1 5 . 0

0 .2 .4 .6 .8 1

Chose Home All Admitted All Denied Lottery Sample David Deming DRAFT – DO NOT CIRCULATE September 2009 37 Figure 3B

Predicted Probability of 4-Year College Attendance by Choice Status, Low-Performing Schools Only 3 2 1 0

0 .2 .4 .6 .8 1

Chose Home Degenerate Lotteries Lottery Sample David Deming DRAFT – DO NOT CIRCULATE September 2009 38 Figure 4

Felony Arrests (Males)

0.15 0.10 0.05 0.00 -0.05 -0.10 -0.15 May Aug Nov Feb May Aug Nov Feb May Aug Nov 06 06 06 07 07 07 07 08 08 08 08

Felonies Lower CI Upper CI Control Mean David Deming DRAFT – DO NOT CIRCULATE September 2009 39

Table 1: School Characteristics by Year and Neighborhood Zone 8th Grade Math Percent Black Enrollment 2001 2002 Home 2001 2002 Home 2001 2002 Home Utilization (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) Providence 0.41 0.73 0.90 15% 10% 4% 680 545 463 118% South Meck 0.19 0.34 0.53 24% 17% 13% 452 553 560 99% Butler -0.09 0.26 0.52 38% 24% 14% 506 545 437 125% Myers Park 0.42 0.49 0.35 28% 30% 36% 632 598 519 115% Hopewell -0.06 0.03 0.12 28% 30% 29% 414 424 476 89% North Meck 0.24 0.15 0.08 23% 32% 36% 329 483 401 120% East Meck -0.02 -0.12 -0.03 36% 49% 38% 565 589 480 123% Olympic -0.17 -0.15 -0.04 45% 43% 45% 260 295 429 69% Independence 0.01 -0.24 -0.22 45% 52% 55% 586 629 657 96% Vance -0.16 -0.32 -0.22 51% 61% 61% 657 537 718 75% West Meck -0.42 -0.49 -0.29 64% 60% 68% 422 428 739 58% Waddell -0.46 -0.39 -0.33 52% 55% 46% 331 254 329 77% Garinger -0.60 -0.79 -0.45 68% 72% 68% 428 286 574 50% West Charlotte -0.63 -0.64 -0.63 76% 88% 87% 447 316 484 65% Harding University 0.06 0.09 62% 73% 393 417 Berry Academy -0.23 76% 433 Northwest Arts -0.37 -0.35 45% 51% 153 162

Notes: The first and second columns report the average 8th grade math scores of rising 9th grade freshmen at each high school in 2001 and 2002 respectively. The third column reports average test scores for students in each school's neighborhood assigment zone. Test scores are reported in standard deviation units where zero is the North Carolina state average. Columns 4 through 6 report the percent of rising 9th grade freshman that are black for 2001, 2002 and home assignment zones respectively. Columns 7 through 9 report entering 9th grade cohort sizes for each school or zone. Column 10 is the utilization rate for each school, defined as enrollment divided by zone size, or Column 8 divided by Column 9. The three bottom rows are magnet schools which do not have home assignment zones. David Deming DRAFT – DO NOT CIRCULATE September 2009 40

Table 2: Choice Patterns of 9th Grade Applicants, by Neighborhood "Home" School All Students Non-Guaranteed Applicants Lottery Sample N % Chose Home N Admitted N Admitted Weight (1) (2) (3) (4) (5) (6) (7) Providence 1,812 95% 98 63% 9 56% 0% South Meck 2,108 76% 502 73% 124 31% 6% Butler 1,648 60% 653 41% 38 39% 2% Myers Park 1,849 75% 465 69% 77 68% 4% Hopewell 1,851 79% 390 49% 66 42% 3% North Meck 1,709 61% 662 43% 50 72% 2% East Meck 1,906 54% 872 57% 190 25% 8% Olympic 1,518 58% 637 62% 149 34% 7% Independence 2,376 68% 771 56% 156 45% 8% Vance 2,591 65% 905 53% 250 52% 13% West Meck 2,383 49% 1,216 55% 319 56% 17% Waddell 1,187 65% 415 60% 138 43% 7% Garinger 1,762 37% 1,104 48% 246 41% 13% West Charlotte 1,542 35% 1,005 67% 204 70% 9% Total 26,242 9,695 2,016

Notes: Rows are organized by the total number of students in each school's neighborhood assignment zone. Column 1 contains the total number of students that submitted a choice and Column 2 is the percent of students in each assignment zone that chose their home school. Column 3 is all lottery applicants in each home zone that chose any non-guaranteed (non-home) school and the percent who were admitted is in Column 4. Column 5 shows the number of students from each neighborhood zone who are in the lottery sample (i.e. that applied to schools where the probability of admission was neither zero nor one) and Column 6 shows the percent that were admitted. Column 7 calculates the total weight applied to students in each zone in an overall regression of the effect of winning a lottery on outcomes. The weight is calculated as (N*(P*(1-P))/ΣN) where N is the total number of students in Column 5 and P is the percent of students admitted in Column 6. David Deming DRAFT – DO NOT CIRCULATE September 2009 41

Table 3: Choice Patterns of 9-12th Grade Applicants, by 1st Choice School All Students Non-guaranteed Applicants Applicants in Lottery Sample N N Admitted Enroll | Admit N Admitted Enroll | Admit Weight (1) (2) (3) (4) (5) (6) (7) (8) Providence 2,189 543 78% 93% 97 24% 96% 4% South Meck 1,877 393 65% 85% 124 33% 82% 7% Butler 1,705 818 61% 94% 147 29% 100% 8% Myers Park 2,267 768 65% 91% 89 13% 100% 3% Hopewell 1,575 311 18% 80% 47 45% 70% 3% North Meck 1,502 465 42% 90% 95 5% 100% 1% East Meck 1,736 749 58% 94% 10 50% 77% 1% Olympic 871 176 81% 87% 7 57% 50% 0% Independence 2,738 1,104 27% 91% 185 12% 85% 5% Vance 2,233 794 33% 96% 5 20% 100% 0% West Meck 1,037 307 70% 84% 14 86% 92% 0% Waddell 912 241 61% 77% 23 17% 100% 1% Garinger 949 457 82% 86% 0% West Charlotte 1,271 623 80% 89% 137 88% 89% 4% Harding University 1,566 662 54% 92% 217 60% 90% 13% Berry Academy 1,007 990 65% 80% 702 62% 88% 42% Northwest Arts 807 294 36% 47% 117 60% 43% 7% Total 26,242 9,695 2,016

Notes: Rows are organized by the total number of students that listed each school as their first choice. Column 1 contains the total number of students that submitted a choice. Column 2 is the number of applicants to each school that were not guaranteed admission, Column 3 is the percent of applicants that were admitted, and Column 4 is the fraction of admitted applicants that subsequently enrolled. Columns 5 through 7 repeat this exercise for the lottery sample (i.e. that applied to schools where the probability of admission was neither zero nor one) and Column 8 calculates the total weight applied to students in each zone in an overall regression of the effect of winning a lottery on outcomes. The weight is calculated as (N*(P*(1-P))/ΣN) where N is the total number of students in Column 5 and P is the percent of students admitted in Column 6. David Deming DRAFT – DO NOT CIRCULATE September 2009 42 Table 4: Characteristics of Lottery Applicants Chose Non-Guaranteed School Any Choice Chose Home Admitted Waitlisted Lottery No Choice (1) (2) (3) (4) (5) (6) Male 50.0% 49.9% 49.7% 48.0% 53.8% 58.3% Black 41.1% 32.0% 53.9% 57.7% 61.1% 61.2% Hispanic 4.8% 4.8% 6.9% 2.5% 4.7% 10.9% Free / Reduced Lunch 26.2% 18.9% 46.4% 26.8% 39.8% 48.9% Special Education 28.9% 31.6% 26.8% 22.6% 22.2% 23.4% Limited Eng Prof. 7.7% 7.1% 13.1% 4.7% 5.6% 14.5% 8th Grade Math 0.03 0.18 -0.23 -0.27 -0.24 -0.73 8th Grad Reading 0.08 0.24 -0.21 -0.18 -0.20 -0.77 Sample Size 26,242 16,547 4,471 3,208 2,016 1,383

Column 1 presents summary statistics for all rising 9th-12th graders in CMS who submitted at least one choice in the lottery conducted during the spring of 2002. Column 2 contains summary statistics for the 63 percent of applicants who listed their guaranteed home school first. Columns 3 through 5 report summary statistics for students who listed a non-guaranteed (i.e. non- home or magnet) school first. Columns 3 and 4 contain all students in priority groups where everyone was admitted or everyone was denied, respectively. Column 5 is students in priority groups with randomization (i.e. where the probability of admission was greater than zero but less than one), or the lottery sample. Column 6 presents summary statistics for the less than 5% of students who did not submit a choice but were enrolled in a CMS high school in the Fall of 2002. David Deming DRAFT – DO NOT CIRCULATE September 2009 43

Table 5: Measures of School Quality Avg. Achievement Value Added Parental Demand 8th Read College English College Simple Cond. Logit (1) (2) (3) (4) (5) (6) Providence 0.69 0.606 -0.030 0.052 0.068 0.048 South Meck 0.34 0.533 0.032 0.070 0.030 0.006 Myers Park 0.33 0.540 0.037 0.060 0.055 0.043 Butler 0.27 0.430 0.098 -0.033 0.031 0.004 North Meck 0.27 0.527 0.021 0.109 0.054 0.002 Independence 0.16 0.347 0.070 -0.025 0.120 0.254 Hopewell 0.08 0.472 -0.021 0.061 0.024 0.141 East Meck -0.04 0.355 0.007 -0.028 -0.007 -0.051 Olympic -0.22 0.331 -0.075 -0.021 -0.081 -0.043 Vance -0.23 0.326 0.043 -0.020 -0.036 0.014 West Meck -0.62 0.156 -0.140 -0.130 -0.134 -0.093 Waddell -0.44 0.249 -0.072 -0.046 -0.044 -0.103 Garinger -0.75 0.212 0.036 -0.049 -0.134 -0.116 West Charlotte -0.80 0.164 -0.094 -0.091 -0.073 -0.112 Harding University 0.02 0.420 0.017 0.009 0.067 0.060 Berry Academy -0.32 0.302 -0.127 -0.036 0.134 0.094 Northwest Arts 0.01 0.364 0.056 -0.017 -0.076 -0.129 Correlations Avg. Read College Eng VA Coll VA Demand 1 Demand 2 Average Reading College 0.951 English VA 0.529 0.481 College VA 0.671 0.916 0.438 Demand 1 0.532 0.626 0.231 0.555 Demand 2 0.245 0.462 0.241 0.418 0.825

Notes: Columns 1 and 2 measure the average 8th grade reading score (in standard deviations units, on a statewide exam) and the percent of students attending at least 4 semesters of college 4among rising 9th grade students in each high school. Columns 3 and 4 estimate value-added measures for the 9th grade english score and college-going respectively. Value added is computed as the school-level mean residual from a student-level regression of each outcome on a 3rd order polynomial in 7th and 8th grade test scores, demographic characteristics and census tract fixed effects. Columns 5 and 6 are measures of parental demand. Column 5 is calculated simply as the fraction of families who list each school as their first choice, weighted by neighborhood size. Column 6 is the school residual from a conditional logistic regression of the probability that each student chooses each school, conditional on a 4th order polynomial in travel time to each school for each student plus home school and choice zone fixed effects. David Deming DRAFT – DO NOT CIRCULATE September 2009 44

Table 6: Validity of the Lottery and First Stage Effect on Enrollment All Initial Applicants Still Enrolled in Fall 2002 All Schools All Schools Low-Performing (1) (2) (3) (4) (5) (6) Male 0.535 0.022 0.532 0.022 0.519 0.057* [0.023] [0.025] [0.033] Black 0.506 0.021 0.518 0.020 0.648 0.064** [0.013] [0.014] [0.031] Hispanic 0.052 -0.007 0.050 -0.005 0.049 -0.010 [0.010] [0.011] [0.016] FRPL 0.330 -0.011 0.333 -0.005 0.479 -0.001 [0.011] [0.015] [0.019] Special Education 0.226 -0.015 0.227 -0.018 0.218 -0.038 [0.018] [0.019] [0.025] LEP 0.060 -0.006 0.060 -0.006 0.052 -0.001 [0.010] [0.010] [0.008] 8th Grade Math -0.238 0.017 -0.190 0.017 -0.381 0.013 [0.027] [0.029] [0.041] 8th Grade Reading -0.169 -0.008 -0.120 -0.013 -0.381 0.039 [0.028] [0.030] [0.054] Days Absent 10.76 0.03 10.47 -0.06 12.00 0.77 [0.67] [0.67] [0.62] Ever Suspended 0.233 0.009 0.230 0.007 0.279 0.001 [0.014] [0.015] [0.026] Predicted Pr(4 yr coll) 0.328 0.012 0.350 0.006 0.262 0.026 [0.010] [0.013] [0.018] Sample Size 2122 2016 907 Enrolled in CMS in Fall 2002 0.937 0.002 0.942 -0.001 [0.005] [0.007]

Notes: Each entry in the even numbered columns represents the point estimate from a regression of the outcome listed in each row on an indicator variable for winning the lottery, lottery fixed effects, and a vector of student demographic characteristics and prior year school fixed effects. The low-performing group consists of students assigned to one of the 4 lowest-scoring schools listed in Table 5 - West Meck, Waddell, Garinger and West Charlotte. Standard errors are in brackets below each estimate and are clustered at the lottery level. The odd numbered columns list the control mean for each outcome. *** = sig. at the 1% level; ** = sig. at the 5% level; * = sig. at the 10% level. David Deming DRAFT – DO NOT CIRCULATE September 2009 45

Table 7: Initial Enrollment and School Characteristics Not Low-Performing Low-Performing First Stage - Fall 2002 (1) (2) (3) (4) Enrolled in 1st Choice 0.415 0.529*** 0.272 0.578*** [0.069] [0.076] Enrolled in Neighborhood School 0.422 -0.374*** 0.413 -0.345*** [0.049] [0.051] Enrolled in Magnet School 0.129 0.261** 0.117 0.314** [0.119] [0.135] Distance to Enrolled School 6.42 2.67*** 6.03 0.13 [0.69] [0.43] Characteristics of School Attended Percent Black 0.387 0.050 0.544 0.003 {0.211} [0.048] {0.215} [0.042] Percent Free Lunch 0.257 0.041 0.392 -0.005 {0.183} [0.036] {0.192} [0.032] Avg. 8th Grade Math 0.058 0.018 -0.238 0.143** {0.380} [0.049] {0.399} [0.060] Graduation Rate 0.735 0.020*** 0.669 0.061*** {0.093} [0.006] {0.121} [0.010] 4-Year College Matriculation Rate 0.444 0.031*** 0.344 0.083*** {0.131} [0.009] {0.131} [0.016] Arrest Rate 0.109 -0.005** 0.144 -0.030*** {0.044} [0.002] {0.068} [0.005] Average Number of Drug Arrests 0.092 -0.010** 0.147 -0.051*** {0.046} [0.005] {0.135} [0.011] Avg. GPA -Math & Sci 1.949 0.057*** 1.746 0.156*** {0.240} [0.014] {0.283} [0.030] Percent Suspended 0.159 -0.012 0.220 -0.042*** {0.094} [0.008] {0.108} [0.010] Average Number of Absences 9.14 -1.24*** 10.74 -2.03*** {3.93) [0.34] {3.54} [0.38]

Notes: Each entry in the even numbered columns represents the point estimate from a regression of the outcome listed in each row on an indicator variable for winning the lottery, lottery fixed effects, and a vector of student demographic characteristics and prior year school fixed effects. The low-performing group consists of students assigned to one of the 4 lowest-scoring schools listed in Table 5 - West Meck, Waddell, Garinger and West Charlotte. Average peer characteristic variables are calculating using students' Fall 2002 enrollment and exclude the lottery sample from the base rate calculation. Standard errors are in brackets below each estimate and are clustered at the lottery level. The odd numbered columns list the control mean for each outcome. *** = sig. at the 1% level; ** = sig. at the 5% level; * = sig. at the 10% level. David Deming DRAFT – DO NOT CIRCULATE September 2009 46

Table 8: Effect of Winning the Lottery on Persistence, Graduation and Transfer All Schools Low Performing Girls Boys Persistence (1) (2) (3) (4) (5) (6) In 1st Choice, Fall 2003 0.359 0.442*** 0.259 0.489*** 0.288 0.497*** [0.060] [0.094] [0.082] Fall 2004 0.246 0.355*** 0.190 0.467*** 0.219 0.366*** [0.037] [0.071] [0.066] Fall 2005 0.176 0.341*** 0.171 0.463*** 0.193 0.346*** [0.032] [0.052] [0.071] Grade Attainment and Repetition Made it to Grade 10 0.859 0.051** 0.841 0.051 0.843 0.063 [0.025] [0.044] [0.053] Grade 11 0.767 0.042** 0.732 0.073 0.713 0.096*** [0.022] [0.072] [0.033] Grade 12 0.707 0.041** 0.680 0.090 0.630 0.094*** [0.018] [0.067] [0.032] Repeat 9th Grade 0.187 -0.027 0.212 -0.093** 0.230 -0.017 [0.028] [0.042] [0.083] Repeat 10th Grade 0.078 -0.004 0.071 -0.023 0.117 -0.023 [0.019] [0.042] [0.048]

Graduation and Transfer Graduated from CMS 0.608 0.041* 0.620 0.113* 0.480 0.134*** [0.023] [0.068] [0.038] Transfer In-State 0.124 -0.020 0.132 -0.081*** 0.145 -0.017 [0.016] [0.021] [0.027] Transfer Out of State 0.074 0.004 0.063 0.051 0.068 -0.008 [0.019] [0.038] [0.036] Dropout 0.132 -0.026 0.112 -0.032 0.163 -0.073 [0.017] [0.040] [0.050] No Code 0.118 -0.007 0.122 -0.080 0.199 -0.023 [0.023] [0.051] [0.043]

Notes: Each entry in the even numbered columns represents the point estimate from a regression of the outcome listed in each row on an indicator variable for winning the lottery, lottery fixed effects, and a vector of student demographic characteristics and prior year school fixed effects. The low-performing group consists of students assigned to one of the 4 lowest-scoring schools listed in Table 5 - West Meck, Waddell, Garinger and West Charlotte. The results for persistence and grade attainment exclude student in rising grades who either would have already graduated or had already attained each grade prior to the lottery (i.e. the results for persistence until the Fall of 2005 and for Grade 10 include only the rising 9th grade sample). Standard errors are in brackets below each estimate and are clustered at the lottery level. The odd numbered columns list the control mean for each outcome. *** = sig. at the 1% level; ** = sig. at the 5% level; * = sig. at the 10% level. David Deming DRAFT – DO NOT CIRCULATE September 2009 47

Table 9: Effect of Winning the Lottery on Postsecondary Outcomes All Schools Low Performing Girls Boys Ever Enrolled (1) (2) (3) (4) (5) (6) Any College 0.668 0.002 0.649 -0.017 0.588 -0.022 [0.018] [0.046] [0.040] 2 Year, Degree-Seeking 0.295 0.058* 0.307 0.065 0.258 0.065 [0.033] [0.064] [0.066] 4 Year 0.340 0.009 0.244 0.126*** 0.249 -0.029 [0.026] [0.035] [0.054] Persistence and Transfer >2 Semesters 0.467 0.007 0.405 0.081 0.371 -0.091* [0.018] [0.049] [0.047] >2 Semesters at a 4 year 0.275 0.012 0.190 0.096*** 0.190 -0.036 [0.015] [0.030] [0.038] Semester FTEs 2.36 -0.011 1.83 0.477** 1.70 -0.379 [0.086] [0.214] [0.248] Semester FTEs at a 4 year 1.60 -0.059 1.00 0.399** 1.11 -0.302 [0.104] [0.182] [0.244] Enrolled in 4 yr, Spring 2008 0.267 -0.012 0.195 0.094*** 0.190 -0.073** [0.017] [0.031] [0.035] 2 to 4 Transfer 0.068 0.050** 0.049 0.074* 0.041 0.018 [0.019] [0.043] [0.013] Selectivity Enrolled in "Competitive" 4 Yr 0.195 -0.022 0.127 0.023 0.104 -0.050 [0.015] [0.028] [0.036] Sample Size 2016 435 472

Notes: Each entry in the even numbered columns represents the point estimate from a regression of the outcome listed in each row on an indicator variable for winning the lottery, lottery fixed effects, and a vector of student demographic characteristics and prior year school fixed effects. The low-performing group consists of students assigned to one of the 4 lowest-scoring schools listed in Table 5 - West Meck, Waddell, Garinger and West Charlotte. FTE stands for full-time equivalent, and is calculated by multiplying the number of semesters enrolled by the numerical equivalent of enrollment status (1 for full-time, 0.5 for half-time and 0.25 for less than half). Spring 2008 is currently the last semester in which students are observed in college. Standard errors are in brackets below each estimate and are clustered at the lottery level. The odd numbered columns list the control mean for each outcome. *** = sig. at the 1% level; ** = sig. at the 5% level; * = sig. at the 10% level. David Deming DRAFT – DO NOT CIRCULATE September 2009 48

Table 11: Test Performance, Course-Taking and Behavior All Schools Low Performing Girls Boys Panel A: Test Scores and GPA (1) (2) (3) (4) (5) (6) English I -0.149 -0.051 -0.209 0.072 -0.486 0.015 9th Only [0.042] [0.096] [0.055] Algebra I -0.781 -0.027 -0.762 -0.010 -0.918 0.304*** 9th Only [0.054] [0.076] [0.093] Geometry -0.607 -0.097* -0.749 -0.078 -0.749 0.027 [0.054] [0.089] [0.075] Algebra II -0.457 -0.066 -0.346 -0.127 -0.784 0.132 [0.057] [0.144] [0.114] Biology -0.349 -0.087 -0.436 -0.073 -0.62 -0.167 [0.061] [0.082] [0.124] Chemistry -0.673 -0.101 -0.883 0.066 -1.059 0.250* [0.065] [0.089] [0.136] Grade Point Average 2.085 0.080*** 2.096 0.205*** 1.773 0.077 [0.027] [0.073] [0.064] GPA - Math and Science 1.696 0.045 1.702 0.175** 1.381 0.121** [0.034] [0.071] [0.056] Panel B: Course-Taking Any EOC Math - 2002-2003 0.715 0.089*** 0.732 0.080* 0.670 0.105** [0.022] [0.042] [0.047] Total Math Credits Earned 2.354 0.164*** 2.101 0.308** 1.962 0.267** [0.053] [0.150] [0.111] Total EOC Science Passed 1.326 0.136*** 1.268 0.165** 1.131 0.170*** [0.039] [0.076] [0.041] AP Math 0.058 -0.008 0.049 -0.016 0.032 0.027 [0.011] [0.047] [0.019] Panel C: Behavior Absences in 2003 12.54 -0.33 14.68 -2.68*** 13.74 -1.02 [0.57] [0.93] [1.31] Absences in 2004 10.86 0.60 12.84 -3.10* 10.77 -0.35 [0.89] [1.67] [1.38] Suspended in 2003 0.268 -0.081*** 0.254 -0.092*** 0.367 -0.106** [0.021] [0.034] [0.046] Suspended in 2004 0.219 -0.046** 0.229 -0.089 0.276 -0.027 [0.020] [0.080] [0.048] Sample Size 2016 435 472 Panel D: Relative Ranking All Schools Low-Performing Low-Performing Rank of 8th Grade Math 0.505 -0.038 0.484 -0.070** 0.474 -0.049** [0.024] [0.032] [0.019] GPA Rank 0.506 -0.021 0.530 0.010 0.446 -0.064* [0.018] [0.030] [0.036] Bottom Quartile of GPA 0.235 0.032 0.182 0.013 0.308 0.101* [0.029] [0.045] [0.054] Top Quartile of GPA 0.239 0.020 0.273 0.079 0.173 -0.060 [0.037] [0.067] [0.037] Sample Size (9th grade) 1230 315 323

Notes: Each entry in the even numbered columns represents the point estimate from a regression of the outcome listed in each row on an indicator variable for winning the lottery, lottery fixed effects, and a vector of student demographic characteristics and prior year school fixed effects. The low-performing group consists of students assigned to one of the 4 lowest-scoring schools listed in Table 5 - West Meck, Waddell, Garinger and West Charlotte. Sample sizes are smaller for each EOC exam since not all students take the exam. Relative rank is calculated by ordering the math scores and GPAs of each student within their assigned school and creating a percentile score from 0 to 100. Standard errors are in brackets below each estimate and are clustered at the lottery level. The odd numbered columns list the control mean for each outcome. *** = sig. at the 1% level; ** = sig. at the 5% level; * = sig. at the 10% level.