A Service of

Leibniz-Informationszentrum econstor Wirtschaft Leibniz Information Centre Make Your Publications Visible. zbw for Economics

Levitt, Steven D.

Article The economics of education

NBER Reporter Online

Provided in Cooperation with: National Bureau of Economic Research (NBER), Cambridge, Mass.

Suggested Citation: Levitt, Steven D. (2003) : The economics of education, NBER Reporter Online, National Bureau of Economic Research (NBER), Cambridge, MA, Iss. Winter 2003/04, pp. 15-17

This Version is available at: http://hdl.handle.net/10419/61839

Standard-Nutzungsbedingungen: Terms of use:

Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Documents in EconStor may be saved and copied for your Zwecken und zum Privatgebrauch gespeichert und kopiert werden. personal and scholarly purposes.

Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle You are not to copy documents for public or commercial Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich purposes, to exhibit the documents publicly, to make them machen, vertreiben oder anderweitig nutzen. publicly available on the internet, or to distribute or otherwise use the documents in public. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, If the documents have been made available under an Open gelten abweichend von diesen Nutzungsbedingungen die in der dort Content Licence (especially Creative Commons Licences), you genannten Lizenz gewährten Nutzungsrechte. may exercise further usage rights as specified in the indicated licence. www.econstor.eu The Economics of Education

Steven D. Levitt*

In recent years, I have written a one in every seven school districts ences. In either of these cases, the number of papers related to the eco- nationally, and in more than a third of increased graduation rates represent nomics of education. This research large districts. Moreover, No Child the true benefits of open enrollment. agenda has three distinct strands. One Left Behind mandates that students in There are, however, scenarios in which set of papers analyzes the impact of underperforming schools be provided the students who take advantage of school choice on student outcomes. A the option to attend other schools in school choice outperform students second line of research investigates the district. who do not, but the differences in out- teacher and administrator cheating on Along with co-authors Julie B. comes do not actually reflect real benefits standardized tests, and explores how Cullen and Brian Jacob, I have written of open enrollment. Higher graduation such behavior responds to the intro- two papers that analyze the impact of rates among those who opt out may be duction of high-stakes testing. Third, I open enrollment policies on student spurious if those who opt out are bet- have examined Black-White test score outcomes in the Chicago Public ter on unobserved dimensions (for differentials and the role that the edu- Schools (ChiPS). ChiPS represents an example, student motivation, parental cational system may play in contribut- excellent laboratory for studying the involvement). In other words, the stu- ing to those differences. I discuss these impact of open enrollment. Chicago dents who opt out may have systemat- three sets of papers in turn. has been among the most aggressive ically done better than other students, cities in implementing this form of even if they had not left their assigned The Impact of Public school choice, with more than half of schools. Also, it is possible that the the students in the system presently graduation gap is attributable not to School Choice on Student opting out of their neighborhood the students who opt out doing better, Outcomes schools. Thus it may provide a window but rather to the students who remain into what the future holds for other behind doing worse, since they have less In recent years, school choice has districts that are moving in the same able and motivated peers. become an increasingly prominent fea- direction. The Chicago data are also Our results suggest that, with the ture of primary and secondary school exceptionally rich, including not only exception of career academies (that is, education. With the passage of new detailed administrative records on vocational schools that focus on prac- federal legislation (No Child Left attainment and test scores, but also tical skills), the benefits of school Behind), there is little doubt that the attitudinal surveys administered peri- choice to students who opt out are illu- trend will continue. School choice odically to students. sory. There are three primary pieces of comes in a variety of flavors. The first of these papers1 starts evidence supporting this claim. First, Vouchers and charter schools are two with the observation that students in a survey administered in eighth types of school choice which have who opt out of their local school to grade that asks students a wide range received a great deal of both academic take advantage of open enrollment are of questions about their expectations and media attention. A third type of 7.6 percentage points more likely to for the future, past educational record, school choice, open enrollment, is graduate from high school than peers and parental involvement, the respons- actually far more prevalent than either who are observationally equivalent in es are strongly correlated with both the vouchers or charter schools. Under eighth grade — off of a baseline grad- likelihood of graduation and with the open enrollment, students within a uation rate of 50 percent. This incre- decision to opt out. This suggests that public school district are able to attend ment to graduation is the same order students who opt out would be expect- schools other than their neighborhood of magnitude as the gap between stu- ed to do better, even if they had to school, including specially designated dents at Catholic and non-Catholic remain in their local school. The sec- magnet schools. As of 1996, open schools in previous studies. ond piece of evidence is that students enrollment was available in more than There are several competing who live in areas with many nearby explanations for why students who opt schools on average should derive the out of their assigned school outper- greatest benefit from the availability of * Levitt is a Research Associate in the form those who stay behind. Higher school choice, because distance to NBER’s Programs on Public Economics, graduation rates among those who opt nearby schools is a strong predictor of Law and Economics, Children, and out may be the result of these students the likelihood that a student will opt out Education. He is also a Professor of attending better schools or finding a of the assigned school. Empirically, we Economics at the University of Chicago. school that better matches their prefer- find that easy access to a career acade-

NBER Reporter Winter 2003/2004 15 my is associated with substantial ed enjoyment of school, availability of student fails to complete a section. increases in graduation likelihood, but computers, expectations for college Our work in this area represents the the same is not true for other types of attendance, and arrest rates. This sug- first systematic attempt to identify schools, including high-achieving gests that schools may be influencing empirically the overall prevalence of schools. Finally, when we compare stu- children in a variety of ways not gen- teacher cheating and to analyze the dent outcomes within a given school (in erally captured by test scores. To the factors that predict cheating. most schools in ChiPS some students extent that these non-traditional meas- To address these questions, we are assigned and some opt in), we find ures help to predict life outcomes such once again turn to data from the that those opting in do the same as as college attendance, labor market Chicago Public Schools, for which we those assigned at career academies, but attachment, wages, and criminal have the question-by-question answers do much better at other schools. Since involvement, an exclusive focus on test given by every student in grades 3-7 all students at a school experience sim- scores will be misleading. taking the Iowa Test of Basic Skills ilar peers and teacher quality, the fact An important caveat to interpret- (ITBS) over an eight year period. In that those opting in far outperform ing the results of both of these papers the first paper,3 we develop and test an those assigned to the school reinforces is that we are only able to evaluate how algorithm for detecting cheating. Our the idea that those who opt in are sys- access to a particular school affects approach uses two types of cheating tematically better than observationally educational outcomes for a student, indicators: unexpected test score fluc- similar students who make other holding constant the existence of a tuations and unusual patterns of schooling choices and would outper- school choice program. We cannot answers for students within a class- form them regardless, except at career estimate the overall impact of intro- room. Teacher cheating increases the academies. ducing a system of school choice, likelihood that students in a classroom Our second paper on this topic2 which might induce changes in residen- will experience large, unexpected exploits the fact that school choice tial location choice or in overall school increases in test scores one year, fol- causes desirable schools in ChiPS to be quality due to increased competition. lowed by very small test score gains (or oversubscribed, and many of these even declines) the following year. schools use randomized lotteries to Teacher Cheating Teacher cheating, especially if done in determine which students gain admis- an unsophisticated manner, is also like- sion. We analyze data from 194 sepa- High-stakes testing, like school ly to leave tell-tale signs in the form of rate lotteries held to gain access to choice, has become an increasingly blocks of identical answers, unusual high school. One drawback of the data prominent feature of the educational patterns of correlations across student is that we only observe student out- landscape. Every state in the country, answers within the classroom, or comes if they enroll in ChiPS. To the except Iowa, currently administers unusual response patterns within a stu- extent that there is selective attrition, state-wide assessment tests to students dent’s exam (for example, a student the inferences drawn from a simple in elementary and secondary school. who answers a number of very diffi- comparison of outcomes of lottery Federal legislation requires states to cult questions correctly while missing winners and losers will be misleading. test students annually in third through many simple questions). Relative to past studies (for example, eighth grade and to judge the perform- Empirically, we find evidence of the Milwaukee voucher experiment), ance of schools based on student cheating in approximately 4 to 5 per- however, attrition rates are low, with achievement scores. cent of the classes in our sample. For over 90 percent of the students The debate over high-stakes test- two reasons, this estimate is likely to be remaining in ChiPS. ing traditionally has pitted proponents a lower bound on the true incidence of Empirically, we find that those arguing that such tests increase incen- cheating. First, we focus only on the students who win the lotteries attend tives for learning and hold schools most egregious type of cheating, what appear to be substantially better accountable for their students’ per- where teachers systematically alter stu- high schools — for example, schools formance against opponents who dent test forms. There are other more with higher achievement levels and argue that the emphasis on testing will subtle ways in which teachers can graduation rates and lower levels of lead teachers to substitute away from cheat, such as providing extra time to poverty. Nonetheless, consistent with teaching other skills or topics not students, that our algorithm is unlikely our first paper discussed earlier, we directly tested on the exam. Along to detect. Second, even when test find little evidence that attending these with Brian Jacob, I have written two forms are altered, our approach is only sought-after programs provides any papers that explore a very different partially successful in detecting illicit benefit on a wide variety of traditional concern regarding high-stakes testing behavior. We then demonstrate that achievement measures, including stan- — cheating on the part of teachers the prevalence of cheating responds to dardized test scores, attendance rates, and administrators. As incentives for relatively minor changes in teacher course-taking patterns, credit accumu- high test scores increase, unscrupulous incentives. The importance of stan- lation, or grades. We do, however, find teachers may be more likely to engage dardized tests in the ChiPS increased evidence that attendance at such in a range of illicit activities, such as substantially with a change in leader- schools may improve non-traditional changing student responses on answer ship in 1996. Schools that scored low outcome measures, such as self-report- sheets, or filling in the blanks when a on reading tests were placed on proba-

16 NBER Reporter Winter 2003/2004 tion and faced the threat of reconstitu- White students on standardized tests. in test scores over time for Blacks and tion. Following the introduction of Even after controlling for a wide range Whites attending the same school, this policy, the prevalence of cheating of covariates including family struc- Black students lose only a third as rose sharply in classrooms with large ture, socioeconomic status, measures much ground as they do relative to numbers of low-achieving students. of school quality, and neighborhood Whites in the overall sample. This In contrast, schools with average or characteristics, a substantial racial gap result suggests that differences in qual- higher-achieving students, which were in test scores persists. ity across schools attended by Whites at low risk for probation, showed no In a paper joint with Roland and Blacks is likely to be an important increase in cheating. Fryer,5 I revisit this topic with a newly part of the story. Interestingly, along Our second paper on this topic4 collected data set, the Early Childhood “traditional” dimensions of school reports on the results of an unusual Longitudinal Study Kindergarten quality (class size, teacher education, policy implementation of our cheating Cohort (ECLS-K). The survey covers computer-to-student ratio, and so on), detection tools. We were invited by a sample of more than 20,000 children Blacks and Whites attend schools that ChiPS to design and implement audit- entering kindergarten in the fall of are similar. On a wide range of “non- ing and retesting procedures imple- 1998. The original sample of students standard” school inputs (for example, menting our methods. Using that has subsequently been re-interviewed gang problems in school, percent of cheating detection algorithm, we in the spring of kindergarten and first students on free lunch, amount of loi- selected roughly 120 classrooms to be grade. tering in front of school by non-stu- retested on the Spring 2002 ITBS. The The results we obtain using these dents, amount of litter around the classrooms retested include not only new data are informative and in some school, whether or not students need cases suspected of cheating, but also cases quite surprising. As in previous hall passes, and PTA funding), Blacks classrooms that had achieved large datasets, we observe substantial racial do appear to be attending much worse gains but were not suspected of cheat- differences in test scores in the raw data: schools. Other explanations for the ing, as well as a randomly selected con- Black kindergartners score on average divergence in Black-White test scores, trol group. As a consequence, the .64 standard deviations worse than such as a greater “summer setback” for implementation also allowed a Whites. In stark contrast to earlier stud- Blacks when school is not in session, or prospective test of the validity of the ies (including those looking at kinder- discrimination by teachers against tools we developed in our first paper gartners), however, after controlling for Blacks, find no support in our data. on the subject. a small number of other observable The results of the retesting pro- characteristics (children’s age, child’s 1 J. B. Cullen, B. Jacob, and S. D. Levitt, vided strong support for the effective- birth weight, a socio-economic status “The Impact of School Choice on Student ness of the cheating detection algo- measure, WIC participation, mother’s Outcomes: An Analysis of the Chicago rithm. Classrooms suspected of cheat- age at first birth, and number of chil- Public Schools,” NBER Working Paper ing experienced large declines in test dren’s books in the home), we essen- No. 7888, September 2000, forthcoming in scores (on average about one grade tially eliminate the Black-White test Journal of Public Economics. equivalent, although in some cases the score gap in math and reading for stu- 2 J. B. Cullen, B. Jacob, and S. D. Levitt, fall in mean classroom test scores was dents entering kindergarten. While “The Effect of School Choice on Student over three grade equivalents) when there are numerous possible explana- Outcomes: Evidence from Randomized retested under controlled conditions. tions for why our results differ so Lotteries,” forthcoming as an NBER In contrast, classrooms not suspected sharply from earlier research, we con- Working Paper. of cheating a priori maintained virtual- clude that real gains by recent cohorts 3 B. Jacob and S. D. Levitt, “Rotten Apples: ly all of their gains on the retest. As a of Blacks are likely to be an important An Investigation of the Prevalence and consequence of these audits and sub- part of the explanation. Predictors of Teacher Cheating,” NBER sequent investigations, disciplinary Despite the fact that we see no Working Paper No. 9413, January 2003, action was brought against a substantial difference in initial test scores for and Quarterly Journal of Economics, number of teachers, test administrators, observationally equivalent Black and 117 (August 2003), pp. 843-77. and principals. White children when they enter kinder- 4 B. Jacob and S. D. Levitt, “Catching garten, their paths diverge once they Cheating Teachers: The Results of an Black-White Test Score are in school. Between the beginning of Unusual Experiment in Implementing kindergarten and the end of first grade, Theory,” NBER Working Paper No. Gaps Early in Life and the Black students lose .20 standard devia- 9414, January 2003, and Brookings- Contribution of Schools tions (approximately .10 standard devi- Wharton Papers on Urban Affairs, ation each year) relative to White stu- 2003. The Black-White test score gap is dents with similar characteristics. The 5 R. Fryer and S. D. Levitt, “Understanding a robust empirical regularity. A simple leading explanation for the worse tra- the Black-White Test Score Gap in the First comparison of mean test scores typi- jectory of Black students in our sample Two Years of School,” NBER Working cally finds Black students scoring is that they attend lower quality Paper No. 8975, June 2002, forthcoming in roughly one standard deviation below schools. When we compare the change Review of Economics and Statistics.

NBER Reporter Winter 2003/2004 17